We are not running SAR full unload. As i mentioned earlier,
we will extract only last 30 mins SAR unloads for every 30 mins SAR unload run.
This is our process:
1. Unload from SAR every 30 minutes using SARBCH utility
2. Extract last 30 minutes data(jobs completed in last 30 mins)
3.Unload the “JESYSMSG” log from SAR (using SARBCH with PRINT command)for all the jobs present in the output file of step 2
4.Using SORT step, extract all the records from JESYSMSG with Jobname, Jobid and filenames(retained, rolled in and deleted) for all the jobs and exclude all library names
5.Using Easytrieve, format the file from step 4. The output file will contain jobname, jobid, filenames and keyword(retained,rolled in,deleted).
6. Using Easytrive, filter out the deleted files(based on keyword ‘deleted’) and write only the existing files into output. If the first instance of the file is retained/rolled in and second instance is deleted, then ignore them. The output file will contain jobname, jobid and filenames(only existing)
7. Using SORT job suggested by Skolusu - it will create N dynamic JCLs depending upon the no of files for which we need to determine the counts . No of files that each dynamic job will handle is 1000.
After the dynamic jobs are submitted, as Skolusu suggested if the
we keep the job card same the dynamic jobs will be submitted sequentially and each time the file counts will be added to same output file.
8. In the next step /next job I need to format the file from dynamic jcls.
since I need to create another file in the loadable format for loading in DB2 table.
Hi Skolusu - Please advise on how to achieve this. I am sorry if I am missing something.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
You are using Syncsort, which is why, as enrico has said, the topic has been moved to JCL from the DFSORT forum. Read enrico's message.
I got the SAR bit OK. Are you using a SAR date/time for your extract? You should, if it is possible.
The file from step 6 goes into step 7? 5000/16 = approx 312 datasets (16 = 8 hours of "peak" batch, x2 for the 30minutes). I don't think you'll often even go over 1000 datasets in 30 minutes? That's why I couldn't understand the big-big sort counter.
If you are already in Easytrieve, you could even generate the Syncsort JCL/cards (if necessary) and ship them off to the INTRDR. Easytrieve can again pick up the file you produce with the counts and match the counts to the formatted extract from SAR.
You've still got problems waiting for you to walk into them. Really think hard about how to make your DB2 flexible. Reruns. Business dates. Timings if no SAR date. "Static" files. Tapes. VSAM shareoptions. Online system coming up. Is all your work going to fit in 30 minutes, if not what happens as the next one kicks in?
Maybe you could have had all the programs changed by now...
Thanks for the solution you have provided,it's working good.I have one more requirement.My job is getting abended if any of the file in the input DSN is not cataloged but I want to skip the files if they are not exists and continue to the next file name.Can you please tell me where should I put this condition in the given code?
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hello,
You need to start a new topic for your question - this topic is a Syncsort topic and Kolusu does not support / answer Syncsort questions (Kolusu - DFSORT Development Team (IBM) ).
You cannot do what you want by modifying the sort control statements. One way to do what you want is to build the jcl for only those datasets that exist and then run your process using that jcl.
Someone please STOP me from getting kicked off of this FORUM.
I do understand that these forums are to help people who have questions about the technical tasks that they have been given.
BUT I also understand, from reading many of your responses, that there is some requirement for the TS to take the task as far as they can take it. They should have "investigated a solution", "performed some work to test their solution", and AFTER they have shown that they are still having issues, they should pose their questions.
I have been working in IT for over 40 years and still have questions and have relied on the expertise of others.
BUT, you folks have ANALYZED, DESIGNED, and CODED and DEBUGGED a solution to this person's problem. In this case you have given his company a system that they are going be able to charge the client for.
I hope this person appreciates that the solution that has been arrived at
has been GIVEN freely by people who have invested many years developing their skills and put forth their own efforts in solving problems.
I don't see here that the T.S. has done anything more than USED this forum to do the work that has been assigned to the T.S.
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hello,
Yup, in spirit many of us completely agree. . .
Not only do many of the requestors not appreciate what they are given but are quite willing to accept credit for this within their organizations.
Many (one must consider how some people were treated as children) expect others to do their bidding.
On the other hand we have many who truly appreciate the effort made because of their request(s). Something else that i find quite encouraging is the number of views registered on many of our topics. I just have to believe that of the many who read these, some take away something useful even if they do not participate in the dialog.
Also, in the little-box world it is quite common for entire solutions to be provided (even before the requirement is clearly understood ). And quite often the "solution" has no chance of working. . .
Fortunately, we have people who post Whoa when a "solution" is not. . .
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
ridgewalker58 wrote:
[...]
BUT, you folks have ANALYZED, DESIGNED, and CODED and DEBUGGED a solution to this person's problem. In this case you have given his company a system that they are going be able to charge the client for.
[...]
As a co-culprit on this topic, I don't think the whole thing went as well as you make out for the TS.
Right from the start up to the finish we mostly warned against such a solution as the TS sought, because it was basically nuts and destined to, at best, cause a whole heap of meaningless "problems" and at worst just not work.
Kolusu did valiant work to deliver a part of the requirement. That is, actually, part of his job. Unfortunately, the TS was all the time using SYNCSORT. A TI (Topic Interloper) later found Kolusu's solution useful. Such is life.
The TS was last seen reviewing solutions against doing the counts in the code for the programs which produced the files. Whether they/their management have learned anything through this process, I don't know.
Why did we (or I, as the only one I can speak for) expend so much on this? I think of someone running towards the clifftop because they want to get to the beach. You stop them, and show them the little, windy, steep, path down to the beach and that you can't see the bottom of the path and there are lots of potential daners, but also explain that there are lots of much better beaches with much better access. Having done that once, what do you do when they start running again? And again? And then get a motorbike? A sportscar?
If they come back again, rocket-propelled, I'm getting out of the way, finally.
Just finally, I didn't get the impression that the client would be buying the system (if it existed). I think the client is looking for a way to manage how the system leaks data. Good idea. Poor solution proposed by the client.
and the idTENts did not even carry on some basic arithmetics
given a conservative estimate of 2400 jobs a day under SAR control
a conservative estimate of 10 sequential datasets for each job
then every half an hour 500 datasets will be counted
and it might happen that in one interval more than 50 jobs will end
which means more datasets, more records to count,
the bean counting might take more than half an hour, and and and and ...
would be nice to be there when sith overflows
a non Politically Correct clone of Anuj signature
They have employed some monkey, given them PCs, made them sit in-front of PC for 8-hrs and called this set-up IT departments!
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Bill Woodger wrote:
[...]
Kolusu did valiant work to deliver a part of the requirement. That is, actually, part of his job. (emphasis added)
[...]
I overstated this. If anyone thinks "Bill Woodger says so" will get them some work done for no effort, think again. Start with the manuals, search the DFSORT forum, search the internet. Still stuck? Post all the details, including what you have done and how it failed.
). And quite often the "solution" has no chance of working. . .
