Rec count for multiple flat files - PS

kitchu84 · New User Joined: 02 Dec 2006 Posts: 33 Location: chennai

Hello Bill,

yes you are right ... its option B ...

Also, since we need to run the jobs automatically through CA7 so suppose we submitted 5 dynamic JCLS each of them creating a different file with counts. I have a challenge to merge and then use it in another Job.
Say there are 5000 file names . Hence the main job creates 5 dynamic JCLS and submits them - each of which in turn creates one output file of file counts .
Now since these dynamic JCLs are not defined in CA7, i need a way to put
dependancy of all these jobs/output files on another final job which merges the 5 output file and then uses the data... I cannot put the dependency of Main job on final job because the final job might abend
due to file not found if the dynamic jcls are still running.

Also, is there a possibility that it might take previous version of the file
into consideration?

I am sorry if this isnt the right place to ask this query.

Please guide ...

Skolusu · Posted: Sat May 14, 2011 2:41 am

Bill Woodger · Posted: Sat May 14, 2011 7:06 am

kitchu84 · New User Joined: 02 Dec 2006 Posts: 33 Location: chennai

Hi Skolusu,

With due respect to you - the next part in my post was intended for you. I apologise for not mentioning your name specifically.

I am getting syntax errors in this line:
INREC IFTHEN=(WHEN=GROUP,RECORDS=1000,
*

WER268A INREC STATEMENT : SYNTAX ERROR
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE

I searched in the forum and found a similar kind of issue: : ibmmainframes.com/viewtopic.php?t=51832&postdays=0&postorder=asc&start=0

This type of error suggests that its because WHEN=GROUP is not available in the current release we are using

... Could you please suggest an alternative.

Hi Bill : we will run the file counter every time after we run the SAR unloads. The SAR unloads run every 30 mins (for entire 24 hrs). Regarding the rerun, the timestamp are actual ... we are planning to load the data at that particular instant of time. So say a job runs at 12:30 pm, the SAR unload running at 1 pm will pick up that job details and we will run file counter for the job. If the job again runs at say 2:50 pm with different job id , the next SAR unload will pick up the details of the job and file counter will count the records at that instant of time which will be loaded with that particular timestamp.

Sorry if I am not clear. Please let me know ...

kitchu84 · New User Joined: 02 Dec 2006 Posts: 33 Location: chennai

Just to add, this release of sort is invoked when i submit the job :

SYNCSORT FOR Z/OS 1.3.0.2R

enrico-sorichetti · Posted: Sun May 15, 2011 12:10 pm

kitchu84 · New User Joined: 02 Dec 2006 Posts: 33 Location: chennai

:'( ... will miss Skolusu's solutions ...

Bill Woodger · Posted: Sun May 15, 2011 1:22 pm

enrico-sorichetti · Posted: Sun May 15, 2011 1:34 pm

Bill, looks like we are just wasting time here, as does the TS organization by counting things every half an hour

they made up their mind/bed let them sleep in it

Bill Woodger · Posted: Sun May 15, 2011 2:01 pm

Bill Woodger · Posted: Sun May 15, 2011 2:52 pm

enrico, are you going to move this one, for completeness? 54401 in DFSORT.

kitchu84 · New User Joined: 02 Dec 2006 Posts: 33 Location: chennai

hi Bill,

We are not running SAR full unload. As i mentioned earlier,
we will extract only last 30 mins SAR unloads for every 30 mins SAR unload run.

This is our process:

1. Unload from SAR every 30 minutes using SARBCH utility
2. Extract last 30 minutes data(jobs completed in last 30 mins)
3.Unload the “JESYSMSG” log from SAR (using SARBCH with PRINT command)for all the jobs present in the output file of step 2
4.Using SORT step, extract all the records from JESYSMSG with Jobname, Jobid and filenames(retained, rolled in and deleted) for all the jobs and exclude all library names
5.Using Easytrieve, format the file from step 4. The output file will contain jobname, jobid, filenames and keyword(retained,rolled in,deleted).

6. Using Easytrive, filter out the deleted files(based on keyword ‘deleted’) and write only the existing files into output. If the first instance of the file is retained/rolled in and second instance is deleted, then ignore them. The output file will contain jobname, jobid and filenames(only existing)

7. Using SORT job suggested by Skolusu - it will create N dynamic JCLs depending upon the no of files for which we need to determine the counts . No of files that each dynamic job will handle is 1000.

After the dynamic jobs are submitted, as Skolusu suggested if the
we keep the job card same the dynamic jobs will be submitted sequentially and each time the file counts will be added to same output file.

8. In the next step /next job I need to format the file from dynamic jcls.
since I need to create another file in the loadable format for loading in DB2 table.

Hi Skolusu - Please advise on how to achieve this. I am sorry if I am missing something.

Thanks for your suggestions.

Thanks,

Bill Woodger · Posted: Mon May 16, 2011 4:21 am

You are using Syncsort, which is why, as enrico has said, the topic has been moved to JCL from the DFSORT forum. Read enrico's message.

I got the SAR bit OK. Are you using a SAR date/time for your extract? You should, if it is possible.

The file from step 6 goes into step 7? 5000/16 = approx 312 datasets (16 = 8 hours of "peak" batch, x2 for the 30minutes). I don't think you'll often even go over 1000 datasets in 30 minutes? That's why I couldn't understand the big-big sort counter.

If you are already in Easytrieve, you could even generate the Syncsort JCL/cards (if necessary) and ship them off to the INTRDR. Easytrieve can again pick up the file you produce with the counts and match the counts to the formatted extract from SAR.

You've still got problems waiting for you to walk into them. Really think hard about how to make your DB2 flexible. Reruns. Business dates. Timings if no SAR date. "Static" files. Tapes. VSAM shareoptions. Online system coming up. Is all your work going to fit in 30 minutes, if not what happens as the next one kicks in?

Maybe you could have had all the programs changed by now...

pullaiah.cts · New User Joined: 02 Sep 2010 Posts: 50 Location: Pune

Hi Kolusuko,

Thanks for the solution you have provided,it's working good.I have one more requirement.My job is getting abended if any of the file in the input DSN is not cataloged but I want to skip the files if they are not exists and continue to the next file name.Can you please tell me where should I put this condition in the given code?

dick scherrer · Posted: Tue Jun 07, 2011 10:04 am

Hello,

You need to start a new topic for your question - this topic is a Syncsort topic and Kolusu does not support / answer Syncsort questions (Kolusu - DFSORT Development Team (IBM) ).

You cannot do what you want by modifying the sort control statements. One way to do what you want is to build the jcl for only those datasets that exist and then run your process using that jcl.

ridgewalker58 · New User Joined: 26 Sep 2008 Posts: 51 Location: New York

Someone please STOP me from getting kicked off of this FORUM.

I do understand that these forums are to help people who have questions about the technical tasks that they have been given.

BUT I also understand, from reading many of your responses, that there is some requirement for the TS to take the task as far as they can take it. They should have "investigated a solution", "performed some work to test their solution", and AFTER they have shown that they are still having issues, they should pose their questions.

I have been working in IT for over 40 years and still have questions and have relied on the expertise of others.

BUT, you folks have ANALYZED, DESIGNED, and CODED and DEBUGGED a solution to this person's problem. In this case you have given his company a system that they are going be able to charge the client for.

I hope this person appreciates that the solution that has been arrived at
has been GIVEN freely by people who have invested many years developing their skills and put forth their own efforts in solving problems.

I don't see here that the T.S. has done anything more than USED this forum to do the work that has been assigned to the T.S.

dick scherrer · Posted: Tue Jun 07, 2011 11:24 pm

Hello,

Yup, in spirit many of us completely agree. . .

Not only do many of the requestors not appreciate what they are given but are quite willing to accept credit for this within their organizations.

Many (one must consider how some people were treated as children) expect others to do their bidding.

On the other hand we have many who truly appreciate the effort made because of their request(s). Something else that i find quite encouraging is the number of views registered on many of our topics. I just have to believe that of the many who read these, some take away something useful even if they do not participate in the dialog.

Also, in the little-box world it is quite common for entire solutions to be provided (even before the requirement is clearly understood

). And quite often the "solution" has no chance of working. . .

Fortunately, we have people who post Whoa when a "solution" is not. . .

Bill Woodger · Posted: Thu Jun 09, 2011 12:34 am

enrico-sorichetti · Posted: Thu Jun 09, 2011 1:11 am

and the idTENts did not even carry on some basic arithmetics

given a conservative estimate of 2400 jobs a day under SAR control
a conservative estimate of 10 sequential datasets for each job

then every half an hour 500 datasets will be counted
and it might happen that in one interval more than 50 jobs will end
which means more datasets, more records to count,
the bean counting might take more than half an hour, and and and and ...
would be nice to be there when sith overflows

a non Politically Correct clone of Anuj signature
They have employed some monkey, given them PCs, made them sit in-front of PC for 8-hrs and called this set-up IT departments!

Anuj Dhawan · Posted: Thu Jun 09, 2011 8:48 pm

oh Enrico I was thinking to keep me as away as possible from this topic and see what you've done!

Bill Woodger · Posted: Thu Jun 09, 2011 8:56 pm