IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Rec count for multiple flat files - PS


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Sun May 15, 2011 2:52 pm
Reply with quote

enrico, are you going to move this one, for completeness? 54401 in DFSORT.
Back to top
View user's profile Send private message
kitchu84

New User


Joined: 02 Dec 2006
Posts: 33
Location: chennai

PostPosted: Mon May 16, 2011 2:50 am
Reply with quote

hi Bill,

We are not running SAR full unload. As i mentioned earlier,
we will extract only last 30 mins SAR unloads for every 30 mins SAR unload run.

This is our process:

1. Unload from SAR every 30 minutes using SARBCH utility
2. Extract last 30 minutes data(jobs completed in last 30 mins)
3.Unload the “JESYSMSG” log from SAR (using SARBCH with PRINT command)for all the jobs present in the output file of step 2
4.Using SORT step, extract all the records from JESYSMSG with Jobname, Jobid and filenames(retained, rolled in and deleted) for all the jobs and exclude all library names
5.Using Easytrieve, format the file from step 4. The output file will contain jobname, jobid, filenames and keyword(retained,rolled in,deleted).

6. Using Easytrive, filter out the deleted files(based on keyword ‘deleted’) and write only the existing files into output. If the first instance of the file is retained/rolled in and second instance is deleted, then ignore them. The output file will contain jobname, jobid and filenames(only existing)

7. Using SORT job suggested by Skolusu - it will create N dynamic JCLs depending upon the no of files for which we need to determine the counts . No of files that each dynamic job will handle is 1000.

After the dynamic jobs are submitted, as Skolusu suggested if the
we keep the job card same the dynamic jobs will be submitted sequentially and each time the file counts will be added to same output file.


8. In the next step /next job I need to format the file from dynamic jcls.
since I need to create another file in the loadable format for loading in DB2 table.



Hi Skolusu - Please advise on how to achieve this. I am sorry if I am missing something.


Thanks for your suggestions.


Thanks,
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon May 16, 2011 4:21 am
Reply with quote

You are using Syncsort, which is why, as enrico has said, the topic has been moved to JCL from the DFSORT forum. Read enrico's message.

I got the SAR bit OK. Are you using a SAR date/time for your extract? You should, if it is possible.

The file from step 6 goes into step 7? 5000/16 = approx 312 datasets (16 = 8 hours of "peak" batch, x2 for the 30minutes). I don't think you'll often even go over 1000 datasets in 30 minutes? That's why I couldn't understand the big-big sort counter.

If you are already in Easytrieve, you could even generate the Syncsort JCL/cards (if necessary) and ship them off to the INTRDR. Easytrieve can again pick up the file you produce with the counts and match the counts to the formatted extract from SAR.

You've still got problems waiting for you to walk into them. Really think hard about how to make your DB2 flexible. Reruns. Business dates. Timings if no SAR date. "Static" files. Tapes. VSAM shareoptions. Online system coming up. Is all your work going to fit in 30 minutes, if not what happens as the next one kicks in?

Maybe you could have had all the programs changed by now...
Back to top
View user's profile Send private message
pullaiah.cts

New User


Joined: 02 Sep 2010
Posts: 50
Location: Pune

PostPosted: Tue Jun 07, 2011 9:24 am
Reply with quote

Hi Kolusuko,

Thanks for the solution you have provided,it's working good.I have one more requirement.My job is getting abended if any of the file in the input DSN is not cataloged but I want to skip the files if they are not exists and continue to the next file name.Can you please tell me where should I put this condition in the given code?
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Jun 07, 2011 10:04 am
Reply with quote

Hello,

You need to start a new topic for your question - this topic is a Syncsort topic and Kolusu does not support / answer Syncsort questions (Kolusu - DFSORT Development Team (IBM) ).

You cannot do what you want by modifying the sort control statements. One way to do what you want is to build the jcl for only those datasets that exist and then run your process using that jcl.
Back to top
View user's profile Send private message
ridgewalker58

New User


Joined: 26 Sep 2008
Posts: 51
Location: New York

PostPosted: Tue Jun 07, 2011 9:05 pm
Reply with quote

Someone please STOP me from getting kicked off of this FORUM.

I do understand that these forums are to help people who have questions about the technical tasks that they have been given.

BUT I also understand, from reading many of your responses, that there is some requirement for the TS to take the task as far as they can take it. They should have "investigated a solution", "performed some work to test their solution", and AFTER they have shown that they are still having issues, they should pose their questions.

I have been working in IT for over 40 years and still have questions and have relied on the expertise of others.

BUT, you folks have ANALYZED, DESIGNED, and CODED and DEBUGGED a solution to this person's problem. In this case you have given his company a system that they are going be able to charge the client for.

I hope this person appreciates that the solution that has been arrived at
has been GIVEN freely by people who have invested many years developing their skills and put forth their own efforts in solving problems.

I don't see here that the T.S. has done anything more than USED this forum to do the work that has been assigned to the T.S.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Jun 07, 2011 11:24 pm
Reply with quote

Hello,

Yup, in spirit many of us completely agree. . .

Not only do many of the requestors not appreciate what they are given but are quite willing to accept credit for this within their organizations.

Many (one must consider how some people were treated as children) expect others to do their bidding.

On the other hand we have many who truly appreciate the effort made because of their request(s). Something else that i find quite encouraging is the number of views registered on many of our topics. I just have to believe that of the many who read these, some take away something useful even if they do not participate in the dialog.

Also, in the little-box world it is quite common for entire solutions to be provided (even before the requirement is clearly understood icon_wink.gif ). And quite often the "solution" has no chance of working. . .

Fortunately, we have people who post Whoa when a "solution" is not. . .
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Thu Jun 09, 2011 12:34 am
Reply with quote

ridgewalker58 wrote:


[...]
BUT, you folks have ANALYZED, DESIGNED, and CODED and DEBUGGED a solution to this person's problem. In this case you have given his company a system that they are going be able to charge the client for.

[...]


As a co-culprit on this topic, I don't think the whole thing went as well as you make out for the TS.

Right from the start up to the finish we mostly warned against such a solution as the TS sought, because it was basically nuts and destined to, at best, cause a whole heap of meaningless "problems" and at worst just not work.

Kolusu did valiant work to deliver a part of the requirement. That is, actually, part of his job. Unfortunately, the TS was all the time using SYNCSORT. A TI (Topic Interloper) later found Kolusu's solution useful. Such is life.

The TS was last seen reviewing solutions against doing the counts in the code for the programs which produced the files. Whether they/their management have learned anything through this process, I don't know.

Why did we (or I, as the only one I can speak for) expend so much on this? I think of someone running towards the clifftop because they want to get to the beach. You stop them, and show them the little, windy, steep, path down to the beach and that you can't see the bottom of the path and there are lots of potential daners, but also explain that there are lots of much better beaches with much better access. Having done that once, what do you do when they start running again? And again? And then get a motorbike? A sportscar?

If they come back again, rocket-propelled, I'm getting out of the way, finally.

Just finally, I didn't get the impression that the client would be buying the system (if it existed). I think the client is looking for a way to manage how the system leaks data. Good idea. Poor solution proposed by the client.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Thu Jun 09, 2011 1:11 am
Reply with quote

and the idTENts did not even carry on some basic arithmetics

given a conservative estimate of 2400 jobs a day under SAR control
a conservative estimate of 10 sequential datasets for each job

then every half an hour 500 datasets will be counted
and it might happen that in one interval more than 50 jobs will end
which means more datasets, more records to count,
the bean counting might take more than half an hour, and and and and ...
would be nice to be there when sith overflows icon_biggrin.gif

a non Politically Correct clone of Anuj signature
They have employed some monkey, given them PCs, made them sit in-front of PC for 8-hrs and called this set-up IT departments!
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Thu Jun 09, 2011 8:48 pm
Reply with quote

oh Enrico I was thinking to keep me as away as possible from this topic and see what you've done! icon_biggrin.gif
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Thu Jun 09, 2011 8:56 pm
Reply with quote

Bill Woodger wrote:

[...]

Kolusu did valiant work to deliver a part of the requirement. That is, actually, part of his job. (emphasis added)

[...]


I overstated this. If anyone thinks "Bill Woodger says so" will get them some work done for no effort, think again. Start with the manuals, search the DFSORT forum, search the internet. Still stuck? Post all the details, including what you have done and how it failed.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM Goto page Previous  1, 2, 3

 


Similar Topics
Topic Forum Replies
No new posts INCLUDE OMIT COND for Multiple values... DFSORT/ICETOOL 5
No new posts To get the count of rows for every 1 ... DB2 3
No new posts Write line by line from two files DFSORT/ICETOOL 7
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts Replace Multiple Field values to Othe... DFSORT/ICETOOL 12
Search our Forums:

Back to Top