Splitting dataset and checking for empty dataset

ssk1711 · New User Joined: 16 Jun 2008 Posts: 40 Location: bangalore

Hi

I got a requirement like-

"One of our Job gives an output dataset, which is processed by a Java application. whenever, the number of records in the dataset is less than 10000, the java application will process it successfully. If the number of records in the dataset is more than 10000, then we have to split the output data set into number of dataset each containing less than 10000 records."

Note:- Also we know, the number of records in output file will not exceed 100,000 records.

solution what I thought about is-
"we will split the output dataset in to 10 dataset in the first step and the next step will delete the empty datasets"

Note:- we have SYNCSORT in our shop

Question:- Will this solution is efficient enough or is there any other simple way of meeting this requirement ?

Thanks a lot.

Aaru · Posted: Tue Nov 11, 2008 4:02 pm

Ssk,

ssk1711 · New User Joined: 16 Jun 2008 Posts: 40 Location: bangalore

I was also thinking that the dynamic approach to splitting the file would be the best to go about this issue. However I am not aware of how to use SYNCSORT to find the number of records and allocate it dynamically.

Would you please guide me to that?

Aaru · Posted: Tue Nov 11, 2008 4:54 pm

Ssk,

Arun Raj · Posted: Tue Nov 11, 2008 6:16 pm

ssk1711 · New User Joined: 16 Jun 2008 Posts: 40 Location: bangalore

Thanks Arun.

I gone thru the link and the JCL. The JCL generates another JCL that will have the split statements (the number of split statements is dynamically generated) in it. This generated JCL needs to be submitted either manually or thru the next step in the first JCL.

But, In our shop, we are not suppose to submit a JCL in production and also, we should not submit a JCL thru another JCL (only the scheduler will submit JCL ) !

So, give this case, how can we split it in a same JOB using SYNCSORT (since we have SYNCSORT in our shop)

Thanks,

Arun Raj · Posted: Wed Nov 12, 2008 12:03 pm

ssk1711,

For dynamic allocation, you'll have to create the DD statements dynamically. That means at least some part of your jcl will be different every time depending on the input record count.

One possibility I see is to split this into two jobs. In the first job write the DD statements into a pds member as well as the sort card for the second job. "INCLUDE" the member in your second job. Even if it is doable I personally feel it a bit complex and difficult to maintain. So going for REXX might be an easy solution for this. It's upto you to decide.

ssk1711 · New User Joined: 16 Jun 2008 Posts: 40 Location: bangalore

Thanks Arun.

Even, one of my team member gave that idea. But finally we have decided to do it with REXX.

Thanks once again.

Aaru · Posted: Wed Nov 12, 2008 5:49 pm

Ssk,

sajjan jindal · New User Joined: 09 Sep 2007 Posts: 60 Location: india

Senthil,

I suppose you must have posted your solution here, so that anyone else looking for the same problem could have found it helpful.

Are we here to only post the question or to give the solutions as well. I believe it should be Give and take,rather than being just take and take.

Aaru · Posted: Mon Jul 06, 2009 5:00 pm

Sajjan,