Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Syncsort Spilt up based on 2 difft fields

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
shilpa.khaire

New User


Joined: 21 Jul 2006
Posts: 39
Location: US

PostPosted: Fri Nov 06, 2009 2:52 am    Post subject: Syncsort Spilt up based on 2 difft fields
Reply with quote

Hi All!

I have one file which I want to split based on date. File 1 and file2.
I again want to split File2 based on some other field(not date) .
Can this be done in a single step?
Back to top
View user's profile Send private message

dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Nov 06, 2009 3:04 am    Post subject:
Reply with quote

Hello,

Suggest you post a few sample records from both files and the output you want from these samples.

Why would a "single step" be an issue?

You could always write a bit of really simple code that had 2 processes (1 for each input file).
Back to top
View user's profile Send private message
shilpa.khaire

New User


Joined: 21 Jul 2006
Posts: 39
Location: US

PostPosted: Fri Nov 06, 2009 3:11 am    Post subject:
Reply with quote

I agree to you that it better to break up the processes.
But I want to check if combining into a single step will reduce the cPU time.
e.g.
Input file - Split on date
2005 A
2006 B
2010 A
2011 B

File 1 - Records prior to 2009
2005 A
2006 B


File 2 - Records after 2009
2010 A
2011 B

File 2 - Split based on Second field
My final output will be:
File 3
2010 A
File 4
2011 B

Thanks!
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Nov 06, 2009 3:20 am    Post subject:
Reply with quote

Hello,

So, if i understand, you want to split file1 into 3 files? Is there a need for the file2 data to be in that file or is it only needed for the second split?

It will help if you post the "rules" for the second split.
Back to top
View user's profile Send private message
shilpa.khaire

New User


Joined: 21 Jul 2006
Posts: 39
Location: US

PostPosted: Fri Nov 06, 2009 7:38 pm    Post subject:
Reply with quote

I dont want the file 1 to be exactly into 3 files. Rules for the second split are - Suppose File 2 is a system-id whcih has values A,B,C and D. In that case, I want the file 2 to be split up by the diffreent values in System-id. So in this case we will have 4 files coming out of the second split.

File 1 - Records prior to 2009
2005 A
2006 B


File 2 - Records after 2009
2010 A
2011 B
2011 A
2012 C
2012 D

File 2 - Split based on Second field
My final output will be:
File 3
2010 A
2011 A

File 4
2011 B

File 5
2012 C

File 4
2012 D

Answer to the 2'nd question is I need file2 seprately as a O/P of the first split. And then I need the outputs of the second split separately too.
Basically my idea is to achieve some saving in the run time by merging the process.

Thanks for looking into this!
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Sat Nov 07, 2009 12:27 am    Post subject:
Reply with quote

Hello,

Quote:
Basically my idea is to achieve some saving in the run time by merging the process.
The only way to save some time is to process less. . .

One way to do this is to read all of the data only once and write all of the output during that read. If the data is read/written/read again/written again as proposed it will take about the same time in one step or 2.

You will need to think thru how the "second split" will be managed. Processes that can create an unspecified number of outputs are often a support/maintenance nightmare.

Something simple that can be looked into is improving the block size(s) of the files.
Back to top
View user's profile Send private message
shilpa.khaire

New User


Joined: 21 Jul 2006
Posts: 39
Location: US

PostPosted: Sat Nov 07, 2009 1:42 am    Post subject:
Reply with quote

Thats the reason why I was thinking of one SORT step...and split it up in one read of the input file.

Can we have a single SORT for this?
Back to top
View user's profile Send private message
shilpa.khaire

New User


Joined: 21 Jul 2006
Posts: 39
Location: US

PostPosted: Mon Nov 09, 2009 10:27 pm    Post subject:
Reply with quote

dick scherrer wrote:
Hello,
Something simple that can be looked into is improving the block size(s) of the files.


The Block size is all 0. If we read the input file once to create the splits by date as well as second field , will it not be faster, since it is just one pass through the input file.
The elapsed time for all these steps is around 3 to 4 hours, and I know its not due to data contention. What could be teh other reason for elapsed time being so high?

Thanks!
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Nov 09, 2009 10:50 pm    Post subject:
Reply with quote

Hello,

Quote:
and I know its not due to data contention. What could be teh other reason for elapsed time being so high?
How could it be known that there is not a contention issue? Contention exists far beyond only the files used in the process. . .

The main reason for the elapsed time is probably due to the volume of data being processed. How many records are there in the original input file? What is their lrecl?
Back to top
View user's profile Send private message
shilpa.khaire

New User


Joined: 21 Jul 2006
Posts: 39
Location: US

PostPosted: Fri Nov 13, 2009 12:27 am    Post subject:
Reply with quote

We found that the elapsed time is high due to files getting migrated.
We cannot move the job creating this file but we can run an IEFBR just before our job(in paralllel with the job which triggers our job).

Is there any solution to this?

Thanks!
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts JCL to Set Return code based on DB2 S... vinu78 DB2 17 Mon Mar 13, 2017 9:47 pm
No new posts DFSORT MUL FIELDS tspr52 DFSORT/ICETOOL 16 Fri Mar 03, 2017 11:53 pm
No new posts Inserting records based on conditions vickey_dw DFSORT/ICETOOL 9 Wed Feb 22, 2017 1:33 pm
No new posts Sort two file having same field, one ... himanshu malik DFSORT/ICETOOL 7 Thu Feb 02, 2017 10:09 am
No new posts SYNCSORT SEQNUM and FI fields. nartcr SYNCSORT 7 Sat Jan 21, 2017 4:02 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us