I would like to achieve 1 specific function by DFSORT, but i don't know how, so could you please help on this?
Thank you in advance.
there are several files as input but with different length:
name: length
File1 20
File2 300
File3 1000
.......
FileN nnnn
1.Adding a sequence number at each record in every these 3 files.
2.output the new record to a new file, but how to handle different file length is the diffcult point.
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
Some questions - just to clarify
Are the shown datasets a concatenation or will they be treated individually.
Is the sequence number of all shown files, or again within the same dataset.
Are the shown datasets a concatenation or will they be treated individually.
Is the sequence number of all shown files, or again within the same dataset.
Thank you for asking!
1.each file is a individual file, they don't have anything related to each other. But they all need to be processed by the function of this DFSORT
2. seq num is generated within each file.
for example:
File1:
00000001AAA...
00000002AAA...
00000003AAA...
......
00000100AAA...
Joined: 10 May 2007 Posts: 2455 Location: Hampshire, UK
Sort only has one input DDNAME so your data sets (they are NOT files) would have to be concatenated. That means that there must be some way of distinguishing between the records of one data set and the records of the next. This identifier must be in the same position for each data set. If there is then you can use that infromation to reset the sequence number between data sets.
If you cannot do this then you will have to run the sort for each data set.
Other information needed - what is the RECFM of these data sets?
It is possible because DD DCB parameters have precedence over DSCB (so to speak, "dataset label")
2) For this specific need the SYNCSORT might be more useful; it supports multiple input files by implemented MULTIIN option.
P.S.
You have not "different file size", but "different record length". File size has nothing to do in your case.
sorry for the misunderstanding.
RECFM is FB, but each file has its own file length.
for example, File1 length is 100, file2 length is 20, file 3 length is 1200.
All i need is, seq number + original file record = output.
File1 -> seq num + original File1 file record => Output1
File2 -> seq num + original File2 file record => Output2
File3 -> seq num + original File3 file record => Output3
.......
FileN -> seq num + original FileN file record => OutputN
the issue i am facing is, all these files are from the upstream which are all different companies/departments. Files arrives in and we process them and put them into db2 tables.
the difficult part is, i don't know how to deal with different file length.
Sort only has one input DDNAME so your data sets (they are NOT files) would have to be concatenated. That means that there must be some way of distinguishing between the records of one data set and the records of the next. This identifier must be in the same position for each data set. If there is then you can use that infromation to reset the sequence number between data sets.
If you cannot do this then you will have to run the sort for each data set.
Other information needed - what is the RECFM of these data sets?
Thank you.
and RECFM=FB
actually it's not 1 sort to process multiple files.
this 1 job which contain this sort will be in opc and this opc will be a file trigger one.
we will set up in opc so opc can be trigger by upstream file and the file will be processed.
and input file name will be passed into as sort input data.
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
javen777 wrote:
RECFM is FB, but each file has its own file length.
for example, File1 length is 100, file2 length is 20, file 3 length is 1200.
All i need is, seq number + original file record = output
javen777,
You could achieve this by listing the data set attributes, ie, get the LRECL using the LISTDSI command, then build a control card dynamically to substitute the LRECL along with the SEQNUM. Finally run your input data set through this control card.
the issue i am facing is, all these files are from the upstream which are all different companies/departments. Files arrives in and we process them and put them into db2 tables.
the difficult part is, i don't know how to deal with different file length.
Don't you have a handshakes with upstream team and without having a solution how can one sign off something like this design? why would you need to add seq# at the beginning and how does it help you? What are you trying to achieve here? Can't Upstream add the same than this patch work?
Moreover, how do you load them into different DB2 tables when input DS changes?
RECFM is FB, but each file has its own file length.
for example, File1 length is 100, file2 length is 20, file 3 length is 1200.
All i need is, seq number + original file record = output
javen777,
You could achieve this by listing the data set attributes, ie, get the LRECL using the LISTDSI command, then build a control card dynamically to substitute the LRECL along with the SEQNUM. Finally run your input data set through this control card.
This is one more example of reaching the simple goal by using a complex approach/tool.
The main problem is not solved: for really significant total number of datasets - how to guess in advance the DSN with maximal LRECL?
I can suggest a seriously simpler approach. Just convert ALL files in OUTFIL statements to RECFM=VB,LRECL=maxvalue, like 32000 or other acceptable value (may depend on further required processing). This DCB would also be the best compatible with DB2 load/unload.
This is one more example of reaching the simple goal by using a complex approach/tool.
The main problem is not solved: for really significant total number of datasets - how to guess in advance the DSN with maximal LRECL?
I can suggest a seriously simpler approach. Just convert ALL files in OUTFIL statements to RECFM=VB,LRECL=maxvalue, like 32000 or other acceptable value (may depend on further required processing). This DCB would also be the best compatible with DB2 load/unload.
thanks! very thoughtful!
But how to fetch the whole original record if VB may i ask?
********************************* TOP OF DATA *****
00000001AAAAAAAAAABBBBBBBBBB
00000002DDDDDDDDDDAAAAAAAAAA
00000003CCCCCCCCCCDDDDDDDDDD
00000004BBBBBBBBBBCCCCCCCCCC
******************************** BOTTOM OF DATA ***
Code:
********************************* TOP OF DATA **
00000001AAAAAAAAAABBBBBBBBBBCCCCCCCCCCDDDDDDDDDD
00000002DDDDDDDDDDAAAAAAAAAABBBBBBBBBBCCCCCCCCCC
00000003CCCCCCCCCCDDDDDDDDDDAAAAAAAAAABBBBBBBBBB
00000004BBBBBBBBBBCCCCCCCCCCDDDDDDDDDDAAAAAAAAAA
******************************** BOTTOM OF DATA
Code:
********************************* TOP OF DATA ***********************
00000001AAAAAAAAAABBBBBBBBBBCCCCCCCCCCDDDDDDDDDDEEEEEEEEEEFFFFFFFFFF
00000002DDDDDDDDDDAAAAAAAAAABBBBBBBBBBCCCCCCCCCCGGGGGGGGGGHHHHHHHHHH
00000003CCCCCCCCCCDDDDDDDDDDAAAAAAAAAABBBBBBBBBBIIIIIIIIIIJJJJJJJJJJ
00000004BBBBBBBBBBCCCCCCCCCCDDDDDDDDDDAAAAAAAAAAKKKKKKKKKKLLLLLLLLLL
******************************** BOTTOM OF DATA *********************