View previous topic :: View next topic
Author
Message
Daniel Prosser New User Joined: 05 Nov 2010Posts: 57 Location: Amsterdam
Hi,
I have a sort which is inefficient and I don't know how to improve it.
I have a VB dataset which I want to sort/reformat a little into another VB dataset, the problem is the outputfile is padded with spaces.
My dataset is getting quite large so this is giving me alot of issues now.
Here is my code. Both datasets are pipe delimited.
Code:
SORT FIELDS=COPY
INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20),
%02=(ENDBEFR=C'|',FIXLEN=10),
...
%11=(ENDBEFR=C'|',FIXLEN=256)),
BUILD=(1,4,
5:%01,JFY=(SHIFT=LEFT),
C'|',
...
%09,JFY=(SHIFT=LEFT))
OUTREC IFTHEN=(WHEN=INIT,
BUILD=(1,4,5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001'))
//*
I think the issue is the BUILD stmt where the reclength 1500 is hard coded, tho what I should replace it with I don't know.
Does anyone have any advice?
I am using SYNCSORT FOR Z/OS 1.4.0.1R
This is the result of the SORT:
Code:
WER108I SORTIN : RECFM=VB ; LRECL= 1500; BLKSIZE= 27998
WER073I SORTIN : DSNAME=my input dataset
WER257I INREC RECORD LENGTH = 1211
WER237I OUTREC RECORD LENGTH = 1504
WER110I SORTOUT : RECFM=VB ; LRECL= 1504; BLKSIZE= 27998
WER074I SORTOUT : DSNAME=my output dataset
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
WER054I RCD IN 10, OUT 10 <---- this is my test data in PROD we have 14 million records
WER169I RELEASE 1.4 BATCH 0520 TPF LEVEL 0.1
WER052I END SYNCSORT
Dan.
Back to top
Marso REXX Moderator Joined: 13 Mar 2006Posts: 1353 Location: Israel
SYNCSORT may be able to recalculate the RDW.
I would try without copying it:
Code:
BUILD=(5,1500,SQZ=(SHIFT=LEFT,...
and see if it works
(Sorry, currently not able to check by myself)
Back to top
Marso REXX Moderator Joined: 13 Mar 2006Posts: 1353 Location: Israel
Finally found this:
Syncsort Programmer’s Guide wrote:
If INREC or OUTREC processing changes the output record length, the contents of the
Record Descriptor Word will be automatically revised by the sort.
It seems to confirm what I wrote before: let Syncsort do the calculation.
Back to top
Daniel Prosser New User Joined: 05 Nov 2010Posts: 57 Location: Amsterdam
@Marso
Seemed like a good idea, unfortunately it gives the following error:
Code:
OUTREC IFTHEN=(WHEN=INIT,
BUILD=(5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001'))
WER276B SYSDIAG= 362785, 4401136, 4401136, 5442460
WER164B 2,052K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
WER164B 16K BYTES RESERVE REQUESTED, 2,028K BYTES USED
WER146B 20K BYTES OF EMERGENCY SPACE ALLOCATED
WER108I SORTIN : RECFM=VB ; LRECL= 1500; BLKSIZE= 27998
WER073I SORTIN : DSNAME=...
WER257I INREC RECORD LENGTH = 1211
WER235A OUTREC RDW NOT INCLUDED
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
******************************* Bottom of Data **********************
Back to top
sergeyken Senior Member Joined: 29 Apr 2008Posts: 2010 Location: USA
Daniel Prosser wrote:
@Marso
Seemed like a good idea, unfortunately it gives the following error:
Code:
OUTREC IFTHEN=(WHEN=INIT,
BUILD=(5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),
IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001'))
WER276B SYSDIAG= 362785, 4401136, 4401136, 5442460
WER164B 2,052K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
WER164B 16K BYTES RESERVE REQUESTED, 2,028K BYTES USED
WER146B 20K BYTES OF EMERGENCY SPACE ALLOCATED
WER108I SORTIN : RECFM=VB ; LRECL= 1500; BLKSIZE= 27998
WER073I SORTIN : DSNAME=...
WER257I INREC RECORD LENGTH = 1211
WER235A OUTREC RDW NOT INCLUDED
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
******************************* Bottom of Data **********************
You MUST specify RDW explicitly for every BUILD= parameter:
BUILD=(1,4, ...whatever else ...
Back to top
Daniel Prosser New User Joined: 05 Nov 2010Posts: 57 Location: Amsterdam
So after 2 days of investigation I worked out how to do it
I added a OUTFIL line to the SYSIN
Code:
SORT FIELDS=COPY
OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'
INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20)
...
Back to top
sergeyken Senior Member Joined: 29 Apr 2008Posts: 2010 Location: USA
If the original requirement is true:
Quote:
I have a VB dataset which I want to sort/reformat a little into another VB dataset
Then
Code:
OUTFIL ...,FTOV,...
will not work
Back to top
Daniel Prosser New User Joined: 05 Nov 2010Posts: 57 Location: Amsterdam
@sergeyken
Well the original stmt is true.
What makes you think it will not work? It appears to be working fine.
Back to top
sergeyken Senior Member Joined: 29 Apr 2008Posts: 2010 Location: USA
Daniel Prosser wrote:
@sergeyken
Well the original stmt is true.
What makes you think it will not work? It appears to be working fine.
You did not post your code in full, so it's difficult to say exactly.
In case the input record is really RECFM=VB, as stated by your INREC statement:
Code:
. . . . BUILD=(1,4,
5:%01,JFY=(SHIFT=LEFT), . . . .
and also your output dataset is RECFM=VB, then conversion FTOV has no meaning; it always caused SYNSORT error message in my experience.
You said you tried "to avoid copying of RDW from input record to the output one", but it seems to me that instead of that, you make even more senseless conversion between VB-->FB-->VB, for no reason. Cannot say without you code in full.
Besides of those hidden parts of your code, it always makes sense to place your statements in the order they are actually applied/executed, for not to create additional mess in readers' mind:
Code:
INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20)
. . . . . .
SORT FIELDS=COPY
OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'
...
Back to top
Daniel Prosser New User Joined: 05 Nov 2010Posts: 57 Location: Amsterdam
@sergeyken -
Adding the command:
Code:
OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'
Certainly works. As does:
Code:
OUTFIL FNAMES=SORTOUT,VLTRIM=X'40'
I didn't realise VLTRIM would work without the FTOV
So thanks for helping out.
Dan.
Back to top
Please enable JavaScript!