IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Inefficient BUILD - VB records padding with spaces to LRECL
IBM Mainframe Forums -> SYNCSORT
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 57
Location: Amsterdam
PostPosted: Thu Dec 14, 2017 3:52 pm
 Reply with quote
Hi,

I have a sort which is inefficient and I don't know how to improve it.

I have a VB dataset which I want to sort/reformat a little into another VB dataset, the problem is the outputfile is padded with spaces.

My dataset is getting quite large so this is giving me alot of issues now.

Here is my code. Both datasets are pipe delimited.

Code:
SORT FIELDS=COPY                             
INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20),   
             %02=(ENDBEFR=C'|',FIXLEN=10),
             ...
             %11=(ENDBEFR=C'|',FIXLEN=256)), 
   BUILD=(1,4, 
           5:%01,JFY=(SHIFT=LEFT),     
           C'|',                       
           ...
            %09,JFY=(SHIFT=LEFT))                   
      OUTREC IFTHEN=(WHEN=INIT,                                     
       BUILD=(1,4,5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),                 
       IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),               
       IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),
       IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001'))
//*                                                               
           


I think the issue is the BUILD stmt where the reclength 1500 is hard coded, tho what I should replace it with I don't know.

Does anyone have any advice?

I am using SYNCSORT FOR Z/OS 1.4.0.1R

This is the result of the SORT:
Code:
WER108I  SORTIN   : RECFM=VB   ; LRECL=  1500; BLKSIZE= 27998 
WER073I  SORTIN   : DSNAME=my input dataset       
WER257I  INREC RECORD LENGTH =  1211                           
WER237I  OUTREC RECORD LENGTH =  1504                         
WER110I  SORTOUT  : RECFM=VB   ; LRECL=  1504; BLKSIZE= 27998 
WER074I  SORTOUT  : DSNAME=my output dataset         
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                 
WER054I  RCD IN         10, OUT         10           <---- this is my test data in PROD we have 14 million records           
WER169I  RELEASE 1.4 BATCH 0520 TPF LEVEL 0.1
WER052I  END SYNCSORT   



Dan.
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1353
Location: Israel
PostPosted: Thu Dec 14, 2017 7:46 pm
 Reply with quote
SYNCSORT may be able to recalculate the RDW.
I would try without copying it:
Code:
       BUILD=(5,1500,SQZ=(SHIFT=LEFT,...
and see if it works
(Sorry, currently not able to check by myself)
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1353
Location: Israel
PostPosted: Thu Dec 14, 2017 7:51 pm
 Reply with quote
Finally found this:
Syncsort Programmer’s Guide wrote:
If INREC or OUTREC processing changes the output record length, the contents of the
Record Descriptor Word will be automatically revised by the sort.

It seems to confirm what I wrote before: let Syncsort do the calculation.
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 57
Location: Amsterdam
PostPosted: Thu Dec 14, 2017 8:48 pm
 Reply with quote
@Marso

Seemed like a good idea, unfortunately it gives the following error:

Code:
     OUTREC IFTHEN=(WHEN=INIT,                                       
      BUILD=(5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),                   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),                 
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001')) 
     
WER276B  SYSDIAG= 362785, 4401136, 4401136, 5442460                 
WER164B  2,052K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,   
WER164B     16K BYTES RESERVE REQUESTED, 2,028K BYTES USED           
WER146B  20K BYTES OF EMERGENCY SPACE ALLOCATED                     
WER108I  SORTIN   : RECFM=VB   ; LRECL=  1500; BLKSIZE= 27998       
WER073I  SORTIN   : DSNAME=...             
WER257I  INREC RECORD LENGTH =  1211                                 
WER235A  OUTREC   RDW NOT INCLUDED                                   
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                       
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                       
******************************* Bottom of Data **********************
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2022
Location: USA
PostPosted: Fri Dec 15, 2017 3:34 am
 Reply with quote
Daniel Prosser wrote:
@Marso

Seemed like a good idea, unfortunately it gives the following error:

Code:
     OUTREC IFTHEN=(WHEN=INIT,                                       
      BUILD=(5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),                   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),                 
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001')) 
     
WER276B  SYSDIAG= 362785, 4401136, 4401136, 5442460                 
WER164B  2,052K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,   
WER164B     16K BYTES RESERVE REQUESTED, 2,028K BYTES USED           
WER146B  20K BYTES OF EMERGENCY SPACE ALLOCATED                     
WER108I  SORTIN   : RECFM=VB   ; LRECL=  1500; BLKSIZE= 27998       
WER073I  SORTIN   : DSNAME=...             
WER257I  INREC RECORD LENGTH =  1211                                 
WER235A  OUTREC   RDW NOT INCLUDED                                   
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                       
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                       
******************************* Bottom of Data **********************

You MUST specify RDW explicitly for every BUILD= parameter:
BUILD=(1,4, ...whatever else ...
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 57
Location: Amsterdam
PostPosted: Fri Dec 15, 2017 9:55 pm
 Reply with quote
So after 2 days of investigation I worked out how to do it icon_rolleyes.gif

I added a OUTFIL line to the SYSIN

Code:
 SORT FIELDS=COPY                           
 OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'     
 INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20)
 ...


icon_biggrin.gif
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2022
Location: USA
PostPosted: Sat Dec 16, 2017 12:25 am
 Reply with quote
If the original requirement is true:
Quote:
I have a VB dataset which I want to sort/reformat a little into another VB dataset

Then
Code:
OUTFIL ...,FTOV,...   
will not work
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 57
Location: Amsterdam
PostPosted: Mon Dec 18, 2017 2:58 pm
 Reply with quote
@sergeyken

Well the original stmt is true.

What makes you think it will not work? It appears to be working fine.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2022
Location: USA
PostPosted: Mon Dec 18, 2017 9:30 pm
 Reply with quote
Daniel Prosser wrote:
@sergeyken

Well the original stmt is true.

What makes you think it will not work? It appears to be working fine.

You did not post your code in full, so it's difficult to say exactly.
In case the input record is really RECFM=VB, as stated by your INREC statement:
Code:
. . . .  BUILD=(1,4, 
           5:%01,JFY=(SHIFT=LEFT), . . . .     

and also your output dataset is RECFM=VB, then conversion FTOV has no meaning; it always caused SYNSORT error message in my experience.

You said you tried "to avoid copying of RDW from input record to the output one", but it seems to me that instead of that, you make even more senseless conversion between VB-->FB-->VB, for no reason. Cannot say without you code in full.

Besides of those hidden parts of your code, it always makes sense to place your statements in the order they are actually applied/executed, for not to create additional mess in readers' mind:
Code:

 INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20)
 . . . . . .
 SORT FIELDS=COPY                           
 OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'     
 ...
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 57
Location: Amsterdam
PostPosted: Mon Dec 18, 2017 10:14 pm
 Reply with quote
@sergeyken -


Adding the command:

Code:
 OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'


Certainly works. As does:

Code:
 OUTFIL FNAMES=SORTOUT,VLTRIM=X'40'


I didn't realise VLTRIM would work without the FTOV

So thanks for helping out.

Dan.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> SYNCSORT

 

Similar Topics
Topic Forum Replies
No new posts Compare 2 files(F1 & F2) and writ... JCL & VSAM 5
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts Pulling a fixed number of records fro... DB2 2
No new posts VB to FB - Finding LRECL SYNCSORT 4
No new posts leading spaces can be removed in trai... DFSORT/ICETOOL 1
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us