Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Inefficient BUILD - VB records padding with spaces to LRECL

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT
View previous topic :: :: View next topic  
Author Message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 46
Location: Amsterdam

PostPosted: Thu Dec 14, 2017 3:52 pm    Post subject: Inefficient BUILD - VB records padding with spaces to LRECL
Reply with quote

Hi,

I have a sort which is inefficient and I don't know how to improve it.

I have a VB dataset which I want to sort/reformat a little into another VB dataset, the problem is the outputfile is padded with spaces.

My dataset is getting quite large so this is giving me alot of issues now.

Here is my code. Both datasets are pipe delimited.

Code:
SORT FIELDS=COPY                             
INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20),   
             %02=(ENDBEFR=C'|',FIXLEN=10),
             ...
             %11=(ENDBEFR=C'|',FIXLEN=256)), 
   BUILD=(1,4, 
           5:%01,JFY=(SHIFT=LEFT),     
           C'|',                       
           ...
            %09,JFY=(SHIFT=LEFT))                   
      OUTREC IFTHEN=(WHEN=INIT,                                     
       BUILD=(1,4,5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),                 
       IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),               
       IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),
       IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001'))
//*                                                               
           


I think the issue is the BUILD stmt where the reclength 1500 is hard coded, tho what I should replace it with I don't know.

Does anyone have any advice?

I am using SYNCSORT FOR Z/OS 1.4.0.1R

This is the result of the SORT:
Code:
WER108I  SORTIN   : RECFM=VB   ; LRECL=  1500; BLKSIZE= 27998 
WER073I  SORTIN   : DSNAME=my input dataset       
WER257I  INREC RECORD LENGTH =  1211                           
WER237I  OUTREC RECORD LENGTH =  1504                         
WER110I  SORTOUT  : RECFM=VB   ; LRECL=  1504; BLKSIZE= 27998 
WER074I  SORTOUT  : DSNAME=my output dataset         
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                 
WER054I  RCD IN         10, OUT         10           <---- this is my test data in PROD we have 14 million records           
WER169I  RELEASE 1.4 BATCH 0520 TPF LEVEL 0.1
WER052I  END SYNCSORT   



Dan.
Back to top
View user's profile Send private message

Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1277
Location: Israel

PostPosted: Thu Dec 14, 2017 7:46 pm    Post subject:
Reply with quote

SYNCSORT may be able to recalculate the RDW.
I would try without copying it:
Code:
       BUILD=(5,1500,SQZ=(SHIFT=LEFT,...
and see if it works
(Sorry, currently not able to check by myself)
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1277
Location: Israel

PostPosted: Thu Dec 14, 2017 7:51 pm    Post subject:
Reply with quote

Finally found this:
Syncsort Programmer’s Guide wrote:
If INREC or OUTREC processing changes the output record length, the contents of the
Record Descriptor Word will be automatically revised by the sort.

It seems to confirm what I wrote before: let Syncsort do the calculation.
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 46
Location: Amsterdam

PostPosted: Thu Dec 14, 2017 8:48 pm    Post subject:
Reply with quote

@Marso

Seemed like a good idea, unfortunately it gives the following error:

Code:
     OUTREC IFTHEN=(WHEN=INIT,                                       
      BUILD=(5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),                   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),                 
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001')) 
     
WER276B  SYSDIAG= 362785, 4401136, 4401136, 5442460                 
WER164B  2,052K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,   
WER164B     16K BYTES RESERVE REQUESTED, 2,028K BYTES USED           
WER146B  20K BYTES OF EMERGENCY SPACE ALLOCATED                     
WER108I  SORTIN   : RECFM=VB   ; LRECL=  1500; BLKSIZE= 27998       
WER073I  SORTIN   : DSNAME=...             
WER257I  INREC RECORD LENGTH =  1211                                 
WER235A  OUTREC   RDW NOT INCLUDED                                   
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                       
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                       
******************************* Bottom of Data **********************
Back to top
View user's profile Send private message
sergeyken

Active User


Joined: 29 Apr 2008
Posts: 303
Location: Maryland

PostPosted: Fri Dec 15, 2017 3:34 am    Post subject:
Reply with quote

Daniel Prosser wrote:
@Marso

Seemed like a good idea, unfortunately it gives the following error:

Code:
     OUTREC IFTHEN=(WHEN=INIT,                                       
      BUILD=(5,1500,SQZ=(SHIFT=LEFT,PAIR=QUOTE))),                   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'"',OUT=C'')),                 
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'6/6/2066',OUT=C'1/1/0001')),   
      IFTHEN=(WHEN=INIT,FINDREP=(IN=C'06/06/2066',OUT=C'1/1/0001')) 
     
WER276B  SYSDIAG= 362785, 4401136, 4401136, 5442460                 
WER164B  2,052K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,   
WER164B     16K BYTES RESERVE REQUESTED, 2,028K BYTES USED           
WER146B  20K BYTES OF EMERGENCY SPACE ALLOCATED                     
WER108I  SORTIN   : RECFM=VB   ; LRECL=  1500; BLKSIZE= 27998       
WER073I  SORTIN   : DSNAME=...             
WER257I  INREC RECORD LENGTH =  1211                                 
WER235A  OUTREC   RDW NOT INCLUDED                                   
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                       
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                       
******************************* Bottom of Data **********************

You MUST specify RDW explicitly for every BUILD= parameter:
BUILD=(1,4, ...whatever else ...
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 46
Location: Amsterdam

PostPosted: Fri Dec 15, 2017 9:55 pm    Post subject:
Reply with quote

So after 2 days of investigation I worked out how to do it icon_rolleyes.gif

I added a OUTFIL line to the SYSIN

Code:
 SORT FIELDS=COPY                           
 OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'     
 INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20)
 ...


icon_biggrin.gif
Back to top
View user's profile Send private message
sergeyken

Active User


Joined: 29 Apr 2008
Posts: 303
Location: Maryland

PostPosted: Sat Dec 16, 2017 12:25 am    Post subject:
Reply with quote

If the original requirement is true:
Quote:
I have a VB dataset which I want to sort/reformat a little into another VB dataset

Then
Code:
OUTFIL ...,FTOV,...   
will not work
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 46
Location: Amsterdam

PostPosted: Mon Dec 18, 2017 2:58 pm    Post subject:
Reply with quote

@sergeyken

Well the original stmt is true.

What makes you think it will not work? It appears to be working fine.
Back to top
View user's profile Send private message
sergeyken

Active User


Joined: 29 Apr 2008
Posts: 303
Location: Maryland

PostPosted: Mon Dec 18, 2017 9:30 pm    Post subject:
Reply with quote

Daniel Prosser wrote:
@sergeyken

Well the original stmt is true.

What makes you think it will not work? It appears to be working fine.

You did not post your code in full, so it's difficult to say exactly.
In case the input record is really RECFM=VB, as stated by your INREC statement:
Code:
. . . .  BUILD=(1,4, 
           5:%01,JFY=(SHIFT=LEFT), . . . .     

and also your output dataset is RECFM=VB, then conversion FTOV has no meaning; it always caused SYNSORT error message in my experience.

You said you tried "to avoid copying of RDW from input record to the output one", but it seems to me that instead of that, you make even more senseless conversion between VB-->FB-->VB, for no reason. Cannot say without you code in full.

Besides of those hidden parts of your code, it always makes sense to place your statements in the order they are actually applied/executed, for not to create additional mess in readers' mind:
Code:

 INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=20)
 . . . . . .
 SORT FIELDS=COPY                           
 OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'     
 ...
Back to top
View user's profile Send private message
Daniel Prosser

New User


Joined: 05 Nov 2010
Posts: 46
Location: Amsterdam

PostPosted: Mon Dec 18, 2017 10:14 pm    Post subject:
Reply with quote

@sergeyken -


Adding the command:

Code:
 OUTFIL FNAMES=SORTOUT,FTOV,VLTRIM=X'40'


Certainly works. As does:

Code:
 OUTFIL FNAMES=SORTOUT,VLTRIM=X'40'


I didn't realise VLTRIM would work without the FTOV

So thanks for helping out.

Dan.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts MXG - Processing Compressed DB2 SMF r... vasanthz All Other Mainframe Topics 2 Thu May 10, 2018 12:47 am
No new posts Incorrect output after build alternat... sandeep kumar302 JCL & VSAM 10 Wed May 02, 2018 2:35 pm
No new posts Sort numeric fields with trailing spaces sancraig16 SYNCSORT 7 Tue Apr 03, 2018 1:21 am
No new posts Extract the records with a PD field's... sudhakar84 DFSORT/ICETOOL 11 Mon Apr 02, 2018 7:26 pm
No new posts Merge 2 records sancraig16 SYNCSORT 19 Tue Mar 27, 2018 8:17 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us