IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

JCL/SORT to Split Records
IBM Mainframe Forums -> SYNCSORT
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Mon Jun 05, 2023 9:58 pm
 Reply with quote
Dear All, I need help .. using below SORT/JCL step to split VB file record based on delimitator X'0A' but getting following error

WER235A SORTOUT OUTREC RDW NOT INCLUDED

here my JCL / SORT Statement

//SPLITR EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=&DSIN.,DISP=SHR
//SORTOUT DD DSN=&SRTOT,DISP=(,PASS),
// SPACE=(CYL,(2,5),,),RECFM=VB,LRECL=32756,BLKSIZE=32760
//SYSIN DD *
OPTION COPY
OUTFIL PARSE=(%01=(ENDBEFR=X'0A',FIXLEN=32752),
%02=(ENDBEFR=X'0A',FIXLEN=32752)),
BUILD=(%01,/,%02)
/*


Thank you for your help
Back to top
View user's profile Send private message
dneufarth

Active User


Joined: 27 Apr 2005
Posts: 420
Location: Inside the SPEW (Southwest Ohio, USA)
PostPosted: Mon Jun 05, 2023 10:51 pm
 Reply with quote
Roberto, welcome to the forums

please look into RDW specification for variable length records

Code:
BUILD=(1,4,5:%01, ...


fyi, please use CODE tag buttons to properly display code, control statements

Code:
//SPLITR EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=&DSIN.,DISP=SHR
//SORTOUT DD DSN=&SRTOT,DISP=(,PASS),
// SPACE=(CYL,(2,5),,),RECFM=VB,LRECL=32756,BLKSIZE=32760
//SYSIN DD *
OPTION COPY
OUTFIL PARSE=(%01=(ENDBEFR=X'0A',FIXLEN=32752),
%02=(ENDBEFR=X'0A',FIXLEN=32752)),
BUILD=(%01,/,%02)
/*


moderator - please move to SYNCSORT forum
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Mon Jun 05, 2023 11:54 pm
 Reply with quote
Please, try to post it in this manner:

roberto2023 wrote:
Dear All, I need help .. using below SORT/JCL step to split VB file record based on delimitator X'0A' but getting following error
Code:
 WER235A  SORTOUT  OUTREC RDW NOT INCLUDED

here my JCL / SORT Statement
Code:
//SPLITR   EXEC PGM=SORT                                               
//SYSOUT   DD  SYSOUT=*                                               
//SORTIN   DD  DSN=&DSIN.,DISP=SHR                                     
//SORTOUT  DD  DSN=&SRTOT,DISP=(,PASS),                               
//             SPACE=(CYL,(2,5),,),RECFM=VB,LRECL=32756,BLKSIZE=32760 
//SYSIN    DD  *                                                       
   OPTION COPY                                                         
   OUTFIL PARSE=(%01=(ENDBEFR=X'0A',FIXLEN=32752),                     
                 %02=(ENDBEFR=X'0A',FIXLEN=32752)),                   
          BUILD=(%01,/,%02)                                           
/*

Thank you for your help

It may be useful also to change:
Code:
. . . . . . .                                                       
   OUTFIL PARSE=(%01=(ABSPOS=5,ENDBEFR=X'0A',FIXLEN=32752),                     
. . . . . . . .                                         
/*

Otherwise there is a chance that X'0A' is one of RDW bytes, and/or your %01 would start with the RDW field...

P.S.
The explanation from the manual is as clear as the bell:
Quote:
WER235A [ddname] {INREC,OUTREC,REFORMAT} RDW NOT INCLUDED

EXPLANATION:
The ddname will be SORTOUT, SORTOFxx, SORTOFx or the ddname provided by an OUTFIL FNAMES parameter.

Four bytes must be provided for the RDW of the variable-length output record in the FIELDS parameter of the INREC, OUTREC, OUTFIL, OUTREC, or REFORMAT specification. These bytes must appear at the beginning of the record and must not be edited. For REFORMAT, the RDW must be specified as coming from a variable-length join input
data set.
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Tue Jun 06, 2023 2:29 am
 Reply with quote
Thank you, this is much better .. now managed to remove the JCL error with updated BUILD as follow

Code:
   OPTION COPY                                               
   OUTFIL PARSE=(%01=((ABSPOS=5,ENDBEFR=X'0A',FIXLEN=32752), 
                 %02=((ABSPOS=5,ENDBEFR=X'0A',FIXLEN=32752), 
          BUILD=(1,4,5:%01,/,1,4,5:%02
)

However, not sure to be on the right track .. my input vb file got 146 x"0A', so the purpose is to have this splitted in 146 records, however from job execution I get the follow:

Code:
OUTFIL WAS USED FOR SORTOUT           
FILESIZE 47,729 BYTES                 
RCD IN          2, OUT          2


.. one more note the input file is in UTF-8

Thank you
Back to top
View user's profile Send private message
dneufarth

Active User


Joined: 27 Apr 2005
Posts: 420
Location: Inside the SPEW (Southwest Ohio, USA)
PostPosted: Tue Jun 06, 2023 3:52 am
 Reply with quote
You need to review ABSPOS & FIXLEN

You probably only want ABSPOS for first parsed field.

FIXLEN should be more in line with the max number of bytes expect between parses within the whole variable length record (are 146 occurrences typical or is it more likely to be 1 or 10 or 200 ...)
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Tue Jun 06, 2023 4:21 am
 Reply with quote
Thank you Dave

.. I updated the SYSIN as follow

Code:
    OPTION COPY                                               
    OUTFIL PARSE=(%01=(ABSPOS=5,ENDBEFR=X'0A',FIXLEN=32752),
                  %02=(ENDBEFR=X'0A',FIXLEN=32752)),         
           BUILD=(1,4,5:%01,/,1,4,5:%02)   


the number of x'0A' in this case is 146 but is not fixed ..max rec length is 32752 .. sorry I am not very clear about FIXLEN usage
Back to top
View user's profile Send private message
dneufarth

Active User


Joined: 27 Apr 2005
Posts: 420
Location: Inside the SPEW (Southwest Ohio, USA)
PostPosted: Tue Jun 06, 2023 4:56 am
 Reply with quote
FIXLEN establishes the fixed length or size of the parsed fields i.e.%01,%02, ...

Try running your job with what you have so far and view the output dataset. See if the results are what you were expecting.
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Tue Jun 06, 2023 9:48 am
 Reply with quote
Assumed the DSN comes per FTP, check the options to transfer it in the correct format. You can specify line terminators and also conversion options for UTF.
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Tue Jun 06, 2023 11:53 am
 Reply with quote
I converted UTF-8 to IBM-1141

rerun JCL with follow SYSIN

Code:
 OPTION COPY                                             
 OUTFIL PARSE=(%01=(ABSPOS=5,ENDBEFR=X'15',FIXLEN=32752),
               %02=(ENDBEFR=X'15',FIXLEN=32752)),       
        BUILD=(1,4,5:%01,/,1,4,5:%02) 


but still getting only two records in output files while I would expect as much as the limited char X'15'
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Tue Jun 06, 2023 12:14 pm
 Reply with quote
Code:
OPTION COPY                                                   
OUTFIL PARSE=(%=(ABSPOS=5),                                   
              %001=(ENDBEFR=X'15',FIXLEN=32752,REPEAT=200)), 
  BUILD=(1,4,%001,/,                                         
         1,4,%002,/,                                         
         1,4,%003,/,                                         
         :                                                   
         1,4,%200)                                           
END

Will give you up to 200 lines (including blank ones when not that much separators are found).
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Tue Jun 06, 2023 1:36 pm
 Reply with quote
.. thank you, can I pls ask if : is allowed

Code:
   OPTION COPY                                                     
    OUTFIL PARSE=(%=(ABSPOS=5),                                     
                   %001=(ENDBEFR=X'15',FIXLEN=32752,REPEAT=200)),   
       BUILD=(1,4,%001,/,                                           
              1,4,%002,/,                                           
              1,4,%003,/,                                           
              :                                                     
              *                                                     
              1,4,%200)                                             
    END                                                             
   
 WER268A  OUTFIL STATEMENT  : SYNTAX ERROR                   
Back to top
View user's profile Send private message
dneufarth

Active User


Joined: 27 Apr 2005
Posts: 420
Location: Inside the SPEW (Southwest Ohio, USA)
PostPosted: Tue Jun 06, 2023 6:38 pm
 Reply with quote
you need to add
Code:
1,4,%nnn,/,
for each parsed field from %004 thru %199
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Tue Jun 06, 2023 8:01 pm
 Reply with quote
One more JCL SORT step may be needed: to eliminate possible multiple empty records.

Code:
 INCLUDE COND=(1,2,BI,GT,+4)   check RDW recsize


For this purpose you may need to truncate initially split fields to their actual parsed size...

Another way, without physical truncation:
Code:
 INCLUDE COND=(5,1,CH,NE,C' ')
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Tue Jun 06, 2023 8:19 pm
 Reply with quote
Thank you .... making great progress

Question: is there a way to include wilde card to apply PARSE and BUILD to all the records and avoid %nn ?

Code:
   OPTION COPY                                                         
   OUTFIL PARSE=(%=(ABSPOS=5),                                         
                  %001=(ENDBEFR=X'15',FIXLEN=32752,REPEAT=20)),         
      BUILD=(1,4,%001,/,                                               
             1,4,%002,/,                                               
             1,4,%003,/,                                               
             1,4,%004,/,                                               
             1,4,%005,/,                                               
             1,4,%006,/,                                               
             1,4,%007,/,                                               
             1,4,%008,/,                                               
             1,4,%009,/,                                               
             1,4,%010,/,                                               
             1,4,%011,/,                                               
             1,4,%012,/,                                               
             1,4,%013,/,                                               
             1,4,%014,/,                                               
             1,4,%015,/,                                               
             1,4,%016,/,                                               
             1,4,%017,/,                                               
             1,4,%018,/,                                               
             1,4,%019,/,                                               
             1,4,%020)                                                 
   END           
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Tue Jun 06, 2023 9:00 pm
 Reply with quote
roberto2023 wrote:
Question: is there a way to include wilde card to apply PARSE and BUILD to all the records and avoid %nn ?

In the ISPF editor it took me about 13 seconds to create as many lines in BUILD as needed.
Just a test JCL, to demonstrate the method of clean splitting.
Code:
//*====================================================================
//FTOV     EXEC PGM=IEBGENER                                           
//SYSPRINT DD  SYSOUT=*                                               
//SYSIN    DD  DUMMY                                                   
//SYSUT1   DD  *       separators are X'0A'                           
1111111111 Name1 Address1 Phone1                                       
2222222222 Name2 Name3 Phone2                                         
3333333333 Name4 Address2 Address3 Phone3 Phone4                       
//*                                                                   
//SYSUT2   DD  DISP=(NEW,PASS),SPACE=(TRK,(10,10)),                   
//             DCB=(RECFM=VB,LRECL=104,BLKSIZE=0),                     
//             DSN=&&FORMATVB                                         
//*                                                                   
//*====================================================================
//SPLIT    EXEC PGM=SYNCSORT                                           
//SYSOUT   DD  SYSOUT=*                                               
//SORTIN   DD  DISP=(OLD,PASS),DSN=&&FORMATVB                         
//*                                                                   
//SORTOUT  DD  DISP=(NEW,PASS),SPACE=(TRK,(10,10)),                   
//             DCB=(RECFM=VB,LRECL=104,BLKSIZE=0),                     
//             DSN=&&SPLITVB                                           
//*                                                                   
//SYSIN    DD  *                                                       
 SORT FIELDS=COPY                                                     
 OUTFIL PARSE=(%=(ABSPOS=5),                                           
              %001=(ENDBEFR=X'0A',FIXLEN=100,REPEAT=100)),             
       BUILD=(1,4,%001,SQZ=(VL,MID=C' '),                             
            /,1,4,%002,SQZ=(VL,MID=C' '),                             
            /,1,4,%003,SQZ=(VL,MID=C' '),                             
            /,1,4,%004,SQZ=(VL,MID=C' '),                             
            /,1,4,%005,SQZ=(VL,MID=C' '),                             
            /,1,4,%006,SQZ=(VL,MID=C' '),                             
            /,1,4,%007,SQZ=(VL,MID=C' '),                             
            /,1,4,%008,SQZ=(VL,MID=C' '),                             
            /,1,4,%009,SQZ=(VL,MID=C' '),                             
            /,1,4,%010,SQZ=(VL,MID=C' '),                             
            /,1,4,%011,SQZ=(VL,MID=C' '),                             
            /,1,4,%012,SQZ=(VL,MID=C' '),                             
            /,1,4,%013,SQZ=(VL,MID=C' '),                             
            /,1,4,%014,SQZ=(VL,MID=C' '),                             
            /,1,4,%015,SQZ=(VL,MID=C' '),                             
            /,1,4,%016,SQZ=(VL,MID=C' '),                             
            /,1,4,%017,SQZ=(VL,MID=C' '),                             
            /,1,4,%018,SQZ=(VL,MID=C' '),                             
            /,1,4,%029,SQZ=(VL,MID=C' '),                             
            /,1,4,%020,SQZ=(VL,MID=C' '),                             
            /,1,4,%021,SQZ=(VL,MID=C' '),                             
            /,1,4,%022,SQZ=(VL,MID=C' '))                             
//*                                                                   
//*====================================================================
//PRINTVBX EXEC PGM=SYNCSORT                                           
//SYSOUT   DD  SYSOUT=*                                               
//SORTIN   DD  DISP=(OLD,PASS),DSN=&&SPLITVB                           
//SORTOUT  DD  SYSOUT=*                                               
//SYSIN    DD  *                                                       
 INCLUDE COND=(1,2,BI,GT,+4)                                           
 SORT FIELDS=COPY                                                     
 OUTREC BUILD=(1,4,                                                   
               C'(',1,2,BI,EDIT=(TTTTT),C',',3,2,BI,EDIT=(TTTTT),C')',
               C':',5)                                                 
//*                                                                   
//*====================================================================


The result is like below:
Code:
********************************* TOP OF DATA ******
(00014,00000):1111111111                           
(00009,00000):Name1                                 
(00012,00000):Address1                             
(00010,00000):Phone1                               
(00014,00000):2222222222                           
(00009,00000):Name2                                 
(00009,00000):Name3                                 
(00010,00000):Phone2                               
(00014,00000):3333333333                           
(00009,00000):Name4                                 
(00012,00000):Address2                             
(00012,00000):Address3                             
(00010,00000):Phone3                               
(00010,00000):Phone4                               
******************************** BOTTOM OF DATA ****
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Wed Jun 07, 2023 1:07 am
 Reply with quote
.. in my case the challenge is that I do not have control on number of records in in input, so if I use limited %nn risk to miss out valid information
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Wed Jun 07, 2023 1:45 am
 Reply with quote
roberto2023 wrote:
.. in my case the challenge is that I do not have control on number of records in in input, so if I use limited %nn risk to miss out valid information

One way to do it:
1) split records to any limited amount of parts, let’s say up to 200 parts. The last parts of each record may include extra unprocessed bytes X’0A’;
2) such records need to be separated, and that intermediate subset can be processed once again, using exactly the same procedure.

If each stage can split up to 200 substrings, the procedure can be repeated until no more records with X’0A’ byte left.

That’s it. Not a rocket science.
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Wed Jun 07, 2023 1:05 pm
 Reply with quote
Thank, would like to ask few clarification

1. REPEAT command ... can I please confim that this repeat the PARSING within the same line ... let"s say I set to 20, then will repeat the pasing within same line 20 times ?

2. The FIXLEN, how is this correlate to LREC ? in my case I have an LRECL=32756 but FIXLEN=32752. what would be the risk of mismatch in this case ? is there a VARLEN for VB files ?
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Wed Jun 07, 2023 1:26 pm
 Reply with quote
roberto2023 wrote:

1. REPEAT command ... can I please confirm that this repeat the PARSING within the same line ... let's say I set to 20, then will repeat the parsing within same line 20 times ?

Yes.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Wed Jun 07, 2023 5:17 pm
 Reply with quote
roberto2023 wrote:

2. The FIXLEN, how is this correlate to LREC ? in my case I have an LRECL=32756 but FIXLEN=32752. what would be the risk of mismatch in this case ? is there a VARLEN for VB files ?

FIXLEN must define your maximum possible substring length (between any pair of X'0A').
For the last used %nnn its FIXLEN must be long enough to fit the rest of the string yet unparsed; to be on safe side it makes sense to make the last FIXLEN be equal to your LRECL.

P.S.
By default any %nnn substring shorter than its FIXLEN is extended to the right with regular blanks. That's why making FIXLEN longer than needed would require more resources when running.
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Thu Jun 08, 2023 11:27 pm
 Reply with quote
// thank you for helping, unfortunately given the matter that I have no control on the input file the matter that I have to define limited REPEAT and limited BUILD my be a constrain for me to use this feature ... I wonder if I should consider EASYTRIEVE
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Fri Jun 09, 2023 12:17 pm
 Reply with quote
..in case I use REPEAT=200, always hit follow error

Code:
WER039A  INSUFFICIENT VIRTUAL STORAGE


how can this be overcome ? I tried with REGION at SORT step level but it doesn"t seems to help . thank you
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Fri Jun 09, 2023 12:41 pm
 Reply with quote
Try REGION=0M
Back to top
View user's profile Send private message
roberto2023

New User


Joined: 05 Jun 2023
Posts: 14
Location: Germany
PostPosted: Fri Jun 09, 2023 1:45 pm
 Reply with quote
Thank you, tried with follow set-up



Code:
//*                                                                     
//SPLITR   EXEC PGM=SORT,REGION=0M                                     
//SYSOUT   DD  SYSOUT=*                                                 
//SORTIN   DD  DSN=&CNVRT.,DISP=SHR                                     
//SORTOUT  DD  DSN=&SRTOT,DISP=(,CATLG,DELETE),                         
//             SPACE=(CYL,(50,50),,),RECFM=VB,LRECL=32756,BLKSIZE=32760
//SYSIN    DD  *                                                       
   OPTION COPY                                                         
   OUTFIL PARSE=(%=(ABSPOS=5),                                         
                  %001=(ENDBEFR=X'15',FIXLEN=32752,REPEAT=200)),       
      BUILD=(1,4,%001,/,                                               
             1,4,%002,/,                                               
             1,4,%003,/,   



but still getting follow issue


Code:
WER410B  6,660K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16-MEGABYTE LINE,
WER410B     0 BYTES RESERVE REQUESTED, 6,892K BYTES USED                     
WER039A  INSUFFICIENT VIRTUAL STORAGE                                         
WER426I  SORT INTERNAL ERROR - RECOVERY ATTEMPT IN PROGRESS
Back to top
View user's profile Send private message
daveporcelan

Active Member


Joined: 01 Dec 2006
Posts: 792
Location: Pennsylvania
PostPosted: Fri Jun 09, 2023 7:31 pm
 Reply with quote
What is the REGION= on the JOB card?

I know this has been discussed many times in many places, but my understanding that the REGION= on the JOB overrides that on the Step.

That seems counter-intuitive to me, but it is what I have read.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> SYNCSORT Goto page 1, 2  Next

 

Similar Topics
Topic Forum Replies
No new posts Compare 2 files and retrive records f... DFSORT/ICETOOL 3
No new posts Compare 2 files(F1 & F2) and writ... JCL & VSAM 8
No new posts Need to set RC4 through JCL SORT DFSORT/ICETOOL 5
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Compare only first records of the fil... SYNCSORT 7
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us