IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Use input file with OMIT rcd keys?


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Time2Live

New User


Joined: 27 Apr 2005
Posts: 43
Location: United States

PostPosted: Fri Jul 28, 2023 4:02 am
Reply with quote

Is there a way to use ICETOOLS (or similar) to copy a file and omit records based on field values in a SYSIN or input file? I would like to use a file because the job runs weekly and the 100+ records to OMIT vary each time the job runs.

This JCL is what I have that works, but would like the OMITs in a file so I dont need to 'implement' new control cards every week. The numeric field I'm matching against is packed. Thank you for any advice.

Code:
//SORTIN    DD DSN=INPUT.FILE
//SORTOF1   DD DSN=OUTPUT.FILE
//SYSIN     DD DATA
    SORT FIELDS=COPY
    OUTFIL FILES=1,OMIT=((2,6,PD,EQ,2004001003),OR,
                         (2,6,PD,EQ,2004001011),OR,
                         (2,6,PD,EQ,2004004014),OR,
                         (2,6,PD,EQ,2004004019))


input file
Code:

2004001003
2004001011
2004004014
2004004019
....plus more keys
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Fri Jul 28, 2023 4:47 am
Reply with quote

Use JOIN option.
Code:
//SYSIN    DD  *
 JOINKEYS F1=SORTIN,FIELDS=(2,6,A)
 JOINKEYS F2=OMIT,FIELDS=(11,6,A)
 JOIN UNPAIRED,F1,ONLY
 . . .
//*
//OMIT      DD  *
2004001003
2004001011
2004004014
2004004019
//*
//JNF2CNTL DD  *
 INREC OVERLAY=(11:1,10,ZD,TO=PD,LENGTH=6)
//*

About details, please RTFM.
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1295
Location: Bamberg, Germany

PostPosted: Fri Jul 28, 2023 10:52 am
Reply with quote

The JOINKEYS operation will sort your data by the key(s). If you need the original sequence, some additional means need to be implemented.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Fri Jul 28, 2023 5:09 pm
Reply with quote

For matching fields PD vs ZD it can be simplified (no field conversion)
Code:
//SYSIN    DD  *
 JOINKEYS F1=SORTIN,FIELDS=(2,6,PD,A)
 JOINKEYS F2=OMIT,FIELDS=(1,10,ZD,A),
          SORTED             (if ordered list)
 JOIN UNPAIRED,F1,ONLY
 . . .
//*
//OMIT      DD  *   
2004001003
2004001011
2004004014
2004004019
//*


To get the final results in desired order, also appropriate SORT may be needed.
JOIN can be performed by the utility in different ways, so the records order after JOIN is generally unpredictable. This sequence can even change unexpectedly depending on the size of data, e.a.
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1295
Location: Bamberg, Germany

PostPosted: Fri Jul 28, 2023 7:23 pm
Reply with quote

sergeyken wrote:
For matching fields PD vs ZD it can be simplified (no field conversion)
Code:
//SYSIN    DD  *
 JOINKEYS F1=SORTIN,FIELDS=(2,6,PD,A)
 JOINKEYS F2=OMIT,FIELDS=(1,10,ZD,A),

This is unfortunately not valid for DFSORT.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Fri Jul 28, 2023 7:58 pm
Reply with quote

Joerg.Findeisen wrote:
sergeyken wrote:
For matching fields PD vs ZD it can be simplified (no field conversion)
Code:
//SYSIN    DD  *
 JOINKEYS F1=SORTIN,FIELDS=(2,6,PD,A)
 JOINKEYS F2=OMIT,FIELDS=(1,10,ZD,A),

This is unfortunately not valid for DFSORT.

Oops... Sorry, I forgot the difference
Back to top
View user's profile Send private message
Time2Live

New User


Joined: 27 Apr 2005
Posts: 43
Location: United States

PostPosted: Fri Jul 28, 2023 11:16 pm
Reply with quote

Thank You So Much sergeyken! It works wonderfully and I dont know how I would have ever figured it out by just RTFM, lol
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Sat Jul 29, 2023 7:57 pm
Reply with quote

BTW, the initial code can be simplified, too:
Code:
//SORTIN    DD DSN=INPUT.FILE
//SORTOF1   DD DSN=OUTPUT.FILE
//SYSIN     DD *
 SORT FIELDS=COPY
 OUTFIL FILES=1,
        OMIT=(2,6,PD,EQ,L(2004001003,2004001011,2004004014,2004004019))
//*


P.S.
Still, RTFM very often may be useful. icon_biggrin.gif
Back to top
View user's profile Send private message
Time2Live

New User


Joined: 27 Apr 2005
Posts: 43
Location: United States

PostPosted: Mon Jul 31, 2023 10:01 pm
Reply with quote

Hi sergeykey,

The users chged the requirements and they want the omitted records written in a 2nd output file. If you can please tell me the 'key word' to look-up in the manual to accomplish this, I can RTFM to look for the structure.

Thank you for understanding, Time2Live
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Mon Jul 31, 2023 10:43 pm
Reply with quote

Code:
// . . . . . . . . . . . . . . .
//SYMNAMES DD  *
Key_Len,6                  key length
SORTIN_Key,2,6,PD          matching key field
SORTIN_Record,1,lrecl,CH   full source record
JOIN_ID,*,1,CH             pairing indicator: B/1/2
OMIT_Key,1,10,ZD           printable omit key
OMIT_Key_PD,*,6,PD         matching omit key
//*
//VALID    DD  SYSOUT=*
//INVALID  DD  SYSOUT=*
//*
//SYSIN    DD  *
 JOINKEYS F1=SORTIN,FIELDS=(SORTIN_Key,A)
 JOINKEYS F2=OMIT,FIELDS=(OMIT_Key_PD,A)
 JOIN UNPAIRED,F1
 REFORMAT FIELDS=(F1:SORTIN_Record,   full original record
                  ?)                  pairing indicator
 . . .
 OUTFIL FNAMES=VALID,
        INCLUDE=(JOIN_ID,NE,C'B'),   select non-matching records
        BUILD=(SORTIN_Record)
 OUTFIL FNAMES=INVALID,
        SAVE,                       all other records
        BUILD=(SORTIN_Record)
//*
//OMIT      DD  *
2004001003
2004001011
2004004014
2004004019
//*
//JNF2CNTL DD  *
* Convert omit key to the SORTIN key type
 INREC OVERLAY=(OMIT_Key_PD:OMIT_Key,TO=PD,LENGTH=Key_Len)
//*
Back to top
View user's profile Send private message
Time2Live

New User


Joined: 27 Apr 2005
Posts: 43
Location: United States

PostPosted: Tue Aug 01, 2023 12:40 am
Reply with quote

WoW sergeykey!! That's a mouthfull! Thanks!
Back to top
View user's profile Send private message
Time2Live

New User


Joined: 27 Apr 2005
Posts: 43
Location: United States

PostPosted: Tue Aug 01, 2023 4:49 am
Reply with quote

Hi sergeykey,
Sorry to be so bothersome, but Im still having a difficult time with this Sort. I have been working on the last sort you posted all day and cant figure out how to stop the input file LRECL=1788 from incrementing by 1.
Code:
ICE414A 0 SORTIN   (F1) REFORMAT FIELD END AT 1789 IS BEYOND LENGTH OF 1788. 

Also I am confused abt these lines --
JOIN_ID,*,1,CH pairing indicator: B/1/2 <-- shld the * be B ?
INCLUDE=(JOIN_ID,NE,C'B'), select non-matching records

What I would like to do is a second sort to include the records omitted on the (working) first sort into a different file. (Maybe I can fit them together later)
I thought it would be as easy as chging 'JOIN UNPAIRED' to 'JOIN PAIRED' but of course that is not valid.

What am I missing? Thank you again for all your help.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Tue Aug 01, 2023 6:10 am
Reply with quote

Time2Live wrote:
Hi sergeykey,
Sorry to be so bothersome, but Im still having a difficult time with this Sort. I have been working on the last sort you posted all day and cant figure out how to stop the input file LRECL=1788 from incrementing by 1.
Code:
ICE414A 0 SORTIN   (F1) REFORMAT FIELD END AT 1789 IS BEYOND LENGTH OF 1788. 


Check that it is defined as:
Code:
SORTIN_Record,1,1788,CH

The output records are truncated to the original size by
Code:
 …BUILD=(SORTIN_Record)

Quote:

Also I am confused abt these lines --
JOIN_ID,*,1,CH pairing indicator: B/1/2 <-- shld the * be B ?
INCLUDE=(JOIN_ID,NE,C'B'), select non-matching records

* - this means: “next position after the previous field”. It cannot be B as per allowed syntax (RTFM).
Quote:

What I would like to do is a second sort to include the records omitted on the (working) first sort into a different file. (Maybe I can fit them together later)

Before adding the second SORT, it would be nice to understand: where is your first SORT?
Quote:

I thought it would be as easy as chging 'JOIN UNPAIRED' to 'JOIN PAIRED' but of course that is not valid.

What am I missing? Thank you again for all your help.

“JOIN PAIRED”
(1) is not allowed by syntax (RTFM), and
(2) it would be the default value, when no JOIN is coded, and
(3) unpaired records are excluded unless explicit “JOIN UNPAIRED” has been coded (RTFM)

Please, demonstrate here YOUR ACTUAL CODE IN FULL, if you really have any problem.

RTFM, RTFM, and RTFM


I RECOMMEND YOU TO COPY-AND-PASTE MY SAMPLE OF CODE PRIOR TO CHANGING IT IN A CRAZY MANNER
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Tue Aug 01, 2023 6:58 pm
Reply with quote

It is a very-very-very bad habit: to tell the story like "nothing is working" without showing any real code, which is not working.

I was not lazy, and re-ran my own copy of the code. The only difference is, my test LRECL=80
Code:
//OMITLIST EXEC PGM=SORT                                       
//SYSOUT   DD  SYSOUT=*                                       
//*                                                           
//SORTIN   DD  *                                               
          - 2004001003                                         
          - 0004001003                                         
          - 2004001011                                         
          - 0004001011                                         
      <   - 2004004014                                         
      <   - 0004004014                                         
      æ°  - 2004004019                                         
       °  - 0004004019                                         
//*-+----1----+----2----+----3----+----4----+----5             
//*                                                           
//SYMNAMES DD  *
* SORTIN fields definitions                                             
Key_Len,6                  key length                         
SORTIN_Key,2,6,PD          matching key field                 
SORTIN_Record,1,80,CH      full source record                 
JOIN_ID,*,1,CH             pairing indicator: B/1/2     
*
* OMIT fields definitions       
OMIT_Key,1,10,ZD           printable omit key                 
OMIT_Key_PD,*,6,PD         matching omit key                   
//*                                                           
//VALID    DD  SYSOUT=*                                       
//INVALID  DD  SYSOUT=*                                       
//*                                                           
//SYSIN    DD  *                                                   
 JOINKEYS F1=SORTIN,FIELDS=(SORTIN_Key,A)                           
 JOINKEYS F2=OMIT,FIELDS=(OMIT_Key_PD,A)                           
 JOIN UNPAIRED,F1                                                   
 REFORMAT FIELDS=(F1:SORTIN_Record,   full original record         
                  ?)                  pairing indicator             
 SORT FIELDS=COPY                                                   
 OUTFIL FNAMES=VALID,                                               
        INCLUDE=(JOIN_ID,NE,C'B'),   select non-matching records   
        BUILD=(SORTIN_Record)                                       
 OUTFIL FNAMES=INVALID,                                             
        SAVE,                       all other records               
        BUILD=(SORTIN_Record)                                       
//*                                                                 
//OMIT      DD  *                                                   
2004001003                                                         
2004001011                                                         
2004004014                                                         
2004004019                                                         
//*                                                                 
//JNF2CNTL DD  *                                                   
* Convert omit key to the SORTIN key type                           
 INREC OVERLAY=(OMIT_Key_PD:OMIT_Key,TO=PD,LENGTH=Key_Len)
//* 


//VALID DD:
Code:
********************************* TOP OF DATA ****
          - 0004001003                           
          - 0004001011                           
          - 0004004019                           
      <   - 0004004014                           
******************************** BOTTOM OF DATA **


//INVALID DD:
Code:
********************************* TOP OF DATA **********
          - 2004001003                                 
          - 2004001011                                 
      <   - 2004004014                                 
          - 2004004019                                 
******************************** BOTTOM OF DATA ********


SORT LOG:
Code:
 . . . . . . . .
  JOINKEYS REFORMAT RECORD LENGTH=   81, TYPE = F                 
  VALID    : RECFM=FB   ; LRECL=    80; BLKSIZE=    80             
  INVALID  : RECFM=FB   ; LRECL=    80; BLKSIZE=    80             
. . . . . . . .
 SORTIN : RCD IN= 8,OMITTED= 0,PAIRED= 4,UNPAIRED= 4         
 OMIT : RCD IN= 4,OMITTED= 0,PAIRED= 4,UNPAIRED= 0     
. . . . . . . .
Back to top
View user's profile Send private message
Time2Live

New User


Joined: 27 Apr 2005
Posts: 43
Location: United States

PostPosted: Tue Aug 01, 2023 9:49 pm
Reply with quote

HI sergeyken!

I finally got it going with my files based on your test! Thank You!!

It was the question mark that threw me. I thought I had to provide a value, but once I saw it in your coding and realized that I had to leave it alone, the output looked great.
Code:
REFORMAT FIELDS=(F1:SORTIN_RECORD,
                 ?)

Thanks again So Much for all your help with this.
Wishing you the best, Time2Live
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2081
Location: USA

PostPosted: Tue Aug 01, 2023 11:11 pm
Reply with quote

Time2Live wrote:
HI sergeyken!

I finally got it going with my files based on your test! Thank You!!

It was the question mark that threw me. I thought I had to provide a value, but once I saw it in your coding and realized that I had to leave it alone, the output looked great.
Code:
REFORMAT FIELDS=(F1:SORTIN_RECORD,
                 ?)

Thanks again So Much for all your help with this.
Wishing you the best, Time2Live

RTFM, RTFM, and RTFM
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts To Omit records based n SORT condition DFSORT/ICETOOL 6
No new posts Format Binary file to EBCDIC JCL & VSAM 4
No new posts Binary File format getting change whi... All Other Mainframe Topics 7
No new posts Compare 2 files and retrive records f... DFSORT/ICETOOL 3
No new posts TRIM everything from input, output co... DFSORT/ICETOOL 1
Search our Forums:

Back to Top