IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Need to OMIT more than 4000 records from a File


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Mon Sep 27, 2010 7:51 pm
Reply with quote

Hi,

How do I Omit around 4000 records in a single SORT Job from a Big file.
I do have the key term in each record in a PS Dataset.

Thanks!
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Sep 27, 2010 8:41 pm
Reply with quote

Hello,

Specify OMIT for the value(s) you do not want in the output file.

If you want more help, you need to provide more info.

Post some sample input data and the output you want from that sample input (use the Code tag for alignment/readability). Post the dsorg and lrecl of the files. Post any rules for getting from the input to the output.
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Tue Sep 28, 2010 1:33 pm
Reply with quote

Thanks D

Here are the details,

Input file is PS file of LRECL 813 -

Code:

0120538201AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 
0127159901AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
0121657801AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
.
.
.


The input file contains 1784989 records
The field 1-10 (Item number) is what I have with me. I have around 4000 Item numbers for which its entire record has to be removed now.

I wanted to know if OMIT - SORT can do this, if so how do i give so many Inputs.

Thanks!
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10873
Location: italy

PostPosted: Tue Sep 28, 2010 1:38 pm
Reply with quote

just a hint, search the forums for more details...

what about using two datasets and use something like the joinkeys approach for paired and unpaired record

in this way You would not have to change the SORT cards just the content of the EXCLUDE/OMIT dataset

see for example
ibmmainframes.com/viewtopic.php?t=47436&highlight=joinkeys
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Wed Sep 29, 2010 3:00 am
Reply with quote

I came to know that DFSORT is present in our installation.
But the PTF UK51707 is still not applied (I got the ICE005A 0 BLANK NEEDED IN COLUMN 1 error)

I have asked system programmer for upgrade too.

In the meantime, What other options can I do this on SORT (DFSORT now)

If ICETOOL, can anyone help me with the JCL?

Thanks a lot,

Regards
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Wed Sep 29, 2010 3:33 am
Reply with quote

Hello,

Your topic has been relocated to the DFSORT part of the forum and Syncsort has been removed form the Subject/Title. Also, some replies that no longer relate were deleted. . .
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Wed Sep 29, 2010 3:43 am
Reply with quote

If you don't have the DFSORT PTF with JOINKEYS, two other options would be:

1) SPLICE
2) Multiple IFTHEN clauses (to overcome the limit for the number of conditions in one logical expression which is about 1600 for the type of constant you're talking about).

For examples of SPLICE, see the "Create files with matching and non-matching records" Smart DFSORT Trick at:

www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094

For an example of the technique with multiple IFTHEN clauses, see the Programmer Response for message ICE151A at:

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CM50/2.2.147?SHELF=&DT=20100618164638&CASE=
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Wed Sep 29, 2010 4:17 pm
Reply with quote

Thanks Frank.
I used your post - ibmmainframes.com/about3049.html too.

But seems, I'm missing something - can you help with this?

My JCL:
Code:

//S1 EXEC PGM=ICETOOL                                   
//TOOLMSG DD SYSOUT=*                                   
//DFSMSG DD SYSOUT=*                                     
//IN1 DD DSN=PUBLIC.STAR.EARL.EXTR120.G4828,DISP=SHR     
//IN2 DD DSN=PUBLIC.STAR.SPAM.ISSUES1,DISP=SHR           
//*USE MOD FOR T1                                       
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(TRK,(5,5)),           
// DISP=(MOD,PASS)                                       
//OUT12 DD SYSOUT=* NAMES IN FILE1 AND FILE2             
//OUT1  DD DSN=PUBLIC.STAR.EARL.EXTR120.G4828Z,         
//           UNIT=(SYSDA,4),SPACE=(CYL,(2369,500),RLSE),
//           DCB=*.IN1,                                 
//           DISP=(NEW,CATLG,DELETE)                     
//OUT2  DD DUMMY    NAMES IN FILE2 ONLY                 
//* ADD '11' IDENTIFIER FOR FILE1 RECORDS.               
//* ADD '22' IDENTIFIER FOR FILE2 RECORDS.               
//* SPLICE TO MATCH UP RECORDS AND WRITE THEM TO THEIR   
//* TO THEIR APPROPRIATE OUTPUT FILES.                   
//TOOLIN DD *                                           
COPY FROM(IN1) TO(T1) USING(CTL1)                       
COPY FROM(IN2) TO(T1) USING(CTL2)                       
SPLICE FROM(T1) TO(OUT1) ON(1,10,CH) WITH(814,1) -       
  KEEPNODUPS USING(CTL3)                                 
/*                                                       
//* MARK FILE1 RECORDS WITH '11'                         
//CTL1CNTL DD *                                         
  INREC OVERLAY=(813:C'11')                             
/*                                                       
//* MARK FILE1 RECORDS WITH '22'                         
//CTL2CNTL DD *                                         
  INREC OVERLAY=(81:C'22')                               
/*                                                       
//* WRITE MATCHING RECORDS TO OUT12 FILE. REMOVE ID.     
//* WRITE FILE1 ONLY RECORDS TO OUT1 FILE. REMOVE ID.   
//* WRITE FILE2 ONLY RECORDS TO OUT2 FILE. REMOVE ID.   
//CTL3CNTL DD *                                         
  OUTFIL FNAMES=OUT12,INCLUDE=(814,2,CH,EQ,C'12'),BUILD=(1,813)
  OUTFIL FNAMES=OUT1,INCLUDE=(814,2,CH,EQ,C'11'),BUILD=(1,813)
  OUTFIL FNAMES=OUT2,INCLUDE=(81,2,CH,EQ,C'22'),BUILD=(1,80)   
/*                                                             


As told earlier in teh post - I need to remove records from IN1 based on the 10 digit key from IN2.
The IN1 file is FB with LRECL 813
The IN2 file is FB with LRECL 80.

The error i got is:

Code:

ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED                                   
                                                                               
ICE650I 0 VISIT http://www.ibm.com/storage/dfsort FOR ICETOOL PAPERS, EXAMPLES AND MORE
                                                                               
ICE632I 0 SOURCE FOR ICETOOL STATEMENTS:  TOOLIN                               
                                                                               
                                                                               
ICE630I 0 MODE IN EFFECT:  STOP                                                 
                                                                               
          COPY FROM(IN1) TO(T1) USING(CTL1)                                     
ICE606I 0 DFSORT CALL 0001 FOR COPY FROM IN1      TO T1       USING CTL1CNTL TERMINATED
ICE602I 0 OPERATION RETURN CODE:  16                                           
                                                                               
ICE630I 2 MODE IN EFFECT:  SCAN                                                 
                                                                               
          COPY FROM(IN2) TO(T1) USING(CTL2)                                     
ICE612I 0 NO ERRORS FOUND IN STATEMENT                                         
                                                                               
          SPLICE FROM(T1) TO(OUT1) ON(1,10,CH) WITH(814,1) -                   
            KEEPNODUPS USING(CTL3)                                             
ICE612I 0 NO ERRORS FOUND IN STATEMENT                                         
                                                                               
                                                                               
ICE601I 0 DFSORT ICETOOL UTILITY RUN ENDED - RETURN CODE:  16                   


Since the IN1 file is huge, i tried increasing T1's space too, but of no use.

Thanks!
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Wed Sep 29, 2010 5:43 pm
Reply with quote

alan_blore,
Will you always have 4000 records in file2? Could you possibly have duplicates in either file1 and/or file2 for the key?

From your sort card, if your file1 is 813 bytes, why are you overlaying 813th byte? Shouldn't you start overlay at 814th position.

If you can't have duplicates in file2 and if it always contain 4000 some records,there may be an easier way.

Thanks,
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Wed Sep 29, 2010 6:04 pm
Reply with quote

Hi Sqlcode1,

Thanks for pointing the overlay in CTL1CNTL - even after i corrected it to 814 I get the same error.

The FILE2 will always have known records count of 4000 odd records.

Let me know the easier way or the error in the SORT above.
(We don't use COBOL here, else I would have picked the sticky program for file-matching from the forum)

Thanks!
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Wed Sep 29, 2010 6:05 pm
Reply with quote

Hi Sqlcode1,

Adding to the above -

File2 doesnt contain any duplicates and has only 4000 (fixed count) records

Thanks!
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Wed Sep 29, 2010 7:10 pm
Reply with quote

Hello,

Quote:
Since the IN1 file is huge, i tried increasing T1's space too, but of no use.
How big is huge? What size/space did you try for t1?

You could actually calculate the size needed for t1 of you know the volume of the files and not just guess. . .
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Wed Sep 29, 2010 7:45 pm
Reply with quote

Hi D,

I gave the same size which used to created the big IN1 - SPACE=(CYL,(2369,500),RLSE) to T1 too, still got the same error.

Thanks!
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Wed Sep 29, 2010 7:54 pm
Reply with quote

alan_blore,
See if below job works for you...
Code:
//STEP01   EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD DISP=SHR,DSN=YOUR FB80 UNIQUE ID FILE                     
//SORTOUT  DD DSN=&&T1,DISP=(MOD,PASS),SPACE=(CYL,(10,5),RLSE)         
//SYSIN    DD *                                                         
  OPTION COPY                                                           
  INREC BUILD=(1,10,813:X)                                             
  OUTFIL REMOVECC,HEADER1=(3C'$'),TRAILER1=(3C'$')                     
//*                                                                     
//STEP0200 EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD DSN=&&T1,DISP=SHR,VOL=REF=*.STEP01.SORTOUT               
//         DD DISP=SHR,DSN=YOUR FB813 FILE                             
//SORTOUT  DD SYSOUT=*                                                 
//SYSIN    DD *                                                         
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'$$$'),PUSH=(814:ID=1))   
  SORT FIELDS=(01,10,CH,A),EQUALS                                       
  OUTREC IFTHEN=(WHEN=GROUP,BEGIN=(814,1,ZD,EQ,1),PUSH=(815:1,10)),     
         IFTHEN=(WHEN=(01,10,CH,EQ,815,10,CH),OVERLAY=(825:C'N'))       
  OUTFIL OMIT=(825,1,CH,EQ,C'N'),BUILD=(1,813)                         
//*                                                                     


Thanks,
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Wed Sep 29, 2010 8:07 pm
Reply with quote

alan_blore,
Regarding your splice job, you have logical errors in your control card. Please try to refer to the link for Smart DFSort Trick which Frank provided earlier and code your job accordingly.

Thanks,
Back to top
View user's profile Send private message
alan_blore

New User


Joined: 01 Aug 2007
Posts: 50
Location: Hosur

PostPosted: Thu Sep 30, 2010 2:55 am
Reply with quote

This worked for me.

Thanks a lot Sqlcode1, Frank, Enirco, Dick and others.

My Bad, I wasn't able to debug the logical error on ICETOOL job though.

Thanks
alan
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Extract the file name from another fi... DFSORT/ICETOOL 6
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts INCLUDE OMIT COND for Multiple values... DFSORT/ICETOOL 5
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
Search our Forums:

Back to Top