IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

How to eliminate duplicates and partial files
IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Fri May 25, 2007 4:48 pm
 Reply with quote
Hi,

I have a requirement like this to eliminate duplicates:

I have an input file which contains headers and trailers repeated, kind of concatenated file...

TT01ABC
TD01456
RT01GHT
TT01DEF
TD01689
TT01MHY
TD01997
RT01NHG

where TT01 is the header record and RT01 is the trailer record(TD01 - detail record).

1) I have to eliminate duplicates by comparing the 3 fields in the header record with that of all the records in the control file(in the above case position 5,6,7). This control file should contain the 3 fields specified above for the last 20 headers. I have to write the control file through this program.

2) Also i have to check whether the header record is having the corresponding trailer record. If you check the second header it does not have a trailer record, so it is not a good one and have to write the entire set to an error file. If the 3 field values are not in the control file also if it has a trailer record then it's a good one and i have to write this to the output file.

3) And the control file which is a flat file should have only 20 records, if it is more i have to delete the old records and maintain only 20 records.

I have already posted this under cobol but wanted to know whether this is possible in DFSORT, if so it would be helpful for me.

Thanks in advance.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Mon May 28, 2007 2:56 am
 Reply with quote
It's not clear what you want from what you've posted. You need to supply more details. Please show an example of the control file, all of the different variations for the groups of records that can appear in the input file, and the expected output records.

I don't understand what you mean by "This control file should contain the 3 fields specified above for the last 20 headers". Please explain what you mean more clearly with an example.

I also don't understand what you mean by number 3). Please explain what you mean more clearly with an example.

Also, give the RECFM and LRECL of the control file and the input file.
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Mon May 28, 2007 9:23 am
 Reply with quote
Hi Frank,

The length of the control file is 3, which should contain fields in position 5,6 and 7 of the input file (of the header- TH01). We are using this file for comparing with the input file also we have to write the header details into this file.

The input file has a record length 80 which is given above. Also the record length of error file and output file is 80.

So the control file should be written like this
ABC
DEF
MHY
and i should take care it contains only 20 records because if i compare the input file headers(position 5,6,7) with that of these 20 records in the control file, it would be sufficient for eliminating duplicates.

So the input file headers fields (position 5, 6,7) should be compared with all the 20 records(position 1,2,3) and check for duplicates, also it should check for the trailer record for the corresponding header.

In the above case the first header does have duplicates, also it has a trailer record, so it should go the output file like
TT01ABC
TD01456
RT01GHT

The second header doesn't have duplicates but no trailer record , so it should go to the error file like
TT01DEF
TD01689

The third header have duplicates (even though it has a trailer record) ,so again it should be written to the output file similar to the first header.

Please let me if i need to provide any other information.

Thanks in advance.
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Mon May 28, 2007 9:50 am
 Reply with quote
The record format is FB for all the files.

Thanks.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Mon May 28, 2007 9:05 pm
 Reply with quote
Quote:
In the above case the first header does have duplicates


Quote:
The second header doesn't have duplicate


I can't figure out what you mean by "duplicates". I thought by "duplicates" you meant a match between the header and control file. The first header has ABC and there's a matching ABC in the control file. But the second header has DEF and there's a matching DEF in the control file, yet you say the second header doesn't have a duplicate. So I can't figure out what it is you mean.

Please explain more clearly what you mean by "duplicate" and why the first header has a duplicate but the second header doesn't.

Please show a more complete example of input records including a header/trailer with a duplicate, a header/trailer without a duplicate, a header/no trailer with a duplicate and a header/no trailer without a duplicate, and show the corresponding output records.

Quote:
and i should take care it contains only 20 records


And what do you want to do if it contains more than 20 records? Keep only the first 20 records? Keep only the last 20 records? Or what?
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Tue May 29, 2007 9:54 am
 Reply with quote
Hi Frank,

The explanation you have given for duplicates is correct. Duplicates meant a match between the header of the input file with that of all the records in the control file.

In the above case,i just gave an example based on assumption for the duplicates(because the first time control file would be be empty), so that i can explain how the output and error file look like.

1) Now let us consider the input file:

TT01ABC
TD01456
RT01001
TT01DEF
TD01689
TT01MHY
TD01997
RT01312
TT01ZSQ
TD01997

and the control file is having 2 records like
MHY
ZSQ

So the header record(position 5,6,7) will be compared with this control file(both the records).

For the first TT01 header there is no matching record in the control file so it is not a duplicate one but still it should check for the trailer RT01 record for the corresponding header. The first header have a corresponding trailer, hence it goes to the output file like
TT01ABC
TD01456
RT01001

The second header also doesn't have a matching record in the control file
but it also doesn't have a trailer record so it is a partial one which will go to the error file like:

TT01DEF
TD01689

The third header have matching record in the control file, so it will go to the error file
TT01MHY
TD01997
RT01312 adding with the second header.

The fourth header also have a matching record in the control file, it will be written to the error file:

TT01ZSQ
TD01997

Basically, first duplicate check should be made, if duplicates are not found we have to check whether it has a corresponding trailer. If duplicates are found, no need to check for the trailer record because we are confirming that it should go to the error file. But it should be written to the error file from header to the corresponding trailer.

2) Regarding your second question, i want to keep only last 20 records in the control file. The control file is a flat file and i want to delete the old records.

3) Also in addition to writing into the output and error file, if a duplicate is not found also if it has a trailer record then the position 5,6,7 of the header record should be written to the control file.

4) At the end the control file will look like:

MHY
ZSQ
ABC

the error file will look like:

TT01DEF
TD01689
TT01MHY
TD01997
RT01312
TT01ZSQ
TD01997

Thanks in advance.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Tue May 29, 2007 11:45 pm
 Reply with quote
So basically you're saying that

1) If the control file has more than 20 records, you want to keep the last 20 records (for example, if the control file has 22 records, you want to keep records 3-22). Right?

2) If a "group" has a header record and a trailer record and if the header record does NOT have a match in the control file, you want to write that "group" to the output file, and you want to add its header string to the control file. And all other records go to the error file. Right?

Does every group have one and only one TD record? Or can a group have no TD records or more than one TD record?

Please run this job and post the //SYSOUT messages - I want to see what level your at:

Code:

//S1    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN DD *
RECORD 1
//SORTOUT DD SYSOUT=*
//SYSIN    DD    *
  OPTION COPY
  INREC OVERLAY=(5:C'A')
/*
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Wed May 30, 2007 9:35 am
 Reply with quote
Hi Frank,

Yes, you are right. I want to keep the last 20 records in the control file.

The second point is also perfect.

And every 'group' will be having many TD records, there are no chances that a group does not have TD records because i am going to process only the TD records in one more program.

I ran the below job given by you and please find the SYSOUT messages below:

Code:

//STEP10  EXEC PGM=ICEMAN   
//SYSOUT    DD SYSOUT=*     
//SORTIN    DD *           
TT01ABC                     
TD01456                     
RT01001                     
TT01DEF                     
TD01689                     
TT01MHY                     
TD01997                     
RT01312                     
TT01ZSQ                     
TD01997                     
//SORTOUT   DD SYSOUT=*     
//SYSIN     DD *           
  OPTION COPY               
  INREC OVERLAY=(5:C'A')   
/*   


SYSOUT mesages:

Code:

...
1ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED     
...                                                                                       
 ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 23:35 ON TUE MAY 29, 2007 -
...     
 ICE201I E RECORD TYPE IS F - DATA STARTS IN POSITION
...
Thanks.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Thu May 31, 2007 4:00 am
 Reply with quote
Well, it isn't pretty and requires lots of passes to do everything you asked for, but here's the DFSORT/ICETOOL job I came up with:

Code:

//S1 EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//CF DD DSN=...  control file (FB/3)
//IN DD DSN=...  input file (FB/7)
//CTL2CNTL DD DSN=&&C2,DISP=(,PASS),SPACE=(TRK,(1,1)),UNIT=SYSDA
//   DD *
  OUTFIL FNAMES=T1,BUILD=(5:1,3,36:X)
  OUTFIL FNAMES=CFOUT,BUILD=(1,3)
/*
//T0 DD DSN=&&T0,UNIT=SYSDA,SPACE=(TRK,(2,2)),DISP=(,PASS)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T2 DD DSN=&&T2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T3 DD DSN=&&T3,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON1 DD DSN=*.T3,VOL=REF=*.T3,DISP=(OLD,PASS)
//     DD DSN=*.T1,VOL=REF=*.T1,DISP=(OLD,PASS)
//T4 DD DSN=&&T4,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T5 DD DSN=&&T5,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON2 DD DSN=*.T4,VOL=REF=*.T4,DISP=(OLD,PASS)
//     DD DSN=*.T5,VOL=REF=*.T5,DISP=(OLD,PASS)
//OUT1    DD DSN=...  output file (FB/7)
//OUT2    DD DSN=...  error output file (FB/7)
//CFOUT DD DSN=...    modified control file (FB/3)
//TOOLIN   DD *
  COPY FROM(CF) USING(CTL0)
  COPY FROM(T0) TO(CTL2CNTL) USING(CTL1)
  COPY FROM(CF) USING(CTL2)
  COPY FROM(IN) USING(CTL3)
  SELECT FROM(T2) TO(T3) ON(21,8,ZD) FIRSTDUP
  SELECT FROM(CON1) TO(T4) ON(5,3,CH) NODUPS USING(CTL4)
  SPLICE FROM(CON2) TO(OUT1) ON(21,8,ZD) KEEPNODUPS KEEPBASE -
    WITHALL WITH(1,7) WITH(37,1) USING(CTL5)
/*
//CTL0CNTL DD *
  OUTFIL FNAMES=T0,NODETAIL,REMOVECC,OUTREC=(8X),
         TRAILER1=(COUNT-20=(TO=ZD,LENGTH=8))
/*
//CTL1CNTL DD *
  INREC IFOUTLEN=80,
    IFTHEN=(WHEN=(1,8,ZD,GT,+0),
          BUILD=(C' OPTION SKIPREC=',1,8)),
    IFTHEN=(WHEN=NONE,
          BUILD=(C'*'))
/*
//CTL3CNTL DD *
  INREC IFOUTLEN=36,
        IFTHEN=(WHEN=INIT,OVERLAY=(21:SEQNUM,8,ZD)),
        IFTHEN=(WHEN=(1,4,CH,EQ,C'TT01'),
                OVERLAY=(21:SEQNUM,8,ZD)),
        IFTHEN=(WHEN=NONE,
                OVERLAY=(29:SEQNUM,8,ZD,
                         21:21,8,ZD,SUB,29,8,ZD,M11,LENGTH=8))
  OUTFIL FNAMES=T2,INCLUDE=(1,4,SS,EQ,C'TT01,RT01')
  OUTFIL FNAMES=T5,OVERLAY=(37:C'VV')
/*
//CTL4CNTL DD *
  OUTFIL FNAMES=T4,INCLUDE=(1,4,CH,EQ,C'TT01'),
    OVERLAY=(37:C'BB')
  OUTFIL FNAMES=CFOUT,INCLUDE=(1,4,CH,EQ,C'TT01'),
    BUILD=(5,3)
/*
//CTL5CNTL DD *
  OUTFIL FNAMES=OUT1,INCLUDE=(37,2,CH,EQ,C'VB'),
    BUILD=(1,7)
  OUTFIL FNAMES=OUT2,INCLUDE=(37,2,CH,EQ,C'VV'),
    BUILD=(1,7)
/*
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Thu May 31, 2007 11:58 am
 Reply with quote
Hi Frank,

Thanks a lot. But i tried running the job, it is failing in 'COPY FROM(CF) USING(CTL2)'. I think CTL2 is missing in between. Please correct me if i am wrong because this is the first time i am using ICETOOL. I am going through the manuals to understand it better.

And now i got the sample input file, hence the record length of the input file, output file and error file is 900. The control file will be of length 24(position 58-81 of the input file) but i want to keep it as 80.

The record in the input file looks like:

TH0101.00999395OA ADF B200607 1323
So, the fields to be compared are 1) the field from position 58-59 2) 60-61 3) 62-81

And the control file will look like
1323
because position 62-81 is blank in the input file.

I hope the changes required for the new record lengths would be:

1) 'SELECT FROM(CON1) TO(T4) ON(5,3,CH) NODUPS USING(CTL4)'
this will be changed to 'SELECT FROM(CON1) TO(T4) ON(58,24,CH) NODUPS USING(CTL4)'

2) 'SPLICE FROM(CON2) TO(OUT1) ON(21,8,ZD) KEEPNODUPS KEEPBASE -
WITHALL WITH(1,7) WITH(37,1) USING(CTL5)'

Will be changed to 'SPLICE FROM(CON2) TO(OUT1) ON(21,8,ZD) KEEPNODUPS KEEPBASE -
WITHALL WITH(1,900) WITH(37,1) USING(CTL5)' .

Please let me know if i have missed something.

Thanks.
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Thu May 31, 2007 2:04 pm
 Reply with quote
I have used the following JCL and the error messages are as below:

Code:

//STEP10  EXEC PGM=ICETOOL                           
//TOOLMSG   DD SYSOUT=*                             
//DFSMSG    DD SYSOUT=*                             
//CF        DD DSN=T3012AU.BCGCDUPS.CNTLFILE(+0),   
//             DISP=SHR                             
//IN        DD DSN=T3012AU.BCGCDUPS.INFILE,         
//             DISP=SHR                             
//CTL2CNTL  DD DSN=&&C2,                             
//             DISP=(,PASS),                         
//             SPACE=(TRK,(1,1)),                   
//             UNIT=SYSDA                           
//   DD *                                           
  OUTFIL FNAMES=T1,BUILD=(5:1,3,36:X)               
  OUTFIL FNAMES=CFOUT,BUILD=(1,3)                   
/*                                                   
//T0        DD DSN=&&T0,                             
//             UNIT=SYSDA,                           
//             SPACE=(TRK,(2,2)),         
//             DISP=(,PASS)               
//T1        DD DSN=&&T1,                   
//             UNIT=SYSDA,                 
//             SPACE=(CYL,(5,5)),         
//             DISP=(,PASS)               
//T2        DD DSN=&&T2,                   
//             UNIT=SYSDA,                 
//             SPACE=(CYL,(5,5)),         
//             DISP=(,PASS)               
//T3        DD DSN=&&T3,                   
//             UNIT=SYSDA,                 
//             SPACE=(CYL,(5,5)),         
//             DISP=(,PASS)               
//CON1      DD DSN=*.T3,                   
//             VOL=REF=*.T3,               
//             DISP=(OLD,PASS)                                   
//          DD DSN=*.T1,                                         
//             VOL=REF=*.T1,                                     
//             DISP=(OLD,PASS)                                   
//T4        DD DSN=&&T4,                                         
//             UNIT=SYSDA,                                       
//             SPACE=(CYL,(5,5)),                                 
//             DISP=(,PASS)                                       
//T5        DD DSN=&&T5,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON2      DD DSN=*.T4,VOL=REF=*.T4,DISP=(OLD,PASS)             
//          DD DSN=*.T5,VOL=REF=*.T5,DISP=(OLD,PASS)             
//OUT1      DD DSN=T3012AU.BCGCDUPS.OUTFILE,                     
//             DISP=(NEW,CATLG,DELETE),                           
//             DCB=(RECFM=FB,LRECL=80),                           
//             SPACE=(CYL,(5,5)),                                 
//             UNIT=SYSDA                                         
//OUT2      DD DSN=T3012AU.BCGCDUPS.ERRFILE,           
//             DISP=(NEW,CATLG,DELETE),               
//             DCB=(RECFM=FB,LRECL=80),               
//             SPACE=(CYL,(5,5)),                     
//             UNIT=SYSDA                             
//CFOUT     DD DSN=T3012AU.BCGCDUPS.CNTLFILE(+1),     
//             DISP=(NEW,CATLG,DELETE),               
//             DCB=(RECFM=FB,LRECL=80),               
//             SPACE=(CYL,(5,5)),                     
//             UNIT=SYSDA                             
//TOOLIN    DD *                                       
  COPY FROM(CF) USING(CTL0)                           
  COPY FROM(T0) TO(CTL2CNTL) USING(CTL1)               
  COPY FROM(CF) USING(CTL2)                           
  COPY FROM(IN) USING(CTL3)                           
  SELECT FROM(T2) TO(T3) ON(21,8,ZD) FIRSTDUP         
  SELECT FROM(CON1) TO(T4) ON(5,3,CH) NODUPS USING(CTL4)           
  SPLICE FROM(CON2) TO(OUT1) ON(21,8,ZD) KEEPNODUPS KEEPBASE -     
    WITHALL WITH(1,7) WITH(37,1) USING(CTL5)                       
/*                                                                 
//CTL0CNTL DD *                                                   
  OUTFIL FNAMES=T0,NODETAIL,REMOVECC,OUTREC=(8X),                 
         TRAILER1=(COUNT-20=(TO=ZD,LENGTH=8))                     
/*                                                                 
//CTL1CNTL DD *                                                   
  INREC IFOUTLEN=80,                                               
    IFTHEN=(WHEN=(1,8,ZD,GT,+0),                                   
          BUILD=(C' OPTION SKIPREC=',1,8)),                       
    IFTHEN=(WHEN=NONE,                                             
          BUILD=(C'*'))                                           
/*       
//CTL3CNTL DD *                                                         
  INREC IFOUTLEN=36,                                               
        IFTHEN=(WHEN=INIT,OVERLAY=(21:SEQNUM,8,ZD)),           
        IFTHEN=(WHEN=(1,4,CH,EQ,C'TT01'),                       
                OVERLAY=(21:SEQNUM,8,ZD)),                     
        IFTHEN=(WHEN=NONE,                                     
                OVERLAY=(29:SEQNUM,8,ZD,                       
                         21:21,8,ZD,SUB,29,8,ZD,M11,LENGTH=8)) 
  OUTFIL FNAMES=T2,INCLUDE=(1,4,SS,EQ,C'TT01,RT01')             
  OUTFIL FNAMES=T5,OVERLAY=(37:C'VV')                           
/*                                                             
//CTL4CNTL DD *                                                 
  OUTFIL FNAMES=T4,INCLUDE=(1,4,CH,EQ,C'TT01'),                 
    OVERLAY=(37:C'BB')                                         
  OUTFIL FNAMES=CFOUT,INCLUDE=(1,4,CH,EQ,C'TT01'),             
    BUILD=(5,3)                                                 
/*                                                             
//CTL5CNTL DD *                                                 
  OUTFIL FNAMES=OUT1,INCLUDE=(37,2,CH,EQ,C'VB'),
    BUILD=(1,7)                                 
  OUTFIL FNAMES=OUT2,INCLUDE=(37,2,CH,EQ,C'VV'),
    BUILD=(1,7)                                 
/*                   


Error messages:

Code:

...
             COPY FROM(CF) USING(CTL2)                                             00060907
 ICE606I 0 DFSORT CALL 0003 FOR COPY FROM CF       TO OUTFIL   USING CTL2CNTL TERMINATED
 ICE602I 0 OPERATION RETURN CODE:  16
...
1ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0003
 ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 04:12 ON THU MAY 31, 2007 -
0          *
             OUTFIL FNAMES=T1,BUILD=(5:1,3,36:X)                                   00047007
             OUTFIL FNAMES=CFOUT,BUILD=(1,3)                                       00048007
 ICE146I 0 END OF STATEMENTS FROM CTL2CNTL - PARAMETER LIST STATEMENTS FOLLOW
           DEBUG NOABEND,ESTAE
           OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL2,SORTIN=CF,DYN*
                          ALLOC
           SORT FIELDS=COPY
 ICE201I E RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE222A 0 3 BYTE FIXED RECORD LENGTH IS NOT EQUAL TO 80 BYTE LRECL FOR CFOUT
 ICE751I 0 C5-K21008 C6-K90007 C7-K90000 C8-K90007 E9-K90007 E7-K11698
 ICE052I 3 END OF DFSORT
...
Back to top
View user's profile Send private message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1436
Location: Bangalore,India
PostPosted: Thu May 31, 2007 2:45 pm
 Reply with quote
Change the statement OUTFIL FNAMES=CFOUT,BUILD=(1,3) in the CTL2CNTL to

Code:
OUTFIL FNAMES=CFOUT,BUILD=(1,3,80:X)
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Thu May 31, 2007 4:20 pm
 Reply with quote
After making the above change the error message is as below:

SELECT FROM(CON1) TO(T4) ON(5,3,CH) NODUPS USING(CTL4)
ICE606I 0 DFSORT CALL 0006 FOR SORT FROM CON1 TO T4 USING CTL4CNTL TERMINATED
ICE602I 0 OPERATION RETURN CODE: 16

Thanks.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Thu May 31, 2007 8:53 pm
 Reply with quote
sanjee27,

You never told me the RECFM and LRECL of your control file and input file, so I assumed they were FB/3 and FB/7 which I documented with the following in my job:

//CF DD DSN=... control file (FB/3)
//IN DD DSN=... input file (FB/7)

I also showed:

//CFOUT DD DSN=... modified control file (FB/3)

but you used:

//CFOUT DD DSN=T3012AU.BCGCDUPS.CNTLFILE(+1),
// DISP=(NEW,CATLG,DELETE),
// DCB=(RECFM=FB,LRECL=80),
// SPACE=(CYL,(5,5)),
// UNIT=SYSDA

The LRECL=80 is what caused the ICE222A error.

CTL2 is NOT missing. CTL2CNTL is present and that's what's needed.

And now you're changing the requirements from what you told me originally.

I don't have time to hit a moving target like this or to evaluate your changes.

You need to look in DFSMSG to see the error message associated with identifier 0006.

But I don't know if fixing that problem will give you what you want since I don't understand your latest changes to the original requirement.
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Fri Jun 01, 2007 9:13 am
 Reply with quote
Hi Frank,

I really apologize for the change in the requirement. The problem is even i got the proper sample input file yesterday only, before that just i know the logic that needs to be implemented.

Now the changes in the requirement are only the record lengths.

The record length of input, error and output file is 900 which was 7 before.
Similarly the record length of control file is 24 which was 3 before. I just said i want to keep the length of control file as 80 so that it will be having data only in 24 bytes and the rest will be fillers.

Also before, the position 5,6,7 of the input file was compared with position 1,2,3 of the control file. Now fields in position 1) 58-59 2) 60-61 3) 62-81 of the input file will be compared with position 1)1-2 2) 3-4 3) 5-24 of the control file.

The change here is the position of the fields which needs to be compared.

Rest all the logic are same.

Thanks.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Fri Jun 01, 2007 10:26 pm
 Reply with quote
Here's a revised DFSORT/ICETOOL job for your changed requirement. DO NOT add any DCB parameters - DFSORT will set the correct LRECLs automatically.

Code:

//S1 EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//CF        DD DSN=T3012AU.BCGCDUPS.CNTLFILE(+0),
//             DISP=SHR     (FB/24)
//IN        DD DSN=T3012AU.BCGCDUPS.INFILE,
//             DISP=SHR     (FB/900)
//CTL2CNTL DD DSN=&&C2,DISP=(,PASS),SPACE=(TRK,(1,1)),UNIT=SYSDA
//   DD *
  OUTFIL FNAMES=T1,BUILD=(58:1,24,918:X)
  OUTFIL FNAMES=CFOUT,BUILD=(1,24,80:X)
//T0 DD DSN=&&T0,UNIT=SYSDA,SPACE=(TRK,(2,2)),DISP=(,PASS)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T2 DD DSN=&&T2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T3 DD DSN=&&T3,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON1 DD DSN=*.T3,VOL=REF=*.T3,DISP=(OLD,PASS)
//     DD DSN=*.T1,VOL=REF=*.T1,DISP=(OLD,PASS)
//T4 DD DSN=&&T4,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T5 DD DSN=&&T5,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON2 DD DSN=*.T4,VOL=REF=*.T4,DISP=(OLD,PASS)
//     DD DSN=*.T5,VOL=REF=*.T5,DISP=(OLD,PASS)
//OUT1      DD DSN=T3012AU.BCGCDUPS.OUTFILE,
//             DISP=(NEW,CATLG,DELETE),
//             SPACE=(CYL,(5,5)),
//             UNIT=SYSDA           (FB/900)
//OUT2      DD DSN=T3012AU.BCGCDUPS.ERRFILE,
//             DISP=(NEW,CATLG,DELETE),
//             SPACE=(CYL,(5,5)),
//             UNIT=SYSDA           (FB/900)
//CFOUT     DD DSN=T3012AU.BCGCDUPS.CNTLFILE(+1),
//             DISP=(NEW,CATLG,DELETE),
//             SPACE=(CYL,(5,5)),
//             UNIT=SYSDA           (FB/80)
//TOOLIN   DD *
  COPY FROM(CF) USING(CTL0)
  COPY FROM(T0) TO(CTL2CNTL) USING(CTL1)
  COPY FROM(CF) USING(CTL2)
  COPY FROM(IN) USING(CTL3)
  SELECT FROM(T2) TO(T3) ON(901,8,ZD) FIRSTDUP
  SELECT FROM(CON1) TO(T4) ON(58,24,CH) NODUPS USING(CTL4)
  SPLICE FROM(CON2) TO(OUT1) ON(901,8,ZD) KEEPNODUPS KEEPBASE -
    WITHALL WITH(1,900) WITH(917,1) USING(CTL5)
/*
//CTL0CNTL DD *
  OUTFIL FNAMES=T0,NODETAIL,REMOVECC,OUTREC=(8X),
         TRAILER1=(COUNT-20=(TO=ZD,LENGTH=8))
/*
//CTL1CNTL DD *
  INREC IFOUTLEN=80,
    IFTHEN=(WHEN=(1,8,ZD,GT,+0),
          BUILD=(C' OPTION SKIPREC=',1,8)),
    IFTHEN=(WHEN=NONE,
          BUILD=(C'*'))
/*
//CTL3CNTL DD *
  INREC IFOUTLEN=918,
        IFTHEN=(WHEN=INIT,OVERLAY=(901:SEQNUM,8,ZD)),
        IFTHEN=(WHEN=(1,4,CH,EQ,C'TT01'),
                OVERLAY=(901:SEQNUM,8,ZD)),
        IFTHEN=(WHEN=NONE,
                OVERLAY=(909:SEQNUM,8,ZD,
                         901:901,8,ZD,SUB,909,8,ZD,M11,LENGTH=8))
  OUTFIL FNAMES=T2,INCLUDE=(1,4,SS,EQ,C'TT01,RT01')
  OUTFIL FNAMES=T5,OVERLAY=(917:C'VV')
/*
//CTL4CNTL DD *
  OUTFIL FNAMES=T4,INCLUDE=(1,4,CH,EQ,C'TT01'),
    OVERLAY=(917:C'BB')
  OUTFIL FNAMES=CFOUT,INCLUDE=(1,4,CH,EQ,C'TT01'),
    BUILD=(58,24,80:X)
/*
//CTL5CNTL DD *
  OUTFIL FNAMES=OUT1,INCLUDE=(917,2,CH,EQ,C'VB'),
    BUILD=(1,900)
  OUTFIL FNAMES=OUT2,INCLUDE=(917,2,CH,EQ,C'VV'),
    BUILD=(1,900)
/*
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Mon Jun 04, 2007 11:36 am
 Reply with quote
Hi Frank,

Thanks a lot for the revised job.

I ran the job but i could not get the desired output file and error file.
I tested with the following input file,

TH0101.00999395OA ADF B200607 9999
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TT0101.00999395OA ADF B200607 9999
TH0101.00999395OB ADF B200607 8765
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TH0101.00999395OC ADF B200607 1323
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TT0101.00999395OC ADF B200607 1323

1) The control file was having the data

1323

Hence the first header record position 58,24 will be compared with the control file, hence the first group should go to the output file since a match is not found.

The second header also does not have a match in the control file but still there is no trailer record, so it will go to the error file.

The third group should go to the error file since a match is found with the control file.

I got the output as:

OUTFILE:

TT0101.00999395OA ADF B200607 9999
TH0101.00999395OB ADF B200607 8765
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TH0101.00999395OC ADF B200607 1323
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001

And the error file as:

TH0101.00999395OA ADF B200607 9999
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TT0101.00999395OC ADF B200607 1323


2) But the required output as per the requirement should be:

OUTFILE:

TH0101.00999395OA ADF B200607 9999
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TT0101.00999395OA ADF B200607 9999

ERRORFILE:

TH0101.00999395OB ADF B200607 8765
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TH0101.00999395OC ADF B200607 1323
BH01998800
RH01998800 T2006-06-01 00:00:002007-12-31 23:59:59
TD019993950000000000100000000008824181562306242401088241815623062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
TD019993950000000000000000000008824181562406242401088241815624062Merchan
RT01998800 T2006-06-01 00:00:002007-12-31 23:59:59
BT0199880000000000000000000001
TT0101.00999395OC ADF B200607 1323

3) Also, since i want to keep the latest 20 good headers so that each header record will be compared with all the records in the control file,i changed the disp parameter of control file to MOD.

Now the control file looks like:

1323
9999

So next time each header of the input file will be compared with the above two records in the control file.

Please suggest me how to fix the problems of output and error file.

Thanks.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Mon Jun 04, 2007 11:15 pm
 Reply with quote
In your first post you said
Quote:
where TT01 is the header record and RT01 is the trailer record
and all of your previous examples followed that.

Now you are showing data with TH01 and TT01 and you wonder why the solution I gave you doesn't work. Well, it was set up for TT01 headers - you never mentioned TH01 headers before. At this point, I don't know what you want any more.
Back to top
View user's profile Send private message
sangee27

New User


Joined: 16 Mar 2007
Posts: 19
Location: Pune
PostPosted: Tue Jun 05, 2007 9:19 am
 Reply with quote
Hi Frank,

Sorry for the confusion. I have mentioned TT01 and RT01 as headers and trailers in my initial post. After that in the sample file i got the headers and trailers as TH01 and TT01 respectively. I should have commmunicated this to you.

Now i have replaced TT01 with 'TH01' and RT01 with 'TT01' in your revised job and now it is working perfectly fine as per the requirement.

Thanks a million again.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Tue Jun 05, 2007 8:10 pm
 Reply with quote
Good. I'm glad the job did what you wanted.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 

Similar Topics
Topic Forum Replies
No new posts Write line by line from two files DFSORT/ICETOOL 7
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts Merge two VSAM KSDS files into third ... JCL & VSAM 6
No new posts Joinkeys - 5 output files DFSORT/ICETOOL 7
No new posts How to append a PS file into multiple... JCL & VSAM 3
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us