Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
how do i compare and concatnate with dropping duplicate

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
neelima.sinha

New User


Joined: 14 Sep 2007
Posts: 30
Location: Bangalore

PostPosted: Wed Jul 30, 2008 3:22 pm    Post subject: how do i compare and concatnate with dropping duplicate
Reply with quote

Hi,

Is there a way to achieve the following using ICETOOL. i tried splice
but not giving completely desired result.

My input dataset:

Input 1:
Record format . . . : FB
Record length . . . : 80

Code:

03000 0 B 17.612.512.312.513.721.709.7
03000 0 B6 17.612.512.312.513.721.709.7
03000 0 B61 18.512.612.312.013.222.109.3
02329 0 B3 18.112.812.513.214.518.710.2
02329 0 B31 18.612.212.212.815.018.610.6


Input 2:

Record format . . . : FB
Record length . . . : 6
Code:

03000A
02339D
02324J


Out put should look like :
Code:

03000 0 B 17.612.512.312.513.721.709.7A
03000 0 B6 17.612.512.312.513.721.709.7A
03000 0 B61 18.512.612.312.013.222.109.3A
02329 0 B3 18.112.812.513.214.518.710.2D
02329 0 B31 18.612.212.212.815.018.610.6D


KEY LENGTH: FIRST 5 CHARACTERS

***************************************************

MY JCL
---------
Code:

//SLGTODAY EXEC PGM=ICETOOL
//IN1 DD DSN=YD65.PTG.AUG,DISP=SHR
//IN2 DD DSN=XR67.PTG.STREAM.AUG,DISP=SHR
//TEMP1 DD DSN=XR67.TEMP1,UNIT=DASD,DISP=SHR
//TEMP2 DD DSN=XR67.TEMP2,UNIT=DASD,DISP=SHR
//CONCAT DD DSN=*.TEMP1,VOL=REF=*.TEMP1,DISP=SHR
// DD DSN=*.TEMP2,VOL=REF=*.TEMP2,DISP=SHR
//COMBINE DD DSN=XR67.PTP.COMBINE,DISP=SHR
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//TOOLIN DD *
* REFORMAT THE BASE RECORD(IN1) FOR SPLICING
COPY FROM(IN1) TO(TEMP1) USING(SRT1)
* REFORMAT THE OVERLAY RECORD(IN2) FOR SPLICING
COPY FROM(IN2) TO(TEMP2) USING(SRT2)
* SPLICE THE REQUIRED DATA FROM IN1 AND IN2 TOGETHER
SPLICE FROM(CONCAT) TO(COMBINE) ON(1,05,CH) WITH(50,1)
/*
//SRT1CNTL DD *
  OUTREC FIELDS=(1,49, IN1 DATA
    55:X) ADD BLANKS FOR SPLICED IN2 DATA
/*
//SRT2CNTL DD *
  OUTREC FIELDS=(1:1,05, IN2 DATA
    50:06,1) IN2 DATA
/*

_________________
Back to top
View user's profile Send private message

expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Wed Jul 30, 2008 3:24 pm    Post subject:
Reply with quote

What result are you getting ?
Back to top
View user's profile Send private message
neelima.sinha

New User


Joined: 14 Sep 2007
Posts: 30
Location: Bangalore

PostPosted: Wed Jul 30, 2008 3:30 pm    Post subject:
Reply with quote

Apologies... I wanted to keep duplicates.....

in results duplicates are getting dropped ...

so getting only two records one for key 03000 and other with 02329

03000 0 B 17.612.512.312.513.721.709.7A
02329 0 B3 18.112.812.513.214.518.710.2D

Regards
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Wed Jul 30, 2008 3:37 pm    Post subject:
Reply with quote

Take a look at using the WITHALL parameter on your SPLICE card

I do have a working example somewhere and will try and find it at some stage today, workload permitting icon_eek.gif
Back to top
View user's profile Send private message
neelima.sinha

New User


Joined: 14 Sep 2007
Posts: 30
Location: Bangalore

PostPosted: Wed Jul 30, 2008 4:01 pm    Post subject:
Reply with quote

WITH ALL does not give the desired details
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Wed Jul 30, 2008 7:08 pm    Post subject:
Reply with quote

Warning: Do NOT use concatenation with referback as shown in this example. Use one MOD data set as shown in Kolusu's example. See my post below for an explanation.

Frank.



I've had a little play and this is what appears to give the correct results, or pretty near icon_biggrin.gif

As for your previous comment about WITHALL, try the code with and without that parameter and spot the difference.

A quick one that maybe Frank / Kolusu can help me to understand ....... I vaguely recalled a similar problem a long time back, and so I reversed the order of the two files concatenated as input to the splice, T2 before T1, and then the results started getting better. I can only assume that the order that the files are read is of some importance where the file without duplicate records was read in first.

Code:

//S1       EXEC PGM=ICETOOL                                     
//TOOLMSG  DD SYSOUT=*                                           
//DFSMSG   DD SYSOUT=*                                           
//IN1      DD *                                                 
03000 0 B   17.612.512.312.513.721.709.7                         
03000 0 B6  17.612.512.312.513.721.709.7                         
03000 0 B61 18.512.612.312.013.222.109.3                         
02329 0 B3  18.112.812.513.214.518.710.2                         
02329 0 B31 18.612.212.212.815.018.610.6                         
//IN2      DD *                                                 
03000A                                                           
02339D                                                           
02324J                                                           
//T1       DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS),
//            RECFM=FB,LRECL=55                                 
//T2       DD DSN=&&T2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS),
//            RECFM=FB,LRECL=55                                 
//CONCAT   DD DSN=*.T2,VOL=REF=*.T2,DISP=(OLD,PASS)             
//         DD DSN=*.T1,VOL=REF=*.T1,DISP=(OLD,PASS)             
//T3       DD DSN=&&T3,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS),
//            RECFM=FB,LRECL=55                                 
//OUT      DD SYSOUT=*                                     
//TOOLIN   DD *                                           
*                                                         
  COPY FROM(IN2) TO(T2) USING(SRT2)                       
*                                                         
  COPY FROM(IN1) TO(T1) USING(SRT1)                       
*                                                         
  SPLICE FROM(CONCAT) TO(T3) ON(1,05,CH) WITHALL WITH(6,40)
*                                                         
  COPY FROM(T3)  TO(OUT) USING(SRT4)                       
/*                                                         
//SRT1CNTL DD *                                           
  OUTREC FIELDS=(1,49,55:X)                               
/*                                                         
//SRT2CNTL DD *                                           
  OUTREC FIELDS=(1:1,49,50:06,1)                           
/*                                                         
//SRT4CNTL DD *                                           
  INCLUDE COND=(50,1,CH,NE,C' ')                           

Output is ......... what I would expect to receive from the data shown in your original post.
Code:

03000 0 B   17.612.512.312.513.721.709.7         A
03000 0 B6  17.612.512.312.513.721.709.7         A
03000 0 B61 18.512.612.312.013.222.109.3         A
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Jul 30, 2008 10:13 pm    Post subject:
Reply with quote

neelima.sinha,

The following DFSORT/ICETOOL Job will give you the desired results. The basic idea is to tag the key from unique key file at the end of every record in the duplicate file and compare the contents at the end with the key

Code:

//STEP0100 EXEC PGM=ICETOOL                                         
//TOOLMSG  DD SYSOUT=*                                             
//DFSMSG   DD SYSOUT=*                                             
//IN1      DD *                                                     
03000 0 B 17.612.512.312.513.721.709.7                             
03000 0 B6 17.612.512.312.513.721.709.7                             
03000 0 B61 18.512.612.312.013.222.109.3                           
02329 0 B3 18.112.812.513.214.518.710.2                             
02329 0 B31 18.612.212.212.815.018.610.6                           
//IN2      DD *                                                     
03000A                                                             
02339D                                                             
02324J                                                             
//T1       DD DSN=&&T1,DISP=(MOD,PASS),SPACE=(CYL,(1,1),RLSE)       
//OUT      DD SYSOUT=*                                             
//TOOLIN   DD *                                                     
  COPY FROM(IN2) USING(CTL1)                                       
  COPY FROM(IN1) USING(CTL2)                                       
  SPLICE FROM(T1) TO(OUT) ON(1,5,CH) WITHALL WITH(01,80) USING(CTL3)
//CTL1CNTL DD *                                                     
  OUTFIL FNAMES=T1,OVERLAY=(81:1,5)                                 
//CTL2CNTL DD *                                                     
  OUTFIL FNAMES=T1,OVERLAY=(81:5X)                                 
//CTL3CNTL DD *                                                     
  OUTFIL FNAMES=OUT,BUILD=(01,80),                                 
  INCLUDE=(1,5,CH,EQ,81,5,CH)                                       
/*
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Jul 30, 2008 11:05 pm    Post subject:
Reply with quote

Quote:
A quick one that maybe Frank / Kolusu can help me to understand ....... I vaguely recalled a similar problem a long time back, and so I reversed the order of the two files concatenated as input to the splice, T2 before T1, and then the results started getting better. I can only assume that the order that the files are read is of some importance where the file without duplicate records was read in first.


Expat,

Please, please don't show examples using concatenation with referback! As I've said many times in this Forum, there's a system restriction that can result in loss of data when you do that. Use one MOD data set as in Kolusu's example above, rather than concatenated data sets with referback as in the example you show. All of the examples in the current DFSORT books and all of the examples Kolusu and I show avoid concatenation with referback.

To answer your question, yes the order of the records is very important. The first record is the base record. The second record and subsequent records are the overlay records. Which record you use as the base record makes a huge difference. If you don't understand why that is, please go back and read the SPLICE documentation paying attention to the descriptions of the base and overlay records.
Back to top
View user's profile Send private message
neelima.sinha

New User


Joined: 14 Sep 2007
Posts: 30
Location: Bangalore

PostPosted: Thu Jul 31, 2008 1:53 pm    Post subject:
Reply with quote

Hi,

the solution offered by expat Global Moderator solves problem to some extent but I am getting output only records with key 03000, but 02329 is getting lost.... I need all of them.

Please help

Regards,
Back to top
View user's profile Send private message
karthikr44

Active User


Joined: 25 Aug 2007
Posts: 235
Location: Chennai

PostPosted: Thu Jul 31, 2008 3:56 pm    Post subject: Reply to: how do i compare and concatnate with dropping dupl
Reply with quote

Hi,

Change

Code:

//IN2      DD *                                                     
03000A                                                             
02339D                                                             
02324J


to

Code:

//IN2      DD *                                                     
03000A                                                             
02329D                                                             
02324J


Regards
R KARTHIK
Back to top
View user's profile Send private message
karthikr44

Active User


Joined: 25 Aug 2007
Posts: 235
Location: Chennai

PostPosted: Thu Jul 31, 2008 4:46 pm    Post subject: Reply to: how do i compare and concatnate with dropping dupl
Reply with quote

Hi,

I have one small doubt. Does the WITH parameter affects any processing in selection of records. Because i tried the SKOLUSU code with changing
the WITH parameter from WITH(01,80) to WITH(02,80). Now i am getting empty output file.

Still now i beleived that only ON fields are used to determine if records match. But WITH field affects the selection process.

Please clarify my doubt...

Regards
R KARTHIK
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Jul 31, 2008 9:12 pm    Post subject: Reply to: how do i compare and concatnate with dropping dupl
Reply with quote

karthikr44 wrote:
I have one small doubt. Does the WITH parameter affects any processing in selection of records. Because i tried the SKOLUSU code with changing
the WITH parameter from WITH(01,80) to WITH(02,80). Now i am getting empty output file.


karthikr44,

yes it does. You have changed the position of the field on the WITH but kept the length to spliced as 80. This pushes the key from file2 to be in 82 position instead of 81st position. The Include condition on the output file is checking if the first 5 bytes match the key at 81st position. Since you changed the position by 1 byte, the 81st byte is now a space and hence none of the records match.
Back to top
View user's profile Send private message
karthikr44

Active User


Joined: 25 Aug 2007
Posts: 235
Location: Chennai

PostPosted: Fri Aug 01, 2008 7:11 pm    Post subject: Reply to: how do i compare and concatnate with dropping dupl
Reply with quote

Hi Skolusu,

Thanks for clarifying the doubt. Now i got the reason for empty o/p file.

Regards
R KARTHIK
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Compare and COPY using DFSORT pshongal DFSORT/ICETOOL 3 Fri Nov 17, 2017 9:49 am
No new posts File Aid tool to compare numeric data balaji81_k Compuware & Other Tools 2 Tue Sep 26, 2017 3:35 am
No new posts Compare yesterday's date to the one o... migusd SYNCSORT 11 Fri Sep 22, 2017 11:35 pm
No new posts compare 2 rows within the same file ram_vizag SYNCSORT 7 Wed Jun 14, 2017 12:34 am
No new posts member list compare jzhardy TSO/ISPF 6 Fri May 12, 2017 3:18 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us