IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

To perform DEDUPE and JOINKEY operation in one step


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Megha Saravagi

New User


Joined: 28 Dec 2011
Posts: 6
Location: India

PostPosted: Sat Jun 09, 2012 4:39 pm
Reply with quote

Hi friends,

I have an Input file which is being processed in two steps using DFSORT.

1) Input file is being deduped using SUM FIELDS=NONE,XSUM ... which creates two files - Unique File, and Duplicate File.

2) In the 2nd step, main Input file is matched with the Duplicate file (created in above step) using JOINKEY feature of DFSORT.


By any method, is it possible to get the whole task done in single step with use of SYNCSORT/ICETOOL ?


Thanks,
Megha
Back to top
View user's profile Send private message
Pandora-Box

Global Moderator


Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy

PostPosted: Sat Jun 09, 2012 5:21 pm
Reply with quote

1.Do you have DFSORT installed in your system for sure ? (if yes request you to paste the output messages of STEP1 and STEP2 of your job??

2.What is the purpose of the above step mentioned in quotes??

Megha Saravagi wrote:

1) Input file is being deduped using SUM FIELDS=NONE,XSUM ... which creates two files - Unique File, and Duplicate File.

2) In the 2nd step, main Input file is matched with the Duplicate file (created in above step) using JOINKEY feature of DFSORT.




3.Also I notice you run first step with SYNCSORT and step2 with DFSORT ?

(Looks very weird to me icon_confused.gif )

4.If my point 3 is true you then both products are installed in your shop

5. If you could explain clearly what you do in step 1 and step 2 and whats the output you need with some sample input and output ( within code tags )

6.Once you explain point 5 please mention the keys for which sorting / JOINKEYS / IDENTIFYING DUPLICATES needs to be done
Back to top
View user's profile Send private message
Megha Saravagi

New User


Joined: 28 Dec 2011
Posts: 6
Location: India

PostPosted: Sat Jun 09, 2012 6:02 pm
Reply with quote

Pandora-Box,

I am sorry. Both the steps use SYNCSORT only.

Here are more details :

Main Input file is FB with LRECL of 817. It has key field of 15 Charactor at column 744.

STEP 1 ) Dedupe on Key field to split main input file in Unique file and Duplicate file.

//STEP1 EXEC PGM=SORT
//SORTIN DD DSN=RACF.INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=RACF.UNIQUE.FILE,DISP=(NEW,CATLG,DELETE)
//SORTXSUM DD DSN=RACF.DUPLIC.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(744,15,CH,A),EQUALS
SUM FIELDS=NONE,XSUM
//*


STEP 2 ) Match main input file and Duplicate file on key field using JOINKEYS. For matched record, 'L' is written in output at column 759. For unmatched record, 'S' is written in output at column 759.

//STEP2 EXEC PGM=SORT
//SORTJNF1 DD DSN=RACF.DUPLIC.FILE,DISP=SHR
//SORTJNF2 DD DSN=RACF.INPUT.FILE,DISP=SHR
//SRTFIL1 DD DSN=RACF.OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
JOINKEYS FILES=F1,FIELDS=(744,15,A)
JOINKEYS FILES=F2,FIELDS=(744,15,A)
JOIN UNPAIRED,F2
REFORMAT FIELDS=(F2:1,817,F1:744,15)
SORT FIELDS=(2,30,BI,A,32,786,BI,A)
SUM FIELDS=NONE

OUTREC IFTHEN=(WHEN=(744,15,CH,NE,818,15,CH),
OVERLAY=(759:C'S')),
IFTHEN=(WHEN=(744,15,CH,EQ,818,15,CH),
OVERLAY=(759:C'L'))

OUTFIL FNAMES=SRTFIL1,
OUTREC=(1:1,817)
//*



Is it a way I can get whole operation done using single sort step ?


Thanks,
Megha
Back to top
View user's profile Send private message
Pandora-Box

Global Moderator


Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy

PostPosted: Sat Jun 09, 2012 6:18 pm
Reply with quote

Hi Megha,

Please be aware SYNCSORT questions need to be posted in JCL forum

Also You are yet to explain the purpose of eliminating duplicates and joining with the duplicates again??

Can you please show us some sample data of step one and step 2?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Sun Jun 10, 2012 3:34 am
Reply with quote

You are doing four sorts. Two are unnecessary (you should specify SORTED on the JOINKEYS).

Does your Syncsort version support JNFnCNTL files? Talk to Syncsort support.

Without that, definitely, since you are doing it that way, with two sorts, you'll need two steps.
Back to top
View user's profile Send private message
Megha Saravagi

New User


Joined: 28 Dec 2011
Posts: 6
Location: India

PostPosted: Tue Jun 12, 2012 2:08 am
Reply with quote

Pandora Box,

Objective here is to identify unique/duplicate record in the input file on a certain key field. I need to mark unique record as 'S' and duplicate record as 'L'.

Could you suggest something different from the above mentioned method, which would be more cost effective in terms of CPU Time?


Thanks,
Megha
Back to top
View user's profile Send private message
Megha Saravagi

New User


Joined: 28 Dec 2011
Posts: 6
Location: India

PostPosted: Tue Jun 12, 2012 2:19 am
Reply with quote

Bill,

I have never used JNFnCNTL. I was trying to get some info on that and found this one below-

http://ibmmainframes.com/archive/o_t__t_58064__removing-duplicates-without-sorting-in-jcl.html

Referring to your last post in this URL location, is it the same you want to suggest here as well?


Thanks,
Megha
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Tue Jun 12, 2012 3:33 am
Reply with quote

To use the JNFnCNTL you'll have to check your version of Syncsort, and talk to Syncsort support if you don't currently have access.

If you can fully describe your processing, including the original order and the new order of the file, there may be something but hard to tell without information.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts Return codes-Normal & Abnormal te... JCL & VSAM 7
No new posts How to append a PS file into multiple... JCL & VSAM 3
No new posts Use of Perform Thru Exit COBOL Programming 6
No new posts convert file from VB to FB and use tr... DFSORT/ICETOOL 8
No new posts Forcing a step to run (even if abended) JCL & VSAM 8
Search our Forums:

Back to Top