Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Remove selective duplicates using SORT

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
senthils

Active User


Joined: 15 Nov 2005
Posts: 117
Location: Chennai, India

PostPosted: Fri Aug 29, 2008 3:24 pm    Post subject: Remove selective duplicates using SORT
Reply with quote

Hi,

My input records looks like this.

file1:
Code:
header1
detail
detail
trailer1


file2:
Code:
header1
detail1
detail1
trailer1


If I Sort these files into one output file i am getting the following output.

Output:
Code:
header1
header1
detail1
detail1
detail1
detail1
trailer1
trailer1


But I want my puput to look like this.
desired output:
Code:
header1
detail1
detail1
detail1
detail1
trailer1


Can anyone please help me out on this?
Back to top
View user's profile Send private message

Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2239
Location: @my desk

PostPosted: Fri Aug 29, 2008 3:41 pm    Post subject:
Reply with quote

senthils,

You need to answer a few questions.

Is there any way to identify the header/trailer/data in the files. What are the input/output file attributes? Are the headers/trailers in both the files the same?

Thanks,
Arun
Back to top
View user's profile Send private message
senthils

Active User


Joined: 15 Nov 2005
Posts: 117
Location: Chennai, India

PostPosted: Fri Aug 29, 2008 3:54 pm    Post subject: Reply to: Remove selective duplicates using SORT
Reply with quote

1. Both the file attributes are same (FB).
2. Headers and trailers in both the files are same.
3. A particular field in the record identifies whether it is a header or trailer or detail record.

For example:
File1:
Code:
0000000H
1111111D
1111111D
9999999T


File2:
Code:
0000000H
1111111D
1111111D
9999999T


Current output:
Code:
0000000H
0000000H
1111111D
1111111D
1111111D
1111111D
9999999T
9999999T


Desired output:
Code:
0000000H
1111111D
1111111D
1111111D
1111111D
9999999T


Hope this answers your questions.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2239
Location: @my desk

PostPosted: Fri Aug 29, 2008 4:29 pm    Post subject:
Reply with quote

senthils,

I have assumed made the below assumptions.
1. 'H' or 'T' in pos-8 to identify the header/trailer record.
2. LRECL of the files=80

The below JCL would work for your requirement. You need to concatenate file-1 and file-2 in SORTIN.
Code:
//STEP1    EXEC PGM=SORT                                       
//SYSOUT   DD SYSOUT=*                                         
//SORTOUT  DD SYSOUT=*                                         
//SORTIN   DD *                                                 
0000000H......file-1                                                       
1111111D                                                       
1111111D                                                       
9999999T         
//  DD *                                             
0000000H......file-2                                                       
1111111D                                                       
1111111D                                                       
9999999T                                                       
//SYSIN    DD *                                                 
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1')),                   
        IFTHEN=(WHEN=(8,1,CH,EQ,C'H'),OVERLAY=(81:SEQNUM,1,ZD)),
        IFTHEN=(WHEN=(8,1,CH,EQ,C'T'),OVERLAY=(81:SEQNUM,1,ZD))
  SORT FIELDS=(1,80,CH,A)                                       
  OUTFIL OMIT=(81,1,ZD,NE,1),BUILD=(1,80)         

SORTOUT
Code:
0000000H
1111111D
1111111D
1111111D
1111111D
9999999T

Thanks,
Arun
Back to top
View user's profile Send private message
senthils

Active User


Joined: 15 Nov 2005
Posts: 117
Location: Chennai, India

PostPosted: Fri Aug 29, 2008 6:11 pm    Post subject: Reply to: Remove selective duplicates using SORT
Reply with quote

thanks Arun for the immediate reply. I will try this and let you know how it turned out. Thanks again.
Back to top
View user's profile Send private message
senthils

Active User


Joined: 15 Nov 2005
Posts: 117
Location: Chennai, India

PostPosted: Fri Aug 29, 2008 7:14 pm    Post subject: Reply to: Remove selective duplicates using SORT
Reply with quote

One extra scenario which came up while testing this.

Input:
File1:
Code:
0000000H
aaaaaaaD
aaaaaaaD
0000000T
1111111H
bbbbbbbD
bbbbbbbD
1111111T
2222222H
cccccccD
cccccccD
2222222T


File2:
Code:
0000000H
aaaaaaaD
aaaaaaaD
0000000T
1111111H
bbbbbbbD
bbbbbbbD
1111111T
2222222H
cccccccD
cccccccD
2222222T


Desired output:
Code:
0000000H
aaaaaaaD
aaaaaaaD
aaaaaaaD
aaaaaaaD
0000000T
1111111H
bbbbbbbD
bbbbbbbD
bbbbbbbD
bbbbbbbD
1111111T
2222222H
cccccccD
cccccccD
cccccccD
cccccccD
2222222T


As per the above SORT the obtained output was

Code:
0000000H
aaaaaaaD
aaaaaaaD
aaaaaaaD
aaaaaaaD
0000000T
bbbbbbbD
bbbbbbbD
bbbbbbbD
bbbbbbbD
cccccccD
cccccccD
cccccccD
cccccccD


So only for the first set the header and trailer was coming. Can you again help me out on this?

Thanks in advance,
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Fri Aug 29, 2008 9:24 pm    Post subject:
Reply with quote

Senthils,

The new input records you show would NOT give the output you say you got because the '0' records would collate after the 'c' records, so the 'H' and 'T' records would be at the bottom of the output file, not surrounding the 'a' records as you show. Also, since you now have groups of records and the records of a group do not have the same value in positions 1-7 (e.g. '0' in the header and trailer, but 'a' in the data), it is more complicated than your original problem.

The actual solution for your problem is very dependent on exactly what the input records look like, so I'd rather not guess at that.

Please show an example of the actual input records and what you expect for output. Indicate how header, data and trailer records are identified. Give the starting position, length and format of the key you want to use to "sort" the records or indicate the "rules" you want to use to combine groups in the output file. Give the LRECL of the input file.

If you are sorting these groups of records successfully with the multiple headers in the right place, please show the control statements you're using to do that.
Back to top
View user's profile Send private message
senthils

Active User


Joined: 15 Nov 2005
Posts: 117
Location: Chennai, India

PostPosted: Mon Sep 01, 2008 12:49 pm    Post subject:
Reply with quote

Actual input data:
File1:

Code:
000000000000000H                   
001100101AAA300H                   
001100101AAA300T                   
001100101AAA300D000TESTING000000051
001100101AAA300D000TESTING000000069
001100101AAA300D000TESTING000000077
001100101AAA300D000TESTING000000085
001100101AAA300D000TESTING000000119
001100101AAA300D000TESTING000000127
001100101AAA300D000TESTING000000135
001100101AAA300D000TESTING000000143
001100101AAA300D000TESTING000000259
001100101AAA300D000TESTING000000317
001100101AAA300D000TESTING000000399
001100101AAA100H                   
001100101AAA100T                   
001100101AAA100D000EXTE000000000029
001100101AAA100D000EXTE000000000045
999999999999999T


File2:
Code:
000000000000000H                   
001100101AAA300H                   
001100101AAA300T                   
001100101AAA300D000TESTING000000044
001100101AAA300D000TESTING000000150
001100101AAA300D000TESTING000000172
001100101AAA300D000TESTING000000176
001100101AAA300D000TESTING000000184
001100101AAA300D000TESTING000000242
001100101AAA300D000TESTING000000400
001100101AAA100H                   
001100101AAA100T                   
001100101AAA100D000EXTE000000000110
001100101AAA100D000EXTE000000000490
001100101AAA100D000EXTE000000000656
001100101AAA100D000EXTE000000000680
001100101AAA100D000EXTE000000000698
999999999999999T


Desired output:
Code:
000000000000000H                   
001100101AAA100H                   
001100101AAA100D000EXTE000000000029
001100101AAA100D000EXTE000000000045
001100101AAA100D000EXTE000000000110
001100101AAA100D000EXTE000000000490
001100101AAA100D000EXTE000000000656
001100101AAA100D000EXTE000000000680
001100101AAA100D000EXTE000000000698
001100101AAA100T                   
001100101AAA300H                   
001100101AAA300D000TESTING000000044
001100101AAA300D000TESTING000000051
001100101AAA300D000TESTING000000069
001100101AAA300D000TESTING000000077
001100101AAA300D000TESTING000000085
001100101AAA300D000TESTING000000119
001100101AAA300D000TESTING000000127
001100101AAA300D000TESTING000000135
001100101AAA300D000TESTING000000143
001100101AAA300D000TESTING000000150
001100101AAA300D000TESTING000000172
001100101AAA300D000TESTING000000176
001100101AAA300D000TESTING000000184
001100101AAA300D000TESTING000000242
001100101AAA300D000TESTING000000259
001100101AAA300D000TESTING000000317
001100101AAA300D000TESTING000000399
001100101AAA300D000TESTING000000400
001100101AAA300T                   
999999999999999T


The records are identified using 16th byte - If 'H' its header, 'T' trailer, 'D' detail record.

Sort key to be used:
SORT FIELDS=(0001,0016,CH,A,0017,0035,CH,A)

LRECL - 80 FB

Do let me know if you need more information.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Mon Sep 01, 2008 10:55 pm    Post subject:
Reply with quote

Here's a DFSORT job that will do what you asked for:

Code:

//S1    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN DD DSN=...  input file1 (FB/80)
//       DD DSN=...  input file2 (FB/80)
//SORTOUT DD DSN=...  output file (FB/80)
//SYSIN    DD    *
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1',SEQNUM,8,ZD)),
    IFTHEN=(WHEN=(16,1,CH,EQ,C'H'),OVERLAY=(81:9C'0')),
    IFTHEN=(WHEN=(16,1,CH,EQ,C'T'),OVERLAY=(81:9C'9'))
  SORT FIELDS=(1,15,CH,A,81,1,CH,A,17,35,CH,A,82,8,ZD,A)
  SUM FIELDS=NONE
  OUTREC BUILD=(1,80)
/*
Back to top
View user's profile Send private message
senthils

Active User


Joined: 15 Nov 2005
Posts: 117
Location: Chennai, India

PostPosted: Tue Sep 02, 2008 10:59 am    Post subject:
Reply with quote

Thanks a lot Frank. I got the output perfectly as required.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Innovations in Your Mainframe Sort Arun Raj PC Guides & IT News 4 Wed Mar 15, 2017 10:38 pm
No new posts SORT VSAM file with each field one by... maxsubrat DFSORT/ICETOOL 6 Tue Mar 14, 2017 1:07 pm
No new posts SORT requirement sjiraga COBOL Programming 6 Thu Mar 09, 2017 9:11 pm
No new posts SORT to append Month and YEAR in MMYY... tisamf DFSORT/ICETOOL 1 Wed Mar 08, 2017 4:46 pm
No new posts JCL sort Match and NoMatch rakeshsekar1987 SYNCSORT 2 Thu Feb 02, 2017 10:26 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us