View previous topic :: View next topic
|
Author |
Message |
anand1204
New User
Joined: 16 Jan 2015 Posts: 12 Location: USA
|
|
|
|
Hi,
I'm looking for a sort control statements (not JCL), to remove duplicates from PS data set (not file) which has an LRECL of [is] 200 bytes and my key is the entire LRECL [file]. But my record order shouldn't change, also the duplicates check should be for particular date only (20180825)
I/P data set (not file)
--------------
Code: |
aaa 20180825 dddd.......ccccc
ccc 20180826 aaaa...... ddddd
aaa 20180825 dddd.......ccccc
ggg 20180227 hhhh.......hhhhh
ccc 20180826 aaaa...... aaaaa
aaa 20180825 dddd.......ccccc
ccc 20180826 aaaa...... aaaaa |
O/p data set (not file)
----------------
Code: |
aaa 20180825 dddd.......ccccc
ccc 20180826 aaaa...... ddddd
ggg 20180227 hhhh.......hhhhh
ccc 20180826 aaaa...... aaaaa
ccc 20180826 aaaa...... aaaaa |
Thanks in advance.
Amended to show correct terminology and use code tags. Stuff in [] is to be ignored. |
|
Back to top |
|
|
dneufarth
Active User
Joined: 27 Apr 2005 Posts: 420 Location: Inside the SPEW (Southwest Ohio, USA)
|
|
|
|
Key is entire record?
Been a few years for me, but read about adding SEQNUM at end of record as well as SUM FIELDS=NONE when sorting
Probably a two step job with the second step putting records back in order of original file and truncating SEQNUM field. (or one step in ICETOOL) |
|
Back to top |
|
|
anand1204
New User
Joined: 16 Jan 2015 Posts: 12 Location: USA
|
|
|
|
dneufarth wrote: |
Key is entire record?
Been a few years for me, but read about adding SEQNUM at end of record as well as SUM FIELDS=NONE when sorting
Probably a two step job with the second step putting records back in order of original file and truncating SEQNUM field. (or one step in ICETOOL) |
hi,
Thanks for the reply.
Do you have a sample JCL to accomplish this |
|
Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
|
|
|
|
Quote: |
duplicates check should be for particular date only (20180825) |
where do you get that date from? |
|
Back to top |
|
|
anand1204
New User
Joined: 16 Jan 2015 Posts: 12 Location: USA
|
|
|
|
Arun Raj wrote: |
Quote: |
duplicates check should be for particular date only (20180825) |
where do you get that date from? |
Hi Arun,
Yes, i'm trying check duplicates only one particular date (20180825) and also i dont need the order sequence to change.
I have also gone thru the previous post from this link below. the last post (solution does change the record sequence)
ibmmainframes.com/about32365.html |
|
Back to top |
|
|
dneufarth
Active User
Joined: 27 Apr 2005 Posts: 420 Location: Inside the SPEW (Southwest Ohio, USA)
|
|
|
|
Don't have JCL example. Most responders here prefer you try something after searching for similar requests/solutions. If there is an issue, then post what you tried and system messages.
Arun and others usually have tremendous knowledge and some tricks up their sleeves (time permitting). |
|
Back to top |
|
|
Rohit Umarjikar
Global Moderator
Joined: 21 Sep 2010 Posts: 3053 Location: NYC,USA
|
|
Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
|
|
|
|
Quote: |
Yes, i'm trying check duplicates only one particular date (20180825) and also i dont need the order sequence to change. |
anand1204,
Where do you get this 'one particular date' from?, I am just trying to make sure I fully understand what you are trying to do |
|
Back to top |
|
|
sergeyken
Senior Member
Joined: 29 Apr 2008 Posts: 2025 Location: USA
|
|
|
|
anand1204 wrote: |
i'm trying check duplicates only one particular date (20180825) and also i dont need the order sequence to change.
I have also gone thru the previous post from this link below. the last post (solution does change the record sequence)
ibmmainframes.com/about32365.html |
Your comments demonstrated that you do not try to understand the recommended algorithm, but just tried to copy-paste the given example as is.
In order to keep your output records in their original order you need to add one extra step at the end (by yourself!), to re-sort them by SEQNUM field created in that example, and probably remove this field at the very end.
I intentionally do not give the ready to copy-paste solutions to those who doesn't try using his own mind to do his own job. |
|
Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
|
|
|
|
anand1204,
Also in your sample data all the duplicate records are identical for the input date - 20180825. Will it be the case always?, or do you expect different data on these duplicate records?. |
|
Back to top |
|
|
Rohit Umarjikar
Global Moderator
Joined: 21 Sep 2010 Posts: 3053 Location: NYC,USA
|
|
|
|
Arun Raj, OP is already loaded with many questions by us to try it by its own and may refer the link provided. Let us wait for it than spending anymore time with unclear requirements. |
|
Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
|
|
|
|
Quote: |
OP is already loaded with many questions by us to try it by its own |
I don't see too many questions asked here, except the one from Dave. |
|
Back to top |
|
|
Nic Clouston
Global Moderator
Joined: 10 May 2007 Posts: 2455 Location: Hampshire, UK
|
|
|
|
Quote: |
I don't see too many questions asked here, except the one from Dave. |
Plus 3 from you, Arun, one of which should not have been asked as the information (is it only that date?) was provided in the original post. Of the other two I do not see the relevance to providing a solution but I do see the relevance of multiple identical records. Can there be 3 duplicates of which 2 are valid and should be retained? Probably not but OP should be made aware of the ramifications.
As I see it, someone mucked up the run that created records for that date - did not back out a failure or something. But that is irrelevant to the solution of getting the data fixed up THIS TIME. The solution to fixing up the data is as dneufarth posted right at the start: add a sequence number as the data is read in - at the end of each record (bytes201-2xx depending), doing a SORT FIELDS=NONE and saving the data. Then sort again - on the added sequence number, doing a build to remove the sequence number from the output data. Examples of all this can be found in the forum and in the manuals.
Note: RECFM should have been provided.
Perhaps the soution could be made to accept the date as a parameter so that the JCL can be kept and reused if a similar event happens again. But, more importantly, the whole application should be designed so that this cannot happen - no appending to a data set but create new records to a temp data set and concatenate after sucessful execution.
Topic has unlocked itself during my cleanup. I have also edited the original post. |
|
Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
|
|
|
|
Thanks Nic. You're right. I have corrected it now.
OP mentioned in a PM while the topic was locked that he needs only one record out of the duplicate-date records. If that is the case then probably we don't have to sort the records or re-sort it.
It could be just assigning a sequence number for records with date = '20180825' (using an IFTHEN..WHEN...OVERLAY) and then omitting records having sequence number > 1 in the output (OUTFIL OMIT....). |
|
Back to top |
|
|
anand1204
New User
Joined: 16 Jan 2015 Posts: 12 Location: USA
|
|
|
|
Arun Raj wrote: |
Thanks Nic. You're right. I have corrected it now.
OP mentioned in a PM while the topic was locked that he needs only one record out of the duplicate-date records. If that is the case then probably we don't have to sort the records or re-sort it.
It could be just assigning a sequence number for records with date = '20180825' (using an IFTHEN..WHEN...OVERLAY) and then omitting records having sequence number > 1 in the output (OUTFIL OMIT....). |
Hi Arun,
Thanks for your response & the logic for this scenario. |
|
Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
|
|
|
|
Anand.
Thanks for posting. Glad it worked for you. |
|
Back to top |
|
|
|