If you need "just the unique records from one of the files" then you don't need the second dataset at all. Eliminate duplicates from the first dataset, and forget about the second one, that's it.
I guess you do not understand your task yet? Or cannot explain it to the public?
Your question as it is now doesn't make sense, at all.
Joined: 31 Oct 2006 Posts: 1042 Location: Richmond, Virginia
I recommend that when posting your code, first remove the commented and other extraneous lines so that others need not wade through them to see what you are doing.
Make it as easy as possible for folks to help you.
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
Jay - Is there anything within the data by which you can identify the source data set? If yes, concatenate both the original data sets and assign a sequence number ONLY for records from one of the data sets (the one you don't want to eliminate dups from). Make sure you have blanks (or some default value) in this field for records from the other data set. Then include the sequence number field in your SORT FIELDS and then eliminate duplicates.
Jay, Sergeyken is right. Understand the task first and then start working on it otherwise its a waste of time.
Arun Raj, Why even involved the other dataset at first place? Just get rid of the dups from one dataset and concatenate later with other dataset .
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
Rohit Umarjikar wrote:
Arun Raj, Why even involved the other dataset at first place? Just get rid of the dups from one dataset and concatenate later with other dataset .
Rohit Umarjikar,
Because from the OP's control statements above, he does sort the other data set as well on the key. Concatenation will not help if this is what the OP wants to do.
Because from the OP's control statements above, he does sort the other data set as well on the key. Concatenation will not help if this is what the OP wants to do.
Sort keys are still same and can be done independently, if I understand the OP's question.
1.Eliminate dups from Dataset 1
2.Concate Dataset from step1, and dataset 2 with SORT ( same key for both).
Since Sort key is same as record length, the whole record has to be a duplicate. If PQR2 needs to be removed then, it doesn't mean anything if it is removed from file1 or file2.
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
Rohit Umarjikar wrote:
Sort keys are still same and can be done independently
Rohit Umarjikar,
You can do it in a single pass with concatenated original input data sets and a single output data set, provided the given conditions satisfy. You don't really need to 'separate' the processing and do it 'independently' if that is the case.
