I have 2 input files
INPUTFILE 1 (PS,FB 80-80) looks like this:
(note: dataset names start in COL1 and can be as long as 44 chars)
BAD.DATASET.NAME1
BAD.DATASET.NAME4
BAD.DATASET.NAME11
BAD.DATASET.NAME305
INPUTFILE 2 (Tape file) looks like this:
(note: dataset names start in COL1 and can be as long as 44 chars)
GOOD.DATASET.NAME2 .......greek......... .... ....
BAD.DATASET.NAME1 ...gibberish........... .... ....
BAD.DATASET.NAME305.. ................ anyting. ................ ...
BAD.DATASET.NAME4 . .. ....... ........... .......... ......
GOOD.DATASET.NAME1000 .... ........sa$@#..........
BAD.DATASET.NAME11 ... ........ ........ ......... . whatever ..
DESIRED PROCESSING: Copy all records from inputfile2, omitting records that have matching dataset names from inputfile1
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
You would use SELECT or SPLICE for this, not OMIT. But I need more information before I can give you the best job for what you want to do.
What is the RECFM and LRECL of input file2?
In your example, there is a match in input file2 for every dsname in input file1. Is that always the case, or can input file1 have a dsname that isn't in input file2 (e.g. BAD.DATASET.NAME12 is in input file1 but not in input file2)?
Frank you're intuitive! That's correct. Records in inputfile1 will always have matching records in inputfile2. So in essence, the resultant output will be an exact copy of inputfile2, but filtering out records that have a match in inputfile1.
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Whether you could do this with OMIT would depend on the number of records in input file1. But SELECT would be a better choice since the number of records in input file1 wouldn't matter. Here's a DFSORT/ICETOOL job using SELECT. I assumed input file2 has RECFM=FB and LRECL=80 (I asked but you didn't say).
Sorry about that.
Input file1 = ps,fb,80-80
Input file2 = ps,vb,13544-32760 (tape file)
Out = ps,vb,13544-32760 (disk file)
Input file1 is usually small, will contain only a few records. Only 1-44 cols matters (DSNs). Inputfile1 records (DSNames) will be coded instream in the JCL (eg: //INPUTFILE DD * )
The COPY operator tells ICETOOL to copy the IN1 data set using the control statements in CTL1CNTL.
FNAMES=T1 gives the ddname of the output data set (T1).
FTOV converts from FB records to VB records.
If you're not familiar with DFSORT and DFSORT's ICETOOL, I'd suggest reading through "z/OS DFSORT: Getting Started". It's an excellent tutorial, with lots of examples, that will show you how to use DFSORT, DFSORT's ICETOOL and DFSORT Symbols. You can access it online, along with all of the other DFSORT books, from:
Ran ok CC=0. Only thing is the output records was sorted ascending. We dont desire any sorting. And more records were deleted than expected (There should be only 2 matches/deletions. I checked this manually before running). Thoughts?
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
I gave you a solution based on what you told/showed me.
You didn't say anything previously about the order of the output so I didn't code for that. If you don't want any sorting, then you'll need to add a sequence number to the original records, and use another pass to sort them back into order by that sequence number and remove the sequence number.
When you say that you got more records deleted than expected, do you mean with the sample input records you showed or with different input records? I would have to know what your input records looked like, what you ran and what you got for output before I could tell you what was different from what you originally asked for. For example, does input file2 have duplicate dsnames within it? Your example didn't show that so I didn't code for it.
If you want me to give you a solution that works for different data then you originally showed, then you need to show me a better example of your data with all its variations and what you want for output.
I can see the (your) confusion. I should have been more precise with my example and requirement.
- I'd like to avoid adding seq# if possible. Is there no other way (recode, products) ?
- There may be duplicate dsnames within file2. Those should remain untouched. Only delete records that have a match in file1.
I can see the (your) confusion. I should have been more precise with my example and requirement.
- I'd like to avoid adding seq# if possible. Is there no other way (recode, products) ?
- There may be duplicate dsnames within file2. Those should remain untouched. However, if the dups have a matching record in file1 then delete them as well.
Only delete ANY or ALL records that have a match in file1.
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Quote:
There may be duplicate dsnames within file2. Those should remain untouched. Only delete records that have a match in file1.
What if the duplicate dsnames within file2 have a match in file1? Wouldn't you want to delete them rather than having them remain untouched? That's why I asked you to show me a better example of your data with all its variations and what you want for output. That way, I don't have to guess what you want.
If you have a maximum of less than about 590 dsnames in input file1, then we could generate an OMIT statement and avoid sorting. Is that the case?
Otherwise, since you have duplicate dsnames in file2, we would need to use SPLICE, not SELECT. SPLICE, like SELECT, requires that you do a sort so you'd need the seqnum to get the records back in the original order.
It works! The stmt set is quite compact and tight, not to mention the BUILD stmt is quite neat.
As you can see there are 4 records in IN1, all 4 have matching dataset names in IN2. One of the records has dups(2) within IN2. Therefore, total deletes expected is 5.
OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL2,SORTIN=IN2,SORTOUT=OUT,DYNALLOC
SORT FIELDS=COPY
RECORD TYPE IS V - DATA STARTS IN POSITION 5
OUTPUT LRECL = 13544, BLKSIZE = 32760, TYPE = VB
INSERT 0, DELETE 5
RECORDS - IN: 3029017, OUT: 3029012
END OF DFSORT
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Good. I guess I should have asked you the maximum number of records in input file1 in the first place instead of assuming there were too many for an OMIT statement.
Frank, again thanks for the assistance. I think I know how this works but in your own words, could you please explain the workings of the stmts.
No rush. I know you're a busy guy constantly juggling and managing your time. btw, You can PM me the answer if you like.
The first COPY operator creates this OMIT statement in the CTL2CNTL data set. It uses an OUTFIL statement in CTL1CNTL to do it as follows:
HEADER1 - creates the first line of the OMIT statement.
BUILD - creates a line for each constant in file1.
TRAILER1 - creates the last line of the OMIT statement.
The second COPY operator uses the OMIT statement in CTL2CNTL against input file2.
Thank you. what about 1,1,NE,11 in the header and trailer?
How are records from IN1 written one by one into CTL2?
How are records in IN1 compared to records in CTL2, and matched records filtered out?
As you can tell Im not a heavy SORT user but Im having real interest in this, I should read up on it more. Quite powerful. I can see using it here more often..I'd have to look around our installation & applications that can benefit from this.
Again no rush. Reply whenever you find time.. thanks
It's easier to use a NOP for the first line and last line then to do the coding to put real constants into those lines. So I use 1,1,NE,1,1 for line1 and linen, and the real constants for the other lines. 1,1,NE,1,1 can never be true, so it has no effect on the records omitted - it's a NOP. Just a trick.
Quote:
How are records from IN1 written one by one into CTL2?
HEADER1 writes an output line before the detail lines.
BUILD writes one output line (detail line) for each record in IN1.
TRAILER1 writes an output line after the detail lines.
Quote:
How are records in IN1 compared to records in CTL2, and matched records filtered out?
I assume you mean IN2 here, not IN1. It's the records in IN2 we want to do the compare against - right? For the second COPY operator, USING(CTL2) tells DFSORT to use the DFSORT statements in CTL2CNTL against the input from IN2. CTL2CNTL has the OMIT statement, so the OMIT is evaluated for each IN2 record.
If you're not familiar with DFSORT and DFSORT's ICETOOL, I'd suggest reading through "z/OS DFSORT: Getting Started". It's an excellent tutorial, with lots of examples, that will show you how to use DFSORT, DFSORT's ICETOOL and DFSORT Symbols. You can access it online, along with all of the other DFSORT books, from: