Hi - Long time lurker that is usually able to find the solution thru search, the manuals or trial and error... or a combination of each.
I want to create a single output record for each unique positional occurrence in an input record which has multiple unique positional occurrences. Not all occurrences may contain data.
My working solution using ICETOOL is to read the input file, and for each positional occurrence, write a record with that unique occurrence to a temp dsn, then pass back thru that temp data to drop out any that are swap spaces. I get four
My question is if this is possible to accomplish using just a single pass of the data instead of two passes? Obviously if that is more efficient, I'd rather explore that solution. I've tried a couple different methods to use one pass but failed miserably...
Input data example:
Code:
//*SAMPLE INPUT (ORIG LRECL=3250,FB, BUT FOR THIS SAY 80)
*XXX,NOTE-OLN TICKET-1 DATA TICKET-2 DATA TICKET-3 DATA TICKET-4
DATA PA111111 A1234567890 DATA
DATA PA222222 A2222222220 DATA B2222222221 DATA C2222222222 DATA D3333333333 DAT
DATA PA222222 E3232323230 DATA F2323232321 DATA G3223223222 DATA
DATA PA333333 M4444444440 DATA T4444444444 DATA
DATA PA999999 W5555555555 DATA X5353535353 DATA A5656565656 DATA N5234456673 DAT
//**
//** NOTE-OLN MAY SPAN ACROSS MULTIPLE RECORDS (IE. RECORDS 2 & 3 above), LEN=8
//** TICKET-# FIELDS ARE UNIQUE, ORIGINAL RECORD HAS UP TO 10 OCCURENCES, LEN=11
//** BUT FOR THE SAMPLE INPUT I DECIDED TO LIMIT TO 4 SINCE THAT
//** SHOULD STILL GET THE IDEA OF THE INPUT ACROSS...
I've tried using IFTHEN to build the unique records to get the output as shown above but I am only getting one record per input record and the output is just the first occurrence. Perhaps my understanding of the IFTHEN function is lacking.... Also, I've tried using the ',/,' in the builds of the IFTHEN but sort gets upset with that. I consider myself pretty capable at sort, but IFTHEN throws me for a loop when I try to use t for other than overlay or findrep functions. Maybe what I want it to do is something that IFTHEN isn't designed to do and I can't get it thru my thick skull. Or there is a different parameter that will accomplish this and I'm barking up the wrong tree...
ICE201I G RECORD TYPE IS F - DATA STARTS IN POSITION 1
However, we do have the most current DFSORT level on our other MVS LPAR's that systems use for upgrades/updates testing (we're z/OS 1.9 on our production MVS system while the other test LPARS are at 1.11, supposedly going to roll out to our current prod system later this year). Can't rush into these things
I did search for previous posts for clues to a similar question/solution, but every key word I tried returned pages and pages of posts that didn't seem to apply, so apologies if this has been discussed before and I missed the post(s)....
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
I guess there are two things.
Firstly, to get the output you want, how about considering RESIZE?
RESIZE to 20 bytes with a USING
In the USING, use INREC to arrange your four occurrences with BUILD as the only data on the record.
Use OUTFIL OMIT=( to get rid of the ones you don't want.
Secondly, the problems with what you have attempted. The Slash Operator (/) is only available on OUTFIL (in BUILD or its old cousin, who should be left creaking on the porch).
No matter how many IFTHENs you have, it is the resultant value of whatever BUILD was last done with the original sources being the input data. With the HIT=NEXT, you are BUILDing using data from the previous build and BUILDing "over" it.
Code:
Input
- BUILD
Build1 record to be treated as output, based on positions from Input.
- BUILD
Build2 record to be treated as output, based on positions from Build1, Build1 is then "gone".
- BUILD
Build3 record to be treated as output, based on positions from Build2, Build2 is then "gone".
- BUILD
Build4 record to be treated as output, based on positions from Build3, Build3 is then "gone".
All IFTHEN's finished. Build4 is output, which has data from Input, via Build1, via Build2, via Build3, so you get the "first occurrence".
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
It would be good to see the SYMNOUT from the step.
Thinking further, I'm not sure how you are getting what you are from the IFTHENs because the fields you are sourcing the BUILD from are not within the "range" of what is by then the current record...
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
John Del wrote:
My question is if this is possible to accomplish using just a single pass of the data instead of two passes? Obviously if that is more efficient, I'd rather explore that solution. I've tried a couple different methods to use one pass but failed miserably...
John,
RESIZE could be used here and eliminate an additional pass but you are running an unsupported release of DFSORT.
When you say unique occurrence, can it span across multiple records ?? for ex: take this sample record
Bill - Thank you, unfortunately, RESIZE isn't available to me (at least not yet in production). I read up on it and tested it on both my prod and test lpars and that would have been the solution I was looking for.
I modified my original code to remove incriminating file field names before posting but I have attached the SYMNAMES & SYMNOUT for my examples.
PB - Thank you, I have tried various inrec and outfil combinations. If I combine the include in the outfil, it would still give me records with blank occurrences.
Skolusu - Thank you. The ARRAY occurrences would be unique to each record (TICKET-1, -2, -3, -4) and would not span across multiple records. The only chance of duplicates are if there is no data there, where the ARRAY occurrence is spaces and I do not need those occurrences in the output.
The key (NOTE-OLN), would be unique to the record but could span across multiple records.
Of course, though I have never seen it happen (knock on wood), if there are dupe ARRAY elements, I would still like to pick them up.
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
John del,
Here is sample job of getting the desired results in a single pass.
I assumed that OLN key of 8 bytes is at position 1
The occurrences start from position 10 for a length of 11 bytes each.
The idea is to create the input record with the key and all the 10 occurrences. So your inrec is reduced to the fields we need.
Now using FINDREP we remove the occurrences with spaces. Once we eliminated the occurrences with spaces, it is just a matter of checking for the number of space records and adding a counter at the end.
We will use that counter to split the records on OUTFIL.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Anyone in the future making use of Kolusu's solution, ensure that you cannot have "partial" fields with both leading and trailing blanks, such that the C' ' could be satisfied if spanning two occurences (this is not possible in the TS's example). If this is the case, use the FINDREP for each individual occurrence, using STARTPOS and ENDPOS.
Yes, I like symbolics. I'm spreading the word where I can, thank you
Hi Kolusu -
Thank you. I followed what you have presented and understand the concept of using the counter to indicate how many occurrences have inrec data, and then querying that counter to create a new record for each on the outfil. I had not thought of overlay in that sense even with the earlier nudge from Bill in that direction.
I signed back on and created input testdata to match your/our symbolic names. I can attach a file of it if anyone wants. However, I'm failing on syntax on this line:
I know its that line because when I comment it out, the sort finishes fine and my output looks good. I've retyped the line, re-pasted the line, checked for funky hex characters after the text and consulted the manual but I am failing to see why I'm getting this error. When I isolate just that IFTHEN in the code above, I get ICE003 continuation line error.
I believe that this line is checking where there is a contiguous 11 bytes of space and replacing with "nil", so the rest of the IFTHENS have the condition to look for(?).
I'll try this out tomorrow with my test production data (lrecl=3250) and tweak as needed. Thanks again.
And in case anyone wants to review the Sysout from the failed sort and someone sees what it is that I'm not picking up on....
I thought it looked a bit close to column 72, but it isn't (or it is, but not close enough).
Perhaps this?
Quote:
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5
I have no idea if you get that message if you have later PTFs applied, but V1R5 itself is too old for FINDREP.
Hi Bill - I've used FINDREP for other things but, this might explain why I'm having a difficult time nailing down how to properly use it.
To confirm that it is my old version causing the error and not one of my many typo's or something else that I've missed in the cards, I copied my test data to the LPAR running z/OS 1.11 with DFSORT V1R10 and the sort job ended successfully using the control cards provided by Skolusu.
Here is my SYSOUT of the production V1R5 level. Even though ICE201I shows 'G', is there a way to tell if there is a PTS missing that can be applied, or is the DFSORT release too old so that we can't do much more in that regard?
Code:
ICE270I 0 PROCESSING SYMNAMES STATEMENTS
ICE280I 1 ORIGINAL STATEMENTS FROM SYSIN FOLLOW
SORT FIELDS=(IN-TICKET,A) 00039100
OUTFIL FNAMES=MYOUT 00039100
ICE282I 0 PERFORMING SYMBOL SUBSTITUTION AS NEEDED
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 09:11 ON FRI MAY 24, 2013 -
SORT FIELDS=(1,11,CH,A)
OUTFIL FNAMES=MYOUT
ICE201I G RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K51706 C6-K51706 C7-K54602 C8-K51706 E4-K51706 C9-BASE E5-K51706 E6-K51706 E7-K51706
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 5 E06JMDNR.STEP012 . , INPUT LRECL = 26, BLKSIZE = 32760, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,6291456,6275072)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6200902,6200902)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,004),ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=0
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
ICE750I 0 DC 0 TC 0 CS DSVVV KSZ 15 VSZ 15
ICE752I 0 FSZ=1 RC IGN=0 E AVG=32 0 WSP=1 C DYN=0 0
ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 27648, ABOVE 16M = 2084864
ICE210I 0 MYOUT : BSAM USED, LRECL = 26, BLKSIZE = 32760, TYPE = FB (SDB)
ICE751I 1 BA-K22788 BB-K44382 BC-K24705 CB-K90013 E8-K51706
ICE080I 0 IN MAIN STORAGE SORT
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 0, OUT: 0
ICE227I 0 MYOUT : DELETED = 0, REPORT = 0, DATA = 0
ICE228I 0 MYOUT : TOTAL IN = 0, TOTAL OUT = 0
ICE174I 0 NO DATA RECORDS FOR AN OUTFIL DATA SET - RC=0
ICE134I 0 NUMBER OF BYTES SORTED: 0
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT STORAGE USED = 1M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT
I assumed that since ICE201I showed the level as 'G', that we were up to date with the available PTF's and were just lacking the 'H' features and enhancements. Ugh.
Thanks - John
Edit to add - in retrospect, all of our other MVS systems are running z/OS 1.11 so I don't think I'd be able to convince the systems people to install anything new to the production system even if there was an available PTF.
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
The acronym is PTF rather than PTS. As it is in the post twice, thought i'd mention it
Probably not the best place to mention this, but when i'm involved with a software upgrade, i push Very hard to have a "sandbox" made available to check the new offering. "Everything" in production gets tested in the sandbox and when everything works as it should, i have the upgrade done in Production FIRST - then the development/qa/whatever environments.
This way the Production target will be able to support anything promoted. Several times i've seen systems they got "wrapped around the axle" because the developers were using some new feature (without realizing it was a new feature and unavailable in production) that when promoted failed. Then there were unhappy users and IT management.
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
John Del,
The error you see is due to a base code bug in symbol processing and it was fixed in the next SPE/release. You are running an unsupported release. You can still run the job without symbols and you would be fine. In the sample job, I showed change the first IFTHEN=(WHEN=INIT to the following
I tweaked the cards for my production LRECL=3250 file layout and field sizes and ran my little sample prod file and it went smoothly. Sysout below for any others that might want it for their review...
Code:
ICE143I 0 BLOCKSET COPY TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 12:44 ON FRI MAY 24, 2013 -
OPTION COPY
* REFORMAT THE INPUT TO THE ELEMENTS WE NEED
INREC IFTHEN=(WHEN=INIT,
BUILD=(141,23,X,254,11,546,11,838,11,1130,11,1422,11,
1714,11,2006,11,2298,11,2590,11,2882,11)),
* SCAN NEWLY FORMATTED RECORDS FOR CONTIGUOUS SPACES TO "BLANKS/NIL"
* CONVERTING THE MATCHING CHARS/SPACES TO "BLANKS/NIL"
* THIS INPUT DATA ELEMENT SHOULD NOT HAVE CONTIGUOUS BYTES OF SPACE
* THAT MEETS THE REPLACE CRITERIA, OTHERWISE WE'D NEED TO SPECIFY
* EACH OCCURS HERE W/FINDREP IN ORDER TO NOT ERRONEOUSLY ELIMINATE
* A VALID ELEMENT/VALUE FROM THE INPUT
IFTHEN=(WHEN=INIT,FINDREP=(STARTPOS=25,INOUT=(C' ',C''))),
* DEFAULT THE COUNTER TO 10 - ASSUME ALL OCCURRENCES ARE POPULATED
IFTHEN=(WHEN=INIT,OVERLAY=(135:C'10')),
* CHECK REFORMATTED OCCURRENCE POSTIONS EQ SPACE, SET CNT FIELD TO
* REFLECT OCCURRENCE (IF 1ST FIELD IS SPACE, WE HAVE 0 OCCURS)
IFTHEN=(WHEN=(0025,110,CH,EQ,C' '),OVERLAY=(135:C'00')),
IFTHEN=(WHEN=(0036,099,CH,EQ,C' '),OVERLAY=(135:C'01')),
IFTHEN=(WHEN=(0047,088,CH,EQ,C' '),OVERLAY=(135:C'02')),
IFTHEN=(WHEN=(0058,077,CH,EQ,C' '),OVERLAY=(135:C'03')),
IFTHEN=(WHEN=(0069,066,CH,EQ,C' '),OVERLAY=(135:C'04')),
IFTHEN=(WHEN=(0080,055,CH,EQ,C' '),OVERLAY=(135:C'05')),
IFTHEN=(WHEN=(0091,044,CH,EQ,C' '),OVERLAY=(135:C'06')),
IFTHEN=(WHEN=(0102,033,CH,EQ,C' '),OVERLAY=(135:C'07')),
IFTHEN=(WHEN=(0113,022,CH,EQ,C' '),OVERLAY=(135:C'08')),
IFTHEN=(WHEN=(0124,011,CH,EQ,C' '),OVERLAY=(135:C'09'))
*BUILD THE OUTPUT QUERYING THE COUNTER & WRITING A RECORD
* FOR EACH OF THE POSITIONAL OCCURS REFLECTED BY THE COUNTER
OUTFIL INCLUDE=(135,2,ZD,GT,0),IFOUTLEN=35,
IFTHEN=(WHEN=(135,2,ZD,EQ,01),
BUILD=(025,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,02),
BUILD=(025,11,X,01,23,/,036,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,03),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,04),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,05),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23,/,069,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,06),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23,/,069,11,X,01,23,/,080,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,07),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23,/,069,11,X,01,23,/,080,11,X,01,23,/,
091,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,08),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23,/,069,11,X,01,23,/,080,11,X,01,23,/,
091,11,X,01,23,/,102,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,09),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23,/,069,11,X,01,23,/,080,11,X,01,23,/,
091,11,X,01,23,/,102,11,X,01,23,/,113,11,X,01,23)),
IFTHEN=(WHEN=(135,2,ZD,EQ,10),
BUILD=(025,11,X,01,23,/,036,11,X,01,23,/,047,11,X,01,23,/,
058,11,X,01,23,/,069,11,X,01,23,/,080,11,X,01,23,/,
091,11,X,01,23,/,102,11,X,01,23,/,113,11,X,01,23,/,
124,11,X,01,23))
ICE201I G RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K51706 C6-K51706 C7-K54602 C8-K51706 E9-K51706 C9-BASE E5-K51706 E7-K51706
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 0 E06JMDNS.STEP0100. , INPUT LRECL = 3250, BLKSIZE = 32500, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,6291456,6270976)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6205820,6205820)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N ,ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=0
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 47104, ABOVE 16M = 91136
ICE210I 0 SORTOUT : BSAM USED, LRECL = 35, BLKSIZE = 35, TYPE = FB
ICE751I 1 EF-K49534 CB-K90013 F0-K49037 E8-K51706
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 403, OUT: 403
ICE227I 0 SORTOUT : DELETED = 0, REPORT = 0, DATA = 1192
ICE228I 0 SORTOUT : TOTAL IN = 403, TOTAL OUT = 1192
ICE052I 0 END OF DFSORT
Thanks again and as an added benefit to myself, now that I know that my DFSORT version is even more behind than I had thought, I won't pull as much hair out when I have trouble with IFTHEN & FINDREP.
Suggest you / your manager explain to Tech or higher management the developer cost because the current release has not been made available. . .
Hi Dick -
You have a point there, especially when I tell them that they have to start cutting checks for the fine people on this board who have helped me out
Seriously, though, you are right and it is a real concern. I've been inquiring nearly every month since last October about when we can get the os upgrade onto production, which means our DFSORT version would be much more current.
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
John Del wrote:
Hi Skolusu -
That did it! Thank you very much!
I tweaked the cards for my production LRECL=3250 file layout and field sizes and ran my little sample prod file and it went smoothly. Sysout below for any others that might want it for their review...
Thanks again and as an added benefit to myself, now that I know that my DFSORT version is even more behind than I had thought, I won't pull as much hair out when I have trouble with IFTHEN & FINDREP.
- John
John Del,
Awesome Job. I really liked the comments and how they explain about the functionality of the Job.
.jpg)
