Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

EQUALs parameter in sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Wed Sep 04, 2013 8:59 am    Post subject: EQUALs parameter in sort
Reply with quote

We made a change to one of our sort step on May and missed the EQUALS parameter in sort step. Last week, one of the successor job has failed due to duplicate data. I do not have any i/p & o/p files available to say what data has created/used. I found one of the successor job input file has data in a random order under its key. So, when I checked this sort card I noted that we missed EQUALS parameter.

I verified last few runs of our job and in SYSOUT ICEXXXI message options EQUALS=N. Could someone tell me how sort has worked without EQUALS option for this time? IS there any other parameter controls that? Now I modified the sort to include EQUALS, but I would like to understand why we had issues all of a sudden.
Back to top
View user's profile Send private message

mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 268
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Wed Sep 04, 2013 11:34 am    Post subject:
Reply with quote

Probably because your site has EQUALS=NO as the default.
And specifying EQUALS in your SORT card temporarily override the default option, so that equally collating records are written in their original order for your job.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Wed Sep 04, 2013 11:41 am    Post subject: Reply to: EQUALs parameter in sort
Reply with quote

With EQUALS the order of duplicate keys on the output from a SORT or MERGE is predictable. For SORT they are in the same order as the input file, for MERGE they are in the same order as on each successive SORTINGnn file.

With NOEQUALS, the order is not predictable, but is whatever happens to be the most convenient when the next record is written.

Both EQUALS and NOEQUALS work. Day in, day out, every time.

Notice that "not predictable" is not the same as "random".

Quote:
Last week, one of the successor job has failed due to duplicate data


If this is an accurate statement, you can't even think to blame SORT for that.

In fact, you can't blame SORT anyway. If it was coded wrong and no-one got it in what should have been several stages of testing, long for blame elsewhere.

Until you explain otherwise, your search for why you had a failure has discovered and unconnected error elsewhere.

You'd best describe the failure and the relevant data in more detail.
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Wed Sep 04, 2013 11:13 pm    Post subject: Reply to: EQUALs parameter in sort
Reply with quote

Hi Bill,

We tested it so many times but we didn't notice any change. This code has been in production and running from May and issue was noticed only on August last week. Our business questions why it took so long to create an issue? I guess that's a valid question.

In my previous shop, if I don't use EQUALS, i will see a difference immediately, but here I do not see that. I cross checked whether there is any default system OPTION as EQUALS=Y, but ICEXXXI message states clearly that EQUALS=N in all previous run.

Sample data is as below
Code:
516CAR1120130813
516CAR32001
516CAR23001
123ABC1120130813
123ABC32001
123ABC32006
123ABC32055
123ABC23062
234DCE1120130813
234DCE32002
234DCE23002

SORT FIELDS=(1,3,CH,A,4,3,CH,A)


The data which we had is in above format where column 7-8 is record type, 11 - header 32 - detail and 23 - trailer. It needs to be in that order. So, EQUALS is a must when we sort on first two fields.

However, if i test our data and above one, i couldn't recreate it. Always it is coming in proper order (11,32,23). Nobody believes me that EQUALS is causing the issue:)
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Sep 04, 2013 11:29 pm    Post subject: Re: Reply to: EQUALs parameter in sort
Reply with quote

kbmk wrote:
Hi Bill,

We tested it so many times but we didn't notice any change. This code has been in production and running from May and issue was noticed only on August last week. Our business questions why it took so long to create an issue? I guess that's a valid question.


kbmk,


Run this JCL and pay attention to the 6th byte, which is in the ascending order. I am sorting on the first 3 bytes which is basically the word "key" for every record
Code:

//STEP0100 EXEC PGM=SORT                               
//SYSOUT   DD SYSOUT=*                                 
//SORTIN   DD *                                       
KEY0 A                                                 
KEY3 B                                                 
KEY4 C                                                 
KEY1 D                                                 
KEY2 E                                                 
KEY3 F                                                 
KEY4 G                                                 
KEY5 H                                                 
KEY6 I                                                 
KEY7 J                                                 
KEY0 K                                                 
KEY0 L                                                 
KEY0 M                                                 
KEY0 N                                                 
KEY0 O                                                 
KEY0 P                                                 
//SORTOUT  DD SYSOUT=*                                 
//SYSIN    DD *                                       
  INREC OVERLAY=(23450:C'KBMK WANTS PROOF FOR EQUALS')
  SORT FIELDS=(1,3,CH,A),NOEQUALS                     
//*


kbmk wrote:
Nobody believes me that EQUALS is causing the issue:)


This is one of those urban legends at your site that just won't die? The "true believers" fall into the category "don't bother me with the facts, my mind is made up".
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Wed Sep 04, 2013 11:42 pm    Post subject:
Reply with quote

Hi skolusu,

I do not notice any difference in output irrespective of whether I had NOEQUALS or EQUALS. Output is same as in input.

SYSOUT with NOEQUALS
Code:
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R10 - 13:03 ON WED SE
            INREC OVERLAY=(23450:C'KBMK WANTS PROOF FOR EQUALS')
            SORT FIELDS=(1,3,CH,A),NOEQUALS
ICE201I G RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K51707 C6-K51707 C7-K54603 C8-K51707 E9-K51707 C9-BASE   E5-K55337
ICE142I 0 DFMSGOUT NOT FOUND - SYSOUT USED
ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
ICE088I 5 TSYDBK1#.STEP001 .        , INPUT LRECL = 80, BLKSIZE = 80, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,12582912,12569694)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (12509168,12509168)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=12582912,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ER
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=SHORT,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=


SYSOUT with EQUALS
Code:
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R10 - 13:06 ON WED SE
            INREC OVERLAY=(23450:C'KBMK WANTS PROOF FOR EQUALS')
            SORT FIELDS=(1,3,CH,A),EQUALS
ICE201I G RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K51707 C6-K51707 C7-K54603 C8-K51707 E9-K51707 C9-BASE   E5-K55337
ICE142I 0 DFMSGOUT NOT FOUND - SYSOUT USED
ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
ICE088I 5 TSYDBK1#.STEP001 .        , INPUT LRECL = 80, BLKSIZE = 80, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,12582912,12569694)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (12509168,12509168)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=12582912,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ER
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=SHORT,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=


Does it give a different output for you?
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Sep 05, 2013 1:48 am    Post subject:
Reply with quote

kbmk wrote:
Hi skolusu,

I do not notice any difference in output irrespective of whether I had NOEQUALS or EQUALS. Output is same as in input.


I need to see the output rather than the sysout. Why is your shop looking DFMSGOUT dd name instead of SYSOUT?

kbmk wrote:
Does it give a different output for you?


I get a different output. I get this output with NOEQUALS.
Code:

KEY0 A
KEY0 L
KEY0 M
KEY0 N
KEY0 O
KEY0 P
KEY0 K
KEY1 D
KEY2 E
KEY3 F
KEY3 B
KEY4 C
KEY4 G
KEY5 H
KEY6 I
KEY7 J
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Thu Sep 05, 2013 1:53 am    Post subject:
Reply with quote

Output with NOEQUALS
Code:
KEY0 A
KEY3 B
KEY4 C
KEY1 D
KEY2 E
KEY3 F
KEY4 G
KEY5 H
KEY6 I
KEY7 J
KEY0 K
KEY0 L
KEY0 M
KEY0 N
KEY0 O
KEY0 P


Output with EQUALS
Code:
KEY0 A
KEY3 B
KEY4 C
KEY1 D
KEY2 E
KEY3 F
KEY4 G
KEY5 H
KEY6 I
KEY7 J
KEY0 K
KEY0 L
KEY0 M
KEY0 N
KEY0 O
KEY0 P


I am not sure why it is looking for DFMSGOUT. I also expect to see a difference in the output when I use NOEQUALS. But I do not see any difference.
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Thu Sep 05, 2013 2:09 am    Post subject:
Reply with quote

I guess you have sorted on 1,4 rather than 1,3 (saying it based on fourth character in your output - it is sorted in ascending order). I tried that. Below is the output and still no difference.
Code:
Input      EQUALS      NOEQUALS   
KEY0   A   KEY0   A   KEY0   A
KEY3   B   KEY0   K   KEY0   K
KEY4   C   KEY0   L   KEY0   L
KEY1   D   KEY0   M   KEY0   M
KEY2   E   KEY0   N   KEY0   N
KEY3   F   KEY0   O   KEY0   O
KEY4   G   KEY0   P   KEY0   P
KEY5   H   KEY1   D   KEY1   D
KEY6   I   KEY2   E   KEY2   E
KEY7   J   KEY3   B   KEY3   B
KEY0   K   KEY3   F   KEY3   F
KEY0   L   KEY4   C   KEY4   C
KEY0   M   KEY4   G   KEY4   G
KEY0   N   KEY5   H   KEY5   H
KEY0   O   KEY6   I   KEY6   I
KEY0   P   KEY7   J   KEY7   J
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Sep 05, 2013 2:28 am    Post subject:
Reply with quote

kbmk wrote:
I guess you have sorted on 1,4 rather than 1,3 (saying it based on fourth character in your output - it is sorted in ascending order). I tried that. Below is the output and still no difference.


Kbmk,

I have sorted only the first 3 bytes and trust me when I say that.

Code:

ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1                         
ICE751I 0 C5-K76982 C6-K90026 C7-K94453 C8-K94453 E9-K60824 C9-BASE   E5-K80744
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED                                 
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 13:14 ON WED SE
            INREC OVERLAY=(23450:C'KBMK WANTS PROOF FOR EQUALS')               
            SORT FIELDS=(1,3,CH,A),NOEQUALS                                     
ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
ICE088I 1 KOLUSUA .STEP0100.        , INPUT LRECL = 80, BLKSIZE = 80, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,6291456,6278238)                                 
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6172656,6172656)                           
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERE
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMI
ICE235I 0 OPTIONS: NULLOUT=RC0                                                 
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y                                         
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTOUT                                   
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN                                   
ICE750I 0 DC 500000 TC 0 CS DSVNN KSZ 3 VSZ 3                                   
ICE752I 0 FSZ=6250 RC  IGN=0 E  AVG=23476 0  WSP=190571 C  DYN=0 0             
ICE751I 1 DE-K83743 D5-K91600 D9-K61787 E8-K94453                               
ICE090I 0 OUTPUT LRECL = 23476, BLKSIZE = 23476, TYPE = FB                     
ICE171I 0 SORTOUT LRECL OF 23476 IS DIFFERENT FROM SORTIN(NN) LRECL OF 80 - RC=0
ICE080I 0 IN MAIN STORAGE SORT                                                 
ICE055I 0 INSERT 0, DELETE 0                                                   
ICE054I 0 RECORDS - IN: 16, OUT: 16                                             
ICE134I 0 NUMBER OF BYTES SORTED: 1280                                         
ICE253I 0 RECORDS SORTED - PROCESSED: 16, EXPECTED: 6250                       
ICE191I 0 DFSORT SVC NOT AVAILABLE - PERFORMANCE MAY BE DEGRADED               
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0             
ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES                         
ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 0M BYTES                         
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES                                   
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES                                   


Please run your job adding //SORTDIAG DD DUMMY to your sort step and send us the complete output to dfsort@us.ibm.com
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Sep 05, 2013 2:38 am    Post subject:
Reply with quote

kbmk wrote:
I am not sure why it is looking for DFMSGOUT. I also expect to see a difference in the output when I use NOEQUALS. But I do not see any difference.


SYSOUT DD identifies the DFSORT message data set. The default ddname is SYSOUT, but you can specify an alternate ddname for the message data set using the MSGDDN installation or run-time option. Always supply a DD statement for the message data set if a catalogued procedure is not used. (If you are invoking DFSORT from a COBOL program and are using the ddname SYSOUT for the message data set, the use of DISPLAY in your COBOL program can produce uncertain printing results.)

So your shop must have changed the defaults. Run this job to list your shop defaults. Please send the output of this job also to the same email address listed

Code:

//STEP0100 EXEC PGM=ICETOOL   
//TOOLMSG   DD SYSOUT=*       
//DFSMSG    DD SYSOUT=*       
//OUT       DD SYSOUT=*       
//TOOLIN    DD *             
  DEFAULTS LIST(OUT)         
/*
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Sep 05, 2013 3:21 am    Post subject:
Reply with quote

kbmk,

I just checked the emails you have sent and unfortunately you are running DFSORT V1R10 which is out of service. We are not supposed to work on releases that are out of service. If your shop has an extended contract with IBM then you can open a PMR and we will look into it.
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Thu Sep 05, 2013 3:25 am    Post subject:
Reply with quote

Hi Skolusu

Thanks. I will check with our administrator on this.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts TOT & DATE parameter in ICEMAN hemanthj642 DFSORT/ICETOOL 4 Mon Nov 14, 2016 5:19 am
No new posts Sort records based on numeric field. Alks SYNCSORT 2 Wed Oct 19, 2016 10:14 pm
No new posts How to delete second instance from Fl... Gunapala CN DFSORT/ICETOOL 6 Tue Oct 18, 2016 11:42 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us