Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

finding duplicates without re-sorting
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Wed Apr 03, 2013 7:33 pm    Post subject: finding duplicates without re-sorting
Reply with quote

I made a search effort but couldn't find the following as a topic.
I've wanted to find duplicates in an already sorted file but couldn't see anything in the manuals that says you can do it without re-sorting the file.
Hence I've had to code

sort fields=(...)
sum fields=none

in order that the duplicates be counted and removed. Of course, what is happening is that the SUM operation needs to be able to recognise duplicates, and the only way to do this is by the SORT statement, telling of the control fields. If there's a better way then I'm all ears!

By the way, regarding searching, the search box is available in the initial forum point but is unavailable after that. Hence, when a search brings back thousands of records you cannot do a further automatic search through the sub-set. You might say that a better search string should be given, but there is often going to be an occasion when searching returns hundreds or thousands of records. Thus, searching should be available for sub-sets.
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Wed Apr 03, 2013 7:58 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

You missed this one.

For your purpose, you'd have the HEADER3 version, the SECTIONS being your "sort key".

You might also want to compare against MERGE FIELDS=(and your key).

You need SORTIN01 instead of SORTIN for the MERGE.

Perhaps you can let us know which method is more zippy?

There are a couple of other possibilities if you're really keen,
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Apr 03, 2013 10:20 pm    Post subject:
Reply with quote

Jim Alton,

You can use SELECT operator with OPTION COPY which will give you the desired results without performing a sort

Code:

//STEP0100 EXEC PGM=ICETOOL                             
//TOOLMSG  DD SYSOUT=*                                   
//DFSMSG   DD SYSOUT=*                                   
//IN       DD *                                         
ABC    MY #1 DUP RECORD                                 
ABC    MY #2 DUP RECORD                                 
ABC    MY #3 DUP RECORD                                 
ABC    MY #4 DUP RECORD                                 
DEF    MY #1 UNIQ RECORD                                 
//OUT      DD SYSOUT=*                                   
//TOOLIN   DD *                                         
  SELECT FROM(IN) TO(OUT) ON(1,3,CH) FIRST USING(CTL1)   
//CTL1CNTL DD *                                         
  OPTION COPY                                           
//*


The output is
Code:

ABC    MY #1 DUP RECORD   
DEF    MY #1 UNIQ RECORD 
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Thu Apr 04, 2013 10:15 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

This is one of the steps in which I tried the first suggestion, and it gave output that tallies with the method I was using previously.

Code:
//SORT1    EXEC PGM=SORT,REGION=4M,TIME=1439
//SYSOUT   DD SYSOUT=*
//SYSABOUT DD SYSOUT=*
//SYSABEND DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SORTIN   DD DSN=*.SORT0.SORTOUT,DISP=(SHR,DELETE)
//SORTOUT  DD DSN=NULLFILE
//SYSIN    DD  *
            OPTION COPY
            OUTFIL SECTIONS=(151,7,HEADER3=(1,9000)),
                NODETAIL,
                REMOVECC
            END


The salient messages given by this step are:

Code:
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 119922, OUT: 119922
ICE227I 0 SORTOUT  : DELETED = 119922, REPORT = 71039, DATA = 0
ICE228I 0 SORTOUT  : TOTAL IN = 119922, TOTAL OUT = 71039
ICE174I 0 NO DATA RECORDS FOR AN OUTFIL DATA SET - RC=0


So there were 119922 input records of which 71039 are unique and 48883 (119922 - 71039) are duplicates that were deleted, but among the 71039 are 'originals' of the deleted duplicates. These figures tally with many runs of this step with the old method I used. Then I tried ICETOOL.

You can see that the JCL is just about the same, but the output counts vary.

Code:
//SORT1    EXEC PGM=ICETOOL,REGION=4M,TIME=1439
//SYSOUT   DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//IN       DD DSN=*.SORT0.SORTOUT,DISP=(SHR,DELETE)
//OUT      DD DSN=NULLFILE
//TOOLIN   DD  *
            SELECT FROM(IN) TO(OUT) ON(157,7,CH) FIRST USING(CTL1)
//CTL1CNTL DD  *
            OPTION COPY


Here are salient output messages, and note that I not only tried FIRST but also FIRST(1),LAST, ALLDUPS, NODUPS, but in every case the counts, except for the total number of records, don't match with what I've had before.

Code:
ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 09:37 ON THU APR 04, 2013 -
                       OPTION COPY
ICE146I 0 END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW
           DEBUG NOABEND,ESTAE
           OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL1,SORTIN=IN,SORTOUT=OUT,DYNALLOC,SZERO,EQUALS,NOVLSHRT,LOCALE=NONE,NOCHECK
           SORT FIELDS=(157,7,CH,A)
           MODS E35=(ICE35DU,12288)

ICE055I 0 INSERT 0, DELETE 29122
ICE054I 0 RECORDS - IN: 119922, OUT: 90800


I hope you can see what I'm getting at, so what is the explanation for these differing counts?

Code'd
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Thu Apr 04, 2013 11:13 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

Jim,

Can you post the full output from both steps please? There is "salient" for the counts and "salient" for what is going on.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Apr 04, 2013 11:29 pm    Post subject:
Reply with quote

Jim Alton,

Sections is using BINARY comparison and select is using Character comparison. DFSORT sorts BI and CH data the same unless there's an option in effect that affects the collating sequence of the CH data like LOCALE or CHALT.

How about changing SELECT also to BI and see if you are still getting different results?

If you are still getting different results then please add the following to your job and re-run the job and send us the complete sysout to dfsort@us.ibm.com.
Code:

//SORTDIAG DD DUMMY
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Fri Apr 05, 2013 4:07 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

In the following I've provided the step jcl and the sort messages. Note that in the ICETOOL SELECT I've used BI instead of CH, as suggested, but it has made no difference; but then it shouldn't since we are not doing a sort in this step but a copy of an already sorted file, and, presumably, ICETOOL is simply making a comparison on the control field to see if the current record's control field is equal to the previous record's control field, and for a simple equals comparison the collating sequence would be immaterial. Note that I've copied and pasted the following from a 3270, 80 byte screen so some of the messages are truncated but I'm hoping that it won't matter.


//SORT1 EXEC PGM=SORT,REGION=4M,TIME=1439
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=*.SORT0.SORTOUT,DISP=(SHR,DELETE)
//SORTOUT DD DSN=NULLFILE
//SORTDIAG DD DUMMY
//SYSIN DD *
OPTION COPY
OUTFIL SECTIONS=(151,7,HEADER3=(1,9000)),
NODETAIL,
REMOVECC
END

From SYSOUT:

ICE805I 1 JOBNAME: L5JALTON , STEPNAME: SORT1
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE858I 0 LA=204416, DA=20856, AA=183560, BA=2009, CP=0, TA=2057
ICE859I 0 LB=3868, DB=6095, AB=967, BB=0, CP=2, RS=4, TB=26
ICE860I 0 F=YN, P=2, M=N, B=2048
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE E5-K76585
ICE143I 0 BLOCKSET COPY TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 04:55 ON FRI AP
OPTION COPY
OUTFIL SECTIONS=(151,7,HEADER3=(1,9000)),
NODETAIL,
REMOVECC
END
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE858I 0 LA=204416, DA=20856, AA=183560, BA=2009, CP=0, TA=2057
ICE859I 0 LB=3868, DB=6095, AB=967, BB=0, CP=2, RS=4, TB=26
ICE860I 0 F=YN, P=2, M=N, B=2048
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE E5-K76585
ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
ICE088I 0 L5JALTON.SORT1 . , INPUT LRECL = 9000, BLKSIZE = 27000, TYPE
ICE093I 0 MAIN STORAGE = (MAX,20971520,20971520)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (20852358,20852358)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=20971520,MAXLIM=6291456,MINLIM=450560,EQUALS=N,LIST=Y,ER
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=20971520,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=0 ,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
ICE889I 0 CT=MAX , SB=3, L=0, D=0000, CCW=1MAM
ICE902I 0 O I JZ11
ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 26624, ABOVE 16M = 2106368
ICE855I 0 SORTOUT : TX=N, R= , L= , B= , BL=3, BR=2, DCT=38, VS=N, RU=X, SB=4
ICE210I 0 SORTOUT : BSAM USED, LRECL = 9000, BLKSIZE = 27000, TYPE = FB
ICE751I 1 EF-BASE CB-K64632 F0-K66717 E8-K79103
ICE900I 0 CON=1,MUV=0,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 119922, OUT: 119922
ICE227I 0 SORTOUT : DELETED = 119922, REPORT = 71039, DATA = 0
ICE228I 0 SORTOUT : TOTAL IN = 119922, TOTAL OUT = 71039
ICE174I 0 NO DATA RECORDS FOR AN OUTFIL DATA SET - RC=0
ICE891I 1 20918512 WMAIN, 9992 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NY QC
ICE892I 1 9000 RIN 27000 BLI 0 BLO 8998 RUN 0 BUN 2817 CPU 00 CVC
ICE804I 1 SORTIN EXCP COUNT: 513
ICE052I 0 END OF DFSORT


//SORT1 EXEC PGM=ICETOOL,REGION=4M,TIME=1439
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//SORTDIAG DD DUMMY
//IN DD DSN=*.SORT0.SORTOUT,DISP=(SHR,DELETE)
//OUT DD DSN=NULLFILE
//TOOLIN DD *
SELECT FROM(IN) TO(OUT) ON(157,7,BI) FIRST USING(CTL1)
//CTL1CNTL DD *
OPTION COPY

From TOOLMSG:

ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED

ICE650I 0 VISIT http://www.ibm.com/storage/dfsort FOR ICETOOL PAPERS, EXAMPLES A

ICE632I 0 SOURCE FOR ICETOOL STATEMENTS: TOOLIN


ICE630I 0 MODE IN EFFECT: STOP

SELECT FROM(IN) TO(OUT) ON(157,7,BI) FIRST USING(CTL1)
ICE606I 0 DFSORT CALL 0001 FOR SORT FROM IN TO OUT USING CTL1CNTL CO
ICE628I 0 RECORD COUNT: 000000000119922
ICE638I 0 NUMBER OF RECORDS RESULTING FROM CRITERIA: 000000000090800
ICE602I 0 OPERATION RETURN CODE: 00


ICE601I 0 DFSORT ICETOOL UTILITY RUN ENDED - RETURN CODE: 00

From DFSMSG:

ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
ICE805I 0 JOBNAME: L5JALTON , STEPNAME: SORT1
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE E5-K76585
ICE143I 0 BLOCKSET COPY TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 04:38 ON FRI AP
OPTION COPY
ICE146I 0 END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW
DEBUG NOABEND,ESTAE
OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL1,SORTIN=IN,SO
TOUT=OUT,DYNALLOC,SZERO,EQUALS,NOVLSHRT,LOCALE=NONE,NOC
ECK
SORT FIELDS=(157,7,BI,A)
MODS E35=(ICE35DU,12288)
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE E5-K76585
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE088I 0 L5JALTON.SORT1 . , INPUT LRECL = 9000, BLKSIZE = 27000, TYPE
ICE093I 0 MAIN STORAGE = (MAX,20971520,20942848)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (16867232,16850848)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=20971520,MAXLIM=6291456,MINLIM=450560,EQUALS=N,LIST=Y,ER
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=20971520,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=0 ,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMI
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y
ICE084I 0 BSAM ACCESS METHOD USED FOR OUT
ICE084I 0 EXCP ACCESS METHOD USED FOR IN
ICE889I 0 CT=MAX , SB=4, L=0, D=0000, CCW=1MAM
ICE902I 0 O AZ00 I JZ11
ICE751I 1 EF-BASE F0-K66717 E8-K79103
ICE900I 0 CON=1,MUV=0,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
ICE090I 0 OUTPUT LRECL = 9000, BLKSIZE = 27000, TYPE = FB
ICE055I 0 INSERT 0, DELETE 29122
ICE054I 0 RECORDS - IN: 119922, OUT: 90800
ICE891I 1 20892672 WMAIN, 10312 CMAIN, MAX CALLOC, N SCN, B BA, 8 AZ, 0 BZ, NN Q
ICE892I 1 9000 RIN 27000 BLI 27000 BLO 8998 RUN 0 BUN 2817 CPU 00 CVC
ICE893I 1 0 XIN 0 WIN 0 GIN NDEN PFP B0D CM000 CIX UPTH LMD VS RUX
ICE804I 1 IN EXCP COUNT: 513
ICE052I 0 END OF DFSORT
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Sat Apr 06, 2013 12:32 am    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

Did you follow Kolusu's advice with the SORTDIAG and ship the output of to DFSORT? Full and unchopped messages. Perhaps XDC it, or some such.
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Sat Apr 06, 2013 9:43 pm    Post subject:
Reply with quote

Dear Bill,

I followed Kolusu's advice and included SORTDIAG in each step, as can be seen by the JCL I included in my last message; because of this, I am puzzled by the meaning of your question: "Did you follow Kolusu's advice with the SORTDIAG and ship the output of to DFSORT?". There is extra information in the output messages that you should recognise, presumably provided because of the SORTDIAG DD statement.
Regarding the truncation of the messages at 80 bytes, if you really can't determine much from what I've already supplied then I'll try to supply the truncated parts as well; however, the method you suggest, XDC, I don't recognise - is it a standard function of an IBM setup?
Back to top
View user's profile Send private message
Nic Clouston

Global Moderator


Joined: 10 May 2007
Posts: 1715
Location: UK

PostPosted: Sat Apr 06, 2013 9:49 pm    Post subject:
Reply with quote

XDC is an SDSF function to output data from the spool to a dataset. What you seem to be missing is that Kolusu asked you to send the sysout to him by e-mail - not post it here.
Back to top
View user's profile Send private message
mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 268
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Sat Apr 06, 2013 10:12 pm    Post subject:
Reply with quote

Click here for the SDSF command to print.
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Sat Apr 06, 2013 10:18 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

Thanks, Nic - I've never exported SDSF output to a dataset, but I'll see if I can do it on Monday. I hope it's an option that is available to me - i.e. that it's a standard function within SDSF that isn't sometimes made unavailable.
Back to top
View user's profile Send private message
mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 268
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Sat Apr 06, 2013 10:23 pm    Post subject:
Reply with quote

On second thought, you can simply change //SYSOUT DD SYSOUT=* to //SYSOUT DD DSN=name.of.file
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Sat Apr 06, 2013 11:21 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

Yes, it was the e-mailing to the DFSORT people I was asking about, sorry to not be clear, Jim.

The SYSOUT to the dataset is another way (requiring a re-run, of course). If you have the output sitting in the queue, ? the job, XDC against the relevant sysouts for the relevant steps, and you should be prompted from there. If that doesn't work, then yes, it's probably been chopped. Perhaps look at "SDSF in batch"?
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Sat Apr 06, 2013 11:31 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

Thanks, Mistah Kurtz - the most obvious solution had to come from someone with second thoughts!
Also, thanks for the link about sending SDSF output to a dataset (XDC). After being informed about XDC I imagined that there would be an SDSF line command or separate panel, but I can see from that link that it would involve coding a REXX exec, referring to SDSF panels, none of which I've done, though I would probably have given it a go, except you've suggested a simpler - though less interesting - solution!
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Sat Apr 06, 2013 11:46 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

I'm in ecstasy about all this advice on XDC.
Thanks, Bill, I'll try issuing it as a line command if the output is still extant.
Back to top
View user's profile Send private message
mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 268
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Sun Apr 07, 2013 1:22 am    Post subject:
Reply with quote

Hi..XDC can be used as line command as below:

Code:
SDSF JOB DATA SET DISPLAY - JOB XXXXXXXX (JOB00001)    LINE 1-11 (11)         
COMMAND INPUT ===>                                            SCROLL ===> CSR 
NP   DDNAME   StepName ProcStep DSID Owner    C Dest               Rec-Cnt Page
     JESMSGLG JES2                 2 XXXXXXX  H LOCAL                   35     
     JESJCL   JES2                 3 XXXXXXX  H LOCAL                   64     
     JESYSMSG JES2                 4 XXXXXXX  H LOCAL                  154     
xdc  SYSOUT   STEP001            107 XXXXXXX  H LOCAL                  104     


And then you get the next Panel:

Code:

                            SDSF Open Print Data Set                         
COMMAND INPUT ===>                                            SCROLL ===> CSR
                                                                             
                                                                             
Data set name  ===> 'XXX.YYYY.ZZZ'                                   
Member to use  ===>                                                           
Disposition    ===> NEW        (OLD, NEW, SHR, MOD)                           
                                                                             
If the data set is to be created, specify the following.                     
Volume serial will be used to locate existing data sets if specified.         
                                                                             
Management class     ===>           (Blank for default management class)     
Storage class        ===>           (Blank for default storage class)         
  Volume serial      ===>           (Blank for authorized default volume)     
  Device type        ===>           (Generic unit or device address)         
Data class           ===>           (Blank for default data class)           
  Space units        ===> TRKS      (BLKS, TRKS, CYLS, BY, KB, or MB)         
  Primary quantity   ===> 5         (In above units)                         
  Secondary quantity ===> 10        (In above units)                         
  Directory blocks   ===> 0         (Zero for sequential data set)           
  Record format      ===> FB                                                 
  Record length      ===> 133                                               


Sorry..I was not very clear earlier..my apologies !!!
Back to top
View user's profile Send private message
Jim Alton

New User


Joined: 31 Dec 2012
Posts: 31
Location: England

PostPosted: Sun Apr 07, 2013 11:24 pm    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

Thanks for the demonstration Mistah Kurtz - as I said earlier, I hope XDC is a standard function that is available to me, and I'll be able to determine that tomorrow, Monday.
By the way, how are you copying your mainframe screen into the forum's reply box? You can see from my earlier replies that copying and pasting from the mainframe screen results in text being imported, but you seem to have saved your screen as an image and pasted that image via the image option of BBCode - correct?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Mon Apr 08, 2013 12:47 am    Post subject: Reply to: finding duplicates without re-sorting
Reply with quote

In a "Post Reply" you'll see lots of buttons to apply the bbcode for formatting.

The Code button does the non-proportional font for screens, data and reports.

Use the Preview button to see what the post will look like.
Back to top
View user's profile Send private message
mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 268
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Mon Apr 08, 2013 8:30 am    Post subject: Re: Reply to: finding duplicates without re-sorting
Reply with quote

Jim Alton wrote:
In the following I've provided the step jcl and the sort messages. Note that in the ICETOOL SELECT I've used BI instead of CH, as suggested, but it has made no difference; but then it shouldn't since we are not doing a sort in this step but a copy of an already sorted file, and, presumably, ICETOOL is simply making a comparison on the control field to see if the current record's control field is equal to the previous record's control field, and for a simple equals comparison the collating sequence would be immaterial. Note that I've copied and pasted the following from a 3270, 80 byte screen so some of the messages are truncated but I'm hoping that it won't matter.


Code:
//SORT1    EXEC PGM=SORT,REGION=4M,TIME=1439
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=*.SORT0.SORTOUT,DISP=(SHR,DELETE)
//SORTOUT  DD DSN=NULLFILE
//SORTDIAG DD DUMMY
//SYSIN    DD  *
            OPTION COPY
            OUTFIL SECTIONS=(151,7,HEADER3=(1,9000)),
                NODETAIL,
                REMOVECC
            END

From SYSOUT:

Code:
ICE805I 1 JOBNAME: L5JALTON , STEPNAME: SORT1
ICE802I 0 BLOCKSET     TECHNIQUE IN CONTROL
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE858I 0 LA=204416, DA=20856, AA=183560, BA=2009, CP=0, TA=2057
ICE859I 0 LB=3868, DB=6095, AB=967, BB=0, CP=2, RS=4, TB=26
ICE860I 0 F=YN, P=2, M=N, B=2048
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE   E5-K76585
ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 04:55 ON FRI AP
                      OPTION COPY
                      OUTFIL SECTIONS=(151,7,HEADER3=(1,9000)),
                          NODETAIL,
                          REMOVECC
                      END
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE858I 0 LA=204416, DA=20856, AA=183560, BA=2009, CP=0, TA=2057
ICE859I 0 LB=3868, DB=6095, AB=967, BB=0, CP=2, RS=4, TB=26
ICE860I 0 F=YN, P=2, M=N, B=2048
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE   E5-K76585
ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
ICE088I 0 L5JALTON.SORT1   .        , INPUT LRECL = 9000, BLKSIZE = 27000, TYPE
ICE093I 0 MAIN STORAGE = (MAX,20971520,20971520)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (20852358,20852358)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=20971520,MAXLIM=6291456,MINLIM=450560,EQUALS=N,LIST=Y,ER
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=20971520,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=0      ,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
ICE889I 0 CT=MAX     , SB=3, L=0, D=0000, CCW=1MAM
ICE902I 0 O       I JZ11
ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 26624, ABOVE 16M = 2106368
ICE855I 0 SORTOUT  : TX=N, R= , L= , B= , BL=3, BR=2, DCT=38, VS=N, RU=X, SB=4
ICE210I 0 SORTOUT  : BSAM USED, LRECL = 9000, BLKSIZE = 27000, TYPE = FB
ICE751I 1 EF-BASE   CB-K64632 F0-K66717 E8-K79103
ICE900I 0 CON=1,MUV=0,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 119922, OUT: 119922
ICE227I 0 SORTOUT  : DELETED = 119922, REPORT = 71039, DATA = 0
ICE228I 0 SORTOUT  : TOTAL IN = 119922, TOTAL OUT = 71039
ICE174I 0 NO DATA RECORDS FOR AN OUTFIL DATA SET - RC=0
ICE891I 1 20918512 WMAIN, 9992 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NY QC
ICE892I 1 9000 RIN  27000 BLI  0 BLO  8998 RUN  0 BUN  2817 CPU  00 CVC
ICE804I 1 SORTIN   EXCP COUNT: 513
ICE052I 0 END OF DFSORT


Code:
//SORT1    EXEC PGM=ICETOOL,REGION=4M,TIME=1439
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//SORTDIAG DD DUMMY
//IN       DD DSN=*.SORT0.SORTOUT,DISP=(SHR,DELETE)
//OUT      DD DSN=NULLFILE
//TOOLIN   DD  *
            SELECT FROM(IN) TO(OUT) ON(157,7,BI) FIRST USING(CTL1)
//CTL1CNTL DD  *
            OPTION COPY


From TOOLMSG:

Code:
ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED

ICE650I 0 VISIT http://www.ibm.com/storage/dfsort FOR ICETOOL PAPERS, EXAMPLES A

ICE632I 0 SOURCE FOR ICETOOL STATEMENTS:  TOOLIN


ICE630I 0 MODE IN EFFECT:  STOP

                      SELECT FROM(IN) TO(OUT) ON(157,7,BI) FIRST USING(CTL1)
ICE606I 0 DFSORT CALL 0001 FOR SORT  FROM IN       TO OUT      USING CTL1CNTL CO
ICE628I 0 RECORD COUNT:  000000000119922
ICE638I 0 NUMBER OF RECORDS RESULTING FROM CRITERIA:  000000000090800
ICE602I 0 OPERATION RETURN CODE:  00


ICE601I 0 DFSORT ICETOOL UTILITY RUN ENDED - RETURN CODE:  00

From DFSMSG:
Code:

ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
ICE805I 0 JOBNAME: L5JALTON , STEPNAME: SORT1
ICE802I 0 BLOCKSET     TECHNIQUE IN CONTROL
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE   E5-K76585
ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 04:38 ON FRI AP
                      OPTION COPY
ICE146I 0 END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW
          DEBUG NOABEND,ESTAE
          OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL1,SORTIN=IN,SO
                         TOUT=OUT,DYNALLOC,SZERO,EQUALS,NOVLSHRT,LOCALE=NONE,NOC
                         ECK
          SORT FIELDS=(157,7,BI,A)
          MODS E35=(ICE35DU,12288)
ICE905I 1 I : RF=144,LR=9000,BLK=27000,BCT=0
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K76982 C6-K90026 C7-K58148 C8-K67572 E9-K60824 C9-BASE   E5-K76585
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE088I 0 L5JALTON.SORT1   .        , INPUT LRECL = 9000, BLKSIZE = 27000, TYPE
ICE093I 0 MAIN STORAGE = (MAX,20971520,20942848)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (16867232,16850848)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=20971520,MAXLIM=6291456,MINLIM=450560,EQUALS=N,LIST=Y,ER
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=20971520,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=0      ,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMI
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y
ICE084I 0 BSAM ACCESS METHOD USED FOR OUT
ICE084I 0 EXCP ACCESS METHOD USED FOR IN
ICE889I 0 CT=MAX     , SB=4, L=0, D=0000, CCW=1MAM
ICE902I 0 O AZ00  I JZ11
ICE751I 1 EF-BASE   F0-K66717 E8-K79103
ICE900I 0 CON=1,MUV=0,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
ICE090I 0 OUTPUT LRECL = 9000, BLKSIZE = 27000, TYPE = FB
ICE055I 0 INSERT 0, DELETE 29122
ICE054I 0 RECORDS - IN: 119922, OUT: 90800
ICE891I 1 20892672 WMAIN, 10312 CMAIN, MAX CALLOC, N SCN, B BA, 8 AZ, 0 BZ, NN Q
ICE892I 1 9000 RIN  27000 BLI  27000 BLO  8998 RUN  0 BUN  2817 CPU  00 CVC
ICE893I 1 0 XIN 0 WIN 0 GIN NDEN PFP   B0D CM000 CIX UPTH LMD VS   RUX
ICE804I 1 IN       EXCP COUNT: 513
ICE052I 0 END OF DFSORT


From your post, I selected the Mainframe code part and clicked on Code button and it's done.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Removing Duplicates based on certain ... chandracdac DFSORT/ICETOOL 8 Fri Dec 09, 2016 4:40 am
No new posts Updating the counters after eliminati... PANDU1 DFSORT/ICETOOL 12 Mon Nov 21, 2016 9:47 am
No new posts Efficient sorting chandracdac DFSORT/ICETOOL 5 Sat Oct 22, 2016 3:23 am
No new posts Stripping and Sorting of VB file G SRINIVASA RAO SYNCSORT 5 Wed Sep 14, 2016 1:34 pm
No new posts DFSORT sorting only specific records akodakka DFSORT/ICETOOL 4 Thu Aug 04, 2016 11:24 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us