IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Compare flat with vsam file using DFSORT


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
magesh23586

Active User


Joined: 06 Jul 2009
Posts: 213
Location: Chennai

PostPosted: Wed Nov 06, 2013 11:13 am
Reply with quote

And the CPU consumption i missed to tell.

After adding Sorted,NOSEQCK and FILSZ

TOTAL TCB CPU TIME= .28 TOTAL ELAPSED TIME= .6

previously it was

TOTAL TCB CPU TIME= .38 TOTAL ELAPSED TIME= .7
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Nov 06, 2013 1:19 pm
Reply with quote

magesh23586 wrote:

2. SORTED, NOSEQCK ==> I think sorted is sufficient, if we mention sorted alone, do the dfsort will perform a sequence check ?


DFSORT is not known for having statements merely for decorative purposes.

Your concern with performance will come down to the number of records on F1. If F1 is greater than 20% of the number of records on F2 I'd expect the JOINKEYS to be faster, generally. If F1 has one record, the process will be horribly slower.

The "tipping point" will be somewhere in between, and only you can identify that by running volume tests where the data is representative of what is expected in production.

DFSORT does no "keyed read" of a KSDS. With one record on F1, your entire KSDS will be read.
Back to top
View user's profile Send private message
magesh23586

Active User


Joined: 06 Jul 2009
Posts: 213
Location: Chennai

PostPosted: Wed Nov 06, 2013 1:52 pm
Reply with quote

Please correct me, if I have misunderstood
if f1 = 100 and F2 = 50 then performance is good
if f1= 50 and F2 =100 then performance will be bad ?

I tried the code as below, no difference, took same CPU.

Code:

//SYSOUT DD SYSOUT=*                                                 
//SORTJNF2 DD DSN=FILE1,DISP=SHR
//SORTJNF1 DD DSN=VSAM1,DISP=SHR               
//SORTOUT DD DSN=JOINKEY.OUT,DISP=OLD                       
//SYSIN DD *                                                         
* CONTROL STATEMENTS FOR JOINKEYS APPLICATION                       
  JOINKEYS FILE=F2,FIELDS=(3,1,A,5,2,A,8,4,A,13,5,A)                 
  JOINKEYS FILE=F1,FIELDS=(1,1,A,2,2,A,4,4,A,8,5,A),SORTED,         
           TYPE=V,NOSEQCK                                           
  JOIN UNPAIRED,F2,ONLY                                             
  REFORMAT FIELDS=(F2:1,1354)                                       
* CONTROL STATEMENTS FOR MAIN TASK (JOINED RECORDS)                 
  SORT FIELDS=COPY                                                   
//JNF1CNTL DD *                                                     
 OMIT COND=(450,1,CH,EQ,C'N')                                       
 OPTION DYNALLOC(,16),FILSZ=E5000000,AVGRLEN=450                     
/*                                                                   
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Nov 06, 2013 2:09 pm
Reply with quote

No. Hopefully with small numbers of records you won't notice anything anyway.

My comment about the JOINKEYS is not about the JOINKEYS itself, but about comparing a two-file-match generally to a keyed READ in COBOL.

For your solution the overhead of reading the entire KSDS searially has to be less than reading the KSDS from the entire key file.
Back to top
View user's profile Send private message
magesh23586

Active User


Joined: 06 Jul 2009
Posts: 213
Location: Chennai

PostPosted: Wed Nov 06, 2013 3:25 pm
Reply with quote

A small changes to be noted, incase if any one refer this code in future.

Variable dataset should start with offset 5, which I missed.

Omit condition need to used with "NE" condition for this requirement.
Code:

//SYSIN DD *                                                 
* CONTROL STATEMENTS FOR JOINKEYS APPLICATION                 
  JOINKEYS FILE=F1,FIELDS=(3,1,A,5,2,A,8,4,A,13,5,A)         
  JOINKEYS FILE=F2,FIELDS=(5,1,A,6,2,A,8,4,A,12,5,A),SORTED,  ==>changed
           TYPE=V,NOSEQCK                                     
  JOIN UNPAIRED,F1,ONLY                                       
  REFORMAT FIELDS=(F1:1,1354)                                 
* CONTROL STATEMENTS FOR MAIN TASK (JOINED RECORDS)           
  SORT FIELDS=COPY                                           
//JNF2CNTL DD *                                               
 OMIT COND=(454,1,CH,NE,C'N')                        ==>changed         
 OPTION DYNALLOC(,16),FILSZ=E5000000,AVGRLEN=450             
/*                                                           
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Nov 06, 2013 4:08 pm
Reply with quote

You can avoid the tortuous OMIT=(...NE...) by instead using INCLUDE=(...EQ...).
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Nov 06, 2013 10:54 pm
Reply with quote

magesh23586,

Code:

//JNF2CNTL DD *                                               
 OMIT COND=(454,1,CH,NE,C'N')                        ==>changed         
 OPTION DYNALLOC(,16),FILSZ=E5000000,AVGRLEN=450     


The DYNALLOC and FILSZ and AVGRLEN are ignored as you used SORTED keyword which makes Joinkeys read the VSAM file with a COPY operation. Remove them. I am not even sure as to why you need to read the VSAM Cluster as Variable when your key is always present and you don't need any other data. Read it TYPE=F and adjust the positions on the JOINKEYS accordingly. Since you don't need any data apart from the key, Just use INREC BUILD=(1,12) in JNFxCNTL which will only build the Key to match. Your INCLUDE condition gets executed before the INREC.

As Bill Mentioned always code for Positive condition rather than negative condition.
Back to top
View user's profile Send private message
magesh23586

Active User


Joined: 06 Jul 2009
Posts: 213
Location: Chennai

PostPosted: Thu Nov 07, 2013 6:12 pm
Reply with quote

SKolusu,

This is Fantastic and awesome, CPU time further reduced to .25 previously .28

Thankyou very much... The fix given by you is worked like a charm.

I can now boldly challenge with my clients icon_biggrin.gif icon_biggrin.gif icon_biggrin.gif

Code:

* CONTROL STATEMENTS FOR JOINKEYS APPLICATION                 
  JOINKEYS FILE=F1,FIELDS=(3,1,A,5,2,A,8,4,A,13,5,A)         
  JOINKEYS FILE=F2,FIELDS=(1,1,A,2,2,A,4,4,A,8,5,A),SORTED,   
           TYPE=F,NOSEQCK                                     
  REFORMAT FIELDS=(F1:1,1354)                                 
* CONTROL STATEMENTS FOR MAIN TASK (JOINED RECORDS)           
  SORT FIELDS=COPY                                           
//JNF2CNTL DD *                                               
 INCLUDE COND=(450,1,CH,NE,C'N')                             
 INREC BUILD=(1,12)                                           
/*                                                           
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Thu Nov 07, 2013 6:43 pm
Reply with quote

For a comparison, why don't you use OPTION COPY/SORT FIELDS=COPY to copy your KSDS to a "flat" file. Then run your step with that flat file. As well as CPU, pay attention to the I-O counts, VSAM vs flat.

Remember, if the number of records you are processing is "small" in relation to the size of the KSDS, your client is not going to be impressed vs a programming language with a direct read.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL Goto page 1, 2  Next

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 7
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
No new posts Modifying Date Format Using DFSORT DFSORT/ICETOOL 9
No new posts Access to non cataloged VSAM file JCL & VSAM 18
Search our Forums:

Back to Top