Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Variable number of input files to merge

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
sclater

New User


Joined: 22 Jun 2007
Posts: 14
Location: South Africa

PostPosted: Tue Mar 04, 2008 3:00 pm    Post subject: Variable number of input files to merge
Reply with quote

Hi, thought I would make my first post my solution to a rather difficult job I've had to do recently (thanks Frank for making sort so powerful icon_biggrin.gif ). Any ideas/advice on how I could possibly do better would be greatly appreciated.

The problem:
We receive a unknown number of files from a external company that we need to merge into one file for another system that actually processes the data. What makes it even more tricky then just not knowing how many files to expect, the files each contain a header record with a value that indicates which institution (bank) the file belongs to and these need to be in bank and then file sequence order for the system to process the files (the files are numbered 1 -> 9999 in the filename). As an example:

File1: header institute 9
File2: header institute 5
File3: header institute 9
File4: header institute 9
File5: header institute 5
File6: header institute 9

Thus the correct sequence of the files for the processing system would be:
File2: header institute 5
File5: header institute 5
File1: header institute 9
File3: header institute 9
File4: header institute 9
File6: header institute 9

The solution:
My solution requires 3 JCL's, one to get the filenames, one to reorganize the filenames using the value from the header record and then one JCL to merge the files into one file.

JCL-1

Code:

//**********************************************************************
//* DO A LISTCAT OF THE FILES                                           
//**********************************************************************
//IDCLCAT  EXEC PGM=IDCAMS,REGION=1500K,ADDRSPC=VIRT                   
//OUTDD    DD DSN=&&CAT,                                               
//            DISP=(NEW,CATLG),                                         
//            UNIT=SYSDA,SPACE=(TRK,(5,1)),                             
//            RECFM=VBA,LRECL=125                                       
//SYSPRINT DD SYSOUT=*                                                 
//AMSDUMP  DD SYSOUT=*                                                 
//SYSUDUMP DD SYSOUT=*                                                 
//SYSOUT   DD SYSOUT=*                                                 
//SYSIN    DD  *                                                       
  LISTCAT OFILE(OUTDD) LVL(DNGG00.CDPAKI)                               
//**********************************************************************
//* SORT THE CLIST TO EXTRACT THE FILENAMES WE WANT TO A MEMBER WE CAN 
//* INCLUDE IN ANOTHER JCL                                             
//* FILES WE WANT:                                                     
//*   XXXXXX.XXXXXX.C4D*.**                                             
//* EXCLUDE THE Z1 AND Z9 FILES:                                       
//*   XXXXXX.XXXXXX.C4D00Z1C.**                                         
//*   XXXXXX.XXXXXX.C4D00Z9C.**                                         
//* FORMAT THE FILES AS JCL INPUT:                                     
//*   //         DD DISP=SHR,DSN=XXXXXX.XXXXXX.C4D*.**                 
//**********************************************************************
//SORT01   EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD DSN=&&CAT,DISP=SHR                                       
//SORTOUT  DD DSN=DNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR               
//SYSIN    DD *                                                         
   OPTION VLSHRT                                                       
   SORT FIELDS=COPY                                                     
   OUTFIL INCLUDE=(36,3,CH,EQ,C'C4D',AND,                               
                   41,3,CH,NE,C'Z1C',AND,                               
                   41,3,CH,NE,C'Z9C'),                                 
          CONVERT,REMOVECC,                                             
          BUILD=(1C'//         DD DISP=SHR,DSN=',22,50,80:X)           
//*                                                                     


Output from the JCL:
Code:
BROWSE    DNCV00.OPSUTIL0.CNTL(SAC4ALL)
Command ===>                                             
********************************* Top of Data ************
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0001D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0002D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0003D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0004D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0005D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0006D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0007D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0008D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0009D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0010D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0011D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0012D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0013D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0014D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0015D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0016D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0017D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0018D.D0.SQ127


JCL-2

Code:
//PROCLIB JCLLIB ORDER=(QNCV00.OPSUTIL0.CNTL)                           
//**********************************************************************
//* 1. COPY THE LIST OF FILES (IN1) AND REMOVE THE FORMATTING FOR THE   
//*    JCL                                                             
//* 2. EXTRACT THE BANK NUMBER FROM THE HEADER RECORD OF EACH FILE (IN2)
//* 3. MERGE THE TWO VALUES TO GIVE US A LIST WE CAN USE               
//* 4. SORT THE FILENAMES USING THE BANK NUMBER AS THE KEY AND THEN THE
//*    FILE NUMBER. DROP THE BANK NUMBER AND REFORMAT THE RECORD TO USE
//*    IN JCL                                                           
//**********************************************************************
//SORT01   EXEC PGM=ICETOOL                                             
//TOOLMSG  DD SYSOUT=*                                                 
//DFSMSG   DD SYSOUT=*                                                 
//IN1      DD DSN=QNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR               
//* INCLUDE THE TEMP FILE AS THE FORMAT OF THE INCLUDE IS NOT CORRECT   
//IN2      DD DSN=&&TEMP1,DISP=(NEW,CATLG),UNIT=SYSDA,                 
//            SPACE=(TRK,(1,1)),RECFM=FB,LRECL=127                     
//    INCLUDE MEMBER=SAC4ALL                                           
//TMP1     DD DSN=&&TEMP2,DISP=(MOD,PASS),UNIT=SYSDA                   
//TMP2     DD DSN=&&TEMP2,DISP=(MOD,PASS),UNIT=SYSDA                   
//OUT      DD DSN=QNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR               
//TOOLIN   DD *                                                         
* REFORMAT THE IN1 DATASET                                             
   COPY FROM(IN1) TO(TMP1) USING(CTL1)                                 
* REFORMAT THE IN2 DATASET                                             
   COPY FROM(IN2) TO(TMP1) USING(CTL2)                                 
* SPLICE RECORDS WITH MATCHING SEQUENCE NUMBERS                         
   SPLICE FROM(TMP1) TO(TMP2) ON(60,8,PD) WITH(51,4) USING(CTL3)       
* SORT THE RECORDS IN THE CORRECT SEQUENCE AND REMOVE THE VALUE FROM THE
* HEADER. BUILD RECORD TO USE IN JCL                                   
   SORT FROM(TMP2) TO(OUT) USING(CTL4)                                 
/*                                                                     
//CTL1CNTL DD *                                                         
* USE OUTREC TO CREATE: |F1FLD|BLANK|SEQNUM|                           
   OUTREC FIELDS=(1:28,50,51:4X,60:SEQNUM,8,PD)                         
/*                                                                     
//CTL2CNTL DD *                                                         
* USE OUTREC TO CREATE: |BLANK|F2FLD|SEQNUM|                           
* ONLY INCLUDE THE HEADER RECORD                                       
   INCLUDE COND=(1,2,CH,EQ,C'01')                                       
   OUTREC FIELDS=(1:50X,51:68,4,60:SEQNUM,8,PD)                         
/*                                                                     
//CTL3CNTL DD *                                                         
* MERGE TWO RECORDS AND KEEP VALUES WE WANT TO USE                     
   OUTFIL FNAMES=TMP2,OUTREC=(1,60,80:X)                               
/*                                                                     
//CTL4CNTL DD *                                                         
* MERGE TWO RECORDS AND KEEP VALUES WE WANT TO USE                     
   SORT   FIELDS=(51,4,CH,A,1,50,CH,A)                                 
   OUTFIL FNAMES=OUT,                                                   
          OUTREC=(1C'//         DD DISP=SHR,DSN=',1,50,80:X)           
/*                                                                     
//*                                                                     


Output from the JCL:
Code:
BROWSE    QNCV00.OPSUTIL0.CNTL(SAC4ALL)                   Line
Command ===>                                             
********************************* Top of Data ************
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0004D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0007D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0009D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0012D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0013D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0014D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0015D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0016D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0017D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0018D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0001D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0002D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0003D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0005D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0006D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0008D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0010D.D0.SQ127
//         DD DISP=SHR,DSN=DNGG00.CDPAKI.C4D0011D.D0.SQ127


So now I have jcl statements that I can include in a repro to merge the files:

Code:
//PROCLIB JCLLIB ORDER=(DNCV00.OPSUTIL0.CNTL)       
//**********************************************************************
//* DELETE OUTPUT FILES                                                 
//**********************************************************************
//DEL01   EXEC PGM=IEFBR14                                               
//OUTFIL  DD  DSN=DNGG00.UCPACK.HADUC8.D0.SQ127,                       
//            DISP=(MOD,DELETE,DELETE),                                 
//            SPACE=(TRK,0)                                             
//*                                                                     
//********************************************************************   
//*    CONCATENATE THE INCOMING FILES INTO ONE OUTPUT FILE               
//********************************************************************   
//CONCAT1  EXEC  PGM=IEBGENER                                           
//SYSPRINT DD  SYSOUT=*                                                 
//SYSIN    DD  DUMMY                                                     
//SYSUT2   DD  DISP=(,KEEP,DELETE),SPACE=(CYL,(20,5),RLSE),             
//             DCB=(BLKSIZE=27940,LRECL=127,RECFM=FB),               
//             DSN=DNGG00.UCPACK.HADUC8.D0.SQ127                     
//SYSUT1   DD DSN=&&TEMP,DISP=(NEW,CATLG),UNIT=SYSDA,                   
//            SPACE=(TRK,(5,1)),                                         
//            DCB=(BLKSIZE=27940,LRECL=127,RECFM=FB)                 
//         INCLUDE MEMBER=SAC4ALL                                       
//*                                                                     
Back to top
View user's profile Send private message

Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Mar 05, 2008 12:15 am    Post subject: Reply to: Variable number of input files to merge
Reply with quote

sclater,

You have the right idea , but you are using one to many passes to get the desired results. if i understand your requirement correctly , it can be done with less no: of passes

what is the LRECL and RECFM of the files involved? is there a way to identify the Header record? If so what is it?

Example assume that you got 3 files like this

File1:
Code:

header institute 9
datarec1
datarec2
...

File2:
Code:

header institute 5
datarec3
datarec4
...


File3:
Code:

header institute 4
datareca
datarecb
...


and you want the output as

Code:

HEADER INSTITUTE 4     
DATARECA               
DATARECB               
HEADER INSTITUTE 5     
DATAREC3               
DATAREC4               
HEADER INSTITUTE 9     
DATAREC1               
DATAREC2               


The following DFSORT ICETOOL job will give you the desired results

Code:

//STEP0100 EXEC PGM=ICETOOL                               
//TOOLMSG  DD SYSOUT=*                                   
//DFSMSG   DD SYSOUT=*                                   
//IN       DD *                                           
HEADER INSTITUTE 9                                       
DATAREC1                                                 
DATAREC2                                                 
HEADER INSTITUTE 5                                       
DATAREC3                                                 
DATAREC4                                                 
HEADER INSTITUTE 4                                       
DATARECA                                                 
DATARECB                                                 
//T1       DD DSN=&&T1,DISP=(,PASS),SPACE=(CYL,(1,1),RLSE)
//OUT      DD SYSOUT=*                                   
//TOOLIN   DD *
//TOOLIN   DD *                                       
  SPLICE FROM(IN) TO(T1) ON(81,8,CH) -               
  WITHALL KEEPBASE WITH(01,80) USING(CTL1)           
  SORT FROM(T1) USING(CTL2)                           
//CTL1CNTL DD *                                       
  SORT FIELDS=COPY                                   
  INREC IFTHEN=(WHEN=INIT,                           
       OVERLAY=(81:SEQNUM,8,ZD)),                     
        IFTHEN=(WHEN=(1,7,CH,EQ,C'HEADER'),           
       OVERLAY=(81:SEQNUM,8,ZD,8,20)),               
        IFTHEN=(WHEN=NONE,                           
       OVERLAY=(89:SEQNUM,8,ZD,                       
                81:81,8,ZD,SUB,89,8,ZD,M11,LENGTH=8))
  OUTFIL FNAMES=T1,                                   
  BUILD=(01,80,89,20)                                 
//CTL2CNTL DD *                                       
  SORT FIELDS=(81,20,CH,A)                           
  OUTFIL FNAMES=OUT,                                 
  BUILD=(01,80)                                       
/*


If that is what you want then let me know and I will post the entire job for you
Back to top
View user's profile Send private message
sclater

New User


Joined: 22 Jun 2007
Posts: 14
Location: South Africa

PostPosted: Wed Mar 05, 2008 1:01 pm    Post subject: Re: Reply to: Variable number of input files to merge
Reply with quote

Skolusu wrote:
sclater,

You have the right idea , but you are using one to many passes to get the desired results. if i understand your requirement correctly , it can be done with less no: of passes


That would be excellent!

Quote:
what is the LRECL and RECFM of the files involved? is there a way to identify the Header record? If so what is it?


LRECL = 127
RECFM = FB
Header can be identified by "01" in positions 1 and 2 (will always only be one header per file).
Key for the insitute is 4 long starting in position 68
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Mar 05, 2008 10:36 pm    Post subject:
Reply with quote

sclater,

Here is DFSORT/ICETOOL job which will give you the desired results. This job will generate a dynamic JCL and submit it via intrdr.
Code:

//**********************************************************
//* DO A LISTCAT OF THE FILES                              *
//**********************************************************
//STEP0100 EXEC PGM=IKJEFT01                     
//SYSTSPRT DD DSN=&&CAT,               
//            DISP=(NEW,CATLG,DELETE),           
//            SPACE=(CYL,(15,5),RLSE),           
//            DCB=(LRECL=80,RECFM=FB,BLKSIZE=0) 
//SYSTSIN  DD *                                 
  LISTCAT LVL(DNGG00.CDPAKI)
/*                             
//**********************************************************
//* CREATE A CONCATENED DD LIST FROM LISTCAT OUTPUT        *
//* INCLUDE IN ANOTHER JCL                                 *
//* FILES WE WANT:                                         *
//*   XXXXXX.XXXXXX.C4D*.**                                *
//* EXCLUDE THE Z1 AND Z9 FILES:                           *
//*   XXXXXX.XXXXXX.C4D00Z1C.**                            *
//*   XXXXXX.XXXXXX.C4D00Z9C.**                            *
//* FORMAT THE FILES AS JCL INPUT:                         *
//* //IN       DD DISP=SHR,DSN=XXXXXX.XXXXXX.C4D*.**       *
//* //         DD DISP=SHR,DSN=XXXXXX.XXXXXX.C4D*.**       *
//* //         DD DISP=SHR,DSN=XXXXXX.XXXXXX.C4D*.**       *
//**********************************************************
//STEP0200 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=&&CAT,DISP=SHR
//SORTOUT  DD DSN=DNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR
//SYSIN    DD *
  INCLUDE COND=(01,07,CH,EQ,C'NONVSAM')
  INREC OVERLAY=(81:SEQNUM,8,ZD)
  SORT FIELDS=COPY
  OUTFIL IFOUTLEN=80,
  INCLUDE=(31,3,CH,EQ,C'C4D',AND,
           36,3,CH,NE,C'Z1C',AND,
           36,3,CH,NE,C'Z9C'),
  IFTHEN=(WHEN=(81,8,ZD,EQ,1),
   BUILD=(C'//IN',7X,C'DD DISP=SHR,DSN=',17,44,80:X)),
  IFTHEN=(WHEN=NONE,
   BUILD=(C'//',9X,C'DD DISP=SHR,DSN=',17,44,80:X))
//*
//**********************************************************
//* CREATE DYNAMIC JCL TO BE SUBMITTED VIA INTRDR          *
//**********************************************************
//STEP0300 EXEC  PGM=SORT
//SYSOUT   DD SYSOUT=*             
//SYSIN    DD *                   
   OPTION COPY                     
/*                                 
//SORTOUT  DD SYSOUT=*             
//SORTIN   DD DATA,DLM=$$         
//TIDXXXXA JOB 'COPY',             
//             CLASS=A,             
//             MSGCLASS=Y,         
//             MSGLEVEL=(1,1),     
//             NOTIFY=TID           
//*                                 
//STEP0100 EXEC  PGM=ICETOOL       
//TOOLMSG  DD SYSOUT=*             
//DFSMSG   DD SYSOUT=*             
//T1       DD DSN=&&T1,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//OUT      DD DSN=OUTPUT OF ALL FILES,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,                                 
//            SPACE=(CYL,(X,Y),RLSE)                     
//TOOLIN   DD *                                           
  SPLICE FROM(IN) TO(T1) ON(128,8,CH) -                   
  WITHALL KEEPBASE WITH(01,127) USING(CTL1)               
  SORT FROM(T1) USING(CTL2)                               
/*                                                       
//CTL1CNTL DD *                                           
  SORT FIELDS=COPY                                       
  INREC IFTHEN=(WHEN=INIT,                               
       OVERLAY=(128:SEQNUM,8,ZD)),                       
        IFTHEN=(WHEN=(1,2,CH,EQ,C'01'),                   
       OVERLAY=(128:SEQNUM,8,ZD,68,04)),                 
        IFTHEN=(WHEN=NONE,                               
       OVERLAY=(136:SEQNUM,8,ZD,                         
                128:128,8,ZD,SUB,136,8,ZD,M11,LENGTH=8)) 
  OUTFIL FNAMES=T1,                                       
  BUILD=(01,127,136,4)                                   
/*                                                       
//CTL2CNTL DD *                                           
  SORT FIELDS=(128,04,CH,A)                               
  OUTFIL FNAMES=OUT,                                     
  BUILD=(01,127)                                         
/*                                                       
$$                                                       
//         DD DNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR
/*


This will generate the JCL needeed to SORT all the files with the header indicator. Look at sortout output from step0300. Once you verify that you have created the JCL correctly, then change the following statement in step0300

Code:

//SORTOUT  DD SYSOUT=*


to
Code:

//SORTOUT  DD SYSOUT=(*,INTRDR)


Hope this helps...

Cheers
Back to top
View user's profile Send private message
sclater

New User


Joined: 22 Jun 2007
Posts: 14
Location: South Africa

PostPosted: Thu Mar 06, 2008 1:23 pm    Post subject:
Reply with quote

Hi Skolusu,

Thanks for the JCL, but something seems to be going wrong icon_sad.gif . Its putting data before the first header record. What I also can see is the incoming files have a space filled records separating every data record (don't ask me why) and with yours it sometimes has two space records between the data records and then a couple of records later it has no space record between the data records. It also seems to be changing the data records order within the file icon_sad.gif

I have also seen a increase in the CPU time when running your's compared to mine (don't really know what the rest mean icon_redface.gif )

Code:
Your JCL's
                                       --TIMINGS (MINS.)--            ----PAGING
STEPNAME PROCSTEP    RC   EXCP   CONN    TCB    SRB  CLOCK   SERV  PG  PAGE  SWA
STEP0100             00     20     14    .00    .00     .0    716   0     0     
STEP0200             00     25     20    .00    .00     .0    301   0     0     
STEP0300             00     34     34    .00    .00     .0    306   0     0     
STEPNAME PROCSTEP    RC   EXCP   CONN    TCB    SRB  CLOCK   SERV  PG  PAGE  SWA
STEP0100             00   3287  54144    .14    .00    1.4   188K   0     0     


My JCL's
                                       --TIMINGS (MINS.)--            ----PAGING
STEPNAME PROCSTEP    RC   EXCP   CONN    TCB    SRB  CLOCK   SERV  PG  PAGE  SWA
IDCLCAT              00     17     16    .00    .00     .0    348   0     0     
SORT01               00     26     22    .00    .00     .0    279   0     0     
STEPNAME PROCSTEP    RC   EXCP   CONN    TCB    SRB  CLOCK   SERV  PG  PAGE  SWA
SORT01               00   1429   5684    .01    .00     .1  16026   0     0     
STEPNAME PROCSTEP    RC   EXCP   CONN    TCB    SRB  CLOCK   SERV  PG  PAGE  SWA
DEL01                00     25     35    .00    .00     .0     29   0     0     
CONCAT1              00  69232  37657    .04    .01     .9   106K   0     0     


Attached is the sysout for the generated sort job

Ps, we are not allowed to submit JCL via intrdr in our Production enviroment as it then runs outside our scheduler. icon_cry.gif
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Mar 06, 2008 10:14 pm    Post subject:
Reply with quote

Quote:
Thanks for the JCL, but something seems to be going wrong . Its putting data before the first header record.


sclater,

I looked at your sysout and it is not exactly the same job as I have given you. You have a NEW temp dataset concatenated with all the other input datasets which is not correct.

As for the wrong results may be you haven't explained the rules of the data properly. I assumed that your Data is follows. The header record has the key of 4 bytes in pos 68


Code:

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
01                                                                 0009
DATAREC3
                                                               
DATAREC4                                                               
01                                                                 0004
DATARECA                                                               

DATARECB                                                               
01                                                                 0005
DATARECC                                                               

DATARECD                                                               
01                                                                 0006
DATAREC1                                                               

DATAREC2                                                               



Looks like your data is different from what you described. Send me first 100 records from 3 of your input files offline
Back to top
View user's profile Send private message
sclater

New User


Joined: 22 Jun 2007
Posts: 14
Location: South Africa

PostPosted: Fri Mar 07, 2008 1:14 pm    Post subject:
Reply with quote

Skolusu wrote:
I looked at your sysout and it is not exactly the same job as I have given you. You have a NEW temp dataset concatenated with all the other input datasets which is not correct.


The problem is that the generated JCL complains about it having no IN dataset, by adding an empty blank dataset I can overcome this problem the quickest.

Skolusu wrote:
As for the wrong results may be you haven't explained the rules of the data properly. I assumed that your Data is follows. The header record has the key of 4 bytes in pos 68


You are 100% correct about how you describe and expect the data, even your example is correct. What is happening is that because the second portion of the job is an actual sort job and it is only using the 4 digit institution code to sort with, the sort is reordering the sequence of the records. By making a change to the build in the splice to add a sequence number and then changing the sort to also sort on this sequence number, I get the record in the correct sequence:

Code:
//TOOLIN   DD *                                           
  SPLICE FROM(IN) TO(T1) ON(128,8,CH) -                   
  WITHALL KEEPBASE WITH(01,127) USING(CTL1)               
  SORT FROM(T1) USING(CTL2)                               
/*                                                         
//CTL1CNTL DD *                                           
  SORT FIELDS=COPY                                         
  INREC IFTHEN=(WHEN=INIT,                                 
       OVERLAY=(128:SEQNUM,8,ZD)),                         
        IFTHEN=(WHEN=(1,2,CH,EQ,C'01'),                   
       OVERLAY=(128:SEQNUM,8,ZD,68,04)),                   
        IFTHEN=(WHEN=NONE,                                 
       OVERLAY=(136:SEQNUM,8,ZD,                           
                128:128,8,ZD,SUB,136,8,ZD,M11,LENGTH=8))   
  OUTFIL FNAMES=T1,                                       
  BUILD=(01,127,136,4,SEQNUM,8,ZD)                         
/*                                                         
//CTL2CNTL DD *                                           
  SORT FIELDS=(128,04,CH,A,132,08,CH,A)                   
  OUTFIL FNAMES=OUT,                                       
  BUILD=(01,127)                                           
/*                                                         
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Fri Mar 07, 2008 10:11 pm    Post subject:
Reply with quote

sclater wrote:

The problem is that the generated JCL complains about it having no IN dataset, by adding an empty blank dataset I can overcome this problem the quickest.


You probably forgot to add the the concatenated DD after the dollar symbol. Copy my JCL as is and you should not have any problem.

Code:

/*                                                       
$$                                                       
//         DD DNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR
/*


sclater wrote:

What is happening is that because the second portion of the job is an actual sort job and it is only using the 4 digit institution code to sort with, the sort is reordering the sequence of the records. By making a change to the build in the splice to add a sequence number and then changing the sort to also sort on this sequence number, I get the record in the correct sequence:


AFAIK you don't need the additional sequence number. Looks like your Shop has NOEQUALS parm as default. Add OPTION EQUALS line in your CTL2CNTL and re run your job without the sequence number. you should get the same results.

Code:

//CTL2CNTL DD *
  OPTION EQUALS                                           
  SORT FIELDS=(128,04,CH,A)                               
  OUTFIL FNAMES=OUT,                                     
  BUILD=(01,127)                                         
/*                     
Back to top
View user's profile Send private message
sclater

New User


Joined: 22 Jun 2007
Posts: 14
Location: South Africa

PostPosted: Mon Mar 10, 2008 4:29 pm    Post subject:
Reply with quote

Skolusu wrote:
sclater wrote:

The problem is that the generated JCL complains about it having no IN dataset, by adding an empty blank dataset I can overcome this problem the quickest.


You probably forgot to add the the concatenated DD after the dollar symbol. Copy my JCL as is and you should not have any problem.

Code:

/*                                                       
$$                                                       
//         DD DNCV00.OPSUTIL0.CNTL(SAC4ALL),DISP=SHR
/*



Hmm, something weird happened, looked at the JCL and everything looked right, it should be generating the DD. Reran the first JCL and it generated the correct JCL. Granted I was messing around with different solutions all writing to SAC4ALL, so I might have overwritten it inbetween running the first and the second JCL.


Skolusu wrote:
AFAIK you don't need the additional sequence number. Looks like your Shop has NOEQUALS parm as default. Add OPTION EQUALS line in your CTL2CNTL and re run your job without the sequence number. you should get the same results.


Aha, that solves teh problem icon_biggrin.gif

Thanks for the help I have learned quite a number of new tricks, now if only the rest of my dev team could learn to use DFsort for more than just sorting a file.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Converting Variable block file to Fix... bhavana yalavarthi DFSORT/ICETOOL 10 Tue May 16, 2017 2:20 pm
No new posts Replace values in the input data Vikas Maharnawar DFSORT/ICETOOL 10 Thu May 11, 2017 2:18 pm
No new posts Unable to create multiple files using... mbattu COBOL Programming 3 Fri May 05, 2017 5:35 pm
No new posts Getting error while opening a variabl... apandey1 COBOL Programming 5 Fri May 05, 2017 12:22 pm
No new posts SMTP the current generation number of... Jyothi Kulunde JCL & VSAM 4 Thu May 04, 2017 4:08 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us