Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Extracting fields from a tape file using SYNCSORT
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Thu Jan 21, 2010 10:16 am    Post subject: Extracting fields from a tape file using SYNCSORT
Reply with quote

Hi,

I have to create a new job which should extract details from a tape file. I do not need all the records & fields from the tape. I am using the below sort

Code:
SORT FIELDS=(1,3,CH,A)
INCLUDE COND=(1,9,CH,LT,C'079')
INREC FIELDS=(....)


The tape file size is 10.28GB. I have used five sortwk files with space as(CYL,(1500,200). But, when i have checked the syslog, no sort work files are used. Also, the job takes more than 2 hours.

I have also tried MAXSORT parameter with JCL as below, but it ran for 4 hours. I might have coded something wrong here.

Code:


//STEP01 EXEC PGM=SORT,PARM='MAXSORT,MAXWKSP=MAX'
//SORTIN DD DSN=TAPE.FILE,DISP=SHR
//SORTOUT DD DSN=DASD.FILE,DISP=SHR (cataloged before)
//SORTBKPT DD DSN=BKDSN.FILE,DISP=(OLD,KEEP) (Cataloged before)
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(1500,100))
...
//SORTWK10 DD UNIT=SYSDA,SPACE=(CYL,(1500,100))
//SORTOU01 DD DSN=&&DSN1,DISP=(NEW,DELETE),
//              SPACE=(CYL,(500,200),RLSE)
...
//SORTOU05 DD DSN=&&DSN5,DISP=(NEW,DELETE),
//             SPACE=(CYL,(500,200),RLSE)
//SYSIN DD *
SORT FIELDS=(1,3,CH,A)
INCLUDE COND=(1,9,CH,LT,C'079')
INREC FIELDS=(....)
/*
//*


Kindly advice me whether there is a chance to reduce the run time. I have only syncsort. Any suggestions to reduce the job run time will be helpful.
Back to top
View user's profile Send private message

dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jan 21, 2010 10:39 am    Post subject:
Reply with quote

Hello,

Quote:
I have only syncsort.
Which is very fast. . .

What tape media is being used? How many tape volumes are read? You might look at the syslog and see how much tiimes was spent waiting on tape mounts.

What is the lrecl for the data and how many records are there that will be "included"?
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu Jan 21, 2010 11:00 am    Post subject:
Reply with quote

do they need to be sorted? instead of COPY?
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Thu Jan 21, 2010 12:15 pm    Post subject:
Reply with quote

LRECL is 1000. No of records is 11043008 (calculated with formula (block-size * number of blocks)/lrecl).

The media type is 36tk (Obtained from TLMS tape inquiry). The extracted output file contains about 11000 records with LRECL as 400.

If Sort fields=copy will reduce the run time, it will be fine to have another step to sort the disk file (will not take more time).
Back to top
View user's profile Send private message
Anuj Dhawan

Senior Member


Joined: 22 Apr 2006
Posts: 6258
Location: Mumbai, India

PostPosted: Thu Jan 21, 2010 3:48 pm    Post subject:
Reply with quote

Quote:
But, when i have checked the syslog, no sort work files are used. Also, the job takes more than 2 hours.
As Dick has asked about copy (probably with the same intentions as mine - and my thoughts are) - sort product don't use sortwork files in "copy-operation" - those 2-hrs might be related to tape mountings or i-o operations, so if sorting is not required just 'option copy".

Suggest you get in touch with Alissa, syncsort point of contact in this forum - she might be able to help you better, as she supports this product.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Jan 21, 2010 4:02 pm    Post subject:
Reply with quote

Hi,

changing the option to copy should not make much difference considering it's only sorting 11000 records with an LRECL of 400.


Gerry
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Jan 21, 2010 4:10 pm    Post subject:
Reply with quote

Hi,

is it possible to show the output from run ?


Gerry
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Thu Jan 21, 2010 4:31 pm    Post subject:
Reply with quote

I guess your input blocksize=lrecl=1000.
What is your output blocksize?
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Jan 21, 2010 4:36 pm    Post subject:
Reply with quote

Hi Peter,

I'd be more concerned with the input blocksize, that's why I asked for the output from the run.


Gerry
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Thu Jan 21, 2010 4:43 pm    Post subject:
Reply with quote

gcicchet wrote:
Hi Peter,

I'd be more concerned with the input blocksize, that's why I asked for the output from the run.


Gerry


Hello Gerry,

I mentioned my doubt about the input blksize in my comment.
So im quite sure the output is also blksize=lrecl=400
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Thu Jan 21, 2010 5:48 pm    Post subject:
Reply with quote

Quote:
As Dick has asked about copy (probably with the same intentions as mine - and my thoughts are) - sort product don't use sortwork files in "copy-operation" - those 2-hrs might be related to tape mountings or i-o operations, so if sorting is not required just 'option copy".

I have used only sort fields=(1,9,ch,a). But still, it has not used the sortwork files.
Quote:
I guess your input blocksize=lrecl=1000.
What is your output blocksize?

Input file block size is 32000.

For some reason, here there is no convention to code blocksize along with DCB parameter for output files. it will be left for the system to decide automatically. I have verified the dataset information, it says 27939 & lrecl=417.

Quote:
is it possible to show the output from run ?

Do you mean by output file? it is difficult to get that.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jan 21, 2010 9:05 pm    Post subject:
Reply with quote

Hello,

Not the data file - just the informational sysout data generated by the run. It is possibly still in the spool. . .

It may also help if you post the complete jcl and control statements.

For the sysout data and the jcl/control statements, please use the "Code" tag for readability.

How many tape volumes were mounted to read the data?
Back to top
View user's profile Send private message
Anuj Dhawan

Senior Member


Joined: 22 Apr 2006
Posts: 6258
Location: Mumbai, India

PostPosted: Thu Jan 21, 2010 9:29 pm    Post subject:
Reply with quote

kbmk wrote:
I have used only sort fields=(1,9,ch,a). But still, it has not used the sortwork files.
(An un-tested assumption is)- may be your file is already sorted on those keys then...

Well, please post the sysout as asked many times - otherwise my empty mind will keep on adding clutters here... (since some days it's just behaving as evil's workshop...hmmmm)
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Fri Jan 22, 2010 8:39 am    Post subject:
Reply with quote

Sorry, my spool output logs are erased. I have to run the job again. I will provide the SYSOUT today.
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Fri Jan 22, 2010 8:02 pm    Post subject:
Reply with quote

sysout data
Code:
************************************************************************************************************************************
 * STEP |   STEP   | PROGRAM  |          TIME IN SECS          |   VIRT STORAGE IN K   |  COMP  |           SERVICE UNITS           *
 *  NO  |   NAME   |   NAME   |  ELAPSED    CPU SRB    CPU TCB |REQUSTD   USED   OVRHD |  CODE  |  STEP     CPU      IO       MSO   *
 ************************************************************************************************************************************
 * 002  | S01      | SORT     |  7145.81       3.15      49.34 |0131072 0003080  00264 |  0000  | 1247776  8273907  1808667        0*
 ************************************************************************************************************************************
 * DDNAME   EXCPS  | DDNAME   EXCPS  | DDNAME   EXCPS  | DDNAME   EXCPS  | DDNAME   EXCPS  | DDNAME   EXCPS  | DDNAME   EXCPS  *
 *SORTIN      55622|SORTOUT         0|SYSOUT          0|SYSIN           0|SORTIN      74168|SORTOUT         0|SYSOUT          0*
 *SYSIN           0|SORTIN     101300|SORTOUT         0|SYSOUT          0|SYSIN           0|SORTIN     102478|SORTOUT         0*
 *SYSOUT          0|SYSIN           0|SORTIN      27773|SORTOUT         1|SYSOUT          0|SYSIN           0|                 *
 *******************************************************************************************************************************

 IEF373I STEP/S01     /START 2010021.1438
 IEF374I STEP/S01     /STOP  2010021.1637 CPU    0MIN 49.34SEC SRB    0MIN 03.15SEC VIRT  3080K SYS   264K EXT    6384K SYS   11504K

 *********************************************************************************************************
 *          |          TIME IN SECS          |                  SERVICE UNITS                  |         *
 * JOBNAME  |  ELAPSED    CPU SRB    CPU TCB |   JOB        CPU        IO       MSO       SRB  | SYS ID  *
 *********************************************************************************************************
 * SORT1234 |  7146.13       3.15      49.49 | 1304995   8331014   1808727         0   1165254 |  SYSD   *
 *********************************************************************************************************

 IEF375I  JOB/SORT1234/START 2010021.1438
 IEF376I  JOB/SORT1234/STOP  2010021.1637 CPU    0MIN 49.49SEC SRB    0MIN 03.15SEC



JCL for the above sysout
Code:
//STEP EXEC PGM=SORT,PARM='IO'
//SORTIN DD DSN=INPUT.TAPE.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,
//      DISP=(NEW,CATLG),
//     UNIT=TESTDA,SPACE=(CYL,(50,5),RLSE)
//SYSIN DD *
 SORT FIELDS=COPY
 INCLUDE COND=(....)
 INREC FIELDS=(....)
/*
//*


The input tape file has 11403008 records with lrecl as 1000 (total of 17 volumes). The output file has 985 records with lrecl as 100 in this case (Because of INREC statement). It took almost 2 hours to complete.

Actually, the output of the above file is used to sum up the fields in a particular position. I have tried to combine both these two steps below.

I have submitted another job with sort fields=(1,9,ch,a) and sum fields=(54,6,zd). Previously when I submit the jobs with DYNALLOC=32 parameter, no sort work files are used. The output contains just 29 records. this job is now running. i will post the details once it is completed.

Kindly advise me how to reduce the job run time.
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Fri Jan 22, 2010 8:13 pm    Post subject:
Reply with quote

2 hours for 17 volumes is not that long (7 minutes a volume), taking in account the time needed for mounting/demounting, sleeping tape operators (if they still exist).
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7913
Location: Bellevue, IA

PostPosted: Fri Jan 22, 2010 8:16 pm    Post subject:
Reply with quote

Quote:
sleeping tape operators (if they still exist).
usually on third shift. icon_biggrin.gif
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10202
Location: italy

PostPosted: Fri Jan 22, 2010 8:20 pm    Post subject: Reply to: Extracting fields from a tape file using SYNCSORT
Reply with quote

Hard to do, I would say that it' s all wait time due to tape handling
also as far as the copy/sort debate it is a non issue
a properly written sort step will pass to the sort process only the selected/included records ,
1000 records will certainly be handled in main storage without the need of workfiles
no reason to split in multiple steps
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Fri Jan 22, 2010 8:21 pm    Post subject:
Reply with quote

Yes Robert, and in day time shifts drinking coffee or reading papers. With the paper lying over the next reel and turning the tape vault over to find it.
Back to top
View user's profile Send private message
kbmk

New User


Joined: 27 Sep 2007
Posts: 24
Location: Chennai

PostPosted: Fri Jan 22, 2010 8:22 pm    Post subject:
Reply with quote

Robert Sample wrote:
Quote:
sleeping tape operators (if they still exist).
usually on third shift. icon_biggrin.gif


Ok. Thanks. But, I want to understand why sort work files are not being used when I am using DYNALLOC=32 with SORT FIELDS=(1,9,CH,A)

Any help in this regard?
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts syncsort: copy lines after the keyword shreya19 SYNCSORT 7 Fri Dec 02, 2016 9:47 am
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm
No new posts File Aid to File Manager conversion murali3955 IBM Tools 4 Thu Nov 24, 2016 3:41 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts CICS Roll back partially - Need to re... dwijadas CICS 4 Wed Nov 16, 2016 4:30 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us