Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Is Concatenation of files essential
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Fri Jun 27, 2008 4:03 pm    Post subject: Is Concatenation of files essential
Reply with quote

Hi,

I want to search for a particular expression in a group of 10 files. Is it essential for me to concatenate all the files before proceeding? The occurence is same in all the files. I am using sort to find the expression.
Back to top
View user's profile Send private message

Garry Carroll

Active Member


Joined: 08 May 2006
Posts: 990
Location: Dublin, Ireland / Edinburgh, Scotland

PostPosted: Fri Jun 27, 2008 4:26 pm    Post subject:
Reply with quote

Concatenate to process in one step or else process each file separately, I'd think. Also, depends on what you mean by
Quote:
before proceeding

Do you mean concatenate in one step and then process in another? If so, that shouldn't be necessary.


Garry.
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10203
Location: italy

PostPosted: Fri Jun 27, 2008 4:28 pm    Post subject: Reply to: Is Concatenation of files essential
Reply with quote

the main issue here is that the concatenation will be seen as a single file
and You will lose track of the file where the search argument will be located
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Fri Jun 27, 2008 4:29 pm    Post subject:
Reply with quote

By
Quote:
Before proceeding
i meant to say, that before i proceed to search for the expression, shall i concatenate the files?

Each file is having a huge bulks of records due to which the job is abending with SE37 space abend error.

I am providing the space parameter as

SPACE=(CYL,(4200,4200),RLSE)
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Fri Jun 27, 2008 4:30 pm    Post subject:
Reply with quote

So, Enrico you suggest me to search individual files instead of concatenating them?
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10203
Location: italy

PostPosted: Fri Jun 27, 2008 4:35 pm    Post subject: Reply to: Is Concatenation of files essential
Reply with quote

all depends on what the result of the search should be

I would go for
single file to understand data occurrence patterns
concatenated files to extract records for further processing
Back to top
View user's profile Send private message
Craq Giegerich

Senior Member


Joined: 19 May 2007
Posts: 1512
Location: Virginia, USA

PostPosted: Fri Jun 27, 2008 5:05 pm    Post subject:
Reply with quote

Is the results of the search a large number of records? Are you sorting the input file before selecting the records?
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Fri Jun 27, 2008 5:23 pm    Post subject:
Reply with quote

The Input files are sorted and the output file is not having large number of records when compared with the number of records in the Input files.
Back to top
View user's profile Send private message
Phrzby Phil

Active Member


Joined: 31 Oct 2006
Posts: 961
Location: Richmond, Virginia

PostPosted: Fri Jun 27, 2008 5:38 pm    Post subject:
Reply with quote

SAS note: SAS provides a simple way of knowing which file (by position number in the list) of a concatenation you are in:

EOV=variable
names a variable that SAS sets to 1 when the first record in a file in a series of concatenated files is read. The variable is set only after SAS encounters the next file. Like automatic variables, the EOV= variable is not written to the data set.
Tip: Reset the EOV= variable back to 0 after SAS encounters each boundary.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Jun 27, 2008 8:27 pm    Post subject:
Reply with quote

Hello,

Quote:
Each file is having a huge bulks of records due to which the job is abending with SE37 space abend error.
I'm confused. . .

If you read millions of records and select a couple of hundred records matching the "expression", how does the space abend occur? Reading should not require "space" icon_confused.gif

You mention "sort", but you might not need to sort the data if you are simply scanning the file for "the expression".
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Sat Jun 28, 2008 6:38 am    Post subject:
Reply with quote

Hi,
you can still sort the file but as long as use the include/omit then the sort will only apply to the selected records.


Gerry
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Thu Jul 03, 2008 5:02 pm    Post subject:
Reply with quote

Actually, I was trying to merge the files using SYNCSORT. But the job got abended. The sysout message that I received is as under:

Code:

ALLOCATION HAS FAILED FOR ALL VOLUMES SELECTED FOR DATA SET           
&NAME..CLAIMDEC.OUT                                             
THERE ARE (732) CANDIDATE VOLUMES OF WHICH (571) ARE ENABLED OR QUIESCED
THERE WERE 1 CANDIDATE STORAGE GROUPS OF WHICH THE FIRST 1           
WERE ELIGIBLE FOR VOLUME SELECTION.                                             
THE CANDIDATE STORAGE GROUPS WERE:S1LRG90 S2LRG90 SGDISTPU                     
161 VOLUMES WERE REJECTED BECAUSE THE SMS VOLUME STATUS WAS DISABLED 
571 VOLUMES WERE REJECTED BECAUSE THEY DID NOT HAVE SUFFICIENT SPACE (041A041D)
SMS ALLOCATED TO DDNAME (SORTOUT )                         
DSN (T3709BTX.G3709T01.TBP.CLAIMDEC.OUT          )         
STORCLAS (STANDARD) MGMTCLAS (MCSTNDRD) DATACLAS (DCDEFLT)
VOL SER NOS= SML1TE                                       


Can any one please help me out in regards to the same.
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Thu Jul 03, 2008 5:03 pm    Post subject:
Reply with quote

The job that I am using is as under:
Code:

//SORT01   EXEC PGM=SORT                                             
//*                                                                 
//SYSPRINT DD   SYSOUT=*                                             
//SYSOUT   DD   SYSOUT=*                                             
//*                                                                 
//SORTIN01  DD DISP=SHR,DSN=&NAME..DEC2005.CMI
//SORTIN02  DD DISP=SHR,DSN=&NAME..DEC2005.CMI
//SORTIN03  DD DISP=SHR,DSN=&NAME..DEC2005.CMI
//SORTOUT   DD DSN=&NAME..CLAIMDEC.OUT,                             
//             DISP=(NEW,CATLG,CATLG),                               
//             DCB=(RECFM=FB,LRECL=5200,BLKSIZE=0,DSORG=PS),         
//             UNIT=3390,SPACE=(CYL,(4200,4200),RLSE)               
//SYSIN     DD DISP=SHR,DSN=&NAME..PULL.SRUNJCL(XXXXXXXX)   
//*                                                                 
Back to top
View user's profile Send private message
Garry Carroll

Active Member


Joined: 08 May 2006
Posts: 990
Location: Dublin, Ireland / Edinburgh, Scotland

PostPosted: Thu Jul 03, 2008 5:08 pm    Post subject:
Reply with quote

This problem is something you should take up with your storage administrators. You are failing to get sufficient space allocated via SMS.

Garry.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7914
Location: Bellevue, IA

PostPosted: Thu Jul 03, 2008 5:09 pm    Post subject:
Reply with quote

Typical 3390 mod 3 disk packs have 3335 cylinders of space. Asking for 4200 cylinders requires a mod 9 or higher. From the output messages, the storage pool you're asking for space from either (1) has no mod 9 devices defined, or (2) has no device with 4200 cylinders available in 5 extents or less.

Cut the space allocation to 3335 cylinders or less, and possibly use more volumes to get the same amount of space.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jul 03, 2008 9:07 pm    Post subject:
Reply with quote

Hello,

Are there multiple questions in this topic?

Quote:
the output file is not having large number of records when compared with the number of records in the Input files.
How much output. . .?

What is being merged that has to do with this:
Quote:
I want to search for a particular expression in a group of 10 files


If there are multiple different questions, you need to use multiple topics. We can help separate these if it is multiple issues. If these are really the same requirement, a sngle approach needs to be worked thru rather than a "shotgun" approach.

From what i see, i'm only confused icon_confused.gif
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Fri Jul 04, 2008 9:17 am    Post subject:
Reply with quote

Hi Dick,

This is a single requirement.

The input files(which are 30 in #) are in tapes. The number of records in each input file is around 100million. I am required to pick up particularly 1800 records from these number of input files.

For this reason previously I was trying to concatenate the files.

Can you guide me through a good approach.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Jul 04, 2008 9:45 am    Post subject:
Reply with quote

Hello,

Will there be 1800 total records selected or will there be 1800 records per input tape selected?

1800 5k records is not a terribly high volume. . . You still might talk with your storage management peole and ask what is the proper device and/or dataclas for your output files. They will best know the storage configuration on yhour system.

How long does it take to read one of these input tapes? Without knowing much about your data and your environment, my first approach would probably be to split the task nto 5 or even 10 jobs and create a smaller selected output from each. Once the separate jobs successfully complete, the selected files could be copied back to one tape file or used in-place on dasd until this requirement to use the data is met and then backed up.

Again, coordinating with the storage management people should help.
Back to top
View user's profile Send private message
swapnadeep.ganguly

Active User


Joined: 21 Mar 2007
Posts: 203
Location: India

PostPosted: Fri Jul 04, 2008 10:31 am    Post subject:
Reply with quote

Thank you Dick for your suggestion.

Is there any method by which we can restore the Tape datasets into DASD?
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Fri Jul 04, 2008 10:46 am    Post subject:
Reply with quote

Hi,
why would you need to copy the tape datasets to disk, you still need to read all of the data and use up lots of disk space.


Gerry
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts High CPU consumption Job using IAM fi... aswinir JCL & VSAM 8 Thu Dec 01, 2016 8:28 pm
No new posts Match or compare two files in VB Format anatol DFSORT/ICETOOL 14 Thu Nov 03, 2016 7:41 pm
This topic is locked: you cannot edit posts or make replies. How to use 2 input files in control c... Gunapala CN DFSORT/ICETOOL 23 Thu Oct 13, 2016 3:42 pm
No new posts Adding records from two files into on... shiitiizz SYNCSORT 4 Mon Sep 19, 2016 8:41 pm
No new posts Icetool or Joinkeys example to compar... ishant chauhan DFSORT/ICETOOL 21 Sat Aug 20, 2016 2:40 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us