IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

How to dynamically decide which one to terse and not?


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
javen777

New User


Joined: 06 Mar 2015
Posts: 31
Location: china

PostPosted: Wed Dec 19, 2018 8:50 am
Reply with quote

Dear friends,

Currently i am going to implement a new function in system.
Which is to receive upstream data files and process them necessarily.

New Function Process Flow:
1.upstream files are generated in our MVS system.
2. Once each file is generated, it will trigger 1 OPC automatically to :
1) unterse the file if it was tersed, if not tersed then skip the unterse step.
2) After 1) , rename the file itself to our naming standard.


upstream file names are like:
FDCAP.WE.RECEIVED.W401F01.TERSED.FTP (terse file has TERSED in qualifiers)
FDCAP.PROD.D0000000.EXTFEED.G101F05.FTP
FDCAP.I.P1.FMSDC.G101.FTP
FDCAP.WS.W401F03.FEED

our naming standard:
TFMS.F.DD.USS.<SYSIDSRCID>

for example:
FDCAP.PROD.D0000000.EXTFEED.G101F05.FTP - >TFMS.F.DD.USS.G101F05
FDCAP.WS.W401F03.FEED ->TFMS.F.DD.USS.W401F03

Could you guys plz help on this?

Thank you very much
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Dec 19, 2018 9:12 am
Reply with quote

amaterse was never supposed to be a general use use utility,
it is not even described in the utilities reference manual

it is a service aid used to exchange information with the IBM support organisation

wiser to investigate a different approach

Quote:
unterse the file if it was tersed, if not tersed then skip the unterse step.

and anyway how are you going to determine if a dataset is tersed ?
Back to top
View user's profile Send private message
javen777

New User


Joined: 06 Mar 2015
Posts: 31
Location: china

PostPosted: Wed Dec 19, 2018 10:51 am
Reply with quote

enrico-sorichetti wrote:
amaterse was never supposed to be a general use use utility,
it is not even described in the utilities reference manual

it is a service aid used to exchange information with the IBM support organisation

wiser to investigate a different approach

Quote:
unterse the file if it was tersed, if not tersed then skip the unterse step.

and anyway how are you going to determine if a dataset is tersed ?


unfortunately files are from upstream which are out of our control, otherwise it would not be a problem.

If a dataset is a tersed one, in dataset name there would be a qualifier as 'TERSED'

e.g. FDCAP.WE.RECEIVED.W401F01.TERSED.FTP
Back to top
View user's profile Send private message
steve-myers

Active Member


Joined: 30 Nov 2013
Posts: 917
Location: The Universe

PostPosted: Wed Dec 19, 2018 10:54 am
Reply with quote

About the only semi reliable way to deduce if a file represents a class of main frame data set is to look at its data length.

Multiple of -
  • 1024 TERSE data set
  • 4160 SYS1.DUMPxx data set
Back to top
View user's profile Send private message
javen777

New User


Joined: 06 Mar 2015
Posts: 31
Location: china

PostPosted: Wed Dec 19, 2018 11:21 am
Reply with quote

steve-myers wrote:
About the only semi reliable way to deduce if a file represents a class of main frame data set is to look at its data length.

Multiple of -
  • 1024 TERSE data set
  • 4160 SYS1.DUMPxx data set


thanks Steve, but what if a non-tersed file length is also 1024?
Back to top
View user's profile Send private message
steve-myers

Active Member


Joined: 30 Nov 2013
Posts: 917
Location: The Universe

PostPosted: Wed Dec 19, 2018 1:03 pm
Reply with quote

I did not say the method is 100% reliable. I just said it's the only method I know of that can serve as a guide. Mainframe data sets containing tersed data are always RECFM=FB, LRECL=1024
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1306
Location: Vilnius, Lithuania

PostPosted: Wed Dec 19, 2018 1:24 pm
Reply with quote

There is no, no, no, no (repeated ad infinitum) 100% reliable way other than to use AMATERSE to (try to) UNPACK the dataset. Just as I can create a dataset that seems to be ISPF PACKED by editing the first few bytes in HEX ON (and making it un-openable with the ISPF editor once it's saved), I can do the same with an FB/LRECL(1024) dataset, where the first 8(?) bytes of the first record contain information AMATERSE needs to properly UNPACK the dataset.

And if a Terse-Bomb can be made, UNPACKing an unknown file that looks like a TERSE'd one is not a good thing...
Back to top
View user's profile Send private message
javen777

New User


Joined: 06 Mar 2015
Posts: 31
Location: china

PostPosted: Wed Dec 19, 2018 1:29 pm
Reply with quote

prino wrote:
There is no, no, no, no (repeated ad infinitum) 100% reliable way other than to use AMATERSE to (try to) UNPACK the dataset. Just as I can create a dataset that seems to be ISPF PACKED by editing the first few bytes in HEX ON (and making it un-openable with the ISPF editor once it's saved), I can do the same with an FB/LRECL(1024) dataset, where the first 8(?) bytes of the first record contain information AMATERSE needs to properly UNPACK the dataset.

And if a Terse-Bomb can be made, UNPACKing an unknown file that looks like a TERSE'd one is not a good thing...


how about writing a REXX to deside whether it's a tersed file? icon_wink.gif

I just don't know how
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1306
Location: Vilnius, Lithuania

PostPosted: Wed Dec 19, 2018 1:42 pm
Reply with quote

Are you stupid, or just pretending? What can't you understand about my previous reply that tells you that only, only, only (repeat ad infinitum) AMATERSE can decide that a file is validly tersed.

And what is "a REXX"? The same as "a COBOL", or "a PL/I"?
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1353
Location: Israel

PostPosted: Thu Dec 20, 2018 9:22 pm
Reply with quote

You can know if a data set is tersed only by looking at its name:

javen777 wrote:
upstream file names are like:
FDCAP.WE.RECEIVED.W401F01.TERSED.FTP (terse file has TERSED in qualifiers)


javen777 wrote:
If a dataset is a tersed one, in dataset name there would be a qualifier as 'TERSED'

e.g. FDCAP.WE.RECEIVED.W401F01.TERSED.FTP

So yes, your new process flow could run a small REXX program that would get the dataset name and find if AMATERSE is needed.
This REXX program can:
1. run AMATERSE by itself, or
2. terminate with a return code (followed by AMATERSE in a step conditioned by the return code).
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Fri Dec 21, 2018 4:26 am
Reply with quote

No it wouldn't, the host name depends on the transmit command. I have uploaded AMATERSEd files to various zos dataset names.
What you can do is (as suggested above) is to
1. If recfm/lrecl if F(B)/1204 run a AMATERSE step to a dummy output dataset.
2. If RC=0 then run a AMATERSE step to a permanent output dataset.
As stated several times above there is no way you can determine if a dataset is AMATERSEd from just looking at the dataset specs and the data, period.
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Fri Dec 21, 2018 4:48 am
Reply with quote

Come to think about it, skip the test for recfm/lrecl, just run an AMATERSE step to a dummy output dataset and if RC=0 then run a AMATERSE step to a permanent output dataset.
Back to top
View user's profile Send private message
javen777

New User


Joined: 06 Mar 2015
Posts: 31
Location: china

PostPosted: Fri Dec 21, 2018 7:13 am
Reply with quote

Willy Jensen wrote:
Come to think about it, skip the test for recfm/lrecl, just run an AMATERSE step to a dummy output dataset and if RC=0 then run a AMATERSE step to a permanent output dataset.


Thank you very much Willy!
Back to top
View user's profile Send private message
steve-myers

Active Member


Joined: 30 Nov 2013
Posts: 917
Location: The Universe

PostPosted: Fri Dec 21, 2018 9:45 am
Reply with quote

Willy Jensen wrote:
Come to think about it, skip the test for recfm/lrecl, just run an AMATERSE step to a dummy output dataset and if RC=0 then run a AMATERSE step to a permanent output dataset.

Oh, you run AMATERSE twice! Granted it's effective and reliable, but for a large data set, it's awfully slow! I'd screen using RECFM=F,LRECL=1024 first, and only run AMATERSE if it passes the screening.
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Fri Dec 21, 2018 3:13 pm
Reply with quote

I did a quick test and found that AMATERSE would very quickly determine if the dataset specs were wrong, or the start of the dataset was not right. In fact I would expect the AMATERSE check to be faster than starting a REXX program to check the dataset specs. Then again, I could be wrong. One advantage with a REXX check is that you could run the AMATERSE from the REXX, thus eliminating the 2nd step.
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Fri Dec 21, 2018 3:39 pm
Reply with quote

Sorry, I see what you mean now. Well, the 2nd step could be an IEBGENER from the temporary dataset, which would be much faster than AMATERSE. My suggestion was because it had earlier been mentioned that a FB/1024 dataset is not neccessarily an AMATERSEd datataset, so you would have to run AMATERSE to determine if it is.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Fri Dec 21, 2018 5:05 pm
Reply with quote

the TS already clarified that ...

Quote:
If a dataset is a tersed one, in dataset name there would be a qualifier as 'TERSED'

e.g. FDCAP.WE.RECEIVED.W401F01.TERSED.FTP



time to lock the topic
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts Dynamically pass table name to a sele... DB2 2
No new posts FD Section to Create FB or Vb File Dy... COBOL Programming 1
No new posts How to replace a string dynamically i... CLIST & REXX 16
No new posts Dynamically switching terminal model TSO/ISPF 9
No new posts Dynamically read input files using BP... COBOL Programming 23
Search our Forums:

Back to Top