|
View previous topic :: View next topic
|
| Author |
Message |
Vinodh S
New User
Joined: 12 Apr 2012
Posts: 28
Location: LA, California
|
|
|
|
Hi
I have a tape file and its huge. Is there a way in SORT to identify if a file is sorted.
Say I want to check if the file is sorted by the position 1 to 10. |
|
| Back to top |
|
 |
Vinodh S
New User
Joined: 12 Apr 2012
Posts: 28
Location: LA, California
|
|
|
|
In the JOINKEYS statement when the file is already sorted, we would give the SORTED Keyword if the file is already sorted. And if the records are not in the sorted order, it Would terminate.
I'm just looking for something like this. Is there any function in SORT that would say whether the file is already SORTED or not. |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
since to determine if the file is sorted You might have to read all the records
there is no advantage in doing it.
just write Jour Joinkeys step assuming that the file is not sorted.
anyway the way to do it is to write a fake MERGE |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
| Ask the owner of the dataset, he/she should know if the file is sorted at creation or in some other job where it is used. Using dataset naming conventions (like jobname, ddname etc. in the dsn). And for the rest what Enrico says. |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
| Quote: |
| In the JOINKEYS statement when the file is already sorted, we would give the SORTED Keyword if the file is already sorted. And if the records are not in the sorted order, it Would terminate. |
hit enter too early ...
do You need the joinkey result or not ...
if You need it just assume the worst
if You do not.. why waste resources checking
or ... fix the whole process to start with |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
.jpg)
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
Add a sequence number say if lrecl is 80 add a sequnce number at 81 and Do a sort on 1,10
Add a sequence number at 91 and check if value from 81-90 = 91-100
If they are not equal the for all records or any first hit then the file is not sorted initially |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
Pandora don' t You think that Your suggestion is a bit too complicated ?  |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
I guess it is considering the tape it is even more 
But what TS could also do is if he s not sure about the data in Tape he could just sort it before processing |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
Whenever I read a file in "key order" I check that it is in "key order": along with the rest of the program, there is little overhead in this compared to the knowledge that the file is OK.
To do this "standalone", as enrico suggested:
| Code: |
MERGE FIELDS=(key-to-be-verified)
//DD SORTIN01 DD your datatest
//DD SORTOUT DD DUMMY or NULLFILE
|
The step will fail if not in sequence. |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
IMO, this kind of questions are just a waste of everybody' s time
see my previous questions and comments
once a process has passed all the acceptance tests no need to check for anything
everybody is allowed to assume that the process works as <certified> ,
if at some point it does not ... time to kick some ass |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
| Thanks Bill and Enrico |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
and to stick deeper the knife into the wound ...
these "trucs et astuces" should be part of the tools chest of any testing team
( have been for ages in any reasonably organized shop )
checking if a file is properly ordered is part of the basics test usually present in most of the checklists |
|
| Back to top |
|
|
Vinodh S
New User
Joined: 12 Apr 2012
Posts: 28
Location: LA, California
|
|
|
|
| Bill Woodger wrote: |
Whenever I read a file in "key order" I check that it is in "key order": along with the rest of the program, there is little overhead in this compared to the knowledge that the file is OK.
To do this "standalone", as enrico suggested:
| Code: |
MERGE FIELDS=(key-to-be-verified)
//DD SORTIN01 DD your datatest
//DD SORTOUT DD DUMMY or NULLFILE
|
The step will fail if not in sequence. |
Thanks Bill. I'll use this to test if my file is already sorted.
The reason I posted this ques is, I have a step in a job where multiple files are concatenated in the SORTIN DD and written out to a VSAM file. The step uses the SORT FIELDS=(1,10,CH,A).
I was thinking if the Files were are already sorted, I could use the Merge command instead of the SORT Command so that the performance of the step would increase. So in order to test if the files are already sorted by the required field I couldn't think of a direct method to do so.
PS: Yes I can always look at the Jobs where the files are getting created and check how the file would be Sorted. But there are many files to reach out. Anyway after checking the files with the idea BILL has mentioned, I Would look onto how the files are created. |
|
| Back to top |
|
|
Akatsukami
Global Moderator
Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL
|
|
|
|
| If in fact all the data sets are concatenated under SORTIN, then for this purpose they can be considered a single data set; not only must each data set itself be sorted, but the last key in the nth data set must be no higher than the first key in the n+1th. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| True for SORT, but for MERGE each has its own DD. A MERGE of a lot of data is going to consume fewer resources than a SORT of the same data. Worth spending the time to check, and even arranging for them all to be in order, if some already are. |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
| Quote: |
| I was thinking if the Files were are already sorted, I could use the Merge command instead of the SORT Command so that the performance of the step would increase. So in order to test if the files are already sorted by the required field I couldn't think of a direct method to do so. |
did You read my previous comments about the correctness of the process
look at the process documentation/specifications and if it tells that each datasets is in the proper sequence just use a merge
there is a bit of miscommunication around here or You are looping around
|
|
| Back to top |
|
|
Gary McDowell
Active User
Joined: 15 Oct 2012
Posts: 139
Location: USA
|
|
|
|
How huge is this tape file? - i.e. number of records, lrecl, blksize?
If this truly is huge, copy the first x-amount of records and see if they are in sorted order. Not full-proof of course but quick and relatively reliable.
Why make this so complicated? |
|
| Back to top |
|
|
Gary McDowell
Active User
Joined: 15 Oct 2012
Posts: 139
Location: USA
|
|
|
|
| ...or copy every n-th record for x-amount and see if in sorted order. |
|
| Back to top |
|
|
|
|
