|
View previous topic :: View next topic
|
| Author |
Message |
veeramanimurugesan
New User
Joined: 03 Nov 2009
Posts: 19
Location: chennai
|
|
|
|
Hi,
Need some assistance with the below.
| Code: |
Input File :
Acc No Balance
A 0
A 0
A 0
B 10
B 0
C 0
D 0
D 0
E 0
E 100
E 1000
Required output file :
Acc No Balance
A 0
A 0
A 0
C 0
D 0
D 0 |
I want to select that group of accounts which does not have any balance (zero) in all occurences. I tried using SUM FIELDS but that would give me only one record whereas I need the whole group in my output file. |
|
| Back to top |
|
 |
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales
|
|
|
|
| Please post the code that you are using, also are the figures really left justified |
|
| Back to top |
|
|
veeramanimurugesan
New User
Joined: 03 Nov 2009
Posts: 19
Location: chennai
|
|
|
|
expat,
Yes the figures are left justified. The field is CHAR in the file though its an Amount field.
I tried the below but it gives me only 1 record of the group whereas i need all records of zero amount groups.
SORT FIELDS=(Account Number)
SUM FIELDS=(Amount,ZD) |
|
| Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales
|
|
|
|
| Then you need to use the INCLUDE COND= rather than the SUM FIELDS statement |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
.jpg)
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
Please refer to manual
the SORT manual
Assuming your field is numeric
| Code: |
//SORTIN DD *
A 0
A 0
A 0
B 10
B 0
C 0
D 0
D 0
E 0
E 100
E 1000
/*
//SORTOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=COPY
INCLUDE COND=(15,4,ZD,EQ,0)
|
|
|
| Back to top |
|
|
veeramanimurugesan
New User
Joined: 03 Nov 2009
Posts: 19
Location: chennai
|
|
|
|
INCLUDE COND would not work because that would work on a record level but my checking needs to be at a GROUP level. That is, account groups with absolutely zero balances.
If i use INCLUDE then i would get account B & E in my output as 1 record of it has zero balance but it would be incorrect because another record for same accounts have balances.
Please advise. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| From the SUM you can end up with a file of "keys" where you include only the zero values. Then take the file of keys and match to the original input file. Don't know if it can be done in one step. |
|
| Back to top |
|
|
sqlcode1
Active Member
Joined: 08 Apr 2010
Posts: 577
Location: USA
|
|
|
|
veeramanimurugesan,
I am assuming input is 80 byte FB and balance amount is 8 byte zd. If amount is not ZD, change it accordingly.
See if below works...
| Code: |
//STEP0100 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//INA DD *
A 0
A 0
A 0
B 10
B 0
C 0
D 0
D 0
E 0
E 100
E 1000
//INB DD *
A 0
A 0
A 0
B 10
B 0
C 0
D 0
D 0
E 0
E 100
E 1000
//SORTOUT DD SYSOUT=*
//SYSIN DD *
OPTION COPY
JOINKEYS F1=INA,FIELDS=(01,1,A)
JOINKEYS F2=INB,FIELDS=(01,1,A)
JOIN UNPAIRED,F2
REFORMAT FIELDS=(F2:1,80,F1:81,8)
OUTFIL INCLUDE=(81,8,ZD,EQ,0),BUILD=(1,80)
//*
//JNF1CNTL DD *
INREC OVERLAY=(81:15,8,UFF,TO=ZD,LENGTH=8)
SUM FIELDS=(81,8,ZD)
//* |
OUTPUT
| Code: |
A 0
A 0
A 0
C 0
D 0
D 0 |
Thanks, |
|
| Back to top |
|
|
veeramanimurugesan
New User
Joined: 03 Nov 2009
Posts: 19
Location: chennai
|
|
|
|
sqlcode1,
Thanks. I ran some tests and it worked perfectly!!
But i'm facing a problem when running with full volume. The file consists of approximately 24 million records. So the job is ending with the below error,
| Code: |
ICE039A 1 INSUFFICIENT MAIN STORAGE - ADD AT LEAST 28K BYTES |
I tried REGION=28K,32K,64K,128K but same problem. In test environment i was able to run with REGION=0M and the job completed fine. But i'm sure i cant use REGION=0M for production.
Also the files are in sorted order so i added the below but still no go.
| Code: |
JOINKEYS F1=INA,FIELDS=(01,1,A),SORTED
JOINKEYS F2=INB,FIELDS=(01,1,A),SORTED |
Can you please provide your suggestions on this to resolve this storage issue? |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
Why don't you look back at my suggestion (not code, but you need to do something...).
At the moment you are reading 24m records twice and doing a join on them. I'm suggesting you find out those that are zero with a simple SUM and then hit the file for the full data knowing the keys you want. |
|
| Back to top |
|
|
sqlcode1
Active Member
Joined: 08 Apr 2010
Posts: 577
Location: USA
|
|
|
|
veeramanimurugesan,
You may be aware that since JOINKEYS application uses three tasks, it can require more storage than a regular DFSORT application.
I am not sure what to tell you without looking at entire message log. You may also wait until someone from DFSort team looks at this more closely and provide you better suggestions. Also, I am not sure why you can't use REGION=0M in production.
| Quote: |
| At the moment you are reading 24m records twice and doing a join on them. |
Bill,
Even with 2 pass approach, OP will be reading 24m records twice but in a separate steps. Regardless with both the approaches, it needs to read 24m records twice unless single pass solution without joinkeys is developed.
Thanks, |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
Yes... I thought it might have to read the file again in the second step... mainly the point of the second step :-)
It is not so much the twice the files, but doing the join (or whatever) only with those keys that are known to be needed, from the SUM in the first step.
I can be completely wrong about this, of course... |
|
| Back to top |
|
|
Frank Yaeger
DFSORT Developer
Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
|
|
|
|
| Quote: |
| I tried REGION=28K,32K,64K,128K but same problem. In test environment i was able to run with REGION=0M and the job completed fine. But i'm sure i cant use REGION=0M for production. |
128K is a tiny REGION and is probably smaller than your default REGION.
If you can't use REGION=0M for some reason, then at least try a large REGION such as REGION=50M. But you may actually need to use REGION=0M.
From the DFSORT Application Programming Guide:
| Quote: |
| Since a JOINKEYS application uses three tasks, it can require more storage than a regular DFSORT application. You may need to use REGION=0M for some JOINKEYS applications. |
|
|
| Back to top |
|
|
veeramanimurugesan
New User
Joined: 03 Nov 2009
Posts: 19
Location: chennai
|
|
|
|
Yahoo  !! I tried 32M,64M and 128M. Job runs fine from REGION=64M and the job's elapsed time was less than 5 minutes.
Thank you all for your help and support. |
|
| Back to top |
|
|
Skolusu
Senior Member
Joined: 07 Dec 2007
Posts: 2205
Location: San Jose
|
|
|
|
veeramanimurugesan,
Here is an alternative solution without sorting. since your data is already sorted, I added SORTED,NOSEQCK parameters to both JNF1 and JNF2. Since your intention is to pick records that have all zeroe balance we use an INCLUDE condition to pick the records that have balance greater than zero and eliminate them using the ONLY on the Join Statement
| Code: |
//STEP0100 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//INA DD *
A 0
A 0
A 0
B 10
B 0
C 0
D 0
D 0
E 0
E 100
E 1000
//INB DD *
A 0
A 0
A 0
B 10
B 0
C 0
D 0
D 0
E 0
E 100
E 1000
//SORTOUT DD SYSOUT=*
//SYSIN DD *
OPTION COPY
JOINKEYS F1=INA,FIELDS=(01,1,A),SORTED,NOSEQCK
JOINKEYS F2=INB,FIELDS=(01,1,A),SORTED,NOSEQCK
JOIN UNPAIRED,F2,ONLY
//*
//JNF1CNTL DD *
INCLUDE COND=(15,8,UFF,GT,0)
//* |
|
|
| Back to top |
|
|
|
|
