Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Select Group of records based on condition

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
veeramanimurugesan

New User


Joined: 03 Nov 2009
Posts: 19
Location: chennai

PostPosted: Tue Nov 01, 2011 6:08 pm    Post subject: Select Group of records based on condition
Reply with quote

Hi,

Need some assistance with the below.

Code:
Input File :
Acc No     Balance
A             0
A             0
A             0
B             10
B             0
C             0
D             0
D             0
E             0
E             100
E             1000

Required output file :
Acc No     Balance
A             0
A             0
A             0
C             0
D             0
D             0


I want to select that group of accounts which does not have any balance (zero) in all occurences. I tried using SUM FIELDS but that would give me only one record whereas I need the whole group in my output file.
Back to top
View user's profile Send private message

expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Tue Nov 01, 2011 6:12 pm    Post subject:
Reply with quote

Please post the code that you are using, also are the figures really left justified
Back to top
View user's profile Send private message
veeramanimurugesan

New User


Joined: 03 Nov 2009
Posts: 19
Location: chennai

PostPosted: Tue Nov 01, 2011 6:52 pm    Post subject:
Reply with quote

expat,

Yes the figures are left justified. The field is CHAR in the file though its an Amount field.

I tried the below but it gives me only 1 record of the group whereas i need all records of zero amount groups.

SORT FIELDS=(Account Number)
SUM FIELDS=(Amount,ZD)
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Tue Nov 01, 2011 7:03 pm    Post subject:
Reply with quote

Then you need to use the INCLUDE COND= rather than the SUM FIELDS statement
Back to top
View user's profile Send private message
Pandora-Box

Moderator


Joined: 07 Sep 2006
Posts: 1529
Location: Andromeda Galaxy

PostPosted: Tue Nov 01, 2011 7:07 pm    Post subject:
Reply with quote

Please refer to manual
the SORT manual


Code:


Include cond



Assuming your field is numeric

Code:

//SORTIN   DD *
A             0
A             0
A             0
B             10
B             0
C             0
D             0
D             0
E             0
E             100
E             1000
/*
//SORTOUT  DD SYSOUT=*
//SYSIN    DD *
  SORT FIELDS=COPY
  INCLUDE COND=(15,4,ZD,EQ,0)

Back to top
View user's profile Send private message
veeramanimurugesan

New User


Joined: 03 Nov 2009
Posts: 19
Location: chennai

PostPosted: Tue Nov 01, 2011 7:14 pm    Post subject:
Reply with quote

INCLUDE COND would not work because that would work on a record level but my checking needs to be at a GROUP level. That is, account groups with absolutely zero balances.

If i use INCLUDE then i would get account B & E in my output as 1 record of it has zero balance but it would be incorrect because another record for same accounts have balances.

Please advise.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Tue Nov 01, 2011 8:55 pm    Post subject: Reply to: Select Group of records based on condition
Reply with quote

From the SUM you can end up with a file of "keys" where you include only the zero values. Then take the file of keys and match to the original input file. Don't know if it can be done in one step.
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Tue Nov 01, 2011 9:10 pm    Post subject:
Reply with quote

veeramanimurugesan,
I am assuming input is 80 byte FB and balance amount is 8 byte zd. If amount is not ZD, change it accordingly.

See if below works...
Code:
//STEP0100 EXEC PGM=SORT                                   
//SYSOUT   DD SYSOUT=*                                     
//INA      DD *                                           
A             0                                           
A             0                                           
A             0                                           
B             10                                           
B             0                                           
C             0                                           
D             0                                           
D             0                                           
E             0                                           
E             100                                         
E             1000                                         
//INB      DD *                                           
A             0                                           
A             0                                           
A             0                                           
B             10                                           
B             0                                           
C             0                                           
D             0                                           
D             0                                           
E             0                                           
E             100                                         
E             1000                                         
//SORTOUT  DD SYSOUT=*                                     
//SYSIN    DD *                                           
  OPTION COPY                                             
  JOINKEYS F1=INA,FIELDS=(01,1,A)                         
  JOINKEYS F2=INB,FIELDS=(01,1,A)                         
  JOIN UNPAIRED,F2                                         
  REFORMAT FIELDS=(F2:1,80,F1:81,8)                       
  OUTFIL INCLUDE=(81,8,ZD,EQ,0),BUILD=(1,80)               
//*                                                       
//JNF1CNTL DD *                                           
  INREC OVERLAY=(81:15,8,UFF,TO=ZD,LENGTH=8)               
  SUM FIELDS=(81,8,ZD)                                     
//*

OUTPUT
Code:
A             0
A             0
A             0
C             0
D             0
D             0

Thanks,
Back to top
View user's profile Send private message
veeramanimurugesan

New User


Joined: 03 Nov 2009
Posts: 19
Location: chennai

PostPosted: Wed Nov 02, 2011 5:03 pm    Post subject:
Reply with quote

sqlcode1,

Thanks. I ran some tests and it worked perfectly!!

But i'm facing a problem when running with full volume. The file consists of approximately 24 million records. So the job is ending with the below error,

Code:

ICE039A 1 INSUFFICIENT MAIN STORAGE - ADD AT LEAST 28K BYTES


I tried REGION=28K,32K,64K,128K but same problem. In test environment i was able to run with REGION=0M and the job completed fine. But i'm sure i cant use REGION=0M for production.

Also the files are in sorted order so i added the below but still no go.
Code:
JOINKEYS F1=INA,FIELDS=(01,1,A),SORTED                         
JOINKEYS F2=INB,FIELDS=(01,1,A),SORTED 


Can you please provide your suggestions on this to resolve this storage issue?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Wed Nov 02, 2011 5:13 pm    Post subject: Reply to: Select Group of records based on condition
Reply with quote

Why don't you look back at my suggestion (not code, but you need to do something...).

At the moment you are reading 24m records twice and doing a join on them. I'm suggesting you find out those that are zero with a simple SUM and then hit the file for the full data knowing the keys you want.
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Wed Nov 02, 2011 8:01 pm    Post subject:
Reply with quote

veeramanimurugesan,
You may be aware that since JOINKEYS application uses three tasks, it can require more storage than a regular DFSORT application.

I am not sure what to tell you without looking at entire message log. You may also wait until someone from DFSort team looks at this more closely and provide you better suggestions. Also, I am not sure why you can't use REGION=0M in production.

Quote:
At the moment you are reading 24m records twice and doing a join on them.

Bill,
Even with 2 pass approach, OP will be reading 24m records twice but in a separate steps. Regardless with both the approaches, it needs to read 24m records twice unless single pass solution without joinkeys is developed.

Thanks,
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Wed Nov 02, 2011 8:10 pm    Post subject: Reply to: Select Group of records based on condition
Reply with quote

Yes... I thought it might have to read the file again in the second step... mainly the point of the second step :-)

It is not so much the twice the files, but doing the join (or whatever) only with those keys that are known to be needed, from the SUM in the first step.

I can be completely wrong about this, of course...
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Nov 02, 2011 10:42 pm    Post subject:
Reply with quote

Quote:
I tried REGION=28K,32K,64K,128K but same problem. In test environment i was able to run with REGION=0M and the job completed fine. But i'm sure i cant use REGION=0M for production.


128K is a tiny REGION and is probably smaller than your default REGION.

If you can't use REGION=0M for some reason, then at least try a large REGION such as REGION=50M. But you may actually need to use REGION=0M.

From the DFSORT Application Programming Guide:

Quote:
Since a JOINKEYS application uses three tasks, it can require more storage than a regular DFSORT application. You may need to use REGION=0M for some JOINKEYS applications.
Back to top
View user's profile Send private message
veeramanimurugesan

New User


Joined: 03 Nov 2009
Posts: 19
Location: chennai

PostPosted: Thu Nov 03, 2011 12:01 am    Post subject:
Reply with quote

Yahoo icon_biggrin.gif !! I tried 32M,64M and 128M. Job runs fine from REGION=64M and the job's elapsed time was less than 5 minutes.

Thank you all for your help and support.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Fri Nov 11, 2011 4:13 am    Post subject: Reply to: Select Group of records based on condition
Reply with quote

veeramanimurugesan,

Here is an alternative solution without sorting. since your data is already sorted, I added SORTED,NOSEQCK parameters to both JNF1 and JNF2. Since your intention is to pick records that have all zeroe balance we use an INCLUDE condition to pick the records that have balance greater than zero and eliminate them using the ONLY on the Join Statement

Code:

//STEP0100 EXEC PGM=SORT                               
//SYSOUT   DD SYSOUT=*                                 
//INA      DD *                                       
A             0                                       
A             0                                       
A             0                                       
B             10                                       
B             0                                       
C             0                                       
D             0                                       
D             0                                       
E             0                                       
E             100                                     
E             1000                                     
//INB      DD *                                       
A             0                                       
A             0                                       
A             0                                       
B             10                                       
B             0                                       
C             0                                       
D             0                                       
D             0                                       
E             0                                       
E             100                                     
E             1000                                     
//SORTOUT  DD SYSOUT=*                                 
//SYSIN    DD *                                       
  OPTION COPY                                         
  JOINKEYS F1=INA,FIELDS=(01,1,A),SORTED,NOSEQCK       
  JOINKEYS F2=INB,FIELDS=(01,1,A),SORTED,NOSEQCK       
  JOIN UNPAIRED,F2,ONLY                               
//*                                                   
//JNF1CNTL DD *                                       
  INCLUDE COND=(15,8,UFF,GT,0)                         
//*
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Removing Duplicates based on certain ... chandracdac DFSORT/ICETOOL 8 Fri Dec 09, 2016 4:40 am
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts Two Selects in a single select query Rohit Umarjikar DB2 1 Fri Nov 04, 2016 8:46 pm
This topic is locked: you cannot edit posts or make replies. How to pass the previous month date i... Suganya87 DFSORT/ICETOOL 5 Mon Oct 31, 2016 4:13 pm
No new posts How to split the records using the am... vnktrrd DFSORT/ICETOOL 24 Fri Oct 28, 2016 7:33 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us