Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

How to increase the condition limit for INCLUDE statement?

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1439
Location: Bangalore,India

PostPosted: Tue Mar 13, 2007 4:16 pm    Post subject: How to increase the condition limit for INCLUDE statement?
Reply with quote

Hi All,

Could it be possible to increase the # of conditions handled by INCLUDE statement? Im looking at 70 to 100 thousand conditions.

I had seen MINCORE option (I guess its a syncsort option) in other shop. I had come to know that it increases the # of conditions handled by included statement. Is the same possible in DFSORT (as my shop uses only DFSORT).

If yes, please let me know the same and also correct me if Im wrong in this regard.

Regards,
Murali
Back to top
View user's profile Send private message

guptae

Moderator


Joined: 14 Oct 2005
Posts: 1187
Location: Bangalore,India

PostPosted: Tue Mar 13, 2007 4:37 pm    Post subject:
Reply with quote

Hi Murali,

Would you please tell me abt MINCORE option ? I am also facing the same issue & my shop uses syncsort.
Back to top
View user's profile Send private message
sril.krishy

Active User


Joined: 30 Jul 2005
Posts: 183
Location: hyderabad

PostPosted: Tue Mar 13, 2007 5:32 pm    Post subject:
Reply with quote

Guptae,
me too had the same problem long back and I got the answer from one of my senior telling me to use the below option.

PGM=SORT,PARM='MINCORE=750K'

You can increase this value as per your requirement.(more than 2M is not recommended).

Thank you
Krishy
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Tue Mar 13, 2007 9:37 pm    Post subject:
Reply with quote

Syncsort has a default storage value for certain operations and requires MINCORE to increase the limit. DFSORT does not have such a default and allows the maximum number of conditions automatically without the need for a parameter. But you can't use 70 to 100 thousand INCLUDE conditions regardless of how much storage is available. The limit is much, much smaller than that.

You might be able to do what you want using SPLICE with a selection file and master file instead.
Back to top
View user's profile Send private message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1439
Location: Bangalore,India

PostPosted: Wed Mar 14, 2007 2:39 pm    Post subject:
Reply with quote

Frank,

Would you let me know, the MAXIMUM number of conditions allowed in INCLUDE/OMIT statement.

This gives me chance of spliting the conditions into 'n' of files dynamically and building a control card at run time.

Thanks for your inputs in advance.

Regars,
Murali
Back to top
View user's profile Send private message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1439
Location: Bangalore,India

PostPosted: Wed Mar 14, 2007 2:42 pm    Post subject:
Reply with quote

In addition to my post above,

Please tell me whether sort handles 125M records (5 flat files of each 25M records). Since the layout is similar for all the 5 files, Im in an idea to concatenate all the 5 files in SORTIN statement.

Regards,
Murali
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Mar 14, 2007 8:32 pm    Post subject:
Reply with quote

Quote:
Would you let me know, the MAXIMUM number of conditions allowed in INCLUDE/OMIT statement.


It depends on what your conditions look like. Give an example.

Note that to increase the number of conditions, you can use the IFTHEN technique shown in the MULT job at:

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CM20/2.2.147?SHELF=&DT=20060721170811&CASE=

Quote:
Please tell me whether sort handles 125M records


Yes, it can.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed Mar 14, 2007 11:26 pm    Post subject:
Reply with quote

Hello,

I believe IFTHEN is a future feature for Syncsort. It is not in the documentation downloaded a couple of months ago.

If you have 70-100k conditions, a better solution would be designing some kind of external "rule" file(s) and processing these rules against your input file.

This is another example of something best not done in a sort even if you find a way (IMHO).
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Thu Mar 15, 2007 1:15 am    Post subject:
Reply with quote

Dick,

Your reference to Syncsort seems irrelevant considering that the original poster said:

Quote:
my shop uses only DFSORT


Quote:
If you have 70-100k conditions, a better solution would be designing some kind of external "rule" file(s) and processing these rules against your input file.

This is another example of something best not done in a sort even if you find a way (IMHO).


Maybe or maybe not. As I said previously
Quote:
You might be able to do what you want using SPLICE with a selection file and master file instead.


That might, in fact, be a good solution using DFSORT.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Mar 15, 2007 2:58 am    Post subject:
Reply with quote

Hi Frank,

Yup,
Quote:
Your reference to Syncsort seems irrelevant considering that the original poster said:


Obviously, i was focusing on
Quote:
I am also facing the same issue & my shop uses syncsort

and
Quote:
MINCORE option (I guess its a syncsort option)

and missed the bit about that shop being DFSORT icon_redface.gif

I believe the SPLICE "selection file and master file" approach is the same thing i was referring to by "rules". What i believe should not be done is to code 50-100k INCLUDES.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Thu Mar 15, 2007 3:42 am    Post subject:
Reply with quote

Quote:
What i believe should not be done is to code 50-100k INCLUDES.


Yes, I agree with that.
Back to top
View user's profile Send private message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1439
Location: Bangalore,India

PostPosted: Thu Mar 15, 2007 2:17 pm    Post subject:
Reply with quote

Frank,

Thanks all for your inputs.

My selection file as well as Master file layout are provided below -

Length - 267; Key length - 14 starts @ 9 column.

Quote:

It depends on what your conditions look like. Give an example.


I dint get the above, but I had given an example below -

Sample conditions:
Code:


include cond=(9,14,CH,EQ,C'1CR0701U035A15',OR,
                      9,14,CH,EQ,C'19A0701C162IEH',OR,
                      9,14,CH,EQ,C'1PY0701U039O19',OR,
                      9,14,CH,EQ,C'1750701C162C05',OR, and so on)


As suggested by you, I had tried IFTHEN technique for 79997 conditions. Fortunately or unfortunately, I had the following storage error.

JESYSMSG
Code:

IEF237I JES2 ALLOCATED TO
IEA995I SYMPTOM DUMP OUTPUT
SYSTEM COMPLETION CODE=80A  REASON CODE=00000010
TIME=06.53.13  SEQ=06877  CPU=0040  ASID=0044
PSW AT TIME OF ERROR  070C1000   81485F06  ILC 2  INTC 0D


JCL/Step used:

Code:


//STEP2   EXEC PGM=SORT,PARM='MINLIM=32M'
//SYSOUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SORTIN DD DSN=ISTEST.W.G.PRODSCHM.POL.PREVMNTH,DISP=SHR
//SORTOUT DD SYSOUT=*,LRECL=267,RECFM=FB
//SYSIN DD *
  SORT FIELDS=COPY
**
  INREC IFTHEN=(WHEN=(9,14,CH,EQ,C'00000000000000',OR,
//   DD DSN=ISTEST.W.G.PRODSCHM.POL.PREVMNTH.IFTHNCRD,DISP=SHR
//      DD *
               9,14,CH,EQ,C'00000000000000'),OVERLAY=(268:C'Y')),
               IFTHEN=(WHEN=NONE,OVERLAY=(268:C'N'))
*
   OUTFIL INCLUDE=(268,1,CH,EQ,C'Y'),
   OUTREC=(1:267)
/*

Where the file ISTEST.W.G.PRODSCHM.POL.PREVMNTH.IFTHNCRD contain the conditions built during run time. I had provided sample file content below -

9,14,CH,EQ,C'1CR0701U035A15',OR,
9,14,CH,EQ,C'19A0701C162IEH',OR,
9,14,CH,EQ,C'1PY0701U039O19',OR,    so on.




Could you please let me know, how to provide more storage to make the run successfully.



Dick,

Quote:
This is another example of something best not done in a sort even if you find a way (IMHO).


Could you please share (some) more inputs on this?
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Mar 15, 2007 7:32 pm    Post subject:
Reply with quote

Hello,

The 80A is because the step needed more memory than it could get (which may be found in the "MVS System Codes" manual, which is linked to this site). If you look there, you will see exactly what the problem was by the reason-code that follows the 80A.

Rather than 80k includes, you might put the 80k "include values" in a file and then user a 2-file match/merge to keep/discard the records you want to "include" or "exclude.

If you look in the forum you will see several examples that Frank has provided to do this.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Thu Mar 15, 2007 9:00 pm    Post subject:
Reply with quote

Murali,

For a 14-byte CH field, you can use about 1360 conditions per INCLUDE statement (or WHEN operand). It really isn't practical to do 80000 conditions with IFTHEN statements - it takes too much storage. A better approach would be to use the SELECT or SPLICE operator of DFSORT's ICETOOL. If you want me to show you how, I just need you to tell me whether or not there are duplicates in the input file (that is, can 1CR0701U035A15 or any other key appear in more than one record of the input file)?
Back to top
View user's profile Send private message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1439
Location: Bangalore,India

PostPosted: Fri Mar 16, 2007 9:04 am    Post subject:
Reply with quote

Actually my master file has duplicate values and I tried the Create files with matching and non-matching records from SORTTRCK pdf. But my peers have not agreed for it saying 'Control cards are not easily understandable for those who have not worked much with sort techniques'. So, I was wondering the same could be achieved in someother way (through INCLUDE etc ....).

Anyway Thanks one and all everyone for your valuable inputs.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Fri Mar 16, 2007 8:44 pm    Post subject:
Reply with quote

Quote:
But my peers have not agreed for it saying 'Control cards are not easily understandable for those who have not worked much with sort techniques'.


This seems to me to be equivalent to saying "we aren't capable of learning new things, even when they can help us do our job". I guess your peers wouldn't have liked the IFTHEN solution either (if it were feasible) since that's something new as well.

Why bother to ask us here how to achieve this; why not just ask your peers to solve it since they seem to have veto power over anything we could propose.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
This topic is locked: you cannot edit posts or make replies. Need help on If condition continuatio... ashok_uddaraju CLIST & REXX 4 Fri May 19, 2017 8:55 am
No new posts Joinkeys with condition scdinesh DFSORT/ICETOOL 10 Tue Feb 14, 2017 12:20 am
No new posts setting new limit Key values for inde... srilata83 DB2 1 Fri Feb 10, 2017 9:24 pm
No new posts ROUNDED Problem with COMPUTE statement shalem COBOL Programming 11 Thu Feb 09, 2017 8:16 pm
No new posts VALIDATE NULL VALUE IN DB2 CASE STATE... useit DB2 5 Thu Feb 09, 2017 4:34 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us