Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Maximum memory that can be allocated using CEEGTST

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming
View previous topic :: :: View next topic  
Author Message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Sat Apr 26, 2008 6:02 pm    Post subject: Maximum memory that can be allocated using CEEGTST
Reply with quote

Hi all,

I would like to know the maximum memory that OS will allocate thru CEEGTST call. My requirement is that I need to load 5 million records and each record is of size 40 Bytes. So total memory requirement is 200 Million Bytes. Is it possible for the OS to allocate this much of contigous memory ? I did not try this out as I feel that there will be memory contention (10% of 2GB limit). Could any one help me out ?

Thanks in advance,
Back to top
View user's profile Send private message

enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10274
Location: italy

PostPosted: Sat Apr 26, 2008 6:18 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Quote:
I did not try this out as I feel that there will be memory contention (10% of 2GB limit)


no reason not to test ,
storage contention does not depend on our reply, but on Your environment
and it will be there as soon You put the program in production

loading a 5 million elements table in memory is a rather unusual approach
but if that' s the requirement icon_question.gif icon_question.gif

a loop of CEEGTST and CEEFRST one after the other one
of increasing size up to the amount needed should not do any harm

searching the net for "CEEGTST maximum allocatable storage" will return a few useful pointers
Back to top
View user's profile Send private message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Sat Apr 26, 2008 6:36 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Thank you for a quick response.

My requirement is that we need to replace one credit card number with encrypted credit card number. The encrypted credit card numbers are stored in a flat file. There are around 5 Million records in the file. So for each subscriber, we need to find an entry in the flat file and replace the current credit card # with the found entry.

Loading the table into memory and do a binary search was one of the options we thought. If this option is not possible, then is it feezible to load the flat file into VSAM with index and do a search ?

Or any other option is available ?

Any help is much appriciated

Thank you,
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10274
Location: italy

PostPosted: Sat Apr 26, 2008 6:45 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

just a couple of guesses..

the card number is the current key in some table

the encrypted card number is going to be the new key ...

all the tables must be synchronized

what is the plan for the cutover from clear to encyphered

one of the basics of updates is that if the updates touch more than
( for somebody is 35% of the records )
then the best performance is given by unload/reload and a sequential two file match approach

all depends on the service level agreement for application downtime

but why in heaven not to have a single program do the cyphering and the update
Back to top
View user's profile Send private message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Sat Apr 26, 2008 7:05 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

The input credit card# is stored in a VSAM file which is having all the data available for the customer and this VSAM file is compacted. One of the application programs is expanding the record at run time. Only after the expansion, the record will fit into the copy book lay out. ie only at the run time, the credit card numbers will be available. So we cannot go for the suggested approach of having a single program to do the cyphering.

We need to run the new program and replace the credit card number as part of the existing application because of the mentioned issue.

If CEEGTST can allocate contigous memory, then this issue can easily be resolved

Thanks,
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10274
Location: italy

PostPosted: Sat Apr 26, 2008 7:13 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

You are just giving useless info and lame excuses for doing it the way You want ...

How do you expect any help if You ignore the questions being asked
Back to top
View user's profile Send private message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Sat Apr 26, 2008 7:35 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Sorry for making you confused.

I was explaining about the application which we have.
This is the answers for the questions that you have asked.

1. the card number is the current key in some table
No

The credit card number to be replaced is availabe in the master file ( which is a VSAM). This has to be replaced with a credit card number available in a flat file. This flat file is having two fileds. One is decrypted and other one is encrypted. So for each Credit card # in the in the master should be searched in the flat file and getting it replaced with encrypted one

2. the encrypted card number is going to be the new key ...
Yes

3. all the tables must be synchronized
I am not sure about the question ( are u referring to DB2 ?)

4. what is the plan for the cutover from clear to encyphered
The new program should be added in the existing application and it will replace the credit card numbers

5.but why in heaven not to have a single program do the cyphering and the update
Both the cyphering and updation will be done in the same program.

Could you please explain more about the two file match approach ?

Thank you,
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Sat Apr 26, 2008 8:37 pm    Post subject:
Reply with quote

Hello Satheesh and welcome to the forums,

Quote:
Could you please explain more about the two file match approach ?

Near the top of this COBOL section of the forum is a "Sticky" with sample code of how to do a 2-file match/merge. This code should do just what you need.

Please download the code, review it, and post back here if you have any questions or problems.
Back to top
View user's profile Send private message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Sat Apr 26, 2008 11:19 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Thank you d.sch

I will try try out the logic and post a response

Regards,
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10274
Location: italy

PostPosted: Sat Apr 26, 2008 11:32 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Quote:
2. the encrypted card number is going to be the new key ...
Yes
........
4. what is the plan for the cutover from clear to encyphered
The new program should be added in the existing application and it will replace the credit card numbers



these two points make me doubtful about the whole approach...

let me ask about point 4 first
my understanding is that encyphering should be done in bulk once in a while...

after that the application should be modified in order that when processing the card number
it gets encrypted and decrypted

why Your statement about adding the new program to the existing application ????

now back to point 2
You are going to change the vsam key ...
will it be in the same position ... probably yes
will it be the same length ... again yes

what if a cyphered key is the same as a clear key,,, ( low probability, but still to be taken into account )

I would run a few performance test and go for a sequential processing and
vsam reload...

there will be a cutover with a service disruption ...
in order to make the rest of the application aware of the encyphering

.... scenario
input 1 - vsam file ordered by key on the clear card number
input 2 - sequential file containing the clear card number and the encyphered card number
program to produce a temporary file with the clear key changed to the encyphered key
sort the temporary file on the encyphered key and reload the vsam

You cannot get away without the sort...

still I do not see any reason to have an intermediate file with the two card numbers

simply plug in the program being used to generate the temporary dataset
the subroutine/function to do the encyphering
Back to top
View user's profile Send private message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Sun Apr 27, 2008 1:37 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Thank you enrico-sorichetti,

That is a valuable suggestion. For VSAM input file, we can follow the suggested approach.

1. As you have told, the cyphering should be done once in a while. This program is going to be a one time correction to encrypt the credit card numbers. After this another program would take care of encryption and decryption and store the encrypted cc# to the file.

2. We need to write the program as there are many input files which are having clear cc#, which are normal VB files (compacted). Atleast 200 files need to be corrected.

Each file is of length 13026 bytes, but when expanded the length will become 80000 bytes. Other than VSAM, none of the files, we will not be knowing the position of the credit card number. So we need to make the program a part of application to get the expanded record.

I guess for replacing the cc# in VB file, we should have a program with an intermediate file.

will DB2 help ?

Thanks,
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10274
Location: italy

PostPosted: Sun Apr 27, 2008 1:57 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

there is a very big security flaw in Your company's approach..
in case of problems lots of people on the management chain might get into serious trouble
( no due diligence on custody/management of sensitive data )

never, never, (repeat never for zillions times) clear and encyphered texts should coexist on any support
( dataset with clear data and encyphered data )
anybody making such a proposal should be kicked out ...
what if ....

also whenever using compressed data (any algorithm )
the key or collating tokens should not be compressed at all
so You should not have the need to uncompress anything
if You have to... the approach is wrong


I stand strongly by my suggestion

for any dataset where the cc#must be changed...

read the input
compute the encyphered cc#
write the output

at the end sort according
reload the production datasets
Back to top
View user's profile Send private message
Satheesh S

New User


Joined: 25 Apr 2008
Posts: 9
Location: Chennai

PostPosted: Mon Apr 28, 2008 10:00 pm    Post subject: Reply to: Maximum memory that can be allocated using CEEGTST
Reply with quote

Thank you very much enrico-sorichetti for all of your valuable suggestions.

Actually the cc# are level1 encryptd and it should be encrypted in level 2 and the new number is provided by Amex itself. So there will not be any security flaw. We have discussed this issue with the clients and they told to go ahead with an assembler program with GETMAIN in it and binary search. They would be needing this program for many a files. I have written that and it is working perfectly fine.

We are going to use your approach for VSAM files.

Thank you again.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Release quantity that was allocated v... Auryn JCL & VSAM 5 Thu May 18, 2017 6:02 pm
No new posts To find out size allocated to a seque... ashek15 JCL & VSAM 15 Thu Apr 27, 2017 9:42 am
No new posts Not enough virtual memory is availabl... Vinay Pushpala IBM Tools 16 Tue Sep 13, 2016 4:37 pm
No new posts Data set is allocated to another job ... Seppe CLIST & REXX 2 Wed Aug 03, 2016 4:31 pm
No new posts What is the maximum number of sort wo... Pravina M SYNCSORT 2 Mon Mar 14, 2016 4:51 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us