GDG version conflict

srini_igsi · New User Joined: 09 Dec 2005 Posts: 30 Location: Pune

Hi,

All of our Jobs are dataset triggered. As soon as a Job is triggered, we first take the back up of the file in a GDG and then we delete the original file in the immediate next step. The Job makes use of the current GDG version(+0) created in the previous step for the rest of the processing.

Let us assume that while processig the first instance of the file, we have received another file. So the next instance of the job gets triggered and as usual it takes the back of the second instance of the file and deletes the same.

So here my doubt is, both the parallel jobs running with milli seonds diff may refer to the current GDG version(+0) created for the first instance of the first file which triggered the first job.

I s my understanding correct?? I jus want to relate the first file to the first instance of the job, second file to the second instance job and so and so.

Aany suggetions, pl let me know.

Thx in advance.

expat · Posted: Sat Sep 22, 2007 3:57 pm

Well if the same dataset is created and the same job is submitted by the schedule package then the two jobs will not run in parallel because they both have the same job name.

srini_igsi · New User Joined: 09 Dec 2005 Posts: 30 Location: Pune

Does it mean that the scheduler itself will restrict the other instance of the job to run while the first instance of the same job is being executed??

The actual issue is whenever the server receives a file, immediately it NDM's the fil to Mainframe with specific name so that the corresponding Job gets triggered and executed. However whenever the server receives another file, the NDM is failing to deliver the file to Mainframe because, still the first file received might be executing in Mainframe there by will restrict the other file to come in. Currently we are deleting the actual file at the end only.

My primary goal is whenever the Mainframe receives more than one file at a time, I shouldn't let the NDM fail to deliver another instance of the file at the same time I should be able to process all the files whatever I come across at any point of time.

Because of this problem Client is suggesting us to move the file deletion step to the top so that we should first take the back of the file then delete the same and then make use of the GDG version created for the rest of the processing.

Pl let me know if you have any suggetions.

murmohk1 · Posted: Sat Sep 22, 2007 10:00 pm

Reddy,

superk · Posted: Sat Sep 22, 2007 10:22 pm

I totally get the problem, and I don't have an easy solution. If it were me, I'd have them send the data to a unique dataset, or directly to a GDG +1 generation for each cycle, and I'd use the static dataset as a dummy dataset name trigger to run the schedule, not to actually hold any real data. This way, there'd never be the chance to inadvertantly overlay the data between cycles.

Any chance you can have the process changed so that it can call the schedule upon completion, and avoid the whole dataset trigger mess completely?

dick scherrer · Posted: Sat Sep 22, 2007 11:24 pm

Hello,

IMHO, using the creation of a dataset to trigger a job or some other process only works well when the frequency of the file creation is very limited. If there is a chance of the "next" file arriving before "this" one is processed, i believe the trigger will not work well - as you are seeing.

What other approaches have been looked at?

srini_igsi · New User Joined: 09 Dec 2005 Posts: 30 Location: Pune

Hi,

To be straight forward I don't have any exposure to Job scheduling.

All the problems started when the NDM was failing to deliver the files to Mainframe. Whenever there are multiple files queued up in the server, the NDM is frequently failing to push the files to Mainframe.

Whenever a file is being excecuted and the sub sequent files are queued up in the server, the NDM is failing and every time we had to log-on to the server and mannually push the files to Mainframe.

With regards to my first post, if we can assume that parllel job processing is allowed, when parallel jobs are running with milli seonds diff, do both the jobs refer to the current GDG version(+0) created for the first instance of the first file which triggered the first job or the first instance of the job will be able to refer to the GDG version created for it and the sencond instance of the job will be able to refer to the version created for for the second instance of the file??

dick scherrer · Posted: Sun Sep 23, 2007 2:12 am

Hello,

One of the difficulties with the current process is that the server can "Push" whenever it wants.

Is there any chance that the mainframe could run a process periodically that would "Pull" whatever was ready?

Another thought might be to change the process to send the server files to a queue and have the mainframe process pull one "file" at a time from the queue. If the process finds data available, process it and re-submit itself via the internal reader. When the process detects no more data, it woulkd just end. This process would be scheduled to run every n minutes processing anythning available or simply ending when there was no data.

I believe trying to use the creation of a generation to trigger the process thru the scheduling system will cause more headscratching than providing something useful.

srini_igsi · New User Joined: 09 Dec 2005 Posts: 30 Location: Pune

Hi Dick,

With regrads to your last point, the GDG version is not going to trigger the Job. The actual file with a standard dataset will first trigger the Job. But as soon as a Job is triggered , our Client is suggesting us to take the original file into a bckup and then delete the same so that the job can make use of the back up taken for the reset of the processing and at the same time the other file which may be queued up in the server can find a way to Mainframe and triggers another instance of the same Job and so and so.

So here what I suspect is, when the first instance of the job is refering to the current version of the GDG, the second instance of the file which triggers the second instance of the job with milli seconds diff might also be refering to the current version of the GDG there by the second instance of the job may refer to the wrong instance of the file (first file) rather the seond instance of the file. This is because the GDG version is updated only after the Job (first job) is succssfully completed.

If my assumption is incorrect, we will not have any issues as each and every instance of the job will be able to refer to its corresponding instance of the file which triggers that job.

However I will talk to our scheduling guys if there is a way to send all the files to a queue and have the mainframe process pull one "file" at a time from the queue. I will also confirm with them whether two instances of the same job can run paralelly.

dick scherrer · Posted: Sun Sep 23, 2007 9:43 pm

Hello,

Srihari Gonugunta · Posted: Mon Sep 24, 2007 10:29 am

My thought:
Split the job into two.
first job: copy the file to gdg and delete
Second job: process the current version of the gdg

srini_igsi · New User Joined: 09 Dec 2005 Posts: 30 Location: Pune

Srihari,

If two or more Jobs are running paralelly and the jobs are done with the back up of diff datasets with milli seconds diff, the Jobs are subjected to refer to the current generation of the dataset taken by the very latest Job . So I don't think this logic is reliable always.