IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

[Solved]Automation need help in sorting the data


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   This topic is locked: you cannot edit posts or make replies.
View previous topic :: View next topic  
Author Message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Fri Oct 13, 2023 1:32 pm
Reply with quote

Scope:

Finacle files manual verificaion
my team will receive below 9 files from XYZ batch team
my team will verify, if these files are created for the current batch or created earlier for the previous batches.
The verification will be done in 2 ways.

Below files will be verified by the header date.
#ENV.Diggi.DAILY.GOANFILE(0)
#ENV.DIGGI.DAILY.KOANFILE(0)
#ENV.DIGGI.DAILY.TRNSFILE(0)


Below files will be verified by the file creation date.
#ENV.DIGGI.DAILY.POANFILE.TRI(0)  
#ENV.DIGGI.DAILY.IOANFILE.TRI(0)
#ENV.DIGGI.DAILY.TRNSFILE.TRI(0)

For this verification, approximately 5 mins will be spent for each batch.
If any issue with the files, next action of the my team is to contact the xyz batch team and request them to send the correct files.
1. Inform the files issue to Bank batch team through mail or teams.
2. Bank batch team to regenerate those files.
3. Inform the same to card batch team.
All this process may take 30 mins approximately.

I want to autoamte this entire process, I have not done any autoamtion before in mainfrmae like this, Any suggastion would be helpful here the date from which we have to compare is available in a file ALtI.Diggi.date.file1 this file as current batch date and previos batch date.

A pseudo code is also fine mostly I will try by myself to complete it with help of pseudocode first.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2000
Location: USA

PostPosted: Fri Oct 13, 2023 6:52 pm
Reply with quote

Please, clarify in plain English the meaning of EACH AND EVERY sentence of your post.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Fri Oct 13, 2023 7:06 pm
Reply with quote

That is plain and very simple English still let me clarify.

Manual process:
Below are file name which are current version of gdg
Diggi.DAILY.GOANFILE(0)
DIGGI.DAILY.KOANFILE(0)
DIGGI.DAILY.TRNSFILE(0)

Step1> My team receives 9 set of files as listed above. for first 4 files My team manually open that file and check for date if it is for current date or for previous date.

Step2>If it is for current date then no action we take if it is for not current date, we send mail to concern team to resend the file with current date.

Now for next 5 files.
DIGGI.DAILY.POANFILE.TRI(0)
DIGGI.DAILY.IOANFILE.TRI(0)
DIGGI.DAILY.TRNSFILE.TRI(0)

Step1> My team receives 5 set of files as listed above. My team manually look at the creation date of the files

Step2>If creation date is current date then no action we take if it is for not current date, we send mail to concern team to resend the file with current date.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2000
Location: USA

PostPosted: Fri Oct 13, 2023 7:24 pm
Reply with quote

Unless you are able to explain your task in that manner understandable by others (except "your team") I doubt if anyone would be able to help you.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Fri Oct 13, 2023 7:59 pm
Reply with quote

I have explained the task clearly, I have few manual steps for which I am trying to automate, the manual steps already I have mentioned above, please let me which part you think is not clear.
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 710
Location: Denmark

PostPosted: Sat Oct 14, 2023 9:06 pm
Reply with quote

Quote:
Below files will be verified by the header date.
Below files will be verified by the file creation date.

Verified against what?
Back to top
View user's profile Send private message
Pedro

Global Moderator


Joined: 01 Sep 2006
Posts: 2541
Location: Silicon Valley

PostPosted: Sat Oct 14, 2023 9:35 pm
Reply with quote

re: receives 9 set of files

It is also not clear what you mean by 'receive'. Do you receive the NAMES of the files that were created previously or are the files transmitted to you and you issue the RECEIVE command?
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 710
Location: Denmark

PostPosted: Sat Oct 14, 2023 10:01 pm
Reply with quote

Ok. I'll take a wild guess that you are comparing the rundate against the previous run.
My preference is REXX, but I guess that any proramming language will do.
In both cases I suggest that you store the latest timestamp in a dataset (please do not call them files, files are for UNIX).
Running in batch with a procedure like this is likely the simplest. The xx.REF dataset is just a sequential dataset with lrecl something like 20. The IF RC JCL statements are off the top of my head, so might not be quite correct.
Code:
//*                                                           
//CHECK    PROC PG=,DS=                                       
//A        EXEC PGM=IKJEFT1B,PARM='&PGM'                     
//SYSEXEC  DD DISP=SHR,DSN=your.exec.lib                     
//REFDS    DD DISP=SHR,DSN=&DS..REF                           
//DATADS   DD DISP=SHR,DSN=&DS.(0)                           
//SYSTSPRT DD SYSOUT=*                                       
//SYSTSIN  DD DUMMY                                           
//         PEND                                               
//*                                                           
//C1       EXEC CHECK,PGM=CHECK1,DS=#ENV.DIGGI.DAILY.GOANFILE
//         IF RC.C1.A EQ 0 THEN                               
  - - - good return, generate and send email - - -           
//         ELSE                                               
  - - - bad return, generate and send email - - -             
//         ENDIF                                 

To send an email, I think that you can use the XMITIP program from CBTTAPE.ORG file 314.

For case 1, comparing a timestamp in the header record, then someting like this:
Code:
 /* Check1                                              rexx 
    Verify header date                                       
 */                                                           
 "execio 1 diskr datads (stem datarec. finis)"                             
 parse var datarec.1 date_in_data .                                         
 "execio 1 diskr refds (refds. finis)"                               
 if date_in_data = refds.1 then exit 8 /* same date */
 refds.1=date_in_data
 "execio 1 diskw refds (refds. finis)"  /* rewrite last-date */                             
 exit 0 


For case 2, comparing a timestamp to the creation date, then someting like this:
Code:
 /* Check2                                              rexx 
    Verify dataset creation date                             
 */                                                           
 cc=BpxWdyn('info dd(datads) inrtdsn(ds)')                   
 cc=ListDsi("'"ds"'")   /* make SYSCREATE var */               
 "execio 1 diskr refds (stem refds. finis)"                               
 if syscreate = refds.1 then exit 8 /* same date */   
 refds.1=syscreate
 "execio 1 diskw refds (refds. finis)"  /* rewrite last-date */                             
 exit 0       
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 710
Location: Denmark

PostPosted: Sat Oct 14, 2023 10:16 pm
Reply with quote

Ok. I'll take a wild guess that you are comparing the rundate against the previous run.
My preference is REXX, but I guess that any proramming language will do.
In both cases I suggest that you store the latest timestamp in a dataset (please do not call them files, files are for UNIX).
Running in batch with a procedure like this is likely the simplest. The xx.REF dataset is just a sequential dataset with lrecl something like 20. The IF RC JCL statements are off the top of my head, so might not be quite correct.
Code:
//*                                                           
//CHECK    PROC PG=,DS=                                       
//A        EXEC PGM=IKJEFT1B,PARM='&PGM'                     
//SYSEXEC  DD DISP=SHR,DSN=your.exec.lib                     
//REFDS    DD DISP=SHR,DSN=&DS..REF                           
//DATADS   DD DISP=SHR,DSN=&DS.(0)                           
//SYSTSPRT DD SYSOUT=*                                       
//SYSTSIN  DD DUMMY                                           
//         PEND                                               
//*                                                           
//C1       EXEC CHECK,PGM=CHECK1,DS=#ENV.DIGGI.DAILY.GOANFILE
//         IF RC.C1.A EQ 0 THEN                               
  - - - good return, generate and send email - - -           
//         ELSE                                               
  - - - bad return, generate and send email - - -             
//         ENDIF                                 

To send an email, I think that you can use the XMITIP program from CBTTAPE.ORG file 314.

For case 1, comparing a timestamp in the header record, then someting like this:
Code:
 /* Check1                                              rexx 
    Verify header date                                       
 */                                                           
 "execio 1 diskr datads (stem datarec. finis)"                             
 parse var datarec.1 date_in_data .                                         
 "execio 1 diskr refds (refds. finis)"                               
 if date_in_data = refds.1 then exit 8 /* same date */
 refds.1=date_in_data
 "execio 1 diskw refds (refds. finis)"  /* rewrite last-date */                             
 exit 0 


For case 2, comparing a timestamp to the creation date, then someting like this:
Code:
 /* Check2                                              rexx 
    Verify dataset creation date                             
 */                                                           
 cc=BpxWdyn('info dd(datads) inrtdsn(ds)')                   
 cc=ListDsi("'"ds"'")   /* make SYSCREATE var */               
 "execio 1 diskr refds (stem refds. finis)"                               
 if syscreate = refds.1 then exit 8 /* same date */   
 refds.1=syscreate
 "execio 1 diskw refds (refds. finis)"  /* rewrite last-date */                             
 exit 0       
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1211
Location: Bamberg, Germany

PostPosted: Mon Oct 16, 2023 5:53 am
Reply with quote

To compare a block of datasets to a date inside another reference dataset can be done per JCL procedure. Include the RC of the previous steps to the mail send step (using JCL IF/ELSE/ENDIF).
Same applies for the CreDt check (involving perhaps IDCAMS/SORT).
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Mon Oct 16, 2023 3:24 pm
Reply with quote

Thank you guys for your response here.

I am working on this and would like to do all using sort and jcl, will let you know once i am done with it. Actually i have a dataset which contains the date record I need to compare the date from that dataset and see. i am working on it. keep you guys updated for solution.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3048
Location: NYC,USA

PostPosted: Mon Oct 16, 2023 4:53 pm
Reply with quote

It seems simple solution to me .. see if you follow
1. Use idcams listcat to get creation date of the gdgs and compare with header dataset that you have using JOINKEYS and if not matched then set rc=4 using nullofl
2. Simple JOINKEYS for incoming datasets with the dataset that has header date and use the same logic of setting rc-4 if not matched .
3. Last step is if any of the previous steps is an rc-4 then send an email to banking team else do nothing .
PS - you can easily get current date in DFSORT and compare with these dataset dates to skip the compares . All that I said above has been on this forum discussed several times , so please give a try and let us know if you don’t get to the finish line .
Good luck !
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Mon Oct 16, 2023 7:44 pm
Reply with quote

Thanks Rohit, I am on it even I decided to go with this type of approach lets see how i accomplish this.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2000
Location: USA

PostPosted: Wed Oct 18, 2023 2:10 am
Reply with quote

Digvijay Singh wrote:
I have explained the task clearly, I have few manual steps for which I am trying to automate, the manual steps already I have mentioned above, please let me which part you think is not clear.


I do not like your way of communication, nevertheless I can give you an idea on the initial part of your task. I hope you can do the rest on your own.

Code:
//*====================================================================
//LISTCAT EXEC PGM=IDCAMS                                             
//*                                                                   
//SYSPRINT DD  DISP=(NEW,PASS),SPACE=(TRK,(10,10)),                   
//             DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0),                   
//             DSN=&&LISTDSN                                           
//SYSIN    DD  *,SYMBOLS=EXECSYS                                       
 LISTCAT ENTRIES(#01TEST.BATCH.XXX.FILE,    -                 
                 #02TEST.BATCH.AAA.FILE,  -                 
                 #03TEST.BATCH.BBB.NEWACCT,  -                 
                 #04TEST.BATCH.CCC.NEWACCT ) -                 
         HISTORY                                                       
//*                                                                   
//*====================================================================
//SELECT  EXEC PGM=SORT,COND=(4,LT),                                   
//            PARM='VLTESTI=2'                                         
//*                                                                   
//SYSOUT   DD  SYSOUT=*                                               
//SORTIN   DD  DISP=(OLD,DELETE),DSN=&&LISTDSN                         
//RESULT   DD  SYSOUT=*                                               
//SYSIN    DD  *                                                       
 INCLUDE COND=(5,80,SS,EQ,L(C'GDG BASE ------ ',                       
                            C'NONVSAM ---- ',                         
                            C'CREATION--------'))                     

 INREC IFTHEN=(WHEN=GROUP,                                             
               BEGIN=(5,80,SS,EQ,L(C'GDG BASE ------ ',                 
                                   C'NONVSAM ---- ')),                 
               PUSH=(67:22,44)),                                       
       IFTHEN=(WHEN=INIT,                                               
               PARSE=(%1=(ABSPOS=67,                                   
                          ENDBEFR=C'.G',                               
                          FIXLEN=44)),                                 
               OVERLAY=(120:&DATE3(.),                                 
                        130:%1))                                       

 SORT FIELDS=(67,44,CH,D)                                               

 OUTFIL  FNAMES=RESULT,                                                 
         VTOF,                                                         
         REMOVECC,NODETAIL,                                             
         INCLUDE=(5,80,SS,EQ,C'CREATION--------'),                     
         SECTIONS=(130,44,                                             
                   HEADER3=(' VERSION (+0): ',67,44,                                     
                          X,'CREATED=',58,8,                           
                          X,'TODAY=',120,8))                           
//*                                                                     
//*=====================================================================


The result looks as follows:
Code:
********************************* TOP OF DATA *********************************************
VERSION (+0): #01TEST.BATCH.XXX.FILE.G0036V00       CREATED=2021.342 TODAY=2023.290
VERSION (+0): #02TEST.BATCH.AAA.NEWACCT.G2766V00    CREATED=2023.290 TODAY=2023.290
VERSION (+0): #03TEST.BATCH.BBB.NEWACCT.G0001V00    CREATED=2021.343 TODAY=2023.290
VERSION (+0): #04TEST.BATCH.CCC.FILE.G0037V00       CREATED=2021.317 TODAY=2023.290
******************************** BOTTOM OF DATA *******************************************
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Wed Oct 18, 2023 9:51 pm
Reply with quote

@sergeyken

Hey, I am sorry for that I deeply value your guidance and expertise. and I understand the importance of clear and concise communication. I had no intention to hurt you it was just I was not getting how i should explain. I will check the part of solution you have provided.

You all are great mentor here and fantastic mainframe buddies and I have huge respect for everyone here.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2000
Location: USA

PostPosted: Thu Oct 19, 2023 12:05 am
Reply with quote

Digvijay Singh wrote:
@sergeyken

Hey, I am sorry for that I deeply value your guidance and expertise. and I understand the importance of clear and concise communication. I had no intention to hurt you it was just I was not getting how i should explain. I will check the part of solution you have provided.

You all are great mentor here and fantastic mainframe buddies and I have huge respect for everyone here.


I recommend you to start from the first JCL step only, with //SYSPRINT temporary changed to SYSOUT=*. Run it with your GDG datasets, and get familiar with the listing it has created.

Next, add the second step to your investigation, and try to find out: how the SORT utility can extract only the data you may need from a full mess of LISTCAT listing.

The final task will be: to implement comparison of creation date, and current date provided by the given SORT step, and to add E-Mail step (usually, it depends on the system settings in your company). The E-Mail step should be controlled by RC returned by SORT utility while performing data fields compare.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Thu Oct 19, 2023 8:10 pm
Reply with quote

Thanks sergeyken for now I am working on first part of the requirement where i have below files.
Below files will be verified by the header date.
#ENV.Diggi.DAILY.GOANFILE(0)
#ENV.DIGGI.DAILY.KOANFILE(0)
#ENV.DIGGI.DAILY.TRNSFILE(0)

Above file has lot of data but i have extracted by sorting and brought it till here where this file has now record counts,header date and trailer date record i need to compare this date with date in another data set and record counts as well in other dataset.

Code:
ssssssssssssssssssssssssssssssssssssssssssssssssssssssss
VIEW       Diggi.DAILY.GOANFILE.OUTPUT5     
Command ===>                                           
=COLS> ----+----1----+----2----+----3----+----4----+----
****** ***************************** Top of Data *******                                       
000001 HEADER2024-07-24                                 
000002 TRAILER2024-07-24000101869
                                           record count starts from here see above record
****** *
*************************** Bottom of Data *****

I have another dataset from which i need to comapre above dates and record counts . if date matches then fine otherwise need to send mail to one team.

Code:
ssssssssssssssssssssssssssssssssssssssssssssssssss
 VIEW       Diggi.UTILITY.CARDLIB(mem1) - 01.00
 Command ===>                                     
 =COLS> ----+----1----+----2----+----3----+----4---
 ****** ***************************** Top of Data *
 000001 00101869  ------record count
 000001 VIRTDATE=2024/09/09 ---date to compare                       
 000002 VIRTTIME=*                                 
 ****** **************************** Bottom of Data
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2000
Location: USA

PostPosted: Thu Oct 19, 2023 8:31 pm
Reply with quote

It makes me crazy...

You are not able to explain clearly: WHERE YOUR FILE CREATION DATE CAME FROM???

People need to spend their time trying to guess what you are trying to keep in secret!!!

I retire from this topic. Good luck.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Thu Oct 19, 2023 8:42 pm
Reply with quote

Thank you for your support. I will take care from here on actually there were two requirements if you go above and see my first post there are 8 files out of 8 files 4 files i need to validate the date and by looking at it's data record header and trailer record not creation date and rest 4 files we need to validate the date against their creation date.

Anyway i am working on it.
Back to top
View user's profile Send private message
Pedro

Global Moderator


Joined: 01 Sep 2006
Posts: 2541
Location: Silicon Valley

PostPosted: Thu Oct 19, 2023 8:44 pm
Reply with quote

Quote:
I need to compare this date with date in another data set


Your example includes two dates. I recommend that you only work with the header date.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Thu Oct 19, 2023 8:44 pm
Reply with quote

I already have a dataset against which we have to validate the dates.
Back to top
View user's profile Send private message
Pedro

Global Moderator


Joined: 01 Sep 2006
Posts: 2541
Location: Silicon Valley

PostPosted: Thu Oct 19, 2023 9:02 pm
Reply with quote

Your requirement is somewhat complex. I think the best option is to use a rexx program to extract each of the dates and to compare them. Several days ago, Willy provided an excellent starting point for a rexx program.

But the two date formats are different. You need to parse the data line to extract the date from one file and convert it to the same format as the other file, then you can do the comparison.

Something like this:
Code:
PARSE VAR HeaderLine 'HEADER' HeaderDate  ' ' .
HeaderDate =  TRANSLATE(HeaderDate, '-', '/')



==========
re: different date formats

It would be better if your company used consistent formats when writing the date.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Thu Oct 19, 2023 9:03 pm
Reply with quote

@Pedro

Let me explain again it just i need to validate 3 things Record count,Header date and trailer which is in dataset1 against the dataset2:

record1=Header date
Record2=Trailer date and at end record count 00101869

Code:
Dataset1:
 Command ===>                       
 =COLS> ----+----1----+----2----+---
 ****** ****************************                 
 000001 HEADER2024-07-24           
 000002 TRAILER2024-07-24000101869 
 ****** ****************************


Validate above dataset against below dataset date and record count, Please note the date is having different format we need to find a way to validate these date and records count.

record1= Record count
record2= Date

Code:
Dataset2:

Command ===>                       
=COLS> ----+----1----+----2----+---
****** ****************************
000001 00101869   
000002 VIRTDATE=2024/09/09         
000003 VIRTTIME=*                 
****** ****************************
Back to top
View user's profile Send private message
Pedro

Global Moderator


Joined: 01 Sep 2006
Posts: 2541
Location: Silicon Valley

PostPosted: Thu Oct 19, 2023 9:13 pm
Reply with quote

Your requirements for case #1 are stated more clearly now. And you have been given advice to read the files with rexx, find the lines that are of interest and to parse out the target pieces of data.

If you have problems in rexx, show us what problems you have in your program. Though, perhaps start a new topic in the rexx forum.
Back to top
View user's profile Send private message
Digvijay Singh

Active User


Joined: 20 Apr 2022
Posts: 138
Location: India

PostPosted: Fri Oct 20, 2023 8:32 pm
Reply with quote

Will do this by sorting itself I am on it and will let the forum know.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   This topic is locked: you cannot edit posts or make replies. View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL Goto page 1, 2  Next

 


Similar Topics
Topic Forum Replies
No new posts Data set Rec-Cnt and Byte-Cnt Testing & Performance 2
No new posts SCOPE PENDING option -check data DB2 2
No new posts Check data with Exception Table DB2 0
No new posts JCL EXEC PARM data in C Java & MQSeries 2
No new posts Fetch data from programs execute (dat... DB2 3
Search our Forums:

Back to Top