Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Generate Checksum in mainframe

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming
View previous topic :: :: View next topic  
Author Message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 1:25 pm    Post subject: Generate Checksum in mainframe
Reply with quote

Hello,

Hope you are all fine. I have a requirement to generate checksum. I give the same input file to unix and mainframe. My objective is to verify if both the systems (Unix and Mainframe) are producing the same output (I am working on a migration project).

So, instead of comparing the full file every time, i believe generating a checksum and comparing the checksum will be more effective.

I got to know that there is no utility available to do this. should i write my own program? if yes, do we have an algorithm that produces checksum.

Note: My objective is not about scurity of the file, its just to check the correctness.

Please advise.
Back to top
View user's profile Send private message

shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 2:34 pm    Post subject:
Reply with quote

I am currently checking SAS MD5, AMASPZAP. But these doesn't give me what i want.

MD5 and ZAP gives me a the checksum for a particular record or i have a limitation of 200 bytes. But i want the checksum for the whole file.
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10203
Location: italy

PostPosted: Tue Jun 17, 2014 6:02 pm    Post subject: Reply to: Checksum mainframe
Reply with quote

You might want to look at FILE 900 at www.cbttape.org
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 6:16 pm    Post subject:
Reply with quote

I am currently checking SAS MD5, AMASPZAP. But these doesn't give me what i want.

MD5 and ZAP gives me a the checksum for a particular record or i have a limitation of 200 bytes. But i want the checksum for the whole file.
Back to top
View user's profile Send private message
Ed Goodman

Active Member


Joined: 08 Jun 2011
Posts: 556
Location: USA

PostPosted: Tue Jun 17, 2014 6:18 pm    Post subject:
Reply with quote

Do you have the Unix stuff available on the mainframe? Like OMVS?

I used that once to generate a checksum. The problem is that you have to convert the mainframe file to ascii to get the same results.
Back to top
View user's profile Send private message
steve-myers

Active User


Joined: 30 Nov 2013
Posts: 462
Location: The Universe

PostPosted: Tue Jun 17, 2014 6:41 pm    Post subject:
Reply with quote

There are two fundamental problems.
  • Text data in a mainframe is in the EBCDIC character set; text data on a work station is usually in a different character set.
  • Data set formats on a mainframe are usually in a different format than a workstation.
So, obtaining a checksum of the data is comparing apples to oranges; you will usually get a different checksum.
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 7:25 pm    Post subject:
Reply with quote

Quote:
Do you have the Unix stuff available on the mainframe? Like OMVS?


I do not have.

I understand that i am trying to compare apple and orange but unfortunately i have this requirement from the customer. i have to find a way to achieve it.

Enrico : I will check the URL and let everybody know what i find.
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 7:27 pm    Post subject:
Reply with quote

If not checksum, what is the way to achieve it?
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 7:31 pm    Post subject:
Reply with quote

I am thinking of writing a record level checksum in another file using MD5 - SAS, and add all the record level checksum to make it one, do the same in unix and compare both.

Will this work? does anybody know?

SAS - MD5 produces checksum for any given string. This is not character set dependent i believe.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7913
Location: Bellevue, IA

PostPosted: Tue Jun 17, 2014 8:08 pm    Post subject:
Reply with quote

From the SAS documentation:
Quote:
The Basics

The MD5 function converts a string, based on the MD5 algorithm, into a 128-bit hash value. This hash value is referred to as a message digest (digital signature), which is nearly unique for each string that is passed to the function.

The MD5 function does not format its own output. You must specify a valid format (such as hex32. or binary128.) to view readable results.

Operating Environment Information: In the z/OS operating environment, the MD5 function produces output in EBCDIC rather than in ASCII. Therefore, the output will differ
so your statement
Quote:
SAS - MD5 produces checksum for any given string. This is not character set dependent i believe.
is not correct. If you wish to use SAS MD5 you need to convert ASCII to EBCDIC (or vice versa) on one platform before using MD5.
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Jun 17, 2014 8:52 pm    Post subject:
Reply with quote

Robert that was very useful. Thanks.

But do you guys agree with the below statements,

1) There is no existing IBM utility that does this.
2) We do not have a sas function that creates a file level checksum.

One more thing,

If you know of any 3rd product that does this, please suggest. My customer is ok to buy a lisence.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Tue Jun 17, 2014 9:15 pm    Post subject: Reply to: Checksum mainframe
Reply with quote

You won't get any type of MD5 checksum to match across one system using EBCDIC and another using ASCII and one system being "Big Endian" and other being "Little Endian".

How many files do you want to do this for? Is the data entirely "textual" (no binary fields, no floating-point, no packed-decilogicallymal, no BCD, no control-characters (other than for line/file end on the *nix)?

If so, you can do it yourself. You have a table where each textual character is assigned a unique numeric value. You use the same table on the Mainframe and on the other system. You look at each record, byte-by-byte, in the table, and accumulate a total across the entire file.

Note some problems. You could mix-up the data on one file, partially or entirely, and still get the same result. Different data will "likely" stand out (can be made more likely by using higher prime numbers for your reference table), but unless you also add "weight" for character-position, you're a little exposed. You also need to "weight" for record-number (else two records could be swapped and not noticed). Getting trickier, eh?

The really simple way, if the files are supposed to be identical, is to transfer one or the other, so they are both on the same character-set. Then compare. If insisting on checksumming, do some weighting yourself for record position.

I suspect whoever has made the decision to checksum is not aware that it is not really going to work.

And that is with textual-only data...

And that is if the files are actually supposed to be logically identical as well.

Phsyically identical on the two different systems, forget it. Physically identical if both files on the same OS? Possible for textual-date only.

Unless you have a tool which checksums a file (that being its task, so it knows the importance of the order of things, one a record, and within a file) and the data is textual, just forget the checksum as a "checksum".

And please, this time, don't wander off leaving us not knowing the outcome.
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Wed Jun 18, 2014 1:11 pm    Post subject:
Reply with quote

At this point i have two feasible solutions,

1) Creating a table with unique value for each character and use it in UNIX and MF, write my own checksum routine.

2) Use MD5 SAS. (MF file needs to be converted into ASCII)

But I have around 3000 files to compare and these files will be created every month.

Both the solutions are time consuming, this will also run for a long time, consumes lot of MIPS. So both are costly solutions and i cannot affort this. Still searching for other ways.
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Wed Jun 18, 2014 1:17 pm    Post subject:
Reply with quote

One more update: I didn't find what i want in www.cbttape.org.
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10203
Location: italy

PostPosted: Thu Jun 19, 2014 1:00 pm    Post subject: Reply to: Checksum mainframe
Reply with quote

Quote:
I didn't find what i want

??? ??? ???

NOBODY will provide code or ready to run solutions for specific needs ...

here is the read me text of file 900

Code:

//***FILE 900 is a set of programs which run under either old MVS   *   FILE 900
//*           or z/OS to calculate MD5 checksums.  Please see the   *   FILE 900
//*           member called @FILEMD5, which contains descriptions   *   FILE 900
//*           of all the pds members found in this file.            *   FILE 900
//*                                                                 *   FILE 900
//*           support:  sbgolob@cbttape.org  or                     *   FILE 900
//*                     sbgolob@attglobal.net                       *   FILE 900
//*                                                                 *   FILE 900
//*      MD5      - Assembler source for MD5 subprogram, which      *   FILE 900
//*                 is a modification of the MD5 program for        *   FILE 900
//*                 REXX.  This must be linked with the MD5SUM      *   FILE 900
//*                 TSO command.                                    *   FILE 900
//*                                                                 *   FILE 900
//*      MD5A     - Assembler source for same MD5 subprogram,       *   FILE 900
//*                 with inline macros changed for IFOX00           *   FILE 900
//*                 assembler.  Intended for MVS370 systems.        *   FILE 900
//*                                                                 *   FILE 900
//*      MD5R     - Assembler source for same MD5 subprogram,       *   FILE 900
//*                 with 8 STCM instructions replaced with 2        *   FILE 900
//*                 STRV.  STRV is comparatively new, from circa    *   FILE 900
//*                 2002.  STRV is like ST but the bytes are        *   FILE 900
//*                 stored in reverse order, as in hex 12345678     *   FILE 900
//*                 being stored as hex 78563412. The MVS 3.8       *   FILE 900
//*                 assembler IFOX00 does not support the STRV      *   FILE 900
//*                 op code.                                        *   FILE 900
//*                                                                 *   FILE 900
//*      MD5COB$  - JCL to compile and run an Enterprise Cobol      *   FILE 900
//*                 program that calls the MD5 subroutine.          *   FILE 900
//*                                                                 *   FILE 900
//*      MD5DATA  - JCL to create test data for MD5SUM, the same    *   FILE 900
//*                 test data used in MD5REXX                       *   FILE 900
//*                                                                 *   FILE 900
//*      MD5FORT$ - JCL to compile and run a Fortran G program      *   FILE 900
//*                 that calls the MD5 subroutine.                  *   FILE 900
//*                                                                 *   FILE 900
//*      MD5PLI$  - JCL to compile and run an Enterprise PL/I       *   FILE 900
//*                 program that calls the MD5 subroutine.          *   FILE 900
//*                                                                 *   FILE 900
//*      MD5REXX  - A copy of the source code for the MD5 for       *   FILE 900
//*                 REXX by Leland Lucius, which was the base       *   FILE 900
//*                 from which member MD5 was created.              *   FILE 900
//*                                                                 *   FILE 900
//*      MD5SUM   - assembler source for the 31-bit MD5SUM TSO      *   FILE 900
//*                 command.  It calls the MD5 subprogram.          *   FILE 900
//*                                                                 *   FILE 900
//*      MD5SUM$  - JCL to assemble MD5SUM and the MD5              *   FILE 900
//*                 subprogram.                                     *   FILE 900
//*                                                                 *   FILE 900
//*      MD5SUM#  - TSO HELP text for the MD5SUM TSO command.       *   FILE 900
//*                                                                 *   FILE 900
//*      MD5SUMA$ - JCL to assemble MD5SUM24 and MD5A using         *   FILE 900
//*                 IFOX00.  Intended for MVS370 systems.           *   FILE 900
//*                                                                 *   FILE 900
//*      MD5SUM2$ - JCL to assemble MD5SUM24 and MD5 using          *   FILE 900
//*                 ASMA90.                                         *   FILE 900
//*                                                                 *   FILE 900
//*      MD5SUM24 - Assembler source for the 24-bit MD5SUM TSO      *   FILE 900
//*                 command, which is the base from which the       *   FILE 900
//*                 31-bit version was converted. Much of this      *   FILE 900
//*                 code was borrowed from the COUNT TSO command    *   FILE 900
//*                 in File 300.  Intended for MVS 3.8 systems.     *   FILE 900
//*                                                                 *   FILE 900
//*      MD5URL   - Information about testing Leland Lucius's code  *   FILE 900
//*                 in MD5REXX                                      *   FILE 900
//*                                                                 *   FILE 900


note the description of the MD5SUM program.

should not be so difficult to modify/adapt it to Your needs
Back to top
View user's profile Send private message
Ed Goodman

Active Member


Joined: 08 Jun 2011
Posts: 556
Location: USA

PostPosted: Thu Jun 19, 2014 7:42 pm    Post subject:
Reply with quote

I built a CRC32 hasher in COBOL, to be able to make a zip-compatible file.

Is that anything at all like what you need?
Back to top
View user's profile Send private message
shankarm

Active User


Joined: 17 May 2010
Posts: 175
Location: India

PostPosted: Tue Oct 07, 2014 4:25 pm    Post subject:
Reply with quote

Just wanted to close this. Below is what we did,

We we transferred the file from mainframe to HDFS and wrote a checksum algorithm in hadoop to compare the files generated in both the environment.
Back to top
View user's profile Send private message
Ed Goodman

Active Member


Joined: 08 Jun 2011
Posts: 556
Location: USA

PostPosted: Tue Oct 07, 2014 8:01 pm    Post subject:
Reply with quote

nice!
Back to top
View user's profile Send private message
ApexNC

New User


Joined: 10 Feb 2006
Posts: 12
Location: USA

PostPosted: Tue Oct 07, 2014 9:45 pm    Post subject: Reply to: Checksum mainframe
Reply with quote

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/DZ9ZR003/7.5.23 - The CHECKSUM instruction is highly precise, but must be coded in Assembler.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
This topic is locked: you cannot edit posts or make replies. Urgent Mainframe Requirments (2-10 yr... Shankar779 Mainframe Jobs 0 Sun Nov 27, 2016 1:48 pm
This topic is locked: you cannot edit posts or make replies. Mainframe Developer requirement in Ku... capricorn Mainframe Jobs 0 Sun Oct 23, 2016 3:49 pm
No new posts Interface from Mainframe to Confluenc... hblun CLIST & REXX 4 Tue Sep 20, 2016 5:47 pm
This topic is locked: you cannot edit posts or make replies. Mainframe Admin Openings with CTS enikhilk Mainframe Jobs 0 Tue Sep 13, 2016 10:13 am
No new posts Cards mainframe developer requirement... capricorn Mainframe Jobs 0 Sun Sep 11, 2016 9:57 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us