JSONVALID to validate the JSON

zmoney · New User Joined: 15 Jan 2021 Posts: 4 Location: PL

I'm writing a PL/I program that reads JSON. I'm using the PL/I builtin function JSONVALID to validate the JSON before processing it. However, this requires the entire input file to be read into a buffer in memory because the first parameter to JSONVALID must be a pointer to a buffer.

I'm quite new to PL/I and I've had trouble finding the answers in manuals.
1) How can I determine the size of an input dataset?
2) What if the input dataset is SYSIN, does that matter?
3) What's the best strategy for copying an entire file to an memory buffer?
4) Does z/OS support memory mapped files?

Garry Carroll · Posted: Fri Jul 02, 2021 12:56 pm

Theoretically, a given dataset could be so large as not to fit into available memory.

I would imaging that you would validate each record in the dataset rather than trying to validate the dataset as a single unit, in which case set the pointer to the record read and set the length to the length of that record. Repeat until EOF.

If I'm correct, you don't need answers to the questions asked.

Garry.

enrico-sorichetti · Posted: Fri Jul 02, 2021 1:14 pm

www.slideshare.net/darintocommIT/pl1-andjsonpeterelderon
could be a good starting point to determine what questions to ask ....

format of the buffer, format of the input data
aftere that is just good old PL1,
lots of string handling, stripping useless whitespace and so on

zmoney · New User Joined: 15 Jan 2021 Posts: 4 Location: PL

After doing some research, I have a better idea but have a new challenge.

Like Garry pointed out, the maximum dataset size could be so large that it would not fit into memory all at once. But SYSIN does have a limit.
In JES2, the maximum length of a record written to the internal reader is 32760 bytes. In JES3, the maximum length is the installation defined buffer size.
www.ibm.com/docs/en/zos/2.3.0?topic=reader-record-length-sysin-data-sets

32760 is actually a reasonable maximum size for JSON, as that format is not optimal for large quantities of data, but is well suited for relatively small arrays of data or data objects representing configurations, and I believe SYSIN is also the standard for passing configuration info to IBM utilities, so it makes even more sense to align with that.

However, JSONVALID parses a UTF8 string.

If x has the CHAR type, the length of UTF8(x) might be two times as large as the length of x.
www.ibm.com/docs/en/SSY2V3_5.2.0/com.ibm.ent.pl1.zos.doc/lrm.pdf

So, converting a 32760 byte buffer of EBCDIC requires a buffer twice as large for the UTF8. But the PL/I compiler limits DCL CHAR(x) to 32767.
I'll have to limit the UTF8 buffer to 32760 and the EBCDIC one to half that, for now, and leave the headache of working around the compiler limit for another day.

prino · Posted: Sat Jul 03, 2021 10:07 pm