IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Using SORT on Unicode data
IBM Mainframe Forums -> SYNCSORT
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Wed Feb 18, 2015 7:12 pm
 Reply with quote
Hi,

I have a situation where we unload data from table (tablespace is UNICODE)
in a flat file using IBM high performance unload in "ASIS" format.

The unloaded file is passed as input to SORT JCL.

Following steps are executed on the unload file.

1. STEP1 - Convert Unicode data to Ebcdic using SORT ALTSEQ.
2. STEP2 - Output of step1 is used to create a output file - This step performs some aggregations/sorting.

I want to know, how can i eliminate STEP1, that is I want to directly process Unicode data.

NOTE: I tried to run the same SORT JCL by eliminating STEP1, i get zero records in output file. So i think SORT only understands EBCDIC data.

Please help.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 8700
Location: Dubuque, Iowa, USA
PostPosted: Wed Feb 18, 2015 7:37 pm
 Reply with quote
Quote:
So i think SORT only understands EBCDIC data.
Not likely, but depending upon what the SORT statements are doing it is entirely possible that you need to convert them from EBCDIC to UNICODE for your sort to work properly.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Wed Feb 18, 2015 8:08 pm
 Reply with quote
Code:

 SORT FIELDS=(01,03,CH,A,                       
              04,06,CH,A,                     
              10,03,CH,A,                     
              13,04,CH,A,                     
              17,10,CH,A,                     
              27,10,CH,A,                     
              38,10,CH,A,                     
              64,19,CH,A,                     
              83,10,CH,D,                     
             109,08,CH,D,                     
             117,01,CH,A)                     
                                                     
 INCLUDE COND=(17,10,CH,GT,C'1900-01-01',AND,         
               118,1,BI,LT,X'C1')                     
                                                     
 OUTFIL FILES=OUT,NODETAIL,REMOVECC,                 
  SECTIONS=(01,03,                     
            04,06,                     
            10,03,                     
            13,04,                     
            17,10,                     
            27,10,                     
           117,01,                     
   TRAILER3=(001:001,03,X,             
             005:004,06,X,             
             012:010,03,X,             
             016:013,04,X,             
             021:017,10,X,             
             032:027,10,               
             042:037,01,X,             
             044:038,10,X,                       
             055:COUNT=(M11,LENGTH=10),X,     
             066:064,19,X,                   
             086:083,10,X,                   
             097:109,08,X,                   
             106:117,01))                     


This is my sort statement... It works perfectly fine when my input data is in EBCDIC.

I want to know is something can be done to achieve the same results on UNICODE data? It will save our time in conversion step1 - UNICODE -> EBCDIC.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Wed Feb 18, 2015 8:25 pm
 Reply with quote
If you show what you have working with ALTSEQ, then the two can be merged.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Wed Feb 18, 2015 8:51 pm
 Reply with quote
Code:
ALTSEQ CODE=(0000,0101,0202,0303,0437,052D,062E,072F,               
0816,0905,0A25,0B0B,0C0C,0D0D,0E0E,0F0F,                             
1010,1111,1212,1313,143C,153D,1632,1726,                             
1818,1919,1A3F,1B27,1C22,1D1D,1E35,1F1F,                             
2040,215A,227F,237B,245B,256C,2650,277D,                             
284D,295D,2A5C,2B4E,2C6B,2D60,2E4B,2F61,                             
30F0,31F1,32F2,33F3,34F4,35F5,36F6,37F7,                             
38F8,39F9,3A7A,3B5E,3C4C,3D7E,3E6E,3F6F,                             
407C,41C1,42C2,43C3,44C4,45C5,46C6,47C7,                             
48C8,49C9,4AD1,4BD2,4CD3,4DD4,4ED5,4FD6,                             
50D7,51D8,52D9,53E2,54E3,55E4,56E5,57E6,                             
58E7,59E8,5AE9,5BAD,5CE0,5DBD,5E5F,5F6D,                             
6079,6181,6282,6383,6484,6585,6686,6787,                             
6888,6989,6A91,6B92,6C93,6D94,6E95,6F96,                             
7097,7198,7299,73A2,74A3,75A4,76A5,77A6,                             
78A7,79A8,7AA9,7BC0,7C4F,7DD0,7EA1,7F07,                             
8043,8120,8221,831C,8423,85EB,8624,879B,                             
8871,8928,8A38,8B49,8C90,8DBA,8EEC,8FDF,                             
9045,9129,922A,939D,9472,952B,968A,979A,                             
9867,9956,9A64,9B4A,9C53,9D68,9E59,9F46,                             
A0EA,A1DA,A22C,A3DE,A48B,A555,A641,A7FE,                             
A858,A951,AA52,AB48,AC69,ADDB,AE8E,AF8D,                             
B073,B174,B275,B3FA,B415,B5B0,B6B1,B7B3,                             
B8B4,B9B5,BA6A,BBB7,BCB8,BDB9,BECC,BFBC,                             
C0AB,C13E,C23B,C30A,C4BF,C58F,C63A,C714,                             
C8A0,C917,CACB,CBCA,CC1A,CD1B,CE9C,CF04,                             
D034,D1EF,D21E,D306,D408,D509,D677,D770,                             
D8BE,D9BB,DAAC,DB54,DC63,DD65,DE66,DF62,                             
E030,E142,E247,E357,E4EE,E533,E6B6,E7E1,                             
E8CD,E9ED,EA36,EB44,ECCE,EDCF,EE31,EFAA,                             
F0FC,F19E,F2AE,F38C,F4DD,F5DC,F639,F7FB,                             
F880,F9AF,FAFD,FB78,FC76,FDB2,FE9F,FFFF)                             
                                                                     
INREC FIELDS=(1,118,TRAN=ALTSEQ)                                     
SORT  FIELDS=COPY                                                     


This is the ALTSEQ STEP1.
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1322
Location: Vilnius, Lithuania
PostPosted: Wed Feb 18, 2015 9:33 pm
 Reply with quote
UNICODE ¬= ASCII (which seems like what you're doing in your conversion)

And how do you translate multi-character UNICODE characters?

Developed in conjunction with the Universal Character Set standard and published as The Unicode Standard, the latest version of Unicode contains a repertoire of more than 110,000 characters covering 100 scripts and multiple symbol sets.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Feb 19, 2015 12:13 am
 Reply with quote
Subject to Prino's point, you can put together your two separate steps into one. On the INCLUDE you will need to specify the CH field as AQ. The LT X'C1' you'd have to decide what that means on the source data. X'C1' is "A" in EBCDIC, but the order of numbers and letters are different between ASCII and EBCDIC, so, what is that field you are testing, and what does the test tell you?
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 19, 2015 3:01 pm
 Reply with quote
Yes, that is A - 118,1,BI,LT,X'C1').

Prino, I dont think we have double byte characters in table... tablespace is defined in unicode format.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Feb 19, 2015 3:59 pm
 Reply with quote
Yes, but A is in a different position in the ASCII collating sequence than in the EBCDIC one, so if you use an ASCII A or X'C1' you'll get different results from your INCLUDE. So you'd need to work out what value to put there which will give you your desired results, if you want it all in one step.

Oh. And check your SyncSORT documentation. I am assuming in this case that the INCLUDE works with AQ the same way that DFSORT does.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 19, 2015 8:10 pm
 Reply with quote
Hi Bill,

I did not understand your point -
Quote:
Yes, but A is in a different position in the ASCII collating sequence than in the EBCDIC one, so if you use an ASCII A or X'C1' you'll get different results from your INCLUDE. So you'd need to work out what value to put there which will give you your desired results, if you want it all in one step.


What exactly you want me to find out?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Feb 19, 2015 8:28 pm
 Reply with quote
Sorry, just realised I'm probably chasing my tail.

If you instead change
Code:

      118,1,BI,LT,X'C1')   


To
Code:

      118,1,AQ,LT,C'A')   


You'll get the correct results (or the expected results from the code, anyway) without having to be concerned about what the X'C1' means in business terms, as the INCLUDE values being tested, for AQ fields, will have the ALTSEQ-translated values.

So, make all the fields in the INCLUDE AQ and you should be OK to merge the two steps into one.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 19, 2015 9:02 pm
 Reply with quote
Hi Bill,

I used the below sort card but did not get the expected results.

Code:

ALTSEQ CODE=(0000,0101,0202,0303,0437,052D,062E,072F,               
0816,0905,0A25,0B0B,0C0C,0D0D,0E0E,0F0F,                           
1010,1111,1212,1313,143C,153D,1632,1726,                           
1818,1919,1A3F,1B27,1C22,1D1D,1E35,1F1F,                           
2040,215A,227F,237B,245B,256C,2650,277D,                           
284D,295D,2A5C,2B4E,2C6B,2D60,2E4B,2F61,                           
30F0,31F1,32F2,33F3,34F4,35F5,36F6,37F7,                           
38F8,39F9,3A7A,3B5E,3C4C,3D7E,3E6E,3F6F,                           
407C,41C1,42C2,43C3,44C4,45C5,46C6,47C7,                           
48C8,49C9,4AD1,4BD2,4CD3,4DD4,4ED5,4FD6,                           
50D7,51D8,52D9,53E2,54E3,55E4,56E5,57E6,                           
58E7,59E8,5AE9,5BAD,5CE0,5DBD,5E5F,5F6D,                           
6079,6181,6282,6383,6484,6585,6686,6787,                           
6888,6989,6A91,6B92,6C93,6D94,6E95,6F96,                           
7097,7198,7299,73A2,74A3,75A4,76A5,77A6,                           
78A7,79A8,7AA9,7BC0,7C4F,7DD0,7EA1,7F07,                           
8043,8120,8221,831C,8423,85EB,8624,879B,                           
8871,8928,8A38,8B49,8C90,8DBA,8EEC,8FDF,                           
9045,9129,922A,939D,9472,952B,968A,979A,                           
9867,9956,9A64,9B4A,9C53,9D68,9E59,9F46,                           
A0EA,A1DA,A22C,A3DE,A48B,A555,A641,A7FE,                           
A858,A951,AA52,AB48,AC69,ADDB,AE8E,AF8D,                           
B073,B174,B275,B3FA,B415,B5B0,B6B1,B7B3,                           
B8B4,B9B5,BA6A,BBB7,BCB8,BDB9,BECC,BFBC,                           
C0AB,C13E,C23B,C30A,C4BF,C58F,C63A,C714,                           
C8A0,C917,CACB,CBCA,CC1A,CD1B,CE9C,CF04,                           
D034,D1EF,D21E,D306,D408,D509,D677,D770,                           
 D8BE,D9BB,DAAC,DB54,DC63,DD65,DE66,DF62,                           
 E030,E142,E247,E357,E4EE,E533,E6B6,E7E1,                           
 E8CD,E9ED,EA36,EB44,ECCE,EDCF,EE31,EFAA,                           
 F0FC,F19E,F2AE,F38C,F4DD,F5DC,F639,F7FB,                           
 F880,F9AF,FAFD,FB78,FC76,FDB2,FE9F,FFFF)                           
                                                                   
 INREC FIELDS=(1,118,TRAN=ALTSEQ)                                   
                                                                   
 SORT FIELDS=(01,03,AQ,A,                       
              04,06,AQ,A,                     
              10,03,AQ,A,                     
              13,04,AQ,A,                     
              17,10,AQ,A,                     
              27,10,AQ,A,                     
              38,10,AQ,A,                     
              64,19,AQ,A,                     
              83,10,AQ,D,                     
             109,08,AQ,D,                     
             117,01,AQ,A)                     
                                               
 INCLUDE COND=(17,10,AQ,GT,C'1900-01-01',AND,                       
               118,1,AQ,LT,C'A')                                   
                                                                   
 OUTFIL FILES=OUT,NODETAIL,REMOVECC,                               
  SECTIONS=(01,03,                   
            04,06,                   
            10,03,                   
           13,04,                   
           17,10,                   
           27,10,                   
          117,01,                   
  TRAILER3=(001:001,03,X,           
            005:004,06,X,           
            012:010,03,X,           
            016:013,04,X,           
            021:017,10,X,           
            032:027,10,             
            042:037,01,X,                      
            044:038,10,X,                   
            055:COUNT=(M11,LENGTH=10),X,     
            066:064,19,X,                   
            086:083,10,X,                   
            097:109,08,X,                   
            106:117,01))                     
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Fri Feb 20, 2015 3:43 am
 Reply with quote
OK, but you're going to have to show some sample input, the output you got, and the output you expect. Post the sysout from the step as well please.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Fri Feb 20, 2015 1:52 pm
 Reply with quote
Hi Bill,

Please find the details:

CASE1: Runs successfully - 2 STEP Process.

STEP010: CONVERT ASCII TO EBCDIC

INPUT DATASET: - SORTIN: SYSREC00 --> This is ASCII format - hence not readable in browse mode. I have shown you hex input.
Code:

 BROWSE    XXXXXX.XXXXXX.XXXXXX.SYSREC00                                                     Line 00000000 Col 039 118
 Command ===>                                                                                          Scroll ===> CSR
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+---
----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+---
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+---
 ------------------------------------------------------------------------------ -------------------------------------
********************************* Top of Data ************************************************************************
----------------------------------------------------------------------------------------------------------------------
........ï...................................................................................................âêìèñâ..+ê
3333333353333333333323323333332332330333323323323323323323333333333333333322222222333323323323323323323333334555443345
15847803710920132013D12D182013D12D1702014D08D25D07E37E55E68252801071291702000000002014D08D25D10E32E06E76553322849210E2
----------------------------------------------------------------------------------------------------------------------
............................................................................................................âêìèñâ..+.
3333333333333333333323323300000000003333323323323323323323333333333333333322222222333323323323323323323333334555443342
15814753973820132013D08D050000000000F2014D08D25D07E37E56E16109301072100079000000001900D01D01D01E01E01E00000022849203E0
----------------------------------------------------------------------------------------------------------------------
............................................................................................................âêìèñâ..+.
3333333333333333333323323300000000003333323323323323323323333333333333333322222222333323323323323323323333334555443342
15814753973820132013D02D010000000000F2014D08D25D07E37E56E16134201072100082000000001900D01D01D01E01E01E00000022849203E0
----------------------------------------------------------------------------------------------------------------------



SORT SYSIN:
Code:

 SYNCSORT FOR Z/OS  1.4.1.0N    U.S. PATENTS: 4210961, 5117495   (C) 2010 SYNCSO
                           
 SYSIN :                                                                       
   ALTSEQ CODE=(0000,0101,0202,0303,0437,052D,062E,072F,                       
   0816,0905,0A25,0B0B,0C0C,0D0D,0E0E,0F0F,                                     
   1010,1111,1212,1313,143C,153D,1632,1726,                                     
   1818,1919,1A3F,1B27,1C22,1D1D,1E35,1F1F,                                     
   2040,215A,227F,237B,245B,256C,2650,277D,                                     
   284D,295D,2A5C,2B4E,2C6B,2D60,2E4B,2F61,                                     
   30F0,31F1,32F2,33F3,34F4,35F5,36F6,37F7,                                     
   38F8,39F9,3A7A,3B5E,3C4C,3D7E,3E6E,3F6F,                                     
   407C,41C1,42C2,43C3,44C4,45C5,46C6,47C7,                                     
   48C8,49C9,4AD1,4BD2,4CD3,4DD4,4ED5,4FD6,                                     
   50D7,51D8,52D9,53E2,54E3,55E4,56E5,57E6,                                     
   58E7,59E8,5AE9,5BAD,5CE0,5DBD,5E5F,5F6D,                                     
   6079,6181,6282,6383,6484,6585,6686,6787,                                     
   6888,6989,6A91,6B92,6C93,6D94,6E95,6F96,                                     
   7097,7198,7299,73A2,74A3,75A4,76A5,77A6,                                     
   78A7,79A8,7AA9,7BC0,7C4F,7DD0,7EA1,7F07,                                     
   8043,8120,8221,831C,8423,85EB,8624,879B,                                     
   8871,8928,8A38,8B49,8C90,8DBA,8EEC,8FDF,                                     
   9045,9129,922A,939D,9472,952B,968A,979A,                                     
   9867,9956,9A64,9B4A,9C53,9D68,9E59,9F46,                                     
   A0EA,A1DA,A22C,A3DE,A48B,A555,A641,A7FE,                                     
  A858,A951,AA52,AB48,AC69,ADDB,AE8E,AF8D,                                     
  B073,B174,B275,B3FA,B415,B5B0,B6B1,B7B3,                                     
  B8B4,B9B5,BA6A,BBB7,BCB8,BDB9,BECC,BFBC,                                     
  C0AB,C13E,C23B,C30A,C4BF,C58F,C63A,C714,                                     
  C8A0,C917,CACB,CBCA,CC1A,CD1B,CE9C,CF04,                                     
  D034,D1EF,D21E,D306,D408,D509,D677,D770,                                     
  D8BE,D9BB,DAAC,DB54,DC63,DD65,DE66,DF62,                                     
  E030,E142,E247,E357,E4EE,E533,E6B6,E7E1,                                     
  E8CD,E9ED,EA36,EB44,ECCE,EDCF,EE31,EFAA,                                     
  F0FC,F19E,F2AE,F38C,F4DD,F5DC,F639,F7FB,                                     
  F880,F9AF,FAFD,FB78,FC76,FDB2,FE9F,FFFF)                                     
                                                                               
  INREC FIELDS=(1,118,TRAN=ALTSEQ)                                             
  SORT  FIELDS=COPY                                                           
WER276B  SYSDIAG= 3352052, 5944027, 5944027, 4574690                           
WER164B  8,872K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,             
WER164B     156K BYTES RESERVE REQUESTED, 2,424K BYTES USED                   
WER146B  24K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=   118; BLKSIZE= 27966                 
WER073I  SORTIN   : DSNAME=XXXXXX.XXXXXX.XXXXXX.SYSREC00                       
WER257I  INREC RECORD LENGTH =   118                                           
WER110I  SORTOUT  : RECFM=FB   ; LRECL=   118; BLKSIZE= 27966                 
WER074I  SORTOUT  : DSNAME=XXXXXX.XXXXXX.XXXXXX.EBCDIC               
WER410B  7,844K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,       
WER410B     0 BYTES RESERVE REQUESTED, 2,280K BYTES USED                       
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER416B  SORTIN   : EXCP'S=546,UNIT=3390,DEV=C930,CHP=(B0B1B2B3B4B59697,1),VOL=
WER416B  SORTOUT  : EXCP'S=552,UNIT=3390,DEV=C72D,CHP=(767778797A7B5859,1),VOL=
WER416B  TOTAL OF 1,098 EXCP'S ISSUED FOR COPYING                             
WER054I  RCD IN    2321815, OUT    2321815                                     
WER169I  RELEASE 1.4 BATCH 0525 TPF LEVEL 1.0                                 
WER052I  END SYNCSORT - XXXXXXXX,STEP010,,DIAG=EC00,734A,C026,004C,A4CA,49E2,06


SORTOUT:
In Plain browse format:
Code:

 BROWSE    XXXXXX.XXXXXX.XXXXXX.EBCDIC               Line 00000000 Col 001 080
 Command ===>                                                  Scroll ===> CSR 
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+---
********************************* Top of Data ***********************************************************************
15847803W10920132013-12-182013-12-17.2014-08-25-07.37.55.68252801071291702        2014-08-25-10.32.06.765533BRXTIB10NR
15814753973820132013-08-05..........?2014-08-25-07.37.56.16109301072100079        1900-01-01-01.01.01.000000BRXTIB03N
15814753973820132013-02-01..........?2014-08-25-07.37.56.16134201072100082        1900-01-01-01.01.01.000000BRXTIB03N


Same SORTOUT in HEX format:
Code:

 BROWSE    XXXXXX.XXXXXX.XXXXXX.EBCDIC               Line 00000000 Col 001 080
 Command ===>                                                  Scroll ===> CSR 
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+---
----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+---
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+---
 ------------------------------------------------------------------------------ -------------------------------------
********************************* Top of Data ***********************************************************************
                                                                                                                     
                                                                                                                     
 ------------------------------------------------------------------------------ -------------------------------------
15847803W10920132013-12-182013-12-17.2014-08-25-07.37.55.68252801071291702        2014-08-25-10.32.06.765533BRXTIB10NR
FFFFFFFFEFFFFFFFFFFF6FF6FFFFFF6FF6FF0FFFF6FF6FF6FF4FF4FF4FFFFFFFFFFFFFFFFF44444444FFFF6FF6FF6FF4FF4FF4FFFFFFCDEECCFFDD
15847803610920132013012018201301201702014008025007B37B55B68252801071291702000000002014008025010B32B06B7655332973921059
 ------------------------------------------------------------------------------ -------------------------------------
15814753973820132013-08-05..........?2014-08-25-07.37.56.16109301072100079        1900-01-01-01.01.01.000000BRXTIB03N
FFFFFFFFFFFFFFFFFFFF6FF6FF00000000006FFFF6FF6FF6FF4FF4FF4FFFFFFFFFFFFFFFFF44444444FFFF6FF6FF6FF4FF4FF4FFFFFFCDEECCFFD4
158147539738201320130080050000000000F2014008025007B37B56B16109301072100079000000001900001001001B01B01B0000002973920350
 ------------------------------------------------------------------------------ -------------------------------------
15814753973820132013-02-01..........?2014-08-25-07.37.56.16134201072100082        1900-01-01-01.01.01.000000BRXTIB03N
FFFFFFFFFFFFFFFFFFFF6FF6FF00000000006FFFF6FF6FF6FF4FF4FF4FFFFFFFFFFFFFFFFF44444444FFFF6FF6FF6FF4FF4FF4FFFFFFCDEECCFFD4
158147539738201320130020010000000000F2014008025007B37B56B16134201072100082000000001900001001001B01B01B0000002973920350
 ------------------------------------------------------------------------------ -------------------------------------   


STEP020: SORT n COUNT (LOGIC).

INPUT - SORTIN: XXXXXX.XXXXXX.XXXXXX.EBCDIC (SORTOUT of STEP010)

SORT SYSIN:


Code:

 SYNCSORT FOR Z/OS  1.4.1.0N    U.S. PATENTS: 4210961, 5117495   (C) 2010 SYNCSO
                             
 SYSIN :                                                                       
   SORT FIELDS=(01,03,CH,A,                       
                04,06,CH,A,                       
                10,03,CH,A,                       
                13,04,CH,A,                       
                17,10,CH,A,                       
                27,10,CH,A,                       
                38,10,CH,A,                       
                64,19,CH,A,                       
                83,10,CH,D,                       
               109,08,CH,D,                       
               117,01,CH,A)                       
                                                 
   INCLUDE COND=(17,10,CH,GT,C'1900-01-01',AND,                                 
                 118,1,BI,LT,X'C1')                                             
                                                                               
   OUTFIL FILES=OUT,NODETAIL,REMOVECC,                                         
    SECTIONS=(01,03,                   
              04,06,                   
              10,03,                   
              13,04,                   
              17,10,                   
             27,10,                   
            117,01,                   
    TRAILER3=(001:001,03,X,           
              005:004,06,X,           
              012:010,03,X,           
              016:013,04,X,           
              021:017,10,X,           
              032:027,10,             
              042:037,01,X,           
              044:038,10,X,           
              055:COUNT=(M11,LENGTH=10),X,       
              066:064,19,X,                     
              086:083,10,X,                     
              097:109,08,X,                     
              106:117,01))                     
WER276B  SYSDIAG= 3349193, 5941165, 5941165, 4574690                           
WER164B  65,540K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,           
WER164B     156K BYTES RESERVE REQUESTED, 65,512K BYTES USED                   
WER146B  24K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=   118; BLKSIZE= 27966                 
WER073I  SORTIN   : DSNAME=XXXXXX.XXXXXX.XXXXXX.EBCDIC               
WER110I  SORTOUT  : RECFM=FB   ; LRECL=   106; BLKSIZE= 27984                 
WER074I  SORTOUT  : DSNAME=XXXXXX.XXXXXX.XXXXXX.OUT                   
WER410B  63M BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,         
WER410B     0 BYTES RESERVE REQUESTED, 63M BYTES USED                         
WER036B  G=193629,B=480,BIAS=00                                               
WER162B  0 PREALLOCATED SORTWORK TRACKS, 5,580 DYNAMICALLY ALLOCATED,         
WER162B     0 ACQUIRED IN SECONDARY EXTENTS, 3,240 RELEASED, TOTAL OF 2,140 TRA
WER045C  END SORT PHASE                                                       
WER055I  INSERT          0, DELETE    1300390                                 
WER405I  SORTOUT  :  DATA RECORDS OUT          0; TOTAL RECORDS OUT      13428
WER493I  ZIIP PROCESSOR USED                                                   
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER416B  SORTIN   : EXCP'S=13,UNIT=3390,DEV=C72D,CHP=(767778797A7B5859,1),VOL=T
WER416B  OUTFIL WAS USED FOR SORTOUT                                           
WER416B  SORTWK01 : EXCP'S=19,UNIT=3390,DEV=BC08,CHP=A6A7A8A9AAAB8889,VOL=WORKD
WER416B  SORTWK02 : EXCP'S=7,UNIT=3390,DEV=C91F,CHP=B0B1B2B3B4B59697,VOL=WORKDE
WER416B  TOTAL OF 26 EXCP'S ISSUED FOR SORTWORKS                               
WER246I  FILESIZE 120,528,150 BYTES                                           
WER054I  RCD IN    2321815, OUT    1021425                                     
WER072I  NOEQUALS, ELAP IN EFFECT                                             
WER169I  RELEASE 1.4 BATCH 0525 TPF LEVEL 1.0                                 
WER052I  END SYNCSORT - XXXXXXXX,STEP020,,DIAG=E600,F958,CA3B,A457,BFC6,49E2,06


SORTOUT:

Plain Browse mode:
Code:

 BROWSE    XXXXXX.XXXXXX.XXXXXX.OUT                  Line 00000000 Col 001 080
 Command ===>                                                  Scroll ===> CSR 
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+-
********************************* Top of Data ************************************************************
158 N74108 106 2013 2013-10-30 ..........? 2014-08-25 0000000001 0105PD58807         1900-01-01 BRXTIB03 N
158 U1362Q AA7 2013 2013-01-31 ..........? 2014-08-25 0000000003 01063975012         1900-01-01 BRXTIB03 N
158 U1362Q AA7 2013 2013-05-01 ..........? 2014-08-25 0000000003 01063975012         1900-01-01 BRXTIB03 N


Same SORTOUT in hex mode:
Code:

BROWSE    XXXXXX.XXXXXX.XXXXXX.OUT                  Line 00000000 Col 001 080
Command ===>                                                  Scroll ===> CSR
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+-
----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+-
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+-
 ------------------------------------------------------------------------------ -------------------------
********************************* Top of Data ************************************************************
                                                                                                         
                                                                                                         
 ------------------------------------------------------------------------------ -------------------------
158 N74108 106 2013 2013-10-30 ..........? 2014-08-25 0000000001 0105PD58807         1900-01-01 BRXTIB03 N
FFF4DFFFFF4FFF4FFFF4FFFF6FF6FF4000000000064FFFF6FF6FF4FFFFFFFFFF4FFFFDCFFFFF444444444FFFF6FF6FF4CDEECCFF4D
15805741080106020130201301003000000000000F0201400802500000000001001057458807000000000190000100102973920305
 ------------------------------------------------------------------------------ -------------------------
158 U1362Q AA7 2013 2013-01-31 ..........? 2014-08-25 0000000003 01063975012         1900-01-01 BRXTIB03 N
FFF4EFFFFD4CCF4FFFF4FFFF6FF6FF4000000000064FFFF6FF6FF4FFFFFFFFFF4FFFFFFFFFFF444444444FFFF6FF6FF4CDEECCFF4D
15804136280117020130201300103100000000000F0201400802500000000003001063975012000000000190000100102973920305
 ------------------------------------------------------------------------------ -------------------------
158 U1362Q AA7 2013 2013-05-01 ..........? 2014-08-25 0000000003 01063975012         1900-01-01 BRXTIB03 N
FFF4EFFFFD4CCF4FFFF4FFFF6FF6FF4000000000064FFFF6FF6FF4FFFFFFFFFF4FFFFFFFFFFF444444444FFFF6FF6FF4CDEECCFF4D
15804136280117020130201300500100000000000F0201400802500000000003001063975012000000000190000100102973920305
 ------------------------------------------------------------------------------ -------------------------



CASE2: As suggested by you - Merging both SORT SYSIN:

INPUT: SORTIN - XXXXXX.XXXXXX.XXXXXX.SYSREC00 (Same input of STEP010 of CASE1)

SORT SYSIN: (Merged)
Code:

 SYNCSORT FOR Z/OS  1.4.1.0N    U.S. PATENTS: 4210961, 5117495   (C) 2010 SYNCSO
 
 SYNCSORT LICENSED FOR CPU SERIAL NUMBER XXXXXXX, MODEL XXXX XXX             LICEN
 SYSIN :                                                                       
   ALTSEQ CODE=(0000,0101,0202,0303,0437,052D,062E,072F,                       
   0816,0905,0A25,0B0B,0C0C,0D0D,0E0E,0F0F,                                     
   1010,1111,1212,1313,143C,153D,1632,1726,                                     
   1818,1919,1A3F,1B27,1C22,1D1D,1E35,1F1F,                                     
   2040,215A,227F,237B,245B,256C,2650,277D,                                     
   284D,295D,2A5C,2B4E,2C6B,2D60,2E4B,2F61,                                     
   30F0,31F1,32F2,33F3,34F4,35F5,36F6,37F7,                                     
   38F8,39F9,3A7A,3B5E,3C4C,3D7E,3E6E,3F6F,                                     
   407C,41C1,42C2,43C3,44C4,45C5,46C6,47C7,                                     
   48C8,49C9,4AD1,4BD2,4CD3,4DD4,4ED5,4FD6,                                     
   50D7,51D8,52D9,53E2,54E3,55E4,56E5,57E6,                                     
   58E7,59E8,5AE9,5BAD,5CE0,5DBD,5E5F,5F6D,                                     
   6079,6181,6282,6383,6484,6585,6686,6787,                                     
   6888,6989,6A91,6B92,6C93,6D94,6E95,6F96,                                     
   7097,7198,7299,73A2,74A3,75A4,76A5,77A6,                                     
   78A7,79A8,7AA9,7BC0,7C4F,7DD0,7EA1,7F07,                                     
   8043,8120,8221,831C,8423,85EB,8624,879B,                                     
   8871,8928,8A38,8B49,8C90,8DBA,8EEC,8FDF,                                     
   9045,9129,922A,939D,9472,952B,968A,979A,                                     
   9867,9956,9A64,9B4A,9C53,9D68,9E59,9F46,                                     
   A0EA,A1DA,A22C,A3DE,A48B,A555,A641,A7FE,                                     
  A858,A951,AA52,AB48,AC69,ADDB,AE8E,AF8D,                                     
  B073,B174,B275,B3FA,B415,B5B0,B6B1,B7B3,                                     
  B8B4,B9B5,BA6A,BBB7,BCB8,BDB9,BECC,BFBC,                                     
  C0AB,C13E,C23B,C30A,C4BF,C58F,C63A,C714,                                     
  C8A0,C917,CACB,CBCA,CC1A,CD1B,CE9C,CF04,                                     
  D034,D1EF,D21E,D306,D408,D509,D677,D770,                                     
  D8BE,D9BB,DAAC,DB54,DC63,DD65,DE66,DF62,                                     
  E030,E142,E247,E357,E4EE,E533,E6B6,E7E1,                                     
  E8CD,E9ED,EA36,EB44,ECCE,EDCF,EE31,EFAA,                                     
  F0FC,F19E,F2AE,F38C,F4DD,F5DC,F639,F7FB,                                     
  F880,F9AF,FAFD,FB78,FC76,FDB2,FE9F,FFFF)                                     
                                                                               
  INREC FIELDS=(1,118,TRAN=ALTSEQ)                                             
                                                 
  SORT FIELDS=(01,03,AQ,A,                     
               04,06,AQ,A,                     
               10,03,AQ,A,                     
               13,04,AQ,A,                     
               17,10,AQ,A,                     
               27,10,AQ,A,                     
               38,10,AQ,A,                     
               64,19,AQ,A,                     
               83,10,AQ,D,                     
              109,08,AQ,D,                     
              117,01,AQ,A)                     
                                                                               
  INCLUDE COND=(17,10,AQ,GT,C'1900-01-01',AND,                                 
                118,1,AQ,LT,C'A')                                             
                                                                               
  OUTFIL FILES=OUT,NODETAIL,REMOVECC,                                         
   SECTIONS=(01,03,                     
             04,06,                   
             10,03,                   
             13,04,                   
             17,10,                   
             27,10,                   
            117,01,                   
    TRAILER3=(001:001,03,X,           
              005:004,06,X,           
              012:010,03,X,           
              016:013,04,X,           
              021:017,10,X,           
              032:027,10,             
              042:037,01,X,           
              044:038,10,X,           
              055:COUNT=(M11,LENGTH=10),X,             
              066:064,19,X,                           
              086:083,10,X,                           
              097:109,08,X,                           
              106:117,01))                             
WER276B  SYSDIAG= 11668784, 13785134, 13785134, 2807635                       
WER164B  65,540K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,           
WER164B     156K BYTES RESERVE REQUESTED, 65,512K BYTES USED                   
WER146B  24K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=   118; BLKSIZE= 27966                 
WER073I  SORTIN   : DSNAME=XXXXXX.XXXXXX.XXXXXX.SYSREC00                       
WER257I  INREC RECORD LENGTH =   118                                           
WER110I  SORTOUT  : RECFM=FB   ; LRECL=   106; BLKSIZE= 27984                 
WER074I  SORTOUT  : DSNAME=XXXXXX.XXXXXX.XXXXXX.SYSREC00.OUT                   
WER410B  63M BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,         
WER410B     0 BYTES RESERVE REQUESTED, 63M BYTES USED                         
WER036B  G=27417                                                               
WER045C  END SORT PHASE                                                       
WER055I  INSERT          0, DELETE    2321815                                 
WER418I  DATASPACE(S) AND/OR ZSPACE USED                                       
WER405I  SORTOUT  :  DATA RECORDS OUT          0; TOTAL RECORDS OUT          1
WER493I  ZIIP PROCESSOR USED                                                   
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER416B  SORTIN   : EXCP'S=24,UNIT=3390,DEV=C930,CHP=(B0B1B2B3B4B59697,1),VOL=T
WER416B  OUTFIL WAS USED FOR SORTOUT                                           
WER246I  FILESIZE 0 BYTES                                                     
WER054I  RCD IN    2321815, OUT          0                                     
WER072I  NOEQUALS, BALANCE IN EFFECT                                           
WER169I  RELEASE 1.4 BATCH 0525 TPF LEVEL 1.0                                 
WER052I  END SYNCSORT - XXXXXXX,STEP010,,DIAG=8100,D258,A81E,A475,C9E3,6CC3,22


SORTOUT:
In Hex browse mode - Output has only 1 record instead of 13428 records.

Code:

----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+-
----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+-
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+-
 ------------------------------------------------------------------------------ -------------------------
********************************* Top of Data ************************************************************
                                                                                                         
                                                                                                         
 ------------------------------------------------------------------------------ -------------------------
                                                      0000000000                                         
444444444444444444444444444444444444444444444444444444FFFFFFFFFF444444444444444444444444444444444444444444
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
 ------------------------------------------------------------------------------ ------------------------- 
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1322
Location: Vilnius, Lithuania
PostPosted: Fri Feb 20, 2015 4:05 pm
 Reply with quote
gylbharat wrote:
Prino, I dont think we have double byte characters in table... tablespace is defined in unicode format.

You are obviously totally clueless about the format of Unicode.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Fri Feb 20, 2015 8:58 pm
 Reply with quote
prino wrote:
gylbharat wrote:
Prino, I dont think we have double byte characters in table... tablespace is defined in unicode format.

You are obviously totally clueless about the format of Unicode.


What i know - unicode supports character set > 255. It includes UTF-8 and UTF-16

UTF-8 is single byte character set and UTF-16 is double byte character set.

What I meant was, although our table is defined in unicode, it does not have any other characters which cannot be converted to EBCDIC/ASCII.
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1322
Location: Vilnius, Lithuania
PostPosted: Fri Feb 20, 2015 9:51 pm
 Reply with quote
gylbharat wrote:
prino wrote:
gylbharat wrote:
Prino, I dont think we have double byte characters in table... tablespace is defined in unicode format.

You are obviously totally clueless about the format of Unicode.


What i know - unicode supports character set > 255. It includes UTF-8 and UTF-16

UTF-8 is single byte character set and UTF-16 is double byte character set.

You amaze me by being even more clueless than I initially thought.

UTF-8 encodes each of the 1,112,064 valid code points in the Unicode code space (1,114,112 code points minus 2,048 surrogate code points) using one to four 8-bit bytes (a group of 8 bits is known as an octet in the Unicode Standard).
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Mon Feb 23, 2015 3:39 pm
 Reply with quote
Thanks Prino... I read about it and now have a better understanding about it.

Bill, any idea how to merge these two statements?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Feb 26, 2015 1:34 am
 Reply with quote
You have situations where you change the collating sequence of XX to YY and then change the collating sequence of YY to ZZ.

This means that an input of XX is treated in the INCLUDE as ZZ.

You can test that this is the case by removing all the changes to what would be EBCDIC numbers (F0-F9), the letter A (C1) and the - (2D).

Your INCLUDE will then "work".

If you genuinely need to change "overlapping" values from one encoding to another I don't think you can do it in one step if you also have the INCLUDE and the SORT. Not with ALTSEQ.

The date you are including on is in ascending order whether in ASCII or in EBCDIC. Without the ALTSEQ you can use a field-type of AC which will automatically convert the literal comparator to ASCII.

That leaves the A. So we're back to "what does that actually mean?".

If you don't know, I'd suggest identifying values with counts for that field, and seeing from the data what you are dealing with. If the A can be represented in the INCLUDE, either as AC or CH, then you can look to using a big FINDREP to do the encoding conversion.

If the amount of data is large doing this would save on passing the file twice. If small, probably not worth putting the two steps together. If in the middle, try both solutions (two steps and one) and see which fits the data better.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 26, 2015 4:47 pm
 Reply with quote
Thanks Bill for your inputs. I will try to merge the steps.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 26, 2015 5:40 pm
 Reply with quote
Hi Bill,

One more question -

Like suppose in SQL i have

SELECT PKEY, MIN(CHARCOL), MAX(DATE) FROM TABLE GROUP BY PKEY

can same be done through sort? Like calculating MIN and MAX on CHAR columns in SORT? I know sections and trailer3 can find min, max, count but only on numeric columns.

How can we calculate MIN/MAX on char columns?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Feb 26, 2015 6:16 pm
 Reply with quote
If MIN and MAX can only be numeric, make them numeric. CH takes account of all 256 bit-values per byte. BI takes account of all 256 bit-values per byte. Define your "CHAR" fields as BI.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 26, 2015 6:55 pm
 Reply with quote
Hi Bill,

I tried this by using BI but getting numeric field error:

Code:

   ALTSEQ CODE=(F030,F131,F232,F333,F434,F535,F636,F737,F838,F939)             
                                                                               
   INCLUDE COND=(17,10,CH,GT,X'313930302D30312D3031',AND,  * 1900-01-01         
                 118,1,BI,LT,X'41')                        * A                 
                                                                               
   SORT FIELDS=(01,03,CH,A,                       
                04,06,CH,A,                     
                10,03,CH,A,                     
                13,04,CH,A,                     
                17,10,CH,A,                     
                27,10,CH,A,                     
                38,10,CH,A,                     
                64,19,CH,D,                     
                83,10,CH,D,                     
               109,08,CH,D,                     
               117,01,CH,A)                     
                                                 
   OUTFIL FILES=OUT,NODETAIL,REMOVECC,                                         
    SECTIONS=(01,03,                     
              04,06,                   
              10,03,                   
              13,04,                   
              17,10,                   
              27,10,                   
            117,01,                     
    TRAILER3=(001:001,03,X,             
              005:004,06,X,             
              012:010,03,X,             
              016:013,04,X,             
              021:017,10,X,             
              032:027,10,               
              042:037,01,X,             
              044:038,10,X,             
              055:COUNT=(M11,LENGTH=10),
              066:MIN=(064,19,BI),X,   
                           *           
              086:083,10,X,             
              097:109,08,X,             
              106:117,01)),             
   BUILD=(1,54,55,10,TRAN=ALTSEQ,65,42,5X)                                     
WER271A  OUTFIL STATEMENT  : NUMERIC FIELD ERROR                               
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 


Is it something I am missing here?
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
PostPosted: Thu Feb 26, 2015 7:06 pm
 Reply with quote
Also, another question - I have specified =
Code:
BUILD=
statement to do TRAN=ALTSEQ of the count field, but it is not working. Is there a way to convert
Code:
055:COUNT=(M11,LENGTH=10)
into this ALTSEQ
Code:
 ALTSEQ CODE=(F030,F131,F232,F333,F434,F535,F636,F737,F838,F939)   

In same sort step?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Feb 26, 2015 7:11 pm
 Reply with quote
OK, so a short-coming in the approach. If you use BI, there is a limit to the length you can use, and 19 busts it.
Back to top
View user's profile Send private message
View previous topic : : View next topic  
Post new topic   Reply to topic All times are GMT + 6 Hours
Forum Index -> SYNCSORT Goto page 1, 2  Next

 

Similar Topics
Topic Forum Replies
No new posts Alternate key data extracttion COBOL Programming 13
No new posts Simplify SORT CARD to combine INREC a... DFSORT/ICETOOL 6
No new posts Sort card to generate the DSN DFSORT/ICETOOL 17
No new posts Sort card to know that creation date ... DFSORT/ICETOOL 10
No new posts Sort to insert record in an empty file. DFSORT/ICETOOL 5
Search our Forums:


Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us