Converting to and from CSV Format

ASN-1Step supports conversion of ASN.1 messages to and from CSV (Comma-Separated Values) format. To represent a structured ASN.1 message in CSV format, ASN-1Step flattens it into a fixed number of columns by recursively expanding complex SET, SEQUENCE, and CHOICE types up to the innermost fields with simple ASN.1 types. The resulting record includes a place for each simple type value, which can be empty for optional fields with absent values.

ASN-1Step supports two CSV formats, as follows:

• A row format, where each row is a record containing values separated by a comma or some other separator.
• A column format, where each value is placed on a separate line, possibly following an optional CSV header name and a column separator.

The CSV format can include an optional CSV header created by parsing the input PDU type definition. The header record consists of complete absolute references or single ASN.1 names that match the values in the rows or columns. The column values are generated by parsing encoded messages or value notation.

Example

ASN.1 type notation

PersonnelRecord ::= SEQUENCE {
              name     PrintableString,
              number   INTEGER OPTIONAL,
              children SEQUENCE OF ChildInformation
     }
     ChildInformation ::= SET {
              name    PrintableString,
              date    GeneralizedTime OPTIONAL
     }

ASN.1 value notation

value PersonnelRecord ::= {
             name "Nancy",
             number 20,
             children {
                     {name "James", date "20020312000000"},
                     {name "Martha", date "20100820000000"}
             }
     }

CSV row format

name,number,children_name,children_date
     "Nancy",20,"James",03/12/02 00:00:00
     "Nancy",20,"Martha",08/20/10 00:00:00

CSV Conversion Restrictions

There are some restrictions imposed on CSV conversion because

• ASN.1 PDU types are usually structured and can be nested to an arbitrary depth.
• Some ASN.1 types can be defined recursively.
• Open types or types with contents constraints can include other type values, known or unknown.

These restrictions are as follows:

• The values for nested recursions are supported only with the full CSV header format, which includes absolute references for all fields. Multiple CSV records of recursive types must have the same number of initialized nesting levels, so all records will have the same number of columns.
• The values of SET OF and SEQUENCE OF types with multiple components are mapped into several CSV records where only one component differs and the rest are repeated. SET OF and SEQUENCE OF types can be nested inside these types. By default, to avoid significant performance degradation, ASN-1Step supports an additional CSV for up to 50 components inside each type. To set the maximum number of components, pass an extra number parameter to the -csv option.
• Automatic decoding of open types is supported only with the full CSV header format, which includes nested PDU names or numbers inside parentheses. When open types are included, at any nesting level, within SET OF or SEQUENCE OF types, their containing PDUs must be the same for all components, so that the CSV structure does not change.
  
  Example

  ASN.1 value:
        ErrorEntry ::=  SEQUENCE {
                errorCode ERROR-CLASS.&code ({ErrorSet}),
                errorInfo ERROR-CLASS.&Type ({ErrorSet}{@.errorCode})
        }
        e ErrorEntry ::= { errorCode 10, errorInfo Int : 20}
  
  CSV:
        errorCode,errorInfo(Int)
        10,20

• The values of types with contents constraints and known containing types are always automatically decoded. If the CSV record includes values of such types in the form of xmlbstring (0101...) or xmlhstring (AB09...), an error is issued.
• By default, CSVs with string type values containing Unicode characters are written to the output file in binary format with UTF-8 character encoding. These CSVs are displayed using \x##, a special C-style character escape sequence. To save CSV in this format, use the text parameter of the -csv option.
  
  Example

  ASN.1 Value:
        myLabelUtf8 LabelUtf8 ::= "Umlaute: äÄöÖüÜ"
  
  CSV in text format:
         LabelUtf8
            "Umlaute: \xc3\xa4\xc3\x84\xc3\xb6\xc3\x96\xc3\xbc\xc3\x9c"

• When the input ASN.1 syntax includes SET or CHOICE types with fields whose tags are not in ascending order as required for PER, encoding and decoding in CSV format should be consistent with any field sorting for PER. So, if a CSV is created while sorting fields by tag because -per is specified (implicitly or explicitly), then -per should also be specified when decoding that CSV. This ensures that the order of the field names in the CSV header is the same as in the related SET or CHOICE internal structure, regardless of whether sorting occurs.
• Parameters in the -csv option affect both -encodeValue and -decodePDU options. When -decodePDU is specified with the input .csv file and -csv <flags> option, the flags are used to decode one or more CSVs from the input .csv file. If decoding succeeds, CSV re-encoding with the same flags is not performed. Conversely, when decoding is successful while using BER, PER, or other encoding rules, re-encoding is performed. If decoding fails, ASN-1Step attempts to determine the format of the input .csv file and decode it. If this decoding succeeds, then the output .csv file is re-encoded using the -csv option flags. If it fails, the error from the first decoding is re-issued.
• The values of SET OF and SEQUENCE OF types with multiple components are mapped into several CSV records where only one component differs and the rest are repeated. SET OF and SEQUENCE OF types can be nested inside these types. By default, to avoid significant performance degradation, ASN-1Step supports an additional CSV for up to 50 components inside each type. To set your own maximum number of components, pass an extra number parameter to the -csv option, which no longer has the limit of 50 as of version 10.1.

Type Mapping and Data Conversion

CSVs are in human-readable format similar to ASN.1 value notation. The following table shows differences sorted by ASN.1 type. Additional representations are available for CSVs by passing special flags to the -csv option.

Conversion of OCTET STRING type values to and from ASCII, BCD, or TBCD format is supported using the -csv option with the ascii, bcd, or tbcd parameter, or by using the OSS.PrintFunctionName directive with an OSS-specific function name for that conversion, for example "ossPrintOctetAsBCDString" for a BCD conversion. Applying the OSS.PrintFunctionName directive to OCTET STRING types converts their values to or from the specified format. For example, you can mark OCTET STRING types for conversion to TBCD, BCD, ASCII, IP address, or a special time stamp format using the following directives:

--<OSS.PrintFunctionName MOD.Seq.*.soctVarBcd  "ossPrintOctetAsBCDString">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarTbcd "ossPrintOctetAsTBCDString">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarIp   "ossPrintOctetAsIPAddress">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarAsc  "ossPrintOctetAsASCII">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarTime "ossPrintOctetAsTimeStamp">--

If the -csv option containing the ascii, bcd, or tbcd parameter is specified, the type values to which the directives are applied will not be affected. However, other OCTET STRING type values will be converted to the format specified in the -csv option.

When using ASCII to convert an OCTET STRING hstring value that starts with one or more space characters (20), the resulting CSV value is enclosed in quotation marks, which indicates that spaces are part of the value. When decoding, ASN-1Step ignores quotation marks that follow space characters.
Example

ASN.1:
      Oct ::= OCTET STRING
      oct Oct ::= '20206C65616469616720737061636573'H

CSV:
      Oct
      "  leading spaces"

---