Polymorph Data Encoding

Jakob Jenkov
Last update: 2024-08-14

The Polymorph Data format uses a compact, fast to read and write, versatile binary encoding. This text specifies how that encoding looks. As mentioned, Polymorph Data uses a binary encoding - also sometimes referred to as Polymorph Binary - or Pinary for short.

Polymorph also contains a textual data format which can be converted to Polymorph binary easily. That way you can more easily create Pinary files. This textual-to-binary data format is also referred to as Textual Binary or Tinary.

Polymorph Data Encoding

The Polymorph Data Encoding uses a simple Type-Length-Value format. All data is encoded into fields. A field has a type. The very first byte of a field tells the type. Depending on the field type (the first byte) the rest of the field's bytes are interpreted differently. Polymorph Data Encoding has the following archetype field encodings:

Type => Value
Type => Length Bytes => Value

Type => Value

Fields that follow the [Type => Value] encoding will have:

1 Type byte
N Value bytes

The number of value bytes depends on the type of the field. For instance, the type INT_POS_1_BYTES means a "positive integer of 1 bytes in length". Thus, the length of the field value is implicitly derived from the field type.

Type => Length => Value

Fields that follow the [Type => Length => Value] encoding will have:

1 Type byte
0 to 15 Length bytes
N Value bytes

The number of length bytes depends on the type of the field. For instance, the type BYTES_2_LENGTH_BYTES means a "byte field with 2 length bytes - meaning a possible field value length of 0 to 65535 bytes".

Little Endian Integer Encoding

All integer numbers that require more than 1 byte to represent, are encoded using a Little Endian encoding, meaning the first byte contains the least significant byte of the number.

Polymorph Data Field Types

Remember, the first byte of a Polymorph Data field is the field type.

Polymorph contains the following field types:

Field Type	Type Code	Description
BOOLEAN_NULL	0	A single byte with the value 0 - representing a boolean field with a null value.
BOOLEAN_TRUE	1	A single byte with the value 1 - representing a boolean field with the value True.
BOOLEAN_FALSE	2	A single byte with the value 2 - representing a boolean field with the value False.
INT_NULL	3	A single byte with the value 3 - representing a null value of type integer.
INT_POS_1_BYTES	4	1 type byte + 1 value byte - containing a positive integer value of 1 byte (0 to 255)
INT_POS_2_BYTES	5	1 type byte + 2 value bytes - containing a positive integer value of 2 bytes (0 to 65535)
INT_POS_3_BYTES	6	1 type byte + 3 value bytes - containing a positive integer value of 3 bytes (0 to 2^24-1)
INT_POS_4_BYTES	7	1 type byte + 4 value bytes - containing a positive integer value of 4 bytes (0 to 2^32-1)
INT_POS_5_BYTES	8	1 type byte + 5 value bytes - containing a positive integer value of 5 bytes (0 to 2^40-1)
INT_POS_6_BYTES	9	1 type byte + 6 value bytes - containing a positive integer value of 6 bytes (0 to 2^48-1)
INT_POS_7_BYTES	10	1 type byte + 7 value bytes - containing a positive integer value of 7 bytes (0 to 2^56-1)
INT_POS_8_BYTES	11	1 type byte + 8 value bytes - containing a positive integer value of 8 bytes (0 to 2^64-1)
INT_NEG_1_BYTES	12	1 type byte + 1 value byte - containing a positive integer representing a negative integer value of 1 byte (0 to 255 meaning -1 to -256)
INT_NEG_2_BYTES	13	1 type byte + 2 value bytes - containing a positive integer representing a negative integer value of 2 bytes (0 to 2^16-1 meaning -1 to -2^16)
INT_NEG_3_BYTES	14	1 type byte + 3 value bytes - containing a positive integer representing a negative integer value of 3 bytes (0 to 2^24-1 meaning -1 to -2^24)
INT_NEG_4_BYTES	15	1 type byte + 4 value bytes - containing a positive integer representing a negative integer value of 4 bytes (0 to 2^32-1 meaning -1 to -2^32)
INT_NEG_5_BYTES	16	1 type byte + 5 value bytes - containing a positive integer representing a negative integer value of 5 bytes (0 to 2^40-1 meaning -1 to -2^40)
INT_NEG_6_BYTES	17	1 type byte + 6 value bytes - containing a positive integer representing a negative integer value of 6 bytes (0 to 2^48-1 meaning -1 to -2^48)
INT_NEG_7_BYTES	18	1 type byte + 7 value bytes - containing a positive integer representing a negative integer value of 7 bytes (0 to 2^56-1 meaning -1 to -2^56)
INT_NEG_8_BYTES	19	1 type byte + 8 value bytes - containing a positive integer representing a negative integer value of 8 bytes (0 to 2^64-1 meaning -1 to -2^64)
FLOAT_NULL	20	1 type byte representing a float field with a null value
FLOAT_4_BYTES	21	1 type byte + 4 value bytes - containing a 4 byte float value. Encoding is the same as in Java.
FLOAT_8_BYTES	22	1 type byte + 8 value bytes - containing an 8 byte float value (AKA double). Encoding is the same as in Java.
BYTES_NULL	23	1 type byte representing a field of type byte sequence - with a null value.
BYTES_0_BYTES	24	1 type byte + 0 value bytes - representing field of type byte sequence - with a zero length.
BYTES_1_BYTES	25	1 type byte + 1 value bytes - representing field of type byte sequence - with a 1 byte length (a 1 byte byte array).
BYTES_2_BYTES	26	1 type byte + 2 value bytes - representing field of type byte sequence - with a 2 byte length (a 2 byte byte array).
BYTES_3_BYTES	27	1 type byte + 3 value bytes - representing field of type byte sequence - with a 3 byte length (a 3 byte byte array).
BYTES_4_BYTES	28	1 type byte + 4 value bytes - representing field of type byte sequence - with a 4 byte length (a 4 byte byte array).
BYTES_5_BYTES	29	1 type byte + 5 value bytes - representing field of type byte sequence - with a 5 byte length (a 5 byte byte array).
BYTES_6_BYTES	30	1 type byte + 6 value bytes - representing field of type byte sequence - with a 6 byte length (a 6 byte byte array).
BYTES_7_BYTES	31	1 type byte + 7 value bytes - representing field of type byte sequence - with a 7 byte length (a 7 byte byte array).
BYTES_8_BYTES	32	1 type byte + 8 value bytes - representing field of type byte sequence - with a 8 byte length (a 8 byte byte array).
BYTES_9_BYTES	33	1 type byte + 9 value bytes - representing field of type byte sequence - with a 9 byte length (a 9 byte byte array).
BYTES_10_BYTES	34	1 type byte + 10 value bytes - representing field of type byte sequence - with a 10 byte length (a 10 byte byte array).
BYTES_11_BYTES	35	1 type byte + 11 value bytes - representing field of type byte sequence - with a 11 byte length (a 11 byte byte array).
BYTES_12_BYTES	36	1 type byte + 12 value bytes - representing field of type byte sequence - with a 12 byte length (a 12 byte byte array).
BYTES_13_BYTES	37	1 type byte + 13 value bytes - representing field of type byte sequence - with a 13 byte length (a 13 byte byte array).
BYTES_14_BYTES	38	1 type byte + 14 value bytes - representing field of type byte sequence - with a 14 byte length (a 14 byte byte array).
BYTES_15_BYTES	39	1 type byte + 15 value bytes - representing field of type byte sequence - with a 15 byte length (a 15 byte byte array).
BYTES_1_LENGTH_BYTES	40	1 type byte + 1 length byte + N length bytes - representing field of type byte sequence with a max length of 2^8-1 bytes.
BYTES_2_LENGTH_BYTES	41	1 type byte + 2 length byte + N length bytes - representing field of type byte sequence with a max length of 2^16-1 bytes.
BYTES_3_LENGTH_BYTES	42	1 type byte + 3 length byte + N length bytes - representing field of type byte sequence with a max length of 2^24-1 bytes.
BYTES_4_LENGTH_BYTES	43	1 type byte + 4 length byte + N length bytes - representing field of type byte sequence with a max length of 2^32-1 bytes.
BYTES_5_LENGTH_BYTES	44	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^40-1 bytes.
BYTES_6_LENGTH_BYTES	45	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^48-1 bytes.
BYTES_7_LENGTH_BYTES	46	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^56-1 bytes.
BYTES_8_LENGTH_BYTES	47	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^64-1 bytes.
UTF_8_NULL	48	1 type byte representing a field of type UTF-8 - with a null value.
UTF_8_0_BYTES	49	1 type byte + 0 value bytes - representing field of type UTF-8 sequence - with a zero length.
UTF_8_1_BYTES	50	1 type byte + 1 value bytes - representing field of type UTF-8 sequence - with a 1 byte length (a 1 byte byte array).
UTF_8_2_BYTES	51	1 type byte + 2 value bytes - representing field of type UTF-8 sequence - with a 2 byte length (a 2 byte byte array).
UTF_8_3_BYTES	52	1 type byte + 3 value bytes - representing field of type UTF-8 sequence - with a 3 byte length (a 3 byte byte array).
UTF_8_4_BYTES	53	1 type byte + 4 value bytes - representing field of type UTF-8 sequence - with a 4 byte length (a 4 byte byte array).
UTF_8_5_BYTES	54	1 type byte + 5 value bytes - representing field of type UTF-8 sequence - with a 5 byte length (a 5 byte byte array).
UTF_8_6_BYTES	55	1 type byte + 6 value bytes - representing field of type UTF-8 sequence - with a 6 byte length (a 6 byte byte array).
UTF_8_7_BYTES	56	1 type byte + 7 value bytes - representing field of type UTF-8 sequence - with a 7 byte length (a 7 byte byte array).
UTF_8_8_BYTES	57	1 type byte + 8 value bytes - representing field of type UTF-8 sequence - with a 8 byte length (a 8 byte byte array).
UTF_8_9_BYTES	58	1 type byte + 9 value bytes - representing field of type UTF-8 sequence - with a 9 byte length (a 9 byte byte array).
UTF_8_10_BYTES	59	1 type byte + 10 value bytes - representing field of type UTF-8 sequence - with a 10 byte length (a 10 byte byte array).
UTF_8_11_BYTES	60	1 type byte + 11 value bytes - representing field of type UTF-8 sequence - with a 11 byte length (a 11 byte byte array).
UTF_8_12_BYTES	61	1 type byte + 12 value bytes - representing field of type UTF-8 sequence - with a 12 byte length (a 12 byte byte array).
UTF_8_13_BYTES	62	1 type byte + 13 value bytes - representing field of type UTF-8 sequence - with a 13 byte length (a 13 byte byte array).
UTF_8_14_BYTES	63	1 type byte + 14 value bytes - representing field of type UTF-8 sequence - with a 14 byte length (a 14 byte byte array).
UTF_8_15_BYTES	64	1 type byte + 15 value bytes - representing field of type UTF-8 sequence - with a 15 byte length (a 15 byte byte array).
UTF_8_1_LENGTH_BYTES	65	1 type byte + 1 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^8-1 bytes.
UTF_8_2_LENGTH_BYTES	66	1 type byte + 2 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^16-1 bytes.
UTF_8_3_LENGTH_BYTES	67	1 type byte + 3 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^24-1 bytes.
UTF_8_4_LENGTH_BYTES	68	1 type byte + 4 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^32-1 bytes.
UTF_8_5_LENGTH_BYTES	69	1 type byte + 5 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^40-1 bytes.
UTF_8_6_LENGTH_BYTES	70	1 type byte + 6 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^48-1 bytes.
UTF_8_7_LENGTH_BYTES	71	1 type byte + 7 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^56-1 bytes.
UTF_8_8_LENGTH_BYTES	72	1 type byte + 8 length byte + N length bytes - representing field of type UTF-8 sequence with a max length of 2^64-1 bytes.
UTC_NULL	73	1 type byte representing a UTC (Universal Time Coordinated) field with the value null.
UTC_0_BYTES	74	1 type byte + 0 value bytes representing a UTC (Universal Time Coordinated) field with an empty value.
UTC_2_BYTES	75	1 type byte + 2 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535).
UTC_3_BYTES	76	1 type byte + 3 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month.
UTC_4_BYTES	77	1 type byte + 4 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month, 1 byte representing the date (day of month)
UTC_5_BYTES	78	1 type byte + 5 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month, 1 byte representing the date (day of month), 1 byte representing hour of day (0-23).
UTC_6_BYTES	79	1 type byte + 6 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month, 1 byte representing the date (day of month), 1 byte representing hour of day (0-23), 1 byte representing minutes.
UTC_7_BYTES	80	1 type byte + 7 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month, 1 byte representing the date (day of month), 1 byte representing hour of day (0-23), 1 byte representing minutes, 1 byte representing seconds.
UTC_8_BYTES	81	1 type byte + 8 value bytes representing a UTC (Universal Time Coordinated) field with an 8 byte timestamp in milliseconds (long format).
UTC_9_BYTES	82	1 type byte + 9 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month, 1 byte representing the date (day of month), 1 byte representing hour of day (0-23), 1 byte representing minutes, 1 byte representing seconds, 2 bytes representing milliseconds.
UTC_9_BYTES	83	1 type byte + 9 value bytes representing a UTC (Universal Time Coordinated) field with 2 bytes representing a year (0 to 65535), 1 byte representing a month, 1 byte representing the date (day of month), 1 byte representing hour of day (0-23), 1 byte representing minutes, 1 byte representing seconds, 3 bytes representing nanoseconds (no milliseconds in this format).
COPY_1_BYTES	84	1 type byte + 1 value byte representing a 1 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_2_BYTES	85	1 type byte + 2 value bytes representing a 2 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_3_BYTES	86	1 type byte + 3 value bytes representing a 3 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_4_BYTES	87	1 type byte + 4 value bytes representing a 4 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_5_BYTES	88	1 type byte + 5 value bytes representing a 5 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_6_BYTES	89	1 type byte + 6 value bytes representing a 6 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_7_BYTES	90	1 type byte + 7 value bytes representing a 7 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
COPY_8_BYTES	91	1 type byte + 8 value bytes representing a 8 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_1_BYTES	92	1 type byte + 1 value byte representing a 1 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_2_BYTES	93	1 type byte + 2 value bytes representing a 2 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_3_BYTES	94	1 type byte + 3 value bytes representing a 3 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_4_BYTES	95	1 type byte + 4 value bytes representing a 4 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_5_BYTES	96	1 type byte + 5 value bytes representing a 5 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_6_BYTES	97	1 type byte + 6 value bytes representing a 6 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_7_BYTES	98	1 type byte + 7 value bytes representing a 7 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
REFERENCE_8_BYTES	99	1 type byte + 8 value bytes representing a 8 byte positive integer. The integer is a relative reference to a field found the integer amount of bytes earlier in the Polymorph Data sequence.
KEY_NULL	100	1 type byte representing a Key field with the value null.
KEY_0_BYTES	101	1 type byte + 0 value bytes representing a Key field with an empty value (empty byte array).
KEY_1_BYTES	102	1 type byte + 1 value bytes representing a Key field with a 1 byte value.
KEY_2_BYTES	103	1 type byte + 2 value bytes representing a Key field with a 2 byte value.
KEY_3_BYTES	104	1 type byte + 3 value bytes representing a Key field with a 3 byte value.
KEY_4_BYTES	105	1 type byte + 4 value bytes representing a Key field with a 4 byte value.
KEY_5_BYTES	106	1 type byte + 5 value bytes representing a Key field with a 5 byte value.
KEY_6_BYTES	107	1 type byte + 6 value bytes representing a Key field with a 6 byte value.
KEY_7_BYTES	108	1 type byte + 7 value bytes representing a Key field with a 7 byte value.
KEY_8_BYTES	109	1 type byte + 8 value bytes representing a Key field with a 8 byte value.
KEY_9_BYTES	110	1 type byte + 9 value bytes representing a Key field with a 9 byte value.
KEY_10_BYTES	111	1 type byte + 10 value bytes representing a Key field with a 10 byte value.
KEY_11_BYTES	112	1 type byte + 11 value bytes representing a Key field with a 11 byte value.
KEY_12_BYTES	113	1 type byte + 12 value bytes representing a Key field with a 12 byte value.
KEY_13_BYTES	114	1 type byte + 13 value bytes representing a Key field with a 13 byte value.
KEY_14_BYTES	115	1 type byte + 14 value bytes representing a Key field with a 14 byte value.
KEY_15_BYTES	116	1 type byte + 15 value bytes representing a Key field with a 15 byte value.
KEY_1_LENGTH_BYTES	117	1 type byte + 1 length byte + N value bytes representing a Key field with an N byte value.
KEY_2_LENGTH_BYTES	118	1 type byte + 2 length bytes + N value bytes representing a Key field with an N byte value.
OBJECT_NULL	119	1 type byte representing a field of type Object with the value null.
OBJECT_1_LENGTH_BYTES	120	1 type byte + 1 length byte + N value bytes (fields nested inside the Object field).
OBJECT_2_LENGTH_BYTES	121	1 type byte + 2 length byte + N value bytes (fields nested inside the Object field).
OBJECT_3_LENGTH_BYTES	122	1 type byte + 3 length byte + N value bytes (fields nested inside the Object field).
OBJECT_4_LENGTH_BYTES	123	1 type byte + 4 length byte + N value bytes (fields nested inside the Object field).
OBJECT_5_LENGTH_BYTES	124	1 type byte + 5 length byte + N value bytes (fields nested inside the Object field).
OBJECT_6_LENGTH_BYTES	125	1 type byte + 6 length byte + N value bytes (fields nested inside the Object field).
OBJECT_7_LENGTH_BYTES	126	1 type byte + 7 length byte + N value bytes (fields nested inside the Object field).
OBJECT_8_LENGTH_BYTES	127	1 type byte + 8 length byte + N value bytes (fields nested inside the Object field).
TABLE_NULL	128	1 type byte representing a field of type Table with the value null.
TABLE_1_LENGTH_BYTES	129	1 type byte + 1 length byte + N value bytes (fields nested inside the Table field).
TABLE_2_LENGTH_BYTES	130	1 type byte + 2 length byte + N value bytes (fields nested inside the Table field).
TABLE_3_LENGTH_BYTES	131	1 type byte + 3 length byte + N value bytes (fields nested inside the Table field).
TABLE_4_LENGTH_BYTES	132	1 type byte + 4 length byte + N value bytes (fields nested inside the Table field).
TABLE_5_LENGTH_BYTES	133	1 type byte + 5 length byte + N value bytes (fields nested inside the Table field).
TABLE_6_LENGTH_BYTES	134	1 type byte + 6 length byte + N value bytes (fields nested inside the Table field).
TABLE_7_LENGTH_BYTES	135	1 type byte + 7 length byte + N value bytes (fields nested inside the Table field).
TABLE_8_LENGTH_BYTES	136	1 type byte + 8 length byte + N value bytes (fields nested inside the Table field).

Boolean Field Type

The Boolean field types are used to encode boolean values such as true and false. It is also possible to encode a null value of type Boolean.

There are only 3 Boolean field types. One type that represents a null value (0), one type that represents the value true (1), and one type that represents the value false (2).

The Boolean fields have no value bytes. There is only the type byte. That is enough to encode the 3 different Boolean values possible (null, true, false).

Boolean Field Encoding Examples

The following encoding examples are shown using hexadecimal notation - meaning 2 digits per byte, and the first (leftmost) digit is the most significant 4 bits of the byte.

A Boolean null encoded:

A Boolean true encoded:

A Boolean false encoded:

Integer Field Type

The Integer field types are used to encode positive and negative integers efficiently. There is also a field type to encode a null Integer field. The positive integer types use only the number of bytes necessary to encode the concrete integer value. The negative integer types encode the value as a positive number, and thus uses only the number of bytes necessary to encode the value that is needed to encode its positive value. After reading a negative value you should negate it (the Polymorph Data APIs do this for you automatically).

Here are the integer field types:

Field Type	Type Code
INT_NULL	3	A single byte with the value 3 - representing a null value of type integer.
INT_POS_1_BYTES	4
INT_POS_2_BYTES	5
INT_POS_3_BYTES	6
INT_POS_4_BYTES	7
INT_POS_5_BYTES	8
INT_POS_6_BYTES	9
INT_POS_7_BYTES	10
INT_POS_8_BYTES	11
INT_NEG_1_BYTES	12
INT_NEG_2_BYTES	13
INT_NEG_3_BYTES	14
INT_NEG_4_BYTES	15
INT_NEG_5_BYTES	16
INT_NEG_6_BYTES	17
INT_NEG_7_BYTES	18
INT_NEG_8_BYTES	19

Integer Field Encoding Examples

The following encoding examples are shown using hexadecimal notation - meaning 2 digits per byte, and the first (leftmost) digit is the most significant 4 bits of the byte.

Integer null encoded:

Integer value 1 encoded:

    04 01

Integer value 65535 encoded:

    05 FF FF

Integer value -1 encoded:

    0C 01

Integer value -65535 encoded:

    0D FF FF

Float Field Type

The Float field type is used to represent floating point numbers of either 4 or 8 bytes - corresponding to Java's float and double types. You can also encode a null of type Float if you need to.

Here are the Float field types:

Field Type	Type Code
FLOAT_NULL	20
FLOAT_4_BYTES	21
FLOAT_8_BYTES	22

Bytes Field Type

The Bytes field type is used to represent sequences of raw bytes - such as a byte array or the contents of a file.

The Bytes field types are:

Field Type	Type Code
BYTES_NULL	23	1 type byte representing a field of type byte sequence - with a null value.
BYTES_0_BYTES	24	1 type byte + 0 value bytes - representing field of type byte sequence - with a zero length.
BYTES_1_BYTES	25	1 type byte + 1 value bytes - representing field of type byte sequence - with a 1 byte length (a 1 byte byte array).
BYTES_2_BYTES	26	1 type byte + 2 value bytes - representing field of type byte sequence - with a 2 byte length (a 2 byte byte array).
BYTES_3_BYTES	27	1 type byte + 3 value bytes - representing field of type byte sequence - with a 3 byte length (a 3 byte byte array).
BYTES_4_BYTES	28	1 type byte + 4 value bytes - representing field of type byte sequence - with a 4 byte length (a 4 byte byte array).
BYTES_5_BYTES	29	1 type byte + 5 value bytes - representing field of type byte sequence - with a 5 byte length (a 5 byte byte array).
BYTES_6_BYTES	30	1 type byte + 6 value bytes - representing field of type byte sequence - with a 6 byte length (a 6 byte byte array).
BYTES_7_BYTES	31	1 type byte + 7 value bytes - representing field of type byte sequence - with a 7 byte length (a 7 byte byte array).
BYTES_8_BYTES	32	1 type byte + 8 value bytes - representing field of type byte sequence - with a 8 byte length (a 8 byte byte array).
BYTES_9_BYTES	33	1 type byte + 9 value bytes - representing field of type byte sequence - with a 9 byte length (a 9 byte byte array).
BYTES_10_BYTES	34	1 type byte + 10 value bytes - representing field of type byte sequence - with a 10 byte length (a 10 byte byte array).
BYTES_11_BYTES	35	1 type byte + 11 value bytes - representing field of type byte sequence - with a 11 byte length (a 11 byte byte array).
BYTES_12_BYTES	36	1 type byte + 12 value bytes - representing field of type byte sequence - with a 12 byte length (a 12 byte byte array).
BYTES_13_BYTES	37	1 type byte + 13 value bytes - representing field of type byte sequence - with a 13 byte length (a 13 byte byte array).
BYTES_14_BYTES	38	1 type byte + 14 value bytes - representing field of type byte sequence - with a 14 byte length (a 14 byte byte array).
BYTES_15_BYTES	39	1 type byte + 15 value bytes - representing field of type byte sequence - with a 15 byte length (a 15 byte byte array).
BYTES_1_LENGTH_BYTES	40	1 type byte + 1 length byte + N length bytes - representing field of type byte sequence with a max length of 2^8-1 bytes.
BYTES_2_LENGTH_BYTES	41	1 type byte + 2 length byte + N length bytes - representing field of type byte sequence with a max length of 2^16-1 bytes.
BYTES_3_LENGTH_BYTES	42	1 type byte + 3 length byte + N length bytes - representing field of type byte sequence with a max length of 2^24-1 bytes.
BYTES_4_LENGTH_BYTES	43	1 type byte + 4 length byte + N length bytes - representing field of type byte sequence with a max length of 2^32-1 bytes.
BYTES_5_LENGTH_BYTES	44	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^40-1 bytes.
BYTES_6_LENGTH_BYTES	45	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^48-1 bytes.
BYTES_7_LENGTH_BYTES	46	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^56-1 bytes.
BYTES_8_LENGTH_BYTES	47	1 type byte + 5 length byte + N length bytes - representing field of type byte sequence with a max length of 2^64-1 bytes.

Bytes Field Encoding Examples

The following encoding examples are shown using hexadecimal notation - meaning 2 digits per byte, and the first (leftmost) digit is the most significant 4 bits of the byte.

A null Bytes field:

A 3 byte long Bytes field:

    1B F3 34 A1

A 16 byte long Bytes field - (with 1 length byte):

    28 10 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

UTF-8 Field Type

The UTF-8 Field type is used to represent sequences of UTF-8 characters. The UTF-8 characters are just stored as bytes - just like in a Bytes field - but the UTF-8 field type tells you that these bytes are to be interpreted as UTF-8 characters.

Here are the UTF-8 field types:

Field Type	Type Code
UTF_8_NULL	48
UTF_8_0_BYTES	49
UTF_8_1_BYTES	50
UTF_8_2_BYTES	51
UTF_8_3_BYTES	52
UTF_8_4_BYTES	53
UTF_8_5_BYTES	54
UTF_8_6_BYTES	55
UTF_8_7_BYTES	56
UTF_8_8_BYTES	57
UTF_8_9_BYTES	58
UTF_8_10_BYTES	59
UTF_8_11_BYTES	60
UTF_8_12_BYTES	61
UTF_8_13_BYTES	62
UTF_8_14_BYTES	63
UTF_8_15_BYTES	64
UTF_8_1_LENGTH_BYTES	65
UTF_8_2_LENGTH_BYTES	66
UTF_8_3_LENGTH_BYTES	67
UTF_8_4_LENGTH_BYTES	68
UTF_8_5_LENGTH_BYTES	69
UTF_8_6_LENGTH_BYTES	70
UTF_8_7_LENGTH_BYTES	71
UTF_8_8_LENGTH_BYTES	72

UTF-8 Field Encoding Examples

A null UTF-8 field:

A 3 byte long UTF-8 field:

    34 41 42 43

A 16 byte long UTF-8 field - (with 1 length byte):

    41 10 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50

UTC Field Type

Copy Field Type

Reference Field Type

Key Field Type

The Key field type is used to represent a "key" in a hashtable, a property name inside an Object field, or a column name inside a Table field.

Key Field Encoding Examples

Here are a few Key field encoding examples - in hexadecimal encoding.

A key field with 4 bytes inside (Ascii values of A, B, C and D):

    69 41 42 43 44

Object Field Type

The Object field type is used to represent objects with named properties + property values, or maps with keys and values. The Object field type is intended to contain other Polymorph Data fields inside it. These nested fields make up the property names and values.

A property name inside an Object field is represented by a Key field + some other field that makes up the value. For instance, an int property would consist of a Key field with the property name, and an Int field with the value.

You can nest Object fields (and Table fields) inside Object fields, to create bigger object graphs.

Object Field Encoding Examples

Here are some examples of encoding a Polymorph Data Object field. The example uses hexadecimal notation.

Here is a null Object field:

Here is an Object with 1 length byte, plus a single property (Key + Int field):

78 05    63 66 31    04 FF

Table Field Type

The Table field is used to represent tabular data - similar to what you can represent in a CSV file or HTML table.

The Table field consists of the Table type code, the length bytes, the number of rows in the table (as an Int field), the Key fields for the column headers, and then all the column value fields - for all rows - with no row separator field.

The number of columns per row is marked by the series of Key fields for the column headers. If there are 3 Key fields in that series, each row has 3 columns.

The length (in bytes) of the Table field should include all nested fields, including the row count Int field and the Key fields for the columns, and of course the row field values.

Table Field Encoding Examples

An example of a null Table field:

Here is a Table field with 1 length byte, 2 rows, and 2 columns (2 Key fields).

81 0F
   02
   63 66 31   63 66 32
   04 FF      04 FE
   04 FD      04 FC

Next: Polymorph Data Language

Tweet
	Jakob Jenkov