Allow for fine-grained byte order specification

[hrstoyanov]

Proposal explained here

[donmendelson]

When the SBE project started, the inclination was to use "network byte order" that was used in internet protocols such as IP and TCP headers. Big-endian was used because when internet protocols were created in the 70s - 90s, big-endian architectures dominated. However, Intel and compatibles (little-endian) later became prominent. My understanding is that ARM can theoretically work either way but all the OS implementations are little-endian. We thought it best not to take sides. You can optimize your messages for your architecture, at least for internal messaging.

What is the use case for mixed byte order?

[hrstoyanov]

Turns out SBE could be useful for describing legacy protocols where no design changes can be made, like the MS SQL Server TDS wire protocol. Not sure why Microsoft mixed Endians, but they did it many years ago, so we are stuck with that ... I know this is a stretch for SBE .

[kleihan]

If the main issue is to use standard meta-data for the description of legacy protocols, I suggest to take a look at Orchestra first. An additional attribute providing further detail to some of the datatypes would be needed.

SBE was aimed at high performance and using big- and little-endian fields in the same message is not something we should promote as an option in the standard.

[donmendelson]

A stream may have mixed byte orders if regions are controlled by different protocols. I have encountered this situation with a header controlled by one protocol in "network byte order" (big endian) and application message in little endian in a different encoding. In that case, each region should be encoded with its own encoding and byte order rather than trying to make one ambidextrous encoder.

[hrstoyanov]

Just to give an example of what I need - the ability to specify default byte order but then be able to override it for certain fields:

<sbe:messageSchema byteOrder="littleEndian">
    <types>
        <composite name="MyType" >
            <type name="a" primitiveType="uint16" />
            <type name="b" primitiveType="uint16" byteOrder="bigEndian"/>
        </composite>
    </types>
    <sbe:message name="MyMessage" id="1" >
        <field name="Field" id="10" type="MyType"/>
    </sbe:message>
</sbe:messageSchema>

[kleihan]

@hrstoyanov, your request is well understood but I do not think it is in the interest of the SBE standard to support such a capability. The objective and purpose of SBE are different.