Proposed SBR guidance on transport and messaging protocols
Context
This paper sets out a draftSBR Messaging Guide (the Guide). The Guide supports best practice in the development of SBR projects.
The Guide has been developed in response to the decision made by the SBR Steering Group late last year to adopt a new, more flexible, framework for SBR (known as SBR evolution). The decision on SBR evolution establishes the SBR Framework and SBR Method. The SBR Framework includes the key SBR components; the SBR dictionary; SBR reporting repository; the SBR business identifier (known as the Australian Business Number) and the SBR business credential (known as AUSkey). The SBR Method sets out an approach for government to work with software developers and agencies to connect business to government systems in a reusable and lowcost manner.
As part of the decision on SBR evolution, the SBR Program is nolongerprescribing any particular message format or transport protocol. However, instead, theSBR program would provide guidance to agencies on recommended implementation patterns for the exchange of information. The Guide is intended to provide this guidance to agencies.
The Messaging Guide
The draft Guide covers two elements of the process of exchanging digital information; the data format and the messaging protocol:
- The data formatrefers to the language used to digitally represent the data being exchanged between two parties (in simple terms, the document). Attachment B sets out the recommended data formats for SBR projects.
- The messaging protocol refers to the way in which that data is enclosed within a digital message and is transported (in simple terms, the envelope). Attachment C sets out the recommended message protocols for SBR projects.
The draft Guide allows end users greater flexibility to choose the best option from a range of options that meets the business outcome they are seeking to achieve. This differs from previous SBR requirements to use a ‘one size fits all’ approach for messaging and data formats. The co-design approach that is inherent in SBR remains a key determinant in exercising that choice. This flexibility supports the expansion of SBR beyond the financial domain.
The draft Guide identifies multiple options for data format and messaging protocol. Each of the options proposed are commonly used and supported by the software industry.
Scope
This guide supports the development of SBR projects. Existing SBR agencies have the flexibility to continue with their existing SBR implementations, or to adopt new message formats or transport standards under this guide in consultation with the software providers.
Authentication is still based on AUSkey as per the SBR Framework, but the methods by which this is applied are not available at this time.
Next steps
Subject to in-principle endorsement of the draft Guide by the SBR Steering Group, the Guide will be used a basis for further consultation with agencies and software developers. Post consultation, the finalised Guide would be put to Steering Group for endorsement at its next meeting.
Attachment ASBR Messaging Guide Cover Sheet
Attachment BData Format
Attachment CMessage Protocol
Page | 1
ATTACHMENT A
SBR Messaging Guide
Digital exchange of information between two parties requires:
- a common meaning of the data;
- an agreed language or format for expressing the data ‘on the wire’; and
- an agreed transport method or protocol for exchanging the data.
The common meaning can be delivered by reference to a common dictionary or through the inclusion of self-described terms for instance extensions. The SBR Program will continue to use dictionaries for the purpose of common meaning as set out in theSBR Framework.
There are a range of commonly used data formats and transport protocols to address a range of business needs. To support their consistent application within the SBR framework and method, a number of guides have been developed. At this time these guides do not address authentication and authorisation.
These guides should be used within the context of the SBR Program Principles, especially:
- Designing the solution with the software community to minimise costs for software developers
- Design for reuse
- Using relevant established International and Australian standards
Page | 1
ATTACHMENT B
Data Format
The data format is the language or syntax used to represent the data being digitally exchanged between two parties. The way in which that data is enclosed within a message and is transported relates to the message protocol.
In simple terms, the dataformat is the document and the message protocol is the envelope. Together they enable the transmission of data between two systems.
The design goals for choice of data format include:
- Compactness: Use the least amount of structure and content to compile the message
- Performance: Use message optimisation to reduce transport and processing overhead
- Ease of use: Simple to understand and tooling and support skills are readily available
There are four commonly used data formats in use. These have a number of unique and overlapping features, so the final choice may also be guided by what is already in use by the agency and the relevant software community.
The four common formats by increasing order of complexity and message size are:
CSV - Comma Separated Values: Similar to a spreadsheet containing a number of row column values. The structure is ordered according to an agreed design. Field values should avoid commas, making it problematic for use with text fields. It is simple to construct and process and is ideally suited to tables containing numeric or boolean data.
JSON – JavaScript Object Notation: Lightweight self-describing format containing attribute - value pairseg{FirstName:“John”}. The freedom provided by its data structure means that is supported by nearly all programming languages for the purpose of data interchange. Easy for humans to read and for machines to process.
XML – eXtensibleMarkup Language: Used for encoding documents over the internet that can be validated against a set of rules. Ability to express relationships between data elements.
XBRL – eXtensible Business Reporting Language: Variant of XML, with semantics. Allows for user defined extensions and provides human and machine comparability of data.
Glossary of terms used in the following matrix.
Term / DescriptionDynamic Ordering / A capability used in changing the ordering of the data items within a message at run-time.
Local Extensions / A capability used to extend the message structure to allow additional definitions to be added.
Semantics / A capability used within a message syntax to describe the meaning of a data item.
Data Typing / A capability to classify various types of data such as an integer, string, or boolean.
Validation / A capability used within a message syntax to validate the structure of message instance.
Natural Comparison / A capability that allows multiple message instances to be compared over time horizons or facts without further translation.
Content / A capability used in defining how data is represented and relates to each other as part of a message structure such as object relationships.
Native Serialisation / Deserialisation / A capability used during the packing and unpacking of a message ranging from manual to automated.
The recommended approach for using the following matrix is to identify the characteristics that are most important to a particular interaction and select the data format that meets them. The matrix is a decision tool so only the differentiating characteristics are listed and does not therefore represent a complete list of format features.
Page | 1
ATTACHMENT C
Message Protocol
The way in which data is enclosed within a message and is transported relates to the message protocol. The data format is the language or syntax used to represent the data being digitally exchanged between two parties.
In simple terms, the data format is the document and the message protocol is the envelope. Together they enable the transmission of data between two systems.
The design goals for choice of message protocol include:
- Compactness: Use the least amount of structure and content to compile the message
- Performance: Use message optimisation to reduce transport and processing overhead
- Ease of use: Simple to understand and tooling and support skills are readily available
There are five main message protocols in use. These have a number of unique and overlapping features, so the final choice may also be guided by what is already in use by the agency and the relevant software community.
The five common protocols by increasing order of complexity:
SCP – Secure Copy Protocol: is a standard that supports the secure transmission of files from a source to a destination. It is used for machine to machine file transfer without the vulnerabilities of FTP.
REST – REpresentational State Transfer: is an architectural style and not a standard. It provides a simple but scalable solution for accessing web resources.
Push – Push Protocol: Used for sending data to client agents that have been set up via a registration process to listen, similar to RSS. As new content becomes available it is pushed to the client application. This protocol is commonly used with mobile apps.
WS* (SOAP) – Web Service (Simple Object Application Protocol): is a web services standard that enables the reliable exchange of structured data between clients and hosts.
AS4 (ebMS3) – AS4 Profile within ebusiness Message Standard: using a subset of the ebMS3 standard to ensure interoperability between two different systems. Useful where sequencing is important and in peer to peer and multi-hop scenarios.
Glossary of terms used in the message protocol choices table.
Term / DescriptionReliability / A capability of guaranteeing to deliver a message to its destination.
Ordering / A capability of guaranteeing a set of messages to be received according to their sending order.
Priority / A capability that the messaging layer can classify messages into different groups and deliver these groups in different ways; e.g., with different resources and in different orders.
One-Way Push / A capability of sending one message from an initiator/sender to one destination.
One-Way Pull / A capability of sending one message from an initiator/sender to one destination.
Two Way Synch / A capability of sending one message from an initiator/sender to one destination and receive a responding message back to the initiator/sender as part of one transaction.
Two Way Asynch / A capability of sending one message from an initiator/sender to one destination as one transaction. The destination can send a separate responding message back to the initiator/sender as part of another transaction.
Multi-cast / A capability of sending one message to multiple destinations.
Multi-hop / A capability of routing a message to its ultimate destination(s) via one or more intermediary nodes.
Pub/Sub / A capability of sending a message to a number of parties who have subscribed to the topic/subject of the message.
Batch Processing / A capability of sending a message that can trigger a batch process on the destination side
Large Messages / A capability of sending very large amounts of data
Patterns referred to in the message protocol choices table
Page | 1