Request/Reply Message Exchange Patterns and MQTT Version 1.0

This isintended as a Non-Standards Track Work Product.

The patent provisions of the OASIS IPR Policy do not apply.

Request/Reply Message Exchange Patterns and MQTT Version1.0

Working Draft 01

15 June2015

Technical Committee:

OASIS Message Queuing Telemetry Transport (MQTT) TC

Chairs:

Raphael J Cohn (), Individual

Richard J Coppen (), IBM

Editor:

Shawn McAllister (), Solace Systems

Related work:

This document is related to:

MQTT Version 3.1.1. Edited by Andrew Banks and Rahul Gupta. Latest version:

Abstract:

This document is a Working Draft document for the MQTT Technical committee. It contains all matter related to adding a request/reply capability forapplications using the MQTT 3.1.1 protocol. This document contains the motivation, requirements and solution options for providing such a capability. It is intended to focus discussion within the MQTT Working Group to allow the group to decide on an appropriate way to address these requirements – whether with changes to the MQTT 3.1.1 specification or via a Committee Note or some other mechanism.

Status:

This Working Draft (WD) has been produced by one or more TC Members; it has not yet been voted on by the TC or approved as a Committee Note Draft. The OASIS document Approval Process begins officially with a TC vote to approve a WD as a Committee Note Draft. A TC may approve a Working Draft, revise it, and re-approve it any number of times as a Committee Note Draft.

URI patterns:

Initial publication URI:

Permanent “Latest version” URI:

(Managed by OASIS TC Administration; please don’t modify.)

All capitalized terms in the following text have the meanings assigned to them in the OASIS Intellectual Property Rights Policy (the "OASIS IPR Policy"). The full Policy may be found at the OASIS website.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published, and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this section are included on all such copies and derivative works. However, this document itself may not be modified in any way, including by removing the copyright notice or references to OASIS, except as needed for the purpose of developing any document or deliverable produced by an OASIS Technical Committee (in which case the rules applicable to copyrights, as set forth in the OASIS IPR Policy, must be followed) or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by OASIS or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and OASIS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY OWNERSHIP RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Table of Contents

1Introduction

1.1 References (non-normative)

2Problem Definition

2.1 Motivation

2.2 Requirements

2.3 Clarifications

2.3.1 Why not use HTTP?

2.3.2 Request/Reply != Synchronous

2.3.3 Request/Reply != Remote Procedure Call (RPC)

3Solution Option #1

4Solution Option #2

Appendix A.Acknowledgments

Appendix B.Some Appendix

B.1 Subsidiary Appendix Section

B.1.1 Sub-subsidiary Appendix Section

Appendix C.Revision History

reqreply-v1.0-wd01Working Draft 0115 June2015

This isintended as a Non-Standards Track Work Product.

The patent provisions of the OASIS IPR Policy do not apply.

1Introduction

This document is a Working Draft document for the MQTT Technical committee. It contains all matter related to adding a request/reply capability for applications using the MQTT 3.1.1 protocol. This document contains the motivation, requirements and solution options for providing such a capability. It is intended to focus discussion within the MQTT Working Group to allow the group to decide on an appropriate way to address these requirements – whether with changes to the MQTT 3.1.1 specification or via a Committee Note or some other mechanism.

1.1References (non-normative)

[MQTT]

MQTT Version 3.1.1. Edited by Andrew Banks and Rahul Gupta. Latest version:

2Problem Definition

This section provides all detail related to defining the request/reply problem intended to be addressed by this document and also in some cases specifically stating what problem is not addressed by this document in order to focus the scope. Only once a problem statement is agreed can potential solutions be assessed.

2.1Motivation

The MQTT protocol [MQTT] supports a publish/subscribe message exchange pattern where a published message is delivered to zero or more consumers based on subscription but where the publisher does receive any message in response to its published message. Publish/subscribe works very well for various types of applications where “notifications” of an event, status change or sensor reading are sent by a publisher and where many consumers can receive this information.

However, there are many common situations where a request/reply message exchange pattern is required and this is not explicitlysupported by [MQTT] nor is guidance provided on how such a message exchange pattern can be performed by applications or APIs using [MQTT]. Application examples of where such a mechanism is needed include:

When an IoT device needs to “read” or download some information from a service running in a datacenter. In this case, a device would like to send a message on a topic and receive one (or perhaps more) replies to this particular message. This is similar to an HTTP GET operation.
When an application in a datacenter needs to “read” some information immediately from a device without waiting some timed period for the new value to be sent from the device. This is like an HTTP GET but in this case it is sent from the datacenter application to the IoT device – the reverse direction of a typical HTTP GET.
When an application in a datacenter needs to “set” some value onto an IoT device and needs confirmation that this set was successful. Somewhat like an HTTP POST but in this case it is sent from the datacenter application to the IoT device – the reverse direction of a typical HTTP POST.

2.2Requirements

The specific requirements of the request/reply mechanism are discretely defined here for discussion and agreement before proceeding to analyze solution options.

We state the requirements in the context of an IoT device D and another application A, where A is typically a datacenter application but could be another IoT device:

Support request/reply interactions with (a) requests originating from D and replies from A as well as (b) requests originating from A and replies from D.
This supports a device D autonomously requesting information from a datacenter application A and allows an application A to request information from D or effect some change on D or cause some action by D as and when A decides to do so, without requiring polling (and coupling) from D to A.
Support request/reply where the request may be routed to many target applications (pub/sub) and where the requester may receive many replies to its one request. This must be supported for when either D or A initiate the request.
From an application’s point of view, it must be possible to support both synchronous (blocking) and asynchronous (non-blocking) request/reply. Many requests may be outstanding at any one time over a single MQTT connection.
Since many requests may be outstanding at the same time on an MQTT connection, the defined mechanisms must allow the requestor to correlate a reply message with the original request (instance) that was sent (or the request state) and must allow for replies to be received in a different order than the order in which the requests were sent.
To ensure scalability, the request/reply interaction must be stateless in the MQTT broker when D and A are both MQTT clients. This means that all state to perform the request reply and any correlation must be maintained at the endpoints.
Both the request portion and the reply portion of the message exchange pattern must be able to use any of QoS 0, 1 or 2 at the application’s discretion and the QoS of the request need not match the QoS of the reply.
Required topologies:

a)D and A are both MQTT clients that are connected to the same or different MQTT brokers. Use of different brokers could be when MQTT bridges are used to support subtended MQTT brokers on gateways, trains, buses, etc.

b)D is an MQTT client but A uses some other non-MQTT protocol and in fact may or may not be connected to the same broker as D.

In this case, it must be possible for the “MQTT-to-otherProtocol” interworking function (IWF) to be NOT application-specific. That is, there must be a standard method for the protocol interworking function to find in the MQTT packet the information required to interwork the MQTT request/reply with the request/reply information of the “otherProtocol”.
Motivation for this is to avoid the need for application+messaging-specific bridges to support request/reply in this topology. Message brokers that implement multiple protocols could then provide request/reply capabilities between MQTT clients and the other protocol clients using the broker in a completely standard manner – without application-specific gateways.
This is an important topology because while it is expected that MQTT will be widely used by IoT devices, it is not expected that it be as widely used by datacenter applications and thus this messaging protocol interworking must be offered as simply as possible “out of the box” by products.
Pictorially, this interworking can be described as follows:
------
D --- MQTT ---| broker + iwf |--- non-MQTT-protocol --- A
------

2.3Clarifications

This section contains a summary of discussions on various topics related to MQTT request/reply in order to document what they were and their outcome.

2.3.1Why not use HTTP?

It was proposed that HTTP be used by devices to provide a request/reply capability whereas MQTT be used by devices when they need pub/sub. The main reason that this option is rejected is because of requirement #1 where a datacenter application needs to send a request to a device. This would require that the device support an HTTP server stack and that its IP address be known to and reachable from the datacenter application.

Another softer reason is that this would indeed require devices to support a dual stack and that the datacenter infrastructure would be significantly more complex with bothMQTT and HTTP servers, duplicate authentication mechanisms, etc. Having a single mechanism for publish/subscribe as well as request/reply overall is simpler.

2.3.2Request/Reply != Synchronous

There was discussion of the complexity and lack of scalability of request/reply mechanisms but this may have been due to a misunderstanding that the request/reply mechanism would be synchronous. In fact as per requirement #3, it must be possible for both synchronous and asynchronous modes to be offered by an MQTT API.

From a protocol point of view, the request and reply messages are “asynchronous” in that after a client sends a request message, any other MQTT packets can be sent on the connectionby the client before the reply is received.

2.3.3Request/Reply != Remote Procedure Call (RPC)

We view the request/reply mechanism defined in this document as beinga superset of the RPC capability and that RPC could be built on top of this request/reply. In particular, some differences are:

-RPC is synchronous, as viewed by the application – request/reply can be synchronous or asynchronous

-RPC receives at most one reply per request whereas the request/reply mechanism could receive many replies for a single request.

As such, RPC will be dealt with separately from the request/reply capability defined in this document but is expected to use the capabilities resulting from this document.

3Solution Option #1

Here we will document at a high level the first candidate solution to the problem. We would need to compare how this solution addresses each of the above requirements and list pros/cons.

4Solution Option #2

Here we will document at a high level the second candidate solution to the problem. We would need to compare how this solution addresses each of the above requirements and list pros/cons.

Appendix A.Acknowledgments

The following individuals have participated in the creation of this specification and are gratefully acknowledged:

Participants:

[Participant Name, Affiliation | Individual Member]

Appendix B.SomeAppendix

Text.

B.1Subsidiary Appendix Section

Text.

B.1.1Sub-subsidiary Appendix Section

text.

Appendix C.Revision History

Revision / Date / Editor / Changes Made
01 / 15 June 2015 / Shawn McAllister / Initial version for discussion. Captured: motivation, proposed requirements.

reqreply-v1.0-wd01Working Draft 0115 June2015