Broker Specification

  • If I wanted to create an OpenDXL broker, are there enough docs/specs on the behavior to do it? Specifically, I'm talking about modification of the base MQTT behavior. I see the open-source `opendxl-broker` project, but not sure where I'd start to implement the same functionality around other MQTT implementation.

  • Hi-


    That is a great question. Unfortunately, at this point there is not a specification for developing a custom broker, but that is definitely something that we should work to address in the future.


    As far as what can be done today, it really depends on what you are wanting to achieve. An OpenDXL client can connect to an MQTT broker today (and send events), but none of the added functionality will be available.


    At a very high-level the main additions to a standard MQTT broker are as follows:

    • RESTful-like service-based model with load-balancing and failover
    • Broker-based filtering of messages to specific clients and brokers
    • Multi-broker hubs to support failover
    • Client and CA-based certificate topic authorization
    • Topic-based routing
    • Multi-tenancy

    See this page for a more detailed comparison between an OpenDXL broker and a standards-based MQTT broker.


    So, one interesting place to start would be to add RESTful-like services support to a standard broker. In the future, other features such as multi-broker fabrics, filtering, routing, etc. could be added.


    The implementation of the service registry in the OpenDXL broker can be found here. The service registry is used to track the services that are available throughout the fabric (as well as those that are on the local broker).


    One of the key aspects of an integration with a broker is that you need to be able to hook into the various points of the message processing flow.


    For Mosquitto, the flow stages for a message are as follows:


    Publish => Store => Insert (per client) => Finalize


    The OpenDXL Broker is able to hook at all these points in a message flow. The majority of the work in the OpenDXL broker occurs in the "Store" stage. At this point the broker can respond to particular topics. For example, it is able to handle service registration messages, and add them to the registry. It is also able to handle service look-ups when a client is attempting to invoke a service.


    All of the handlers can be found in the following location:


    https://github.com/opendxl/ope…brokerlib/message/handler


    Some of the critical classes to look at related to services are as follows:


    ServiceRegistryRegisterRequestHandler: Handles an incoming service registration request

    ServiceLookupHandler: Handles an incoming request from a client to invoke a service


    This is obviously just a starting point, and I totally agree that we need to put together a specification for both brokers and clients. We have been working on putting together specifications that can be used by OpenDXL solutions to document the functionality that they expose (more on this in the near future). After that, we can look at adding specifications for the client and brokers themselves.


    Thanks again for the great question,

    Chris

  • Outside of a spec? I couldn’t find an integration test suite for the broker. The closes thing I found was the examples portion of the Python client. Is there an integration test suite for the broker somewhere else?

  • Hi-


    The integration tests are part of the Java client which is currently being updated to be released as an open source project (should be released fairly soon). The Python tests do perform a subset of the integration tests. Until the Java client releases, the Python tests probably serve as the best integration tests available at this time.


    Thanks,

    Chris