Is .encode()/.decode() required for message payloads?

isaiahT · Jul 26th 2017

In the samples for the OpenDXL Python Client, and all of the solutions I've seen, the payload for any type of DXL message (event/request/response) has .encode() called on it before sending. And anyone that receives the message calls .decode() on it immediately. See below for an example of what I'm referring to:

Python: event_example.py

# ... skipping some code
# Create and add event listener
class MyEventCallback(EventCallback):
def on_event(self, event):
with event_count_condition:
# Print the payload for the received event
print "Received event: " + event.payload.decode()
# Increment the count
event_count[0] += 1
# Notify that the count was increment
event_count_condition.notify_all()
# ... skipping some code
# Loop and send the events
for event_id in range(TOTAL_EVENTS):
# Create the event
event = Event(EVENT_TOPIC)
# Set the payload
event.payload = str(event_id).encode()
# Send the event
client.send_event(event)
# ... skipping some code

Display More

Why is it even necessary to encode and decode the payload if we know the data type ("string", for example)? Is there a reason to do this every time?

ncolter · Jul 26th 2017

Yes, in almost every case outgoing DXL messages should be encoded and incoming messages should be decoded.

Because the DXL message payload is simply binary data, failing to encode data types that have system dependencies (or other factors) can result in an unexpected format for the data when it is received. By using .encode(), the message sender standardizes the format of the payload so that the receiver(s) can use a matching .decode() call and smoothly interpret the payload as intended.

To help simplify the process of writing your DXL client code, we recommend using OpenDXL Bootstrap. The bootstrapper includes the MessageUtils class, which provides methods for basic encoding/decoding objects and managing Python dictionaries as JSON payloads.

pcrook · Aug 14th 2017

In most cases encode/decode will help ensure that the message payload is delivered and parsed by the receiver correctly. I ran across a case with numpy arrays where I needed to use serialization to send the data. In rare cases, you may want to consider using cPickle (its a built-in Python library for serialization)... you can import cPickle, and use cPickle.dumps to load your payload in the sender code; then use cPickle.loads on the receiver. Just remember that the max message size on DXL is currently 1 MB.

sample snippets:

Python: Sender

from dxlclient.client import DxlClient
from dxlclient.client_config import DxlClientConfig
from dxlclient.message import Event
import cPickle, numpy
# Import common logging and configuration
CONFIG_FILE = "/usr/local/etc/opendxl/dxlclient.config"
config = DxlClientConfig.create_dxl_config_from_file(CONFIG_FILE)
EVENT_TOPIC = "/open/event/numpy"
numpypayload= "<numpy array here>"
with DxlClient(config) as client:
client.connect()
event = Event(EVENT_TOPIC)
event.payload = cPickle.dumps(numpypayload)
logger.info("Publishing XML to %s", EVENT_TOPIC)
client.send_event(event)

Display More

Python: Receiver

import dxlclient, cPickle
# Import common logging and configuration
CONFIG_FILE = "/usr/local/etc/opendxl/dxlclient.config"
config = DxlClientConfig.create_dxl_config_from_file(CONFIG_FILE)
EVENT_TOPIC = "/open/event/numpy"
...
with dxlclient.client.DxlClient(config) as client:
print "Connecting to DXL..."
client.connect()
print "Connected to DXL & waiting for events!"
class numpy(dxlclient.callbacks.EventCallback):
def on_event(self, event):
numpypayload= cPickle.loads(event.payload)
print "Received: %r" %numpypayload
client.add_event_callback(EVENT_TOPIC, numpy())
while True:
time.sleep(60)

Display More

Is .encode()/.decode() required for message payloads?

Share

Tags