I really like using JSON encoding as a way of transferring messages between processes as it is machine and language independent. Plus, it is very well suited to stream processing networks (such as rt-ai Edge) as arbitrary fields can be added to existing JSON messages and passed along. Contrast this with compiled IDLs which typically have no flexibility whatsoever.
One problem though is that binary data cannot be included in JSON messages directly. Typically base64 encoding is used to convert binary data into text. However, this is inefficient, especially in a stream processing network where base64 decoding and encoding might have to be done several times.
There are a variety of modifications to JSON around but it is very simple to just add binary data on to the end of a JSON message to form a complete message that can be transferred via MQTT for example.
In Python, an MQTT message can be published like this:
import json import struct ... def publish(topic, jsonData, binData = None): jsonDump = json.dumps(jsonData) jsonString = struct.pack('>I', len(jsonDump)) + jsonDump + binData MQTTClient.publish(topic, jsonString) ...
Here, jsonData contains the normal JSON message text, binData contains the binary data to be sent along with it. To receive the message, use something like this:
import json import struct ... def onMessage(client, userdata, message): jsonLength = struct.unpack('>I', message.payload[0:4]) jsonData = json.loads(message.payload[4:4+jsonLength]) binData = message.payload[4+jsonLength:] ...