Sending and receiving binary data using JSON encoding, Python and MQTT

I really like using JSON encoding as a way of transferring messages between processes as it is machine and language independent. Plus, it is very well suited to stream processing networks (such as rt-ai Edge) as arbitrary fields can be added to existing JSON messages and passed along. Contrast this with compiled IDLs which typically have no flexibility whatsoever.

One problem though is that binary data cannot be included in JSON messages directly. Typically base64 encoding is used to convert binary data into text. However, this is inefficient, especially in a stream processing network where base64 decoding and encoding might have to be done several times.

There are a variety of modifications to JSON around but it is very simple to just add binary data on to the end of a JSON message to form a complete message that can be transferred via MQTT for example.

In Python, an MQTT message can be published like this:

    import json
    import struct
    def publish(topic, jsonData, binData = None):
        jsonDump = json.dumps(jsonData)
        jsonString = struct.pack('>I', len(jsonDump)) + jsonDump + binData
        MQTTClient.publish(topic, jsonString)

Here, jsonData contains the normal JSON message text, binData contains the binary data to be sent along with it. To receive the message, use something like this:

    import json
    import struct
    def onMessage(client, userdata, message):
        jsonLength = struct.unpack('>I', message.payload[0:4])[0]
        jsonData = json.loads(message.payload[4:4+jsonLength])
        binData = message.payload[4+jsonLength:]

Speeding up Apache NiFi compilation

The normal way to build Apache NiFi from source on Linux is to use:

mvn -T C2.0 clean install

More info is here incidentally. One issue with this is that it also runs all the tests which gives rise to a couple of problems. One is that some of the tests take a while and slow down the build process. The other is that, should some arcane test in code that isn’t interesting fail, the build aborts. To avoid this, build with tests turned off:

mvn -T C2.0 clean install -Dmaven.test.skip=true

Saves quite a bit of time!

Developing Electron apps with Visual Studio Code

I have been trying out Electron as a way of developing some WebRTC apps to work with the Janus gateway. In the end I decided that Visual Studio Code was a good route to take for Javascript code development. One thing that wasn’t at all obvious though was how to get breakpoints to work. I found this blog entry that had the answer – no way I would have been able to work it out myself so go to that link for the original source (reproduced here for my convenience).

First thing is to install the Debugger for Chrome extension for VS Code – instructions are here. Then, the .vscode/launch.json file should look something like this:

  // Use IntelliSense to learn about possible Node.js debug attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit:
  "version": "0.2.0",
  "configurations": [
      "type": "node",
      "request": "launch",
      "name": "Main Debug",
      "runtimeExecutable": "${workspaceRoot}/node_modules/.bin/electron",
      "windows": {
        "runtimeExecutable": "${workspaceRoot}/node_modules/.bin/electron.cmd"
      "program": "${workspaceRoot}/main.js",
      "protocol": "legacy"
      "name": "Renderer Debug",
      "type": "chrome",
      "request": "launch",
      "runtimeExecutable": "${workspaceRoot}/node_modules/.bin/electron",
      "runtimeArgs": [
      "sourceMaps": false

Using this launch file, to debug in the main process use Main Debug, to debug in the renderer process use Renderer Debug.

Developing Unity projects for Moverio BT-300 AR glasses on Windows

Since the Moverio BT-300 AR glasses run Android 5.1 using an Atom processor, it is possible to run Unity projects on them. The starting point is the instructions here on setting up Unity for the Android platform. One problem with this is that the android command is not included in Android Studio apparently so Unity builds will fail. So, to get Unity builds for Android to work, it is necessary to download and unzip the command line tools from the bottom of this page. This will create a directory tree that includes a tools directory. This should be used to replace the original tools directory in the Android Studio install, usually found at:


Incidentally, that is also the path that Unity needs to know in order to perform its builds.

There is a Unity plugin that provides support for 3D on the BT-300. For instructions on how to use the plugin, read:

 Assets > MoverioBT300UnityPlugin > MoverioController > README

The plugin includes a scene called MoverioTutorial that can be used as a starting point. It demonstrates some of the features of the plugin.

After the package name has been set in Player > Other Settings, it should then be possible to build, deploy and run on the BT-300 directly from Unity. I had a few problems with the tutorial with regard to SDK functionality but the Unity part seemed to work well (although I had to set 3D mode and disable the 2D camera manually sometimes). I am sure that I am doing something wrong – I’ll update the post when I work out what is happening.

Connecting a webcam to a VirtualBox guest OS

I am running Ubuntu 16.04 in a VirtualBox VM on a Windows 10 machine and wanted to access the laptop’s webcam from a Python script running in the Ubuntu VM. The trick (as described here) is to enter this line on the host while the VM is running:

VboxManage controlvm "vmname" webcam attach .0

where vmname is the name of the VM to be modified.

There doesn’t seem to be any need to add a USB filter for the webcam – doing that doesn’t seem to help at all.

The only problem with this is that the change isn’t permanent – it has to be run each time the VM is started. Simplest way to deal with that is to start the VM from a batch file:

cd "c:\Program Files\Oracle\VirtualBox"
VboxManage startvm "vmname"
VboxManage controlvm "vmname" webcam attach .0

Incidentally, this attaches the default webcam. Individual ones can be specified using .1, .2 etc. Use:

VboxManage list webcams

to get a list of webcams and aliases.

Using zeroconf to ssh into VirtualBox VM instances on Windows

If a VM is started in headless mode with bridged networking and DHCP, it’s kind of tricky to work out what IP address the VM is using in order to ssh into it. The simplest way is to use the zeroconf .local style address (i.e. <hostname>.local) but Windows by default doesn’t support this. However, installing Bonjour Print Services from the Apple Support website solves the problem.

Building the HoloToolkit.Sharing Library

Collaboration using AR is a fascinating area with many potential applications. The HoloToolkit is a very handy resource in general and includes the HoloToolkit.Sharing library to assist with collaboration. The HoloToolkit-Unity actually contains built versions of the Server, SessionManager and Profiler but it seemed like a good idea to build from scratch.

There are a few pre-requisites:

  • Windows SDK 10.0.10240
  • Windows SDK 10.0.10586
  • Common Tools for Visual C++
  • Windows 8.1 SDK and Universal CRT SDK
  • Java 8 SDK

An easy way to get the Windows SDKs is to run the BuildAll.bat script which will exit with an error if something is missing. Then use the solution file for the element that failed to start VS2015. VS2015 will then install the missing components. The Java SDK needs to be installed manually and requires environment variables JAVA_BIN that points to the JDK bin directory and JAVA_INCLUDE that points to the JDK include directory. The BuildAll.bat script should then complete successfully.

The Server is run using SharingService.exe and the user needs administrator permission to install as a service. This can be done by opening a command window in administrator mode and running the command from that for example. It’s actually useful to run the server using the -local flag (as a command line program) as then it’s easy to see status and error messages. The SessionManager displays current server state including connected clients.