AIY Vision Kit + MobileNet+SSD: a smart camera for rt-ai Edge

I decided that it would be fun to try out a Google AIY Vision Kit as a sort of warm-up for the potentially much more significant Edge TPU.

The Vision Kit is basically the same configuration as the ZeroSensor camera except with an extra board in the camera path that can perform inference on the captured images. The kit comes with some frozen graphs that can be used to detect a few things but I thought it would be interesting to try training a MobileNet SSD network with the Pascal VOC 2012 training data which can identify 20 different objects. The instructions for how to do this are here.

Once that was all running, the next step was to integrate it with rt-ai Edge. It’s pretty similar to the earlier full-blown TensorFlow version so it didn’t take too long to get working.

The design is much the same as usual except with the new VisionKit object detection SPE instead of TFObjectDetect or Deeplab. Note that the PiCam and VisionKit SPEs are running on the AIY Vision Kit, whereas the MediaView SPE is running on a desktop.

This is the output from the MediaView SPE. The metadata has been formatted to look exactly the same as the previous TensorFlow detector so that they can be used interchangeably in stream processing networks. I am getting about 2 fps with 640 x 360 images which is actually better than I expected.

Smart spaces and IoT data – the challenge is what to do with it

A while back I built some add-on cards for Raspberry Pis to do some environmental monitoring around the house. This is one of them.

The project starting collecting dust when I couldn’t really think of good ways of using the data, beyond triggering an alarm under some conditions or something. However, it’s often interesting just to see what’s going on around the place so I have revived the sensors (a good use for old first generation Pis). The screen capture shows a simple but actually quite effective way of using the data that’s being generated, providing a display that’s adjacent to the camera feed from a webcam on the same Pi. Between the two streams, you can get good confidence on what’s happening in the smart space.

One day, I’d like to get the HoloLens integrated with this so that I can see the data when I am in the smart space. That would be even more fun.

picam – a Python rtndf video pipeline processing element for the Raspberry Pi/PiCamera

IMG_5370rtndf now has a simple Python script that allows a Raspberry Pi fitted with PiCamera to be used as a PPE to generate an MJPEG video stream over MQTT. The resulting stream looks identical to that generated by uvccam (which works with UVC webcams) so picam and uvccam can be used interchangeably as video sources in rtndf pipelines.

imu and imuview – adding IMU sensing to rtndf data flow pipelines

imuviewUp to now the only data sources in rtndf were video and audio. imu is a new Python PPE that can be used to stream IMU data (fused pose, sensor readings etc) into an rtndf data flow pipeline. Another new PPE is imuview, this time a C++ PPE, that can display the resulting stream. The screen capture above shows the data being streamed from a Raspberry Pi SenseHat which is a full 11-dof sensor.

One of the nice things about using a pub/sub system like MQTT is that it is possible to hook into any of the pipeline links to see what data is flowing. To this end, a future PPE will be a generic viewer. The user just gives it the topic and it determines the type of data and displays it appropriately. A very handy debugging tool!

Changing the Raspberry Pi/Pi2 I2C bus speed, device tree style

Pi2v1Recent releases of Raspbian have started to use the device tree which has the effect of altering the procedure for controlling the I2C bus speed (amongst other things). The new procedure involves a device tree overlay or device tree parameter. These instructions have been distilled from this page.

On the latest Raspbian releases, the required overrides are already present. All that’s needed is to set a device tree parameter. To do this, edit /boot/config.txt and add the line:


Reboot and then you should see a message ( or use the dmesg command) indicating that the I2C speed is indeed 400000. If the release is older, a device tree overlay is needed to add the overrides.

Continue reading “Changing the Raspberry Pi/Pi2 I2C bus speed, device tree style”

Using the Raspberry Pi’s serial port

DB-25Yes, that is a DB-25 plug I found in a box of old stuff. Anyway, I wanted to connect a GPS via a serial interface to a Raspberry Pi. Somehow it had escaped my attention that there is a handy serial port available on the Pi’s connector – I had been wondering what /dev/ttyAMA0 was!

Continue reading “Using the Raspberry Pi’s serial port”