It wasn’t too hard to go from the inline rt-ai Edge Stream Processing Element using the Coral Edge TPU accelerator to an embedded version running on a Raspberry Pi 3 Model B with Pi camera. The rt-ai Edge test design for this SPE is pretty simple again:
As can be seen, the Pi + Coral runs at about 4 fps with 1280 x 720 frames which is not too bad at all. In this example, I am running the PiCoral camera SPE on the Raspberry Pi node (Pi7) and the View SPE on the Default node (an i7 Ubuntu machine). Also, I’m using the combined video and metadata output which contains both the detection data and the associated JPEG video frame. However, the PiCoral SPE also has a metadata-only output. This contains all the frame information and detection data (scores, boxes etc) but not the JPEG frame itself. This can be useful for a couple of reasons. First, especially if the Raspberry Pi is connected via WiFi, transmitting the JPEGs can be a bit onerous and, if they are not needed, very wasteful. Secondly, it satisfies a potential privacy issue in that the raw video data never leaves the Raspberry Pi. Provided the metadata contains enough information for useful downstream processing, this can be a very efficient way to configure a system.
I decided that it would be fun to try out a Google AIY Vision Kit as a sort of warm-up for the potentially much more significant Edge TPU.
The Vision Kit is basically the same configuration as the ZeroSensor camera except with an extra board in the camera path that can perform inference on the captured images. The kit comes with some frozen graphs that can be used to detect a few things but I thought it would be interesting to try training a MobileNet SSD network with the Pascal VOC 2012 training data which can identify 20 different objects. The instructions for how to do this are here.
Once that was all running, the next step was to integrate it with rt-ai Edge. It’s pretty similar to the earlier full-blown TensorFlow version so it didn’t take too long to get working.
The design is much the same as usual except with the new VisionKit object detection SPE instead of TFObjectDetect or Deeplab. Note that the PiCam and VisionKit SPEs are running on the AIY Vision Kit, whereas the MediaView SPE is running on a desktop.
This is the output from the MediaView SPE. The metadata has been formatted to look exactly the same as the previous TensorFlow detector so that they can be used interchangeably in stream processing networks. I am getting about 2 fps with 640 x 360 images which is actually better than I expected.
A while back I built some add-on cards for Raspberry Pis to do some environmental monitoring around the house. This is one of them.
The project starting collecting dust when I couldn’t really think of good ways of using the data, beyond triggering an alarm under some conditions or something. However, it’s often interesting just to see what’s going on around the place so I have revived the sensors (a good use for old first generation Pis). The screen capture shows a simple but actually quite effective way of using the data that’s being generated, providing a display that’s adjacent to the camera feed from a webcam on the same Pi. Between the two streams, you can get good confidence on what’s happening in the smart space.
One day, I’d like to get the HoloLens integrated with this so that I can see the data when I am in the smart space. That would be even more fun.
Easy but not particularly memorable, enter this to set output volume to 60% for example:
amixer set Master 60%
rtndf now has a simple Python script that allows a Raspberry Pi fitted with PiCamera to be used as a PPE to generate an MJPEG video stream over MQTT. The resulting stream looks identical to that generated by uvccam (which works with UVC webcams) so picam and uvccam can be used interchangeably as video sources in rtndf pipelines.
Up to now the only data sources in rtndf were video and audio. imu is a new Python PPE that can be used to stream IMU data (fused pose, sensor readings etc) into an rtndf data flow pipeline. Another new PPE is imuview, this time a C++ PPE, that can display the resulting stream. The screen capture above shows the data being streamed from a Raspberry Pi SenseHat which is a full 11-dof sensor.
One of the nice things about using a pub/sub system like MQTT is that it is possible to hook into any of the pipeline links to see what data is flowing. To this end, a future PPE will be a generic viewer. The user just gives it the topic and it determines the type of data and displays it appropriately. A very handy debugging tool!
Recent releases of Raspbian have started to use the device tree which has the effect of altering the procedure for controlling the I2C bus speed (amongst other things). The new procedure involves a device tree overlay or device tree parameter. These instructions have been distilled from this page.
On the latest Raspbian releases, the required overrides are already present. All that’s needed is to set a device tree parameter. To do this, edit /boot/config.txt and add the line:
Reboot and then you should see a message ( or use the dmesg command) indicating that the I2C speed is indeed 400000. If the release is older, a device tree overlay is needed to add the overrides.
Continue reading “Changing the Raspberry Pi/Pi2 I2C bus speed, device tree style”