In order to avoid the whole garbage in, garbage out problem, I want to make sure that the long term data that I am collecting from the ZeroSensors and IP cameras is actually what I think it is. rtaiView is an rt-ai Edge app that allows both real-time streams and historic stored data to be reviewed in a convenient way. It can be used to check both raw data and extracted metadata for each stream. The screen capture above shows an rtaiView instance monitoring the real-time streams from two external IP cameras and the three streams (video, audio and multi-channel sensor) from a ZeroSensor. I am going to add a second ZeroSensor in a second internal location and then let the whole thing run for a while. Just to make life complicated I am using the straight TensorFlow object detector for the external cameras and DeepLabv3 segmentation in color map mode for the internal cameras so that I don’t upset the other inhabitants who aren’t keen on me putting cameras everywhere.
Focus now is moving on to extracting interesting sequences from the stored data and then using these to train an anomaly detector. The exact architecture of the anomaly detector is still a bit unclear – definitely a research project.