It’s only a step on the road to the ultimate goal of AR headset support for sentient spaces but it is a start at least. As mentioned in an earlier post, passing data from rt-ai Edge into Manifold allows any number of ad-hoc uses of real time and historic data. One of the intended uses is to support a number of AR headset-wearing occupants in a sentient space – the rt-ai Edge to Manifold connection makes this relatively straightforward. Almost every AR headset supports Unity so it seemed like a natural step to develop a Manifold connection for Unity apps. The result, an app called 3DView, is shown in the screen capture above. The simple scene consists of a couple of video walls displaying MJPEG video feeds captured from the rt-ai Edge network.
The test design (shown above) to generate the data is trivial but demonstrates that any rt-ai Edge stream can be piped out into the Manifold allowing access by appropriate Manifold apps. Although not yet fully implemented, Manifold apps will be able to feed data back into the rt-ai Edge design via a new SPE to be called GetManifold.
Next step for the 3DView Unity app is to provide visualization for all ZeroSensor streams correctly physically located within a 3D model of the sentient space. Right now I am using a SpaceMouse to navigate within the space but ultimately this should work with any VR headset with appropriate controller. AR headsets will use their spatial mapping capability to overlay visualizations on the real space so they won’t need a separate controller for navigation.