We chose Unity 3D engine to develop ARchy the Rabbit application because it was the only cross-platform tool supporting augmented reality development, Vuforia, and HoloLens development at the time the project was developed.
Basically, to use an AR app, the user should hover the smartphone or tablet camera over the special marker so the application can capture this marker and run the AR content. We used Vuforia for marker recognition in the application. We had to optimize the app load time, and it took only a few seconds for a final product to launch. Also, we had to experiment with the FPS of the AR content and to optimize the 3D model to ensure the app is working smoothly and avoid crashes.
Unlike the augmented reality application, the HoloLens app is much more immersive and realistic since the virtual content appears in the real world as its natural part.
Also, we have extended the app functionality and added new features: - an ability to locate the rabbit on any surface in the room using the "environment understanding" sensors of HoloLens;
- an ability to see the 3D model and animations from any angle;
- an educational tutorial for new users;
- a possibility to use voice control instead of navigating through UI.
Basic HoloLens functions and features, such as voice recognition, spatial mapping, and spatial sound have been realized with the help of Unity. In addition, our specialists created additional shaders for transparency and to enforce the humanoid animation so that Archy can look at the users and follow their gaze. A combination of HoloToolKit and Unity allowed us to achieve flawless gaze and gesture recognition. As a result, we became one of the first development companies to provide turnkey application development for Microsoft HoloLens and mixed reality.
