Monado - GSoC Ideas 2023

Project Ideas

Portable and convenient config UI

Monado has a lot of configurable options and currently, there is no convenient visual means to update them. This project consists in implementing a config UI in Monado and could be broken down into three project phases. First, choose a UI technology that satisfies the below UI requirements. Second, Integrate that UI technology into Monado codebase. Third, implement the required UI menus and elements using the integrated UI codebase.

UI toolkit requirements:

Deliverables:

Requirements:

Difficulty: easy

Project Size : Small (~150 hours)

General UI

There is no convenient visual feedback about and way to control the state of the currently running Monado instance. This project consists in implementing a general UI in Monado and could be broken down into three project phases. First, choose a UI technology that satisfies the below UI requirements. Second, Integrate that UI technology into Monado codebase. Third, implement the required UI menus and elements using the integrated UI codebase. This idea differs from “Portable and convenient config UI” in that it is less about manipulating configuration values, and more about providing ongoing feedback and controls such as

UI toolkit requirements:

Deliverables:

Requirements:

Difficulty: easy

Project Size : Small (~150 hours)

Frame Timing Tools

In VR, stutters, judders and jitters in rendering are a much bigger deal than on desktop monitors. Instead of being just annoying, they can cause disorientation, and even motion sickness. Most importantly the VR runtime must push frames to the VR hardware at the native hardware refresh rate. To that end it must keep track of the frames submitted by client applications, reproject old frames if the client application missed an interval, and push the frames to the VR hardware while sharing GPU hardware resources with the client application. Such a system benefits greatly from good visual frame timing tools. A good frame timing tool would gather timing information from both the frames submitted by the client application and the frames submitted by the compositor to the hardware and relate them in a visual way. The visualization could be either done with an existing visualization tool like Perfetto, but it could also be a custom built UI for Monado.

Deliverables

Requirements:

Difficulty: medium

Project Size : Large (~350 hours)

Immersive Dashboard

Monado is an OpenXR runtime used to run immersive OpenXR applications. When the Monado server is first launched, the user wearing any AR or VR headset ends up in a very simple space without any means for directly seeing runtime options or launching applications. Apps need to be executed from another means, such as the Host’s console and so, this project Idea consists in implementing a full-blown Immersive System Dashboard within Monado. This visually appealing dashboard should not only let the user interact with applications’ lifecycle (start, pause, stop), but also allows for seeing debug variables in realtime (frame statistics, timing, etc…). The dashboard as an application in itself should be built with lovr (https://lovr.org/), a framework coded in lua language that allows developers to create very nice immersive objects and scenes very easily and efficiently.

Deliverables:

Requirements:

Difficulty: medium

Project Size : Small (~150 hours)

Visual-inertial tracking module expansion

This is mostly an integration task that will make the visual-inertial tracking module more versatile and usable for more people. The student will learn a lot about build systems, dependency management, CMake, C++, etc, for Linux, Windows, and Android.

One of the halves of this project is integrating two tracking systems: DM-VIO and SVO Pro. DM-VIO (GPL) is one of, if not the best monocular-IMU system, even competing with stereo-camera systems. If we plan to make our tracking work on phones, this looks like a very good starting point. SVO Pro looks like one of the SLAM systems with more polishing out there implementing very cool techniques. I think it has a good chance of competing or even surpassing what we have currently, and would be nice to have that as a comparison point for our own system.

The other half is dedicated to platform support. Currently, our tracking module only runs on Linux. Basalt is the underlying SLAM system we use and we need it running on Windows and Android; many Monado users have asked for this. The idea is to run camera-IMU datasets through the entire monado-basalt pipeline on both platforms.

If time remains, and as a stretch goal, expanding and adapting the current Android driver in Monado to make it feasible to run the just-integrated Basalt tracking would be awesome!

Deliverables:

Requirements:

Difficulty: medium

Project Size : Large (~350 hours)

Passthrough

Passthrough for VR is an essential feature in current headsets which have multiple cameras available on them. It lets the VR application project the video feed from the cameras into meshes so that the user can be still be aware of its surroundings and interact with them while they are in the VR experience.

This is also one of the base features needed to be able to have a proper room setup in Monado. Furthermore, if we manage to do full FoV passthrough, that will be a key way of observing drift and accuracy in our visual-inertial tracking systems. The student will learn about OpenXR, Monado, C, C++, and some computer graphics/vision basics.

Deliverables:

Requirements:

Difficulty: medium

Project Size : Large (~350 hours)

Your own idea

We greatly welcome a student to come up with their own project idea, we could not possibly cover ever single idea that is possible for Monado on our own. Please join our chats to discuss them.