OmniComponent: A Modular, Composable, and Reactive UI Extension for NVIDIA Omniverse

Introduction

‍

Managing software complexity is crucial to creating maintainable and scalable applications. One popular approach is to use a component-based system for organizing user-interface (UI) code. This involves standardizing the encapsulation of individual UI components that can be reused throughout the codebase. By following this standard, developers can ensure that their code is interoperable and familiar to other developers on the project. This results in a codebase that is easy to maintain and build on, regardless of development team size. This practice is commonplace among other UI development frameworks; it is particularly successful in web application development and has been popularized by the Angular and React frameworks, among others.
‍

When HEAVY.AI started to build an application on top of NVIDIAs Omniverse, we quickly realized that we needed a framework to guide us to the best practices described above.  UI developers in Omniverse should define and implement practices that have a proven record of success. Standing on the shoulders of giants results not only in a better product, but an improved, easier development process as well. In this article, we will explore a component-based UI extension we’ve created for this purpose in more detail.
‍

Background and Context

Omniverse is an incredible innovation built on Universal Scene Description (USD), that provides a platform on which vastly different industries can create and collaborate in virtual, three-dimensional spaces across the planet. NVIDIA offers many Omniverse Apps, catering to artists and engineers alike, but the true power and potential of the Omniverse platform comes from the Omniverse Kit SDK. Using Omniverse Kit, developers can create specialized Omniverse extensions, microservices, and plugins, and can develop specialized apps for any industry or use case. The vast majority of these extensions and applications require a UI that must be intuitive and easy to use. A well-designed, custom UI can greatly enhance the user experience and make the application more effective, flexible, and efficient.
‍

To develop a UI for NVIDIA Omniverse, developers use the omni.ui python package, which provides a library of atomic elements (i.e. Label, Button, ProgressBar) and element containers used to position these elements (i.e. Frame, Placer, Spacer). These low-level building blocks are highly flexible and configurable, enabling developers to create powerful and complex user interfaces by combining them in a myriad of ways. The composition of these elements in a UI is done programmatically and not through a markup language (i.e. HTML); without standardizing the UI development process, each developer is likely to impose their own approach to composing UI code.

‍

‍

When the HEAVY.AI team set out to create the Telco Digital Twin Kit Application, HeavyRF, we quickly encountered challenges as our custom UI grew in complexity. These issues were familiar to us from our experience building web applications, and we typically solved them by leveraging the React UI library. However, without the maturity of the Javascript ecosystem, we had to develop our solution in Omniverse using Python. Although some of these challenges were specific to our application, the solutions for the following issues will benefit the wider Omniverse developer community:
‍

• How to encapsulate and reuse similar groupings of omni.ui elements in a standardized way

• How to selectively re-render portions of the user interface instead of all of it

‍

Challenge #1: Reusing User-Interface Code

‍

Early in development, we decided to utilize Python functions to group UI elements to reduce code redundancy. Initially, this allowed us to utilize the function’s arguments to customize the behavior and appearance of elements. The variables returned by the function were stored and referenced. For form fields, we returned the `AbstractValueModel` responsible for accessing a field’s value as it is modified by the user; this satisfied the majority of our early needs. We created functions for new UI elements and added arguments to existing functions when they could be adapted for new features that were similar to previously implemented code. 

‍

‍

Eventually, shortcomings of this approach became apparent. First and foremost, we found it difficult to manage the internal state of these function-based components. Passing and returning closures responsible for getting/setting these state variables left us recreating the same closures whenever these function-based components were used, forcing us to write redundant code the functions were intended to eliminate. In addition to better state management, by this stage of development we had accumulated a wishlist of requirements for a new and improved component system:
‍

• Can reference the component using a single variable
• Can easily add behavior that is shared by all components
• Can be easily extended to accommodate new functionality
• Standardizes the interface for creating and interacting with components
  ‍

Solution: Class-based Components

We addressed our needs by devising a superclass from which all components would be derived: the OmniComponent. Once instantiated, a new component calls a method, `render`, that constructs the UI represented by the component. By utilizing class variables and methods, these components can manage their own internal state as well as provide an interface other components can manipulate programmatically. The arguments passed to our deprecated function-based components are instead provided to the class-based components as arguments on instantiation. Using the superclass to define behavior common to each component, we enforce the documentation of these attributes using Python type annotations; if an argument is passed to an OmniComponent that is not annotated, an error is thrown, and development cannot continue until remedied.
‍

‍

Challenge #2: Selective Re-rendering

‍

Certain properties of `omni.ui` elements can be updated without re-instantiating them. For example, by saving a reference to a Label element, the text displayed by the Label can be updated after it is created. However, as a higher-order abstraction, components encompass many elements and require the capability to conditionally render different subsets of elements in response to changes to their internal states. Updating a component’s state via reference was accommodated by our new class-based component system, but one last complication blocked our implementation: because `omni.ui` elements are positioned dependent on where they are nested within their parent containers, and this placement is determined by the order in which the components are initially rendered, how can we re-render a specific component without losing the position at which it was initially rendered? 

‍

Solution: Component-level Containers

We solved this problem by leveraging the ability of `omni.ui` containers to clear their contents without losing the position at which they were initially placed. OmniComponent provides access to a container that is unique to each instantiated component. This container is constructed on component instantiation and cleared of its contents on subsequent calls to the render method. For convenience, OmniComponent adds an `update` method to each component that can be called to re-render the component as necessary. This also allows components to maintain subscriptions and reactively update themselves in response to events within Omniverse.

‍

How-To
‍

‍

‍

Many code examples demonstrating various functionality can be found on the extension repository’s documentation. 
‍

Conclusion

In conclusion, the OmniComponent extension for NVIDIA Omniverse provides a powerful component system that helps developers organize their UI code using separation of concerns. This makes it easier to update and maintain individual components without impacting the rest of the UI, and also allows teams of developers to work on different parts of the UI simultaneously without impacting each other’s work. If you're interested in learning more about how the OmniComponent extension can help you create better, more maintainable user interfaces, be sure to check out the code repository. The extension is open source and contributions are welcome! The extension can also be found in the Omniverse extension manager by searching for “Heavy”. We encourage you to give Omniverse and the OmniComponent extension a try and see for yourself the benefits of this powerful tool.

‍