Runtime Assets

TLDR

I created a system which permits the saving/loading of arbitrary meshes/materials/transform-hierarchies. It’s backed by multiple levels of cache and quality to support real-time streaming of assets.

Problem

We want to allow users to use 3D files created in external programs and bring them into their exhibitions for all to see. This presents a major challenge as all of Unity’s Editor’s 3D file importers/exporters and serializers are not available at runtime.

A solution for adding post-launch content would typically involve AssetBundles, but this isn’t possible because AssetBundles can only be created within a Unity Editor. This may be fine in a game like VRChat which targets a tech literate audience and whose core product is providing social spaces and not user-generated content tools. In our case, however, we want anybody to be able to create exhibitions using our own artist-friendly editor on mobile, rather than Unity’s on desktop. A work around would be to have a Unity Editor running on the Cloud which artists could send 3D files to to be converted, but this presents several large downsides. It would significantly increase costs as our architecture would no longer be “serverless”. It would dramatically increase the complexity and overhead of maintaining our development stack as server scaling and spinning up would no longer be handled for us. It would mean that artists would first have to upload and then download any assets they wanted to use in an exhibition, which would be a hugely tedious workflow bottleneck, and would require a constant network connection.

Solution

For all these reasons I manage the de/serialization of artist’s 3D assets myself and convert them into a custom, Unity-friendly format at runtime.

Streaming

Mesh, Textures and any other large blobs are all loaded asynchronously. This allows artists to keep working and exhibition viewers to keep exploring as assets are streamed in. All these operation’s progressions are tracked at a fine-grain level so detailed feedback can be provided to the end user in the form of smooth, and accurate loading bars/circles/etc..

Caching

There’s a three-tiered caching system which includes main memory, local disk, and the cloud. Whenever a client requests an asset, it’s retrieved from the closest available location that the asset is cached.

Varying qualities

Each Blob is saved at multiple qualities. For textures this means multiple resolutions, for Meshes this means multiple LODs. If clients want to preview an asset they’re not forced to download the full resolution thing from the cloud, waiting for tens of seconds and incurring significant network charges, instead they can download the thumbnail versions of the blobs in a fraction of the time. Even when the client requests the full quality asset, the smallest, preview quality blobs are always streamed in first so we can get something on the screen as quickly as possible before loading the final resolutions.