Version Control For 3D Assets

Version Control For Game Development

Version control management (VCM) is integral to any software development project. VCM is the tool that allows for tracking and managing changes to code, while also allowing large numbers of developers to work on the same project simultaneously. While the original predecessors of modern version control began to emerge in the 80’s (PlasticSCM), today, there are dozens of solutions for VCM, with the two most popular for game developers being Git and Perforce (Diversion):

• Git: a free and open-source distributed version control system that was created by Linus Torvalds, the creator of Linux. GitHub and GitLab are two companies that utilize Git and have created businesses around VCM. Git was rapidly adopted across the wider software ecosystem for its new features, including its distributed nature (not all stored on a central server), its approach to collaboration, and the UX improvements from Github (Hackernoon). It is important to note that Git was originally created as a version control for Linux and, therefore, was built for text-based files (human-readable) instead of binary files (not intended to be read or edited directly by humans).

• Perforce: founded in 1995 and is the gaming industry standard for VCM, with 19 of 20 of the top AAA publishers using it (Perforce). Perforce has taken an enterprise approach to VCM and deeply integrates into existing workflows helping to solve complex problems typically associated with VCM including large file storage and locking files to avoid multiple users editing the same file.

Collaboration: Allowing multiple users to work on the same project at the same time and deciding which versions to keep, especially when there is a conflict (disagreement between developers or artists on which version should be used) is another pillar of VCM.

• Large Files: Large files are more complicated than text because of how they are stored. An asset created in Maya (a popular 3D creation tool) for example will include the body, textures, and color of the 3D model. All of these features are saved together in a FBX file, which is not human readable, but instead a series of digits that represent the 3D object. If this was a text file, it would be simple to view the changes made to the file, decide which ones to keep, and store only those changes in order to save space; but with this FBX file its extremely difficult to ascertain the changes that were made without opening it up in a 3D modeling software that can interpret that file type, and then comparing the versions side by side.
• Code: When using VCM for code, the files are stored in a way that is readable to the human eye (i.e. a line was added to make the character move forward). This allows the developer to easily identify the changes and incorporate the ones that she deems correct, making collaboration straightforward.

In summary: Compared to traditional text-based files, large non-text-based files are cumbersome to store and difficult to compare.

The Problem With Existing Solutions

While many users have turned to existing solutions like Git for large files, Git’s original focus on text-based files is apparent in their piecemeal solutions for large files. So much so, that some users avoid uploading their assets to Git at all and supplement their workflows with Google Drive, Dropbox, and Email, despite having this approach forgo a single source of truth, while also adding another software to manage.

Perforce and other larger enterprise focused VCM solutions have stepped up attempting to alleviate some of the pain points:

• Storage & Access: One of the ways that these companies solve this large file storage issue is by allowing for storage in on-premise servers and thereby alleviating the costs associated with storing large assets in the cloud.
• Collaboration: Because many of these large files require the actual CAD software to render (view in a virtual environment), instead of attempting to build a rendering tool for every type of file (impossible in some cases like the “U” Unreal Engines file format which can only be used on Unreal Engine), some companies have opted only to support specific file types, while others have attempted to create plugins for the CAD software that allows for the direct integration of their VCM.

These solutions address the symptoms, but not the core problem. On-premise servers are a band aid solution for massive files that cannot be stored efficiently on the cloud, while plugins are another point of friction in the user experience instead of the more centralized and streamlined approach we see with Github.

Artists are still frustrated, pointing to issues like the need to learn complex processes and lack of collaboration tools, not to mention the immense cost and complex implementation at the outset. As seen below, artists are still spending large amounts of time trying to manage their assets (Unity).

What’s Next And What’s At Stake?

Solving this problem is more important now than ever given the influx in large files as AR, VR, and virtual worlds become more complex. The next generation of version control could come in many forms offering multiple features:

• Storage: More efficient file storage to reduce the quantity of data stored.
• Cloud: Cloud native storage could allow for more instant collaboration and dynamic locking of files, but would require improved storage efficiency in order to reduce egress and storage costs.
• Focus on VR: Native 3D previews, VR walkthroughs of design changes.
• Cost & Implementation: Alleviating the cost and upfront complexity of implementing these services would open up the long-tail usage of these platforms.
• Workflow Management: VCMs are continuing to expand outside of just version control into the rest of the development process.

Innovations in other parts of the industry will also likely impact how game developers use version control. Most notable is the Alliance for OpenUSD (AOUSD) which is attempting to enhance the interoperability of 3D tools and data, across software applications using their open-sourced software called Universal Scene Description (USD). As more assets are created across more programs in more file types, collaboration across software will become increasingly important. Although USD is not necessarily a VCM, it does enable more real-time collaboration between artists.

Another potential source of disruption is the vertical software providers themselves (Maya, Blender, Unreal Engine, etc). Given they own their own rendering pipeline, it would be more likely that Unreal Engine could create an internal tool that highlights differences in their large files.

We believe that existing solutions today, namely Perforce, do a good job of managing large files for artists and development teams (despite introducing cost and complexity). Still, there are unique areas of technological innovation that have the ability to disrupt the space, leaving the door open for innovative startups.

Takeaway: Version control systems have been pivotal in managing code, but as the digital landscape pivots towards more complexity, traditional version control platforms show their limitations. While solutions like Perforce have made strides in adapting to these needs, there remains room for improvement and innovation. As we look to the future, integrating features such as improved storage and cloud-native functionality could significantly enhance efficiency and collaboration.

The race is on for innovators and established players alike to craft a service that not only meets the current demands of large file asset management but also sets the stage for future advancements, potentially transforming how we store, manage, and analyze digital assets in a rapidly evolving digital world.