Summary
The Outer Worlds 2 is a not yet released sequel to The Outer Worlds.
Given this project is still in development I cannot talk in depth too much about some of the stuff I’ve worked on. However, most of my work on The Outer Worlds 2 shifted to be more technical/engine work that doesn’t reveal any specific details about the game if discussed- so I can still talk a bit about my work.
Engine Upgrades
During the development of The Outer Worlds 2 I have been responsible for engine upgrades, this included the upgrade to Unreal 5 as well. The upgrade process has involved merging the engine upgrade with our code and existing engine modifications, fixing modifications (when possible) that aren’t mergeable due to change from Epic, ensuring stability of the resulting merge (fixing any bugs the upgrade introduced), and finally actually merging the final merge to our main development stream. This process has required my to be quite familiar with most of the modifications our team has made to the engine as I’ve had to touch most of them at one time or another while merging.
World Partition
As I mentioned before we upgraded to Unreal 5 during the development of The Outer Worlds 2, as part of this upgrade (some time after the initial merge) we also made the swap to World Partition (from World Composition) for our levels. I was responsible for most of this process and led a small team (taking people from other teams as needed/scheduled) to get this done.
Some of the major work involved with this included the actual conversion of maps to World Partition (and while Epic had a converter out of the box for maps, we needed to make many fixes to it to get our maps converted properly). We had to fix multiple issues that came up during conversion (namely with actors losing refs during conversion- destroying our content) and had to ensure our navigation/AI systems (many of which we had rolled a bit too custom) works in World Partition.
One of the other main focuses of this upgrade was how we script our world in World Partition. Our worlds used lots of level scripts in the world composition days and those workflows worked for our studio (and I would argue were largely the ‘correct’ way to do things) and were largely missing in World. This lead to creation of World Script Actors.
‘World Script Actors’
World Script Actors was a plugin that I built for how we would script in World Partition on The Outer Worlds 2 (and this plugin ended up being shared across projects and shipped Avowed as well). World Script Actors, put very basically, are placeable level script actors (level script actors normally exist per sublevel and are just embedded into the level).
While there are still improvements we would love to make to this plugin, it is functional / shippable (and as mentioned it’s already shipped a game!)- the main implementation details were:
- World Script Actors just use regular placeable blueprints- they are a blueprint actor like any other normal actor.
- Via various editor extensions you can add ‘bindings’ to actors in a world to the world script actor (using an actor picker or creating bindings to selected actors in the editor)- this creates a soft reference to the actor that, on a placed instance, ‘binds’ to the actor via a hard actor reference blueprint variable. (A blueprint variable was used because then you can use that variable to script with the actor very similarly to when you drop an actor reference directly onto a level script actor).
- Through additional editor extensions we readded the ability to use stuff like context menus to add events from actors (such as on overlap) to a world script actor, find existing events on a script, open scripts, etc. The context menus for these auto-populate with any world script blueprints that are loaded or are actively being edited.
Additionally world partition specific functionality was made-
- As you can guess, this approach to scripting can create large actor clusters that are bad for streaming- the class has the ability to estimate its cluster bounds and throw map check warnings/errors based on their size. We also have tools to visualize the size and help call out actors causing issues to optimize streaming.
- When we first converted to World Partition the resulting conversion had any levels using level scripting in the World Composition version of the map using Level Instances. In order to help design and _significantly_ speed up our conversion to World Partition I ended up adding the ability to convert a Level Instance with a level script to a World Script Actor- this involved extracting the blueprint for the level script to a new blueprint that was a World Script Actor, replacing any hard actor reference nodes with automatically generated ‘bindings’, copying the actors in the level instance into the normal world partition world and destroying the level instance, spawning the new world script blueprint in the world in place of the level instance, and then finally validating everything linked up and worked correctly. This ended up being massively successful and took what could have been months of manually porting scripts to World Script Actors into a day spent by designers converting their level instances and ensuring they worked properly with the World Script Actor.
Tick Manager
The tick manager is a system that we use to bundle and prioritize ticks- based on this video from Rare. The manager was implemented as a plugin so it could be shared, but The Outer World’s 2 uses generic engine mods that allow using the Tick Manager in place of the normal tick built into actors/components easy (with it still calling the same tick function as the Engine, just via the manager).
This has helped us out for the same reason it helped Rare- by bundling ticks of the same type we see performance improvements via keeping the instruction cache hot. It’s also surprisingly helpful when profiling a frame as it makes it easy to see how much a certain component/actor’s tick is taking in aggregate. And finally, the prioritization does help- however the first implementation allows prioritization to either run on the game thread or in parallel (using a parallel for) from the game thread. The overhead of prioritization could actually be quite costly vs the ticks that run for certain objects (making prioritization not appropriate for all objects).
In order to fix the prioritization issues (/claw back some performance to help us hit 60 FPS) we ended up adding the ability for the prioritization step to run on workers, independent of the actual tick (and ticks can be fully async now too with this change). This change required a lot of work making most of the system thread safe (e.g. needing to potentially register/unregister actors while the tick group is calculating/ticking on a worker, etc). The task management itself was simple within the engine as Epic already has a nice system for tick functions (using the task graph) to set up dependencies, tick groups, and their ability to run async- leveraging this system allowed us to easily split up the work and let UE schedule it across threads for us as appropriate. Now we can calculate priority on a worker early in the frame and execute prioritized ticks on a subset of objects later in the frame on the game thread. Where prioritization and ticking can occur, how to tick objects, how to prioritize objects, budgets, and more are all defined by settings set per ‘tick set’ that is passed into the manager. All in all, this has been a powerful system that has helped us squeeze performance for higher framerate modes and makes utilizing more workers a bit easier for gameplay work.
‘AutoPlayer’
I previously mentioned the autoplayer for The Outer Worlds as it was originally developed there (to do basic automated testing for the game). Given its use on the original game we have spent further time improving it for The Outer Worlds 2- some examples of features that I implemented include:
- Additional stat tracking such as VT Pool info, Nanite Pool Info, all stats (timer, counter, memory) can now be tracked, Virtual Shadow Map Usage, and more.
- The autoplayer from The Outer Worlds would track what areas it had visited previously and try to not revisit them- this has further been extended on The Outer Worlds 2 to impact pathfinding- with the autoplayer running path finding with visited areas scored negatively, causing it to prefer visiting new areas with pathing- and actively pathing around already visited areas.
- The autoplayer now tracks and remembers map coverage (e.g. if it’s covered all the nav in a map before, how many times it’s done it, etc) and uses that to score maps to select when deciding to move to new maps (preferring maps that need more testing)- this information is also written to disk on occasion and is restored on start up so if the autoplayer crashes it will continue with the coverage information from the previous autoplayer.
- It can now generate clips using debug APIs on certain platforms/consoles when certain events occur- for example when memory usage peaks it will capture a clip of a configurable length a configurable duration after the event (this was useful for us to identify what was causing memory usage spikes in our game to help resolve them).
- We have built scripts/systems to automatically run the autoplayer on a console if it crashes- this allows us to now kick the autoplayer and harvest data as well as crashes over extended periods of time.
Memory
Like when shipping The Outer Worlds 1, a large part of my focus on The Outer Worlds 2 has been memory- both in analyzing memory usage for content to help pinpoint areas as well as continuing to build and improve systems to help analyze memory. Some of these systemic improvements include-
Custom Memory Report
Expanding upon the custom memory report command we built for The Outer Worlds 1- really any time we found a new type of memory issue we added something to the report to help call it out. These improvements ended up including adding reports for: Chaos Geometry Memory, ISM Component Information, LLM Tags (if enabled), Nanite Pool Usage, Static Mesh Collision, Virtual Texture Info, World Partition Streaming Info, and more. Scripts were also built to be able to parse these reports and generate quick delta reports- used to make comparing reports easy as well as for sending out semi-automated memory report emails for specific content areas to track memory changes over time.
Memory Leaks
Tracking down memory leaks also continued to be a major focus/specialty for me- Unreal Insights now includes tools to help with this that we didn’t have back on The Outer Worlds 1, but we have found that Insights is not good for long term memory captures- meaning our custom LLM tooling was still useful. Late in the project we made a push for fixing memory leaks, and I was able to get our memory growth in the 12-48 hour uptime range (we ignore the first ~12 as a lot of systems in UE just grow memory over time naturally and flatten their usage out) down from ~500 megs in 36 hours to ~50 megs in 36 hours. These fixes were spread across engine fixes (which you can find UDNs authored by me for) as well as project fixes. These improvements were critical for us to stay in memory long term on more memory constrained platforms.