Mid February Update
Greetings everyone!
This is Derek Czerkaski, Head of Studio, reporting for duty once again! I wanted to provide an update to the community about our progress over the last few weeks! As I had paused development, this blog will again be production-oriented. I’ll keep it short and sweet!
As mentioned in my previous blog post, we’ve been moving over to our new toolsets and workflows. This took a little longer than we expected, but we’re wrapping up the final steps now. We expect to have everything, including our hiring page, up and running by the time this post goes live! We’re recruiting for a wide variety of roles, and if you’re a talented developer, artist, or social media expert who would be interested in joining our team, please connect with us on TeamTailor! :)
To share specifics on how these production moves impact Dead Matter in a positive way, we prototyped as a small team to hone in on focus points and to improve our own internal communication. It helped us spend some time with simple game flow to identify game-breaking problems that we needed to fix. As a result (and example of how valuable these sessions were and will continue to be), we started prototyping small-scale combat situations as a way of revisiting our fundamentals.We’ve also fully integrated JIRA into our workflow which is already helping us keep track of all the work that needs to be done. These two moves go a long way to improving our processes by quickly shining a light on problems and keeping those pain points prioritized correctly.
Another big update for you all is: We will be resuming full development February 28th! In our next blog, I’m hopeful we can have one of our designers talk more about what we’re setting out to do to kick things off!
We made substantial stability and performance gains, which I’m going to invite Nik Zorko to talk about in more detail here. Note that he goes pretty deep into the weeds as far as game-dev jargon is concerned, but if you’re a game developer (or an aspiring one), you may find this stuff super interesting.
So to bring everyone up to speed, we’ve been working on a few different things on the code side. First up, we’ve been working on stabilising the dedicated server. We had to add origin shifting support to a few actors. You don’t have to accommodate for origin shifting if your code is primarily run in singleplayer, but you have to manually accommodate for it when writing code for multiplayer.
Origin shifting is essentially the world moving around you to combat issues that stem from being far away from the exact centre of the world. You can see the effects on our procedural animations as there will be jittering when you are far away from the exact centre of the world. This effect gets worse with the distance between your camera and the centre of the world.
Playtests revealed that movement felt off, with constant micro-rubberbanding and poor server performance. In fact, our server would be rendered unplayable within minutes of playtesting which hindered the team working on prototypes. This lead to an entire can of worms that needed to be dissected.
As you can see in the images above, one of the issues we were immediately able to spot was that the time to update our navigation was causing tons of small hitches on the server that would last anywhere from 120ms to 150ms, this is unacceptable for keeping the server stable.
Once we realised that this was one of the contributing factors we were able to correct the issue by tweaking how our navigation mesh was being generated on the server.
We then needed to address the hard-hitting hitches pictured above, which got worse with the time that the server was online for, we initially couldn’t pin down what was causing the issue but after some iterative testing we were able to pin it down on player ragdoll behaviour after a player was killed. This was pretty easily resolved by simply moving the ragdolls to be simulated on clients only, with the server doing none of the physics simulation. We were also able to resolve an issue where player bodies would collapse in on themselves after death.
Lastly we also moved the server away from using Search and Prune for the physics post-update work and moved to using Multi Box Pruning which reduced the impact of simulating physics on the server. Pictured above is a breakdown of a single frame from where the hitching was very bad, this is what we addressed by implementing Multi Box Pruning.
After all of these targeted improvements were made, our server finally ran in a stable state in terms of performance, with GC being the only remaining issue to try and tackle. Though the hitching from GC seems to have a minimal impact, we also need to properly measure the impact via a shipping build since all of our testing was done in development and test builds.
Regardless I’m happy to let sleeping dogs lie on this one so long as it is not detrimental to gameplay. As they say, “it’s a problem for future me”.
Pictured above on the right is the outcome of the changes made to improve server performance and pictured above on the left is where we started.
In terms of things that were detrimental to gameplay as well as performance, there’s an issue we felt was better addressed sooner rather than later which is the IBL lighting system that we use for interiors.
As a refresher, the image above is the current implementation of our IBL lighting system from 0.7. It’s on the left, whereas the lighting setup that is similar to 0.6 is on the right. You can see that it addresses some of the issues with DFAO being a complete monster to our lighting. However, it came with a few bugs that were detrimental to gameplay. Such as eye adaptation not correctly playing with the system. It also came with some bugs that were related to translucency, glass and other translucent objects simply were not supported by the old system.
So I went ahead, and created a brand new implementation of the system from scratch with a few slightly different design goals in mind for the V2 of the system. One of my design goals was allowing artists to have a much easier time blending the different ambient lighting samples together, as well as allowing for in-editor scene capturing that can be used to approximate the ambient lighting present in whichever scene they are sampling.
I also wanted the system to properly support auto-exposure, as well as allowing elements of the previous system to be preserved. Primarily for masking out weather effects and some of the lighting information from the global illumination system we use for outdoor environments. I also wanted to reduce the rendering overhead of the previous system by taking advantage of tiled deferred culling via a compute shader.
With all of this in mind, I had roughly a week to get the new system implemented to replace the old lighting system. Changes to the rendering pipeline are pretty daunting and there is very little publicly available information on where to go in order to do certain things. I had little time and an intimidating lack of clarity from how few people dive this far into the engine.
For three days I didn’t really have any results until I was successfully rendering some of the data that was piped through all of the underlying structure that I had created between Monday and Wednesday. On Thursday I focused on getting all of the shader setup working correctly, and on Friday I implemented the tiled culling as well as cleaning up a bug related to exposure and how the lighting was applied to the diffuse lighting environment. Below are the results from the entire process.
I was actually quite shocked at how I was able to reproduce a quality that was near the baked lightmaps along with the ability to easily swap out IBL data based on whether or not light sources in the scene are active. Keep in mind that in the above screenshots, all forms of screen space ray tracing such as screen space reflections and screen space global illumination have been completely disabled.
However, upon enabling Screen Space Raytracing such as Screen Space Reflections and Screen Space Global Illumination, while there are some differences, the lighting looks similar enough to the point where it is more than acceptable in terms of fidelity. I am extremely happy with the results and I cannot wait to see what our art team does with the system.
Lastly, I’ve also been working on setting up forums for our Backer community via https://forums.qi.games/ I was able to get Backer roles automatically assigned and I also added a few personal touches to the forums. You can also sign in via Steam and if you own a copy of Dead Matter you will automatically be assigned the Backer role, granting you access to a hidden forum specifically for the game and for us to more easily engage with our community.
Thanks for your continued encouragement and support, and we’ll be excited to talk more about the next update in our next blog post!
