Progress Report July 2020

Written by Morph and GoldenX86 on August 04 2020

Hey yuz-ers, what is up and welcome back to the… progress report. In this monthly report, we offer you major rewrites to critical components, another successful attack on shader compilation times, tons of game fixes, and the promise of a very bright future ahead. Let’s get started!

Paper Mario: The Origami King fixes

To begin with, Rodrigo implemented several fixes to make Paper Mario: The Origami King playable:

 Now in A4 format! (Paper Mario: The Origami King)

Now in A4 format! (Paper Mario: The Origami King)

  • Some graphical artifacts affecting OpenGL can be fixed in GLASM by executing the shader instruction BAR inside control flow.

That is one big sheet of paper! (Paper Mario: The Origami King)

Lights, please! (Paper Mario: The Origami King)

New GPU Virtual Memory Manager

bunnei brings the big guns. The complete rewrite of the GPU Virtual Memory Manager fixes several bugs in many games, with the two strongest examples being Mario Kart 8 Deluxe and Super Smash Bros. Ultimate. Now several graphical bugs are fixed in Mario Kart 8, including tracks that show invisible terrain or missing graphics. In Smash’s case, there are no more vertex explosions now.

This rewrite also should reduce committed memory a bit, hopefully reducing the size of the required page file, and opens the possibility to implement new features in the future. We have to clarify that this doesn’t fix the high VRAM usage in Vulkan.

That's not what I meant when I said Anti-Gravity! (Mario Kart 8 Deluxe)

Bug fixes and improvements

Accompanying the recent addition of 32-bit emulation support, bunnei implemented support for the creation of random and default Miis. This does not include full support for custom ones for now. With Miis in place, games like Mario Kart 8 Deluxe and New Super Mario Bros. U Deluxe can now get past their menus and become playable.

For this service to work, the user needs to dump the system archives from their Nintendo Switch. The process requires the Goldleaf homebrew.

To obtain the files, you must open Goldleaf from the Homebrew menu, select “Console and Goldleaf settings,” select “Firmware and updates,” pick “Export update,” confirm on the “Directory” option, and the files will be dumped in the switch\Goldleaf\dump\update location. After that, just copy the files to %AppData%\yuzu\nand\system\Contents\registered.

Ports, ports everywhere! (Mario Kart 8 Deluxe & New Super Mario Bros. U Deluxe)

This is tradition by now. Morph fixed another bug in Kirby Star Allies, this time by implementing MirrorOnceClampOGL. This is done by using the GL_EXT_texture_mirror_clamp extension on the OpenGL drivers that support it. The Intel driver doesn’t, so, with those GPUs, yuzu falls back to GL_MIRROR_CLAMP_TO_EDGE.

Improvements to the texture cache by implementing additional checks, done by Rodrigo, fixed crashing issues on Turing GPUs (RTX 2000 and GTX 1600 series) in The Legend of Zelda: Breath of the Wild. Now you can load saves normally! So get moving and save the princess!

Our GLASM users experienced black textures on top of the vegetation in The Legend of Zelda: Link’s Awakening. By using the extensions NV_shader_buffer_load and NV_shader_buffer_store, Rodrigo solved this issue for good.

Beware of the dark weeds, legend says rupoors lay there (The Legend of Zelda: Link's Awakening)

We talked about this in the previous June progress report, and we’re happy to announce that Rodrigo finished implementing support for VK_EXT_extended_dynamic_state. This fixes the “triangle glitch” that affects Super Mario Odyssey on all Vulkan drivers that support this new extension.

Thanks to help from gdkchan and Blinkhawk, Rodrigo added support for copying 2D arrays of pixels into 3D voxels, fixing rendering bugs in games like LEGO Marvel Super Heroes 2 and possibly others.

toastUnlimited, along with Rodrigo, started preliminary work to fix crashes only happening in recent Nvidia Linux Vulkan drivers. Through trial and error, it was found that increasing the Vulkan device allocable size mitigates this issue on most GPUs, but some still remain affected for now.

epicboy is working on implementing support for GameCube adapters, allowing players to use original GameCube controllers! Veteran Smash players will surely enjoy it.

Project Prometheus introduced so many changes that we are still working on reaching feature parity with the original single-core implementation. ogniK improved the audio timing, fixing audio bugs that were introduced with the implementation of the new multicore and single-core emulation methods.

Since its introduction, enabling multicore was the only way to reach stable gameplay on Linux systems, as the single-core option would lead to a crash in every game. Thanks to comex’s report, Lioncache fixed it by using the return value of Lock() in the nvflinger surface compositor. Thanks to this fix, games that are not yet stable with multicore, like Luigi’s Mansion 3, can now be safely played in the Tux OS (Linux).

Project Kobra

Six months ago, Morph proposed an implementation of a batch installer to yuzu, which allowed users to install many updates and DLCs at the same time. This was a boon for many users, but it remained in Early Access for a long time due to several issues and anticipation of a new VFS implementation dubbed Project Viper. With renewed hope and motivation from the shark (ogniK), Morph went on to fix all of its remaining issues, allowing it to finally be merged into the master/Mainline branch of yuzu.

 I can finally install all the DLCs of Super Smash Bros. Ultimate!

I can finally install all the DLCs of Super Smash Bros. Ultimate!

Additionally, as Project Viper has been delayed until further notice, Morph took it upon himself to fix the list of Virtual File System (VFS) bugs that had been plaguing yuzu.

It was well known that yuzu had problems with installing Updates and DLC into the NAND and reading them correctly. Due to the inability to easily remove individual game files, users had to delete all their installed game files just to ensure that their new Update or DLC got detected properly.

 Not again, my precious write cycles! *sobs*

Not again, my precious write cycles! *sobs*

Upon closer inspection of the inner workings of the VFS, Morph found that the previous Updates or DLC were not removed when a newer one was being installed, leading to issues with the detection of these files in yuzu. He solved this by properly removing the previous update or DLC if it exists on installation. In addition to this, he also corrected a small oversight in the way yuzu stores autogenerated titlekeys. Now invalid titlekeys will no longer be written into the title.keys_autogenerated file, ensuring that only valid ones are written. If your game, update or DLC is not being detected by yuzu, ensure you have the latest key files dumped from your switch, as newer game updates and DLC require newer firmware keys to decrypt. With these fixes in place, yuz-ers no longer have to delete all installed game files to install a newer update or DLC as this properly overwrites the previously installed ones.

Next, he looked into fixing one of the oldest and well known VFS bugs - the Luigi's Mansion 3 autosave issue. Every time an autosave was triggered, the game would softlock. This had also affected other games such as KATANA KAMI: A Way of the Samurai Story and Okami HD. Building upon the initial research by theboy181 (the creator of the mod that bypassed the Luigi’s Mansion saving issue), Morph dug into the VFS code and found that files were not closed prior to being renamed or moved, leading to a simple fix for any games that use this function.

Keeping the ball rolling, he went on to find out why Diablo III: Eternal Collection was not able to get past the EULA screen and thus needing a save file to bypass it. It turned out that an old PR was causing folders instead of files to be created. After this discovery, the PR was immediately removed from Mainline/Early Access and closed as it was no longer needed.

 EULAs can't stop me! (Diablo III)

EULAs can't stop me! (Diablo III)

Looking for the next meal, Morph took a bite at the saving issues of the Bioshock Trilogy. These games were unable to save due to checking for free space prior to saving. This was fixed by ensuring that the NAND partition sizes were set to their defaults and are read correctly by the relevant filesystem service commands.

Game saves are for the weak! (Bioshock Remastered)

After this little treat, the Kirby embarked on an adventure to fix the last remaining issues with the VFS. After 3 days of debugging the Pokemon Sword and Shield RomFS weather bug, he discovered that these games contained a 0 byte file called global_setting.bin which caused the file right after it called weather_data.bin to be skipped entirely. And you may have guessed by now - this file contains all the data for weather and without it, no weather can be generated within the game! This eureka moment led to an extremely simple fix which accommodates for 0 byte files during RomFS building.

Let it snow, let it snow, let it snow! (Pokemon Sword and Shield)

Morph’s adventures have led him to save Zelda Link from the eternal softlocks after awakening from his slumber, seems like Ganon may have known Zelda’s plan after all. Our Kirby traveled to the shattered Kingdom of Hyrule and foiled Ganon’s plans by creating the save subdirectories that would allow Link to save his progress. Finally, he can save the princess without wasting his time eternally stuck after his slumber.

Hearing stories of Nook Inc’s deserted islands, Kirby flew off to visit some of them, only to be met by a barrier preventing access to the other realms. He managed to bypass the barrier by stubbing two filesystem service commands which were blocking access to the islands. Now he can view all the latest treats added in the islands since the last update.

Wrapping up his adventures, he went and brought peace to Dreamland once again by adding an option to remove individual NAND or SDMC installed games, updates, and DLCs and fixing the last remaining issues with the game list, such as opening the Save/Mod Data Location of installed titles, game list subdirectory scanning, and game list metadata detection.

 Removal of installed game files has never been easier!

Removal of installed game files has never been easier!

The old guard

When yuzu started using Conan, most of yuzu’s dependencies were migrated to it. Unfortunately, our audio decoding dependency, Opus, had bad settings in the Conan version. These settings forced the use of the AVX/AVX2 instruction set even on old CPUs that don’t support it, such as AMD Phenom CPUs or any Intel Pentium, Celeron, or Atom CPU. This caused some games to crash when processing audio on the affected CPUs. ogniK fixed this by replacing our included Opus version.

Merry finished the preliminary work needed to fix a performance regression affecting CPUs that lack the FMA instruction set. This includes AMD Phenom series or older, Intel 3rd generation Core-i series or older, and all Pentium, Celeron and Atom CPUs. For now this shows up as a debug CPU configuration in yuzu’s settings. We recommend that our users not play with these settings, as altering them will only produce a worse experience than the default settings.

UI changes

A very requested feature from our users. toastUnlimited rose to the challenge and decided to add per-game configurations to yuzu! For now it lacks support for input settings, but this will be added in the future. To access this new feature, right click a game in yuzu’s game list and select Properties.

 No need to remember specific settings before booting a game!

No need to remember specific settings before booting a game!

Morph added a new theme for our anti-light themes users, Midnight Blue! You can access it in Configuration > General > UI.

 Oooh, Spooky

Oooh, Spooky

Fixing small annoyances is something that is overlooked most of the time, but very appreciated when done. Tobi removed the need to confirm which Switch profile to use when there is only a single one created. Thank you so much Tobi.

 Bye little fella!

Bye little fella!

The Sharkman, episode one

ogniK has been working for a long time on a major audio rewrite for yuzu. To avoid a very long explanation of the complexity of Nintendo’s audio renderer systems, we recommend our more tech savvy readers to read the Pull Request’s description where you will find more detailed information.

This is “Part 1,” several fixes and more complex (or annoying to code) features will be finished at a later date. Right now the results speak for themselves. Tons of games with broken audio of any kind - like missing sound effects, fading music, mixing errors, crackling or even no sound at all - are either running perfectly or are starting to sound a lot more like the native hardware. Some bugs are to be expected for now, for example, Shovel Knight and Fast RMX experience choppy audio, and the voices in Animal Crossing: New Horizons will play from only one ear.

You would think that this is enough, but ogniK has another fantastic gift for us. Up until now, when several new shaders had to be built during gameplay, they could only be done one at a time. Thanks to some good help provided by Exzap, Cemu’s lead developer, yuzu can now decode shaders asynchronously! This means that, depending on the amount of threads your CPU has, the shader compilation will happen on several threads at the same time. Thus cutting down shader build time significantly, especially if you can also use assembly shaders.

To top it off, epicboy added async shader support for Vulkan!

Thanks to BSoD Gaming for the video comparison (Super Smash Bros. Ultimate)

Due to limitations in the AMD Windows OpenGL driver, we recommend that you avoid using asynchronous shaders in OpenGL if you have a Radeon GPU, as it will only produce frame skipping and lead to a very poor gameplay experience. The Linux OpenGL drivers are unaffected by this, as well as any of the Vulkan drivers.

The number of threads used for asynchronous shader compilation (along with example CPUs to illustrate the number of CPU threads) are as follows:

CPU Threads Shader Threads Example CPUs
1 - 7 1 R3-1200, i3-6100
8 - 11 2 R3-3300X, i7-7700K
12 - 15 3 R5-3600, i7-8700K
16+ 4 R7-3700X, i7-10700K

Future projects

Rodrigo started rewriting the Texture Cache and once the work is finished, this should positively affect both OpenGL and Vulkan by introducing bug fixes, better performance, and a better experience in Vulkan in particular.

There are also some other yet undisclosed but very impressive projects that we will reveal soon enough. Stay tuned!

That’s all for now, folks! Until the August progress report!


Please consider supporting us on Patreon!
If you would like to contribute to this project, check out our GitHub!