EPISODE 1735

[INTRODUCTION]

[00:00:01] ANNOUNCER: Darío and Wiseguy are Nintendo 64 hackers and modders. They are the creators of multiple projects, including RT64, which is an N64 graphics renderer for enhancing games, and emulators, and native ports. This year, they released N64 Recompiled, which is a tool to statically recompile N64 binaries into C code that can be compiled for any platform. The advance promises to usher in a revolution in N64 native ports for PC. To demonstrate the tools capability, it was used to produce a working copy of The Legend of Zelda: Majora's Mask on PC. Darío and Wiseguy joined the podcast to talk about their journeys in the N64 hacking scene, the N64 Recompiled project, obscure N64 game bugs, and more. 

Joe Nash is a developer, educator, and award-winning community builder who has worked at companies including GitHub, Twilio, Unity, and PayPal. Joe got a start in software development by creating mods and running servers for Garry's Mod. And game development remains his favorite way to experience and explore new technologies and concepts.

[INTERVIEW]

[00:01:18] JN: Welcome. Thanks so much for joining me on the show today. 

[00:01:20] W: Thanks for having me.

[00:01:21] D: Hello. Thanks for having me. 

[00:01:23] JN: Let's dive into - I guess your background is where I want to start. I'm always fascinated when we get into the N64 modding topic or N64 hacking topic about how people got into this. I guess to start with Darío. What was your journey in the N64 scene? 

[00:01:37] D: My start is actually very recent, because while I do work as a software developer in general and used to mod other games, I actually only started looking at N64 stuff when the Super Mario 64 PC port released. Basically, I only got interested in that because a friend challenged me with the idea of adding ray-tracing to that. And I didn't know how to do ray-tracing through hardware or anything about N64 or anything about the compilations. I'm like, "Sure. Let's try to learn all of those things at once." 

[00:02:10] JN: What was the background to that challenge? How does a friend just go, "You don't really seem like you know about this? You should do it." 

[00:02:16] D: Well, I do have an interest in graphics programming got. But I wasn't super up-to-date on that stuff. Because, usually what your clients usually ask of you, it's not usually the latest up-to-date stuff. We're talking 2020. Ray-tracing was still really new. People complained that the mod was exclusive to that because not many people had ray-tracing GPUs yet. Now that's no longer the case. But it was very noble technology at the time. And now it's starting to be more integrated into games luckily. 

But, yeah, we also like the idea because we saw stuff like Quake 2 RTX, which was like a really good showcase of, "Oh, let's take this old game and just do path tracing and path tracing it entirely and make all of these fancy lights. And, supposedly, that's because the game is super lightweight. It sounds like it's easier and cheaper to do that. It's actually very much not the case. It's probably just as expensive as rending any game to be honest. But, yeah, that's how the idea came around.

[00:03:19] JN: Awesome. Yeah. That makes sense. That was your kind of trial by fire first entry into the N64 landscape. And I assume that that worked and that happened.

[00:03:27] D: Yeah. It taught me the very, very basics. Because as it turns out, and as Wiseguy maybe quickly find out, Super Mario 64 is a really basic N64 game. I had to learn a lot, thanks to him, to actually emulate other games correctly. But it did give me a quick intro of how some stuff work at least. And much of the framework of that render is still around to this day actually. 

[00:03:56] JN: That's awesome. And it's great to hear, Wiseguy, that you played a role in this intro. How did you get your start? 

[00:04:01] W: Well, around the time that COVID hit, I was given, let's just call it, an extended period of time off from work. And I used a lot of that time to investigate the Mario 64 decompilation, which had come out a few months ago at the time. And it was something I was interested in looking at. I started making mods, ROM hacks that ran in an emulator or on console at the time. Nothing PC port-related. 

And from there, I got really interested in it. Because as it turns out, programming for the Nintendo 64 is very similar to embedded development in a lot of ways. Embedded firmware, software, which is something I've done professionally for a while. And software in general. I've also done desktop software. 

And so, you're talking a lot of statically allocated memory, linear allocators. Really bare-bones direct on the metal programming. Because of that, it came pretty easily compared to what I was expecting. And I got really into it. I made a lot of interesting technical showcases. I released a mod or two that were actually finished. 

And then I started getting into the decompilation scene as a whole. There were a lot of projects spinning up around that time. Because Mario 64 had been a pretty big inspiration to a lot of people to start other similar projects. And so, I started one of an old game that I'm sure will come up later called Rocket: Robot on Wheels. I worked on that one in my spare time a little bit. But I also just joined on a lot of other projects to just help in areas where I could. 

And from there, I ended up getting interested in homebrew as well. There was a homebrew competition hosted by a group called N64 Brew. And I took part in that. Had to write basically from scratch an N64 game. Between all of those different experiences, it really gave me a good overview of the console as a whole. And that ended up leading into me getting involved with helping out in RT64. Mainly in the accuracy department of it. Making sure that the graphics were rendered correctly, the baseline graphics before you start talking about ray-tracing, and text replacement, and everything. 

And then from there, while helping out with that part of RT64, I had seen other similar projects in the past that did Static recompilation. And the idea came to me of, "Well, how easy would it be to hook something like that up to RT64 and use that for graphics?" And then it turned out actually pretty easy depending on the game. Starting with a simple game, the one that I mentioned earlier, Rocket Robot. A super simple game. And then from there I realized, "Oh, I could get a lot of other games working with this recompiler by slowly adding features onto it." And then that led me to Majora's Mask, which is a game that's been notoriously hard to emulate for a very long time. Nowadays, you can do it. Because emulators have gotten a lot better. But I think it was a nice showcase of what the tool could really do to start with a game as "hard to emulate" as that one. 

[00:07:30] JN: That's awesome. Yeah. Oh, God. Okay. Where to go from here? You've dug into the compilation a lot and the origin of the N64 Recompile project, which I think is where I'd love to start. And part of the reason I'd love to start there is I think I recall first - I don't know if it was the actual launch or just when I first saw it. But I'm pretty sure that project launched near enough when we recorded our episode with Ethan and Mark for the decompilation. I was like had just learned about this crazy Nintendo 64 decompilation effort and all the things all of y'all have been doing for all these years. And then this project came out and I was just straight away like, "Oh, wow. That looks like an incredible approach to this problem. That's really interesting." Can you start kind of telling us at a high level what is N64 recompiled and I guess how does it work? 

[00:08:10] W: Okay. I guess the place to start would be the main how it works overview. Basically, in the process of decompilation, which we'll tie into this a little bit, one of the first things you need to do when you're setting up a decompilation is tear the ROM apart. You got to figure out where all the code is in the ROM. Where all the data and assets are. In the process of finding where those are, you can actually extract the code into assembly language and then reassemble that assembly language back into the ROM. 

And so, N64 Recompiled sort of injects itself right after that step, where you've figured out where stuff is in the ROM. You haven't necessarily started matching code. Or maybe you have. You don't have to have started matching the code into C. But you know where all the code in the ROM is and you have it extracted into assembly and then generating back the original ROM. 

N64 Recompiled takes that data, the layout data of the ROM, and it goes through instruction by instruction in every function. And it converts those on a one-by-one basis to some equivalent C. Now the big difference is that it doesn't try to match the original binary output when run through a compiler. That's one of the main differences between a matching decompilation like what Ethan and Mark talked about and what N64 Recomp does. Instead, it's matching the behavior one-to-one. 

And so, because of that, you're not limited to only running on the original hardware out of the box. Because you can match that behavior on other platforms. You can generate C code that is not dependent on specific hardware but has that same behavior. From there, there are a couple parts of the process that need to be a little manual. For example, games that have multiple code sections, they're called overlays, that can be loaded or unloaded at runtime. You have to identify when the game loads, those overlays, and then add a little bit of extra code to trigger the load in the recompiler output as well. But for the most part, you have one function in the original game. It turns into one function in the recompiler's output. 

[00:10:27] JN: Okay. Awesome. That makes a lot of sense. And, yeah, I really appreciate the note about the difference between matching compilation, and what you're doing here, and how that influences platforms it runs on. Where this tool fits in in the overall process of creating something like a port. Because I think that's a lot of what's been - you mentioned it there, it's one of the strengths of this approach. And, also, one of a lot of things I've seen discussed about N64 recomp so far. Once you've done that initial process, your output then is a bunch of recompiled C code, right? What then does happen to make that into something approaching like your Majora's Mask project? 

[00:11:04] W: Yeah. After you've got that C code that's output by the recompiler, you have to then fill in the blanks, we'll call it. The gaps where the recompiler is not able to or I chose not to try to recompile the original binary. And , o that's mostly areas like the system library, the SDK that they would originally use to develop and ship these games. 

A lot of those touch harder registers directly. Memory-mapped registers that control harder behavior at the lowest level you can get when you're dealing with a console like this. And recompiling those is doable. In fact, emulators do it in the process of dynamic recompilation, which I can talk a little bit more about the difference between static and dynamic later on. But it adds a lot of cost to do it that way. And it also sort of defeats the point of the project if you were to recompile even the system library. Because then, a lot of the changes that get made to the game happen in the course of upgrading or modifying the behavior of the system library. 

And so, all the system library functions that are not handled by the recompiler need to be implemented manually in C, or C++, or whatever you choose. And then those get linked up to the code that the recompiler generates. And so, I wrote basically a drop in place replacement for the system library that is meant to run on modern systems. I called it the Modern Runtime. N64 Modern Runtime. 

And it's mostly agnostic. You can use it with more than just the specific game it was made to run with. And that's why other games are doable with this with fairly little modifications if any needed to that. And then from there, you also want the new things that are added onto a port beyond just getting it running. 

For example, the menus that were added to the Zelda recompiled project for configuring controller mappings or all the different setting, graphic settings that it has that hook up to RT64, those are implemented in another layer that basically just adds the UI on top of the game. And it adds the launcher menu that lets you pick the ROM and lets you set those settings and all of that.

[00:13:23] JN: Awesome. Cool. Okay. Going to go sideways a little bit. Because I want to cover the full landscape. And you just mentioned RT64. RT64, as you mentioned, is - you were doing a lot of work initially on the accuracy of that. Comes in the Majora's Mask project in a really interesting way that I want to touch on later. But I guess, Darío, to kick us off, can you talk us through a little bit about what RT64 is? 

[00:13:43] D: Yeah. Sure. Well, originally, when it started as the renderer for the Super Mario 64 port, that was only for ray-tracing. That was the only mod basically. Later, the idea came to me to try to - because there were a lot of things I was doing on top of an interface that the PC port exposed, which was pretty inefficient. Because it did a lot of stuff that I wasn't very happy about for performance reasons. For example, every vertex you saw on the screen was transformed by the CPU, which got really bad. Because if you remember that PC port, people loved adding those high polygon models. And it got so bad to the point that I had to multi-thread that workload because it was taking up to like 20 or 30 milliseconds just to process all those vertices on each frame. 

But a lot of that got carried over to a plugin. Because I felt it was possible to do that process on an emulator instead of just a PC port. Because at the end of the day, the PC ports use a very similar process to translate the graphics calls. I just decided to try that. And I got a lot of errors, obviously. The first time I saw anything on the screen, it was really corrupted. Because, obviously, the PC port had different endianness, for example. The bite order was completely different. Everything was really corrupted. But I worked through that with Wiseguy. He taught me a lot about how the internals work. 

And on the process, we ended up not only having the ray-tracing mode there but the regular rendering mode, the rasterization mode, I call it, which is just the regular graphics. And there was a kind of personal interest from my part as well. Because I used to be a kid in the 2000, which was like the era where you had all those N64 emulators which was super cool that we had all the stuff. And I'm really impressed that we had stuff that early that actually worked. 

But I remember all those bugs that like every time you try a game it will be like, "No. This emulator - this glitches out one way. Or another emulator glitches out this other way." And this other emulator, it gives you a seizure. It's like what's the exact reason behind the glitches in all these games? 

We actually went and really discovered a lot of why all these emulators have problems. And like Wiseguy said, their situation is much better nowadays. But even when dealing with high-level emulation, it's still a bit difficult to get everything right. We thought it was very important to try to do it right. You may have seen that there's other renderers like Parallel RDP. You might have seen it, which is an excellent plugin. But it's actually so low-level that emulates the behavior of the [inaudible 00:16:28] rasterizer through a compute shader. It doesn't even use the native functionality your GPU has to render the triangles. It straight up emulates a rasterizer through a compute shader pixel-by-pixel, which is great for accuracy but not so much for performance. 

[00:16:46] JN: Yeah. Absolutely that kind of lands me, I guess, in a topic that I feel comes up a lot or has come up a lot in our episodes about this kind of scene, which is the extent to which - I guess the philosophy and the goal and the extent to which what you're doing is capturing the original historical nature of the game, and its functioning, and the console, and its interaction with the hardware versus like trying to make a playable artifact. I guess for both projects, that is a factor. And I think - well, I want to start with the renderer. But I'll come back to you in a minute why it's got out. Where the static compilation fits in amongst emulators and matching decomp as well?

When you're looking at projects like that that are doing the slow level emulation, like down to mimicking how the hardware works and thinking about your approach, how do you land on that historical accuracy through the playability spectrum? Does that make sense, Darío? 

[00:17:34] D: Yeah. No. It makes complete sense. You have to make some sacrifice at some point. We are talking about our project doesn't try to reach that level of accuracy because we actually want it to render fast. It would be great if I could. But we realized there's like a few tradeoffs we would have to make. We take a lot of careful decisions to never stray too far from the accuracy. It's actually like really accurate. But maybe the very last few details are things that most people don't actually care about. 

We are talking, for example, how we're dithering. For example, do people want 16-bit color on their screen? Or do they want 32-bit color? Our goal is if something like parallel already fulfills the role of that one-to-one accuracy, then there's room for something more oriented around enhancements. And, obviously, we're talking about a plugin that not only features that but something crazy like ray-tracing the entire thing. We're already way past the point of keeping it one-to-one on that level. 

But, also, the other goal we had, which is very important, is the frame rate interpolation. And we're talking very, very high frame rates. When you're targeting stuff like 360 hertz, you can't really afford to go down the route of not using what the hardware has. 

[00:18:57] JN: That makes sense. Going back to the accuracy. The accuracy, I found really interesting. I think this is in the read me for the Zelda Recon project that you mentioned the emphasis on replicating the original N64 effects and how that had been, I guess, a challenge in other projects previously. Could you talk briefly about that and what that takes? Whichever one of you is good to start on that. 

[00:19:17] W: There's a lot of - especially among older emulators. Not so much with the newer ones that we've talked about about how the scene has gotten better. But especially with those older ones, there are a lot of emulators that have hard-coded behavior targeting specific effects in specific games. Maybe it was not known at the time how to emulate this correctly. Or the hardware wasn't up to the task of emulating that effect correctly. But basically, they will rewrite the effect effectively in the emulator itself. 

And so, I guess one of the most concrete examples I can give of that in Majora's Mask would be the - actually, in Ocarina of Time especially. But both games have it. The pause screen background. The way the pause screen works is it captures the frame buffer the moment you pause. It renders one extra frame. Captures the frame buffer for that frame and then displays it in the background of the image as a texture. And then in front of that, it renders the actual pause screen. And that's why when you pause, the background stays still and has an image of what was there before the pause menu came up. 

In a lot of emulators, you'll see frame buffer effects like that are not emulated directly. Instead, there's special handling that says, "Oh, I recognize this game that's currently running is Ocarina of Time. I know what it's doing right now. It's trying to capture the background so that it can display to texture. I'm just going to -" because that is not possible to emulate with the graphics API that I'm using or whatever, this is how it would have been done in the past. They would just manually capture the image into a frame buffer in the native API. Direct3D 9, for example, would have been what they might have been using at the time. 

And then later on when the game goes to render that frame buffer texture, it says, "Oh, I recognize you're rendering the texture that you captured earlier. Let me just bypass that and render the native texture instead." And one of the goals that we had is to never do that. There's not a single line of code that detects a specific game that's running in RT64 or the recompiler anywhere. Because we wanted to make sure it's completely agnostic so that every game can be run through it eventually.

[00:21:32] JN: Awesome. Cool. That makes a lot of sense. Okay. That was a stated goal. And that's some of the situations you're running into. How do you go about fulfilling that goal, Darío? 

[00:21:40] D: Sorry. Which goal? 

[00:21:42] JN: Meeting that accuracy. Making sure that you never had to check for a specific version of the game and you're able to implement these effects. 

[00:21:49] D: Well, mostly the entire process is understanding what the game is doing. Because you don't necessarily have the code of the game. Some of these games are already decompiled and you can read what they do. And that helps a lot. But sometimes you encounter these games that nobody has decompiled yet. So you don't have much of a choice. But to try to understand what it's doing. 

One of the most important parts we did on this process was actually making a graphics debugger, which shows every single command that we detected that it does every operation, every change it does, and trying to understand through that what the game is trying to do. If we found any errors on that process. Because, usually if there's an error, the only way we spot it is visually. That's usually the case there. 

That involves obviously trying to find footage of the game, or trying to run it on console, or trying to understand some of the code that's coming in. And then once we understand it, we delve into how the hardware works and we refer to as much documentation as we have as possible. And, obviously, we also have these excellent plugins like Parallel RDP that I mentioned. Sometimes the problem might be on the very lowest level of the emulation we have with the rasterizer. Or on how we emulate a certain micro-code. But that's usually how it goes. And we had a lot of success with that. 

And every time it turned out, what we were weren't emulating properly was some general hardware behavior. We try to understand as much as possible what general behavior we have to follow to replicate that correctly. Not trying to just fix the problem for that one game. And obviously making sure it didn't break anything else. But that process obviously landed us that, in some games, we had to understand some really complex stuff. Like what I was mentioning, something like the pause frame buffer. I'm not sure if he wants to go into other effects of Majora's Mask that are way more complex than that. Because, honestly, the pause screen is one of the simplest things you can emulate. And there's other effects in that game that, even to this day, there are certain emulators that still have hacks to work around those effects. Because it goes really deep into the features, the hardware uses to achieve that effect.

[00:24:09] W: Yeah. The Lens of Truth is probably the most complicated effect in the game. It's not at all something you can replicate directly by just using modern graphics APIs as intended. You have to very much come up with certain paths to emulate this hardware behavior that just doesn't have a direct analog in modern hardware.

[00:24:28] JN: Awesome. Okay. We've spoken about emulators a lot. I want to take kind of a step back towards recomp for a minute, which is I guess to ask kind of a fundamental question. You compared this approach to matching decomp. Why N64 recompilation versus using an emulator? Why have these native binaries in the first place? What do we gain from doing this? 

[00:24:46] W: Yeah. This is a question that comes up a lot. This is a great chance for me to finally give some concrete answers here.

[00:24:54] JN: Excellent. Happy to provide.

[00:24:56] JN: I mentioned earlier static recompilation, which is what N64 Recompile does. And dynamic recompilation, which is what emulators do. The main difference between the two is that static recompilation is done ahead of time. And dynamic recompilation is done when the game code is being run in the emulator. And the most surface level benefit to doing it ahead of time is performance. 

When you have the entire code of the game, how it's all laid out ahead of time, how it's all split up into individual functions, and sections, and all of that, you can create much more optimized running code than dynamic recompilation, which basically just whatever couple instructions are being run at this given point, that's what you're recompiling into static, into a native code. 

And so, in the case of N64 Recompiled, I actually get the full suite of optimizations that a modern optimizing compiler like Clang would do because I'm generating C code from it. And so, it ends up being very optimized and running very fast. It was such a small amount, I couldn't even profile it. But it takes less than a fraction of a millisecond to run a game frame worth of logic in the recompiler or rather the recompiled output of it. Versus with an emulator, especially if your emulator is getting to very low-level accuracy like dealing with how the original CPU cache worked, it can be to the point where simply just emulating the CPU can get into being the bottleneck for running the game at full speed. 

That's probably the easiest one to sort of digest is that performance difference. But there actually a lot of other reasons why as well. The next biggest one I would say is enhancements. With the recompiler, one of the things you can do is just replace an entire piece of code with a new piece of code during the recompilation process or during the process of linking the port together. 

With an emulator, if I wanted to take, say, Mario 64 and add new code that lets me jump a second time in the air, for example, I'd have to go in and I'd have to make room for that new code somewhere. And I have to find where the original code that handles what you do when you're in the air is. And I'd have to basically inject some sort of place in that function to hook and call the new code that I've added. And then when the new code that I've added is done running, it has to then jump back to the original function. 

With matching decompilation, you can just edit the C function. But not every game has a matching decompilation. And so, that process is a lot more difficult when you're not able to just modify C code from a game and put it back into the game. With static recompilation though, one of the things I can just do is I can take the assembly from that function, or the C code if there's a decompilation of it, and I can just add new lines of code into it. And I pass that through the recompiler and replaced the original function with it. 

Because I don't have to worry about shifting the contents of the original game around because it doesn't actually end up in the original ROM. It sort of is on the side. You can think of it like it's to the side. And so, it becomes very easy to just add new code or remove code from the game much simpler than it would be to do that by editing the original ROM. And that doesn't even get into the world of having to deal with compressed code, for example. 

Some games on the system actually store their code compressed in the ROM. You'd have to decompress that, edit it, recompress it. And make sure in that whole process of doing that, that you're not moving some other data around in the ROM. That the game is expecting to be in a certain place. The enhancement potential is actually really big with recompilation. 

[00:28:51] JN: Right. That makes a l sense. I guess on the topic of enhancement. I guess two things I want to go in here. You mentioned this part of the process that requires you to identify the functions and label some stuff. There's always going to be some element of decompilation process that has to have happened or like some understanding of the ROM assembly. Is that correct? 

[00:29:13] W: Yeah. That's pretty much the case. There are tools. The tools that I'm sure were mentioned in the episode of Ethan and Mark. But Splat. Yeah. Splat and the underlying technology it goes into Splat. There's another library that it uses called spimdismasm, which is MIPS backwards is SPIM. They're able to do a lot of the heavy lifting for you. 

If you say I know that there's code in this region of the ROM. Actually, those tools can go in and tell you, "Okay. I've taken this region of code and split it up into the individual functions." A lot of it is automatic to some extent. You need to know how to find where there is code in the ROM. And there are a couple different ways to do that. You can do that by just looking in the ROM and knowing what to look for. You can do that by watching how the game runs in an emulator and seeing how new code is loaded in and unloaded at runtime. But, yeah, there is at least for now a manual aspect to setting up the recompilations. It's not just a plug-and-play solution. But it's a lot less work than decompiling a game in full definitely. And it's a lot of overlap work in setting up a decompilation.

[00:30:25] JN: Yeah. And that overlap is really cool. I like the Journey you discovered where you can learn from a decompiled project and then the recompilation is not necessarily an alternate way of getting a port of that project, but like a next step for a decompiler to then take in their workflow of actually using the knowledge they've gained through decompilation. I think that's really cool. That's a really nice overlap. I guess one question that is I think I already know the answer to this. But what is the state of the output code? It's not human-readable, right? It couldn't possibly be human-readable. 

[00:30:56] W: No. Not at all. Now the good part is you never have to look at it, right? Even if you're making enhancements, you're not editing the output code. You're recompiling your enhanced code and then you're replacing one recompiler output gibberish with other recompiler output gibberish. Unless you're debugging it, which is usually just what I'm doing, because there's a problem in the recompiler, for example. If you're debugging the output code, that's the only case you're ever really going to be looking at the output. Otherwise, you don't have to care about it.

[00:31:26] JN: Cool. Okay. And this may be a very silly just how do compilers question. But you mentioned earlier that you're looking at the - with matching decompilation, how the matching decompilation is very clear in my head, we've got this piece of assembly. The historical assembly, we know what it is. We know what that call is. We're looking for that same assembly to be output. You mentioned that you're looking for function and not necessarily the exact same output. But you're working with code you don't know what the function necessarily is. It's weird versions of compilers as well. What is the process there? How is the compiler actually compiling, I guess, is my question? 

[00:31:59] W: Yeah. Probably, I think the best option here is to go with another example. An example of a simple function might be it's a function that takes in a pointer to a struct and then it sets some member on the struct. It sets it to one or whatever. That code might compile down into taking in an argument, that is the pointer, in some register. And then it will offset from that register to get to the member of the struct it's changing. And then it'll do a store to that offset address. And then it will return because it's done. 

The recompiler translates that very literally. While matching decompilation, you would go, "Okay, this is clearly writing to a struct. Let me try to figure out how the struct is laid out." And put together back the information of all the different members in the struct and say, "Okay. Well, this is a member. It's eight bytes from the start of it and it's four bytes wide. It's probably an integer." 

The recompiler goes, "I don't care that this was a struct in the slightest." There's a piece of memory. It's offsetting from that memory and then it's writing to memory. All the recompiler does is it goes, "Okay, I see an instruction that adds to this pointer." Or, actually, doesn't even need to know that's a pointer. Because pointers are just integers. The recompiler goes, "I see one instruction that adds some integer to another integer. I'm going to output a piece of C code that adds two integers together." And those integers may happen to be pointers originally. Or maybe they were something else entirely. It doesn't really matter. 

And so, the recompiler, it's a blessing and a curse. Because you don't need to know what the point of the original code was to do static recompilation. But at the same time, you don't get a whole lot of info about what it's doing. And so, that's why it's another area where decompilation and static recompilation go hand-in-hand is that, "Okay, I can get the 95% of the game that I don't really care about the details of to run without ever touching those functions." But then that last 5%, the stuff that I want to change, where the areas where I want to add enhancements or I want to add new stuff into the game, I can just focus on those for decompilation. I can say, "These are the functions I care about changing. And therefore, I'm going to decompile these. Edit them. Recompile the output of that changed decompiled code." And then, again, the recompiler doesn't need to know what it's doing. That's I guess the main difference is the high-level information is never recovered in static recompilation because it doesn't have to be.

[00:34:36] JN: Right. Nice. That makes total sense. Kind of a later topic, this definitely came out of the conversation with Ethan and Mark was kind of my biggest takeaway from that was just like the chaos of all of the different versions of compilers that these ROMs were compiled with over the course of the console's lifetime. To what extent does that affect this project? Are there certain MIPS, flavors, or whatever that this project just can't touch? Is that an issue at all? 

[00:35:01] W: MIPS is a family of CP architectures. To answer your question immediately, yes. There are other MIPS architectures that are not supported by the project. But all N64 games use the same architecture obviously because it's the same hardware that's running on. And they all use the same ABI, application binary interface, which decides exactly how functions interact with each other basically. 

It says, "Okay, arguments get passed in through these registers. The return address is usually stored in this register and so on." The ABI actually doesn't matter a whole lot to the recompilation project. The instruction architecture is the main thing that matters. And so, any N64 game is going to be using the same architecture should work. 

Now there are nuances with different compilers. Absolutely. In that they can do the same thing in different ways. The project actually also has to work with mod recompilers too. Because when you're writing that modified versions of the original code and then injecting those back in basically, you're not going to compile your code that you wrote in 2024 with a compiler from 1994. You're going to compile it with modern claim or modern GCC. And so, there are definitely behaviors that needed to take special care when translating them over so that they work with all different flavors of MIPS compiler. 

Now with that said, most of the instructions are so simple because it's a risk architecture that it doesn't matter a whole lot for most of them. But then when it comes to - it's mostly just when it comes to things like branches. Because MIPS is a little tricky when it comes to branches. But for the most part, yeah, the compiler itself doesn't matter a whole lot. But it only Targets this one specific flavor of MIPS that the N64 uses. 

[00:36:52] JN: Perfect. I guess going back slightly, I think this probably takes us slightly back towards rendering RT64. One of the I guess things I saw a lot of excitement about with recompilation was I guess the ever-present topic with N64 emulation or preserving these games anyway, which is being able to play in widescreen and being able to play outside 4:3 and wider higher frame rates, which I know something that RT64 supports. But why has that been such a difficult thing? I imagine there are myriad answers to that question. But, Darío, is that something you're able to speak on? 

[00:37:28] D: Well, the widescreen in particular has been an option in a lot of plugins actually. It does require some hacks usually. Some of them actually didn't support arbitrary aspect ratios. Sometimes older emulators just were like, "All right. This is 16x9. And that's it." But I think the key thing here is that it's not super different from what other emulators do for the widescreen stuff. But when you actually launch a project like recomp that already accounts for all the stuff that the widescreen needs to work properly. And it actually uses some of the functionality that we expose for doing some stuff the console normally wouldn't allow. You saw the hub, the 2D elements go actually outside of the 4x3 bounds. That's because we actually expose a custom command set that allows that. Because, normally, you wouldn't be able to go outside of the screen mount with those coordinates. 

But we do expose that. And, obviously, that command set is fake. It wouldn't run on real hardware. But we're not running on real hardware. We're running a PC port. Recomp already knows that and sends these new commands that the renderer recognizes. And it's able to patch a lot of the things that people care about usually. And it also includes all these custom patches, for example, for stuff being called. For performance reasons, obviously, they don't draw actors that are outside of the 4x3 bound. And recomp goes like, "Well, we are running on much stronger hardware. So let's just get rid of that check. It doesn't make sense." And it obviously can't run it because it's running something much faster and the renderer can take it. It's no problem at all. 

We are able to take a nice approach, which is packaging what is basically the functionality that the emulator offered in the case of altering the projection to support widescreen stuff. As well as the game patches that are needed for that to work properly. That's the nice aspect of what makes the package whole, in my opinion. Because, normally, if you would run this widescreen hacks on other emulators, sure. Sometimes they will do the work. But they would extend the screen. But you would have all sorts of rendering glitches. And the experience isn't quite nice. The fact that we are able to distribute a version of the game that already has all of that included is what makes the presentation that much better. 

[00:39:56] JN: Awesome. I have a question about the directions of this game in the moment. But first, I guess I want to come back to - you mentioned ultramodern. From chatting with James Lambert about Portal 64, I kind of got the impression that alternatives to LibUltra were a large barrier to a lot of porting efforts and I guess a lot of, let's say, legally-allowed/gray area porting efforts. And then I got into your projects and just came across like, "Oh, we just casually made our own runtime. And his ultramodern it's fine. It's just there." Have I completely misunderstood how difficult making a port of LibUltra is? Or have there been some other tricks here? How has this come about? It seems like a big achievement to me I guess is what I'm getting at.

[00:40:33] W: I'm actually glad this has come up because it's an area that I don't get to talk about very much. Because it is so behind the scenes, it's not something even processes for a lot of people that there is this backing library that exists that is responsible for so much of the heavy lifting to actually get the game running was not easy. 

It took quite a long time. I went through a lot of different revisions of how many different parts of it work. In the video by Narrow where the project was originally introduced, the video that came out alongside the project, he mentioned in there that I had rewritten the audio system alone three times to get it up to the standards I was willing to ship, call it. That is not by far the only piece that I went through many iterations on. 

Actually, interestingly, the ultramodern project started before N64 Recompiled was even a thing. I was helping out with the Paper Mario decompilation. And while a port was never really - for most decompilations, it's actually not - it's almost always the case that a port is not a main goal or even a goal at all of the decompilation. Because a lot of time, the interest doesn't overlap as much between who's interested in making a port and who's interested in doing the decompilation. 

But in that case, there was a little bit of interest in doing a native port. I toyed with the idea of, "Okay. Well, how much of LibUltra could be reimplemented from scratch on a modern system?" And I put together a little like proof of concept LibUltra at the time to run on PC. It was very different than the approach the existing ports have taken. In that rather than change the game code to not require a lot of LibUltra, it is to go straight to reimplementing the actual interface that LibUltra provides in almost its entirety. 

And it's basically sat dormant for a while. And then when I started working on N64 Recompiled, I remembered I had it. And it was like I could probably retool this to work for this project. And I did. But I think by this point, it's been basically entirely thrown out in all the rewrites that have happened. 

The biggest difficulty I would say is how the threading works. It's not necessarily very different from modern threading. But the main difficulty is that the N64 is a single-threaded console. I mean, a single CPU core console. And so, if you were to just naively implement the original games threads as modern CPU threads, you'd be running many different parts of the games code at once that were not designed to be run at the same time. And you'd have a ton of race conditions everywhere all at once. And it would not run. 

Actually, the hardest part in writing the threading code is actually making sure that only one thread is running at a time. What's it making it "worse" is the hardest part of getting those threads to run. There are a lot of other things. A lot of modern systems have a ton of different synchronization primitives you can use. Atomics, and semaphores, and things like that. There's only one in LibUltra, thankfully, the message queue. 

But there's a lot of specifics about how the message queues have to work that are not necessarily things that modern message queue libraries can do. Having a fixed capacity and actually stalling the write when another - when you have filled up that capacity, normally you stall on reads for the most part or receives rather. 

And so, a lot of little nuances like that, they add up over time when you're trying to reimplement the system library. Because you have to get it very accurate. Because, games, they don't always use the library correctly. We'll put it that way. And this is true for graphics as well. You have to recreate not only the correct behavior of the library. But a lot of times, you're worrying about recreating the incorrect behavior when the library is used the wrong way. 

Well, it turns out, these games that were made in a year or two-time span and incredible crunch, they didn't always get it right. It would be nice if we only had to care about the correct case. But it's almost never what actually happens. 

[00:44:46] JN: Yeah. Absolutely. I never thought about it before. I've often said that developers really enjoy using a tool the wrong way. And that's often the goal is to like use a tool in the way it's not meant to be used. And I never really thought about that from the perspective of trying to reimplement a historical tool that developers have used and try to make their broken stuff work on it. Incredible. 

Okay. Get back to the Zelda project. I initially thought this was very much like a proof of concept of the recompiled project. But it seems like an end in itself. The goal of the Zelda recompile project is to make a high-quality playable port. Is that correct? 

[00:45:19] W: Yeah. And it's still getting updated too. Currently, the main thing that's being worked on is on the RT64 side there's HD texture support, which will apply to all recompilations and any emulators that implement RT64. And then on the recompiler side is mod support. The ability to load new code from the launcher. Make your own mods that change the games behavior. Distribute those around without having to rebuild the entire port every time you want to change how something works. 

And it is in some ways a test bed of those new features. Because it's just the one that came out first. But it's not by any means only a test bed or proof of concept. It is very much a fully-fledged port in and of itself. That happens to also be where a lot of these new features come out first simply by virtue of only being the only one right now that I've made that have been released at least. 

[00:46:17] JN: Awesome. Perfect. And in regards to I guess both N64 Recomp and RT64, what's next to these projects? Anything coming over the horizon you're particularly excited about? 

[00:46:26] D: I wanted to clarify something on one question you asked me before. And you asked me about the goal of the accuracy. I wanted to bring up that something was very important, because I do love the term test ribbon development. Because every time I do it, it proves to be right. Wiseguy mentioned that he has experience doing homebrew. He actually made a lot of ROMs that they're not games. They're basically just menus where you control a few settings of whatever we were interested in testing. That actually help - that's probably where 90% of the accuracy comes from more than the actual software games. Because we had a really focused scenario where we just tested every single parameter that we could try on the graphics processor and see what the output was and tried to replicate that. 

It even went as far as doing something that just loaded a bunch of memory into the texture memory unit and used a way to dump it back into RAM just so we could see it. And we could go over and replicate all the weird behavior that it did for loading the texture memory, which was really interesting. And I think we actually ended up with one of the most accurate and readable implementations out there. Because even on the stuff that is supposedly basically a reference, there stuff that, say, for example, it crashes and it doesn't actually crash on console. 

We were actually able to find some much cases that some plugins currently aren't accounting for. But we found them by making these tests run. And that's something that I wanted to emphasize. Because, obviously, I'm thankful to Wiseguy for doing those. But it also proves that you can get a lot of process done far better by just doing with that approach than just testing random games.

[00:48:19] JN: Yeah. Absolutely. That's pure demo scene. I love that. That's super cool. Yeah, as you say, rather than go hunting through, get to the right part of the game play to see like one effect, just mash a bunch of effects together. That's awesome.

[00:48:31] D: Yes. And we actually have a few ideas of some ROMs we want to do next for some of the more obscure effects we haven't actually tested yet. Because there's a lot of higher functionality that works in really weird ways. For example, you can tell the graphics processor to work in a certain mode that only copies raw bytes. There's a lot of weird behavior - that's actually a very common mode that the games use. But if you go by just purely what the command says, the output is not exactly what you expect. Maybe it might be misaligned by a few pixels. Or if the format doesn't match, it does something weird. You need to get it right. You need to sometimes even reimplement parts of how you thought that, "Oh, maybe this is just drawing the texture into the screen." And, no. It turns out it's doing something like copying the value of the pixel and then you have to reinterpret it the right way.

[00:49:27] W: I know we're a little off from what the original question was. 

[00:49:30] JN: The floor is yours. Take me where you want. 

[00:49:33] W: Yeah. There's one specific example of that where I'd love to talk about it. In the the game Paper Mario, which does a very similar thing to what I mentioned earlier with the frame buffer in Zelda where it will capture the frame buffer and then it showed us the background of the pause screen so that you can see the world while the game is paused. There are two bugs in the pause screen code. They are definitively bugs. There's no doubt. Anybody who knows how the hardware works would tell you that's a bug. But it just so happens that the two bugs work perfectly in unison to not cause an issue in the visual output. 

One triggers an off-by-one error in the hardware of the console. And then the other one, the developers must have seen that their pixels are shifted one pixel over and said, "Oh, I guess we got something wrong in the texture coordinates. Let's just fudge the texture coordinates a little bit to account for that. Maybe we did our math wrong." 

But, no. In reality, their texture coordinates are wrong if they hadn't made the other bug. And so, if you emulate very accurately the texture sampling but then don't also emulate this bug in the rendering, you end up with the pause screen shifted one pixel over to the right whenever you pause. And so, you have to emulate the actual bug in that case for it to look correct. And that's an area where even the Parallel RDP, which I don't - I don't want to make it sound like I don't think Parallel RDP is an absolute wonder of engineering. Because it's honestly so cool. But even Parallel RDP doesn't get this behavior correct. And so, that's one area where the test ROM sort of reveal. They show all. You can't hide from boiling it down to a single test ROM that gives you exactly one behavior. 

[00:51:21] JN: That's so cool. I mean, anecdotes like that about mutually destructive bugs are the kind of things that make me question whether computers truly are the pinnacle of human invention and really beautiful. Or whether we've all made a terrible mistake. Saying this is a life path that's just wild. Yeah, thank you for sharing both those examples. Those are both wonderful. 

[00:51:40] D: There's another one - in both Zelda games, actually the sun is really screwed up.

[00:51:45] JN: Oh. 

[00:51:49] D: This is a fun one. The sun is like they got the texture format wrong for it. I think it was a 4-bit texture loaded as 8-bit, I think. It's basically like it had twice the pixels but it loads only half of them. They clearly couldn't tell what was going on. Something was going wrong. But instead of fixing the texture coordinates to match that, they use some other flag that disables perspective correction, which means the texture coordinates are twice as big. And then the sun ends up looking sort of correct. It's a sun. But they got there by doing two errors and canceling each other basically. Only it's at half resolution. 

[00:52:34] JN: I think we've just invented a new podcast we need to do just in the N64 Discord scene, which is cursed bugs that you all are finding. These are great.

[00:52:43] W: Yeah. The hole in Mario Golf. I know remember the details on that. But that one took a while to figure out why it wasn't working correctly. 

[00:52:49] D: Yeah. I think that one was the - yeah, it was basically like it had blending turned off. But, apparently, they had other read that register anyway. I think it was something along those lines. 

[00:52:59] W: Right. Yeah. It uses the input from even when the blender is off. It uses the input from the blender as the output. And so, the blender configuration affects the output even if the blender is turned off. 

[00:53:12] JN: And when you say hole, it's the literal golf hole. The thing that's going - the center of all gameplay that appears everywhere. Perfect. Excellent.

[00:53:18] D: Yeah. We've had most of the other things that are way harder to emulate on Mario Golf correctly. Except the hole didn't show up.

[00:53:27] JN: Okay. This brings me back to a question, which you're producing playable games. For a lot of people that have seen, part of the emphasis is on making preserving these games, getting these games playable. Along the way, you're just also producing the longest tale of incredible tools and weird single-purpose artifacts like test ROMs. What are your personal motivations? Where is the fun you two in doing this? 

[00:53:51] D: I mean, personally, like I said I have some sort of personal vendetta, you could say. I'm used to playing a lot of N64 stuff on emulators before. Now it's like the phase I get to discover why a lot of the stuff was broken. It's like a super fun puzzle to figure out and solve in a way. If I want the perfect emulation, there's already perfect emulation like we said before. Something like Parallel RDP. You already basically get one-to-one. But there's still some interest on me on knowing what exactly a lot of these games work in a certain way. And it's not like we have a solution yet for these games being upscale, for example. Because while Parallel RDP is great at doing that one-to-one accuracy, if the game does anything more complex with all these effects that rely on frame buffers and stuff, that usually doesn't get upscaled at all. 

One challenge that the render has is like it has to involve a lot of high-level logic to figure out when the developers did these effects by tracking where in the RAM they stored - where did they write in RAM? What part of the RAM they loaded? And which tile exactly? I track that information. And then if the game decides to render that tile and it seems like if the conditions are right, then I can just replace that with a high-resolution version of it, which is how most of the frame buffer effects works at a level where we don't even need to be aware of what the game is doing for high-risk frame buffer emulation to work at the moment. 

None of the stuff on the Majora's Mask port was modified at all for that to work. It just does because all the automatic detection does the job. And, obviously, the other goal is the rate ray-tracing is really cool. If I can get both of that and now clarify that this functionality is not going to be you have to choose one or the other, you can actually just press a key and turn it on and off, which is like one of my favorite things to do with remasters like that. 

My goal is just to have the game look as nice as possible. But I also appreciate the original look. There's room for both things. And I think they're under the covers that both of those areas are pretty well. 

[00:56:24] W: I think for me, I love taking apart. I think you probably - if I remember right, you got a similar answer from Ethan and Mark when you asked them about why they're interested in decompilation. But I do just love taking some game that I am very fond of and then just spending a lot of time figuring out why is it doing it. And while I did mention earlier you don't get a lot of that information from making the actual recompilation, you still end up getting a lot of it as you go and you add enhancements to the game. Or you go and you get the renderer working without any issues if there are issues at this point. Fewer and fewer every day. 

But there's also just the aspect of, yeah, being able to experience these games with all the niceties you've grown to expect from gaming at this point. High frame rates and wide screen. It's really cool. But also, there's just an aspect of like is it possible? You ask yourself that question and like, "Well, decompilation, there are some tools that are available for it that sort of help the process go along." Not necessarily automated. But Ghidra and MIPS to C, which is a tool that gets used in decompilations a lot for N64. Sorry. It's machine to C now since it does more textures. 

But they're like to some extent a little bit of doing that some of it automatically. And then the logical question you go is, "Okay. Well, can you do it entirely automatically?" And then you're like, "Well, other static recompilation exist. Why couldn't one exist here?" There is just an intellectual curiosity aspect to it of just like can it be done at all? And it's always fun when the answer is yes. And here it is working. It's really interesting. That's the summary. It is very interesting to me. 

[00:58:18] JN: Perfect. And before I let you both go, are there any particularly obscure or niche games you're excited to see get this treatment? But aside from - what was it? Rocket: Robot on Wheels, which is one I'd never heard of either? But looked really good [inaudible 00:58:30]. 

[00:58:32] D: We actually got a recompilation for game recently that neither of us has played.

[00:58:37] JN: Okay. 

[00:58:38] D: If you have seen the news. 

[00:58:39] W: Oh, yeah. Somebody else made a recompilation and released one that was not - they didn't even ask anything about. They just took the tool on their own. Figured out how to use it based on the Zelda repo and then got it running. 

[00:58:53] JN: The open source dream. Your docs did the trick. 

[00:58:56] W: They could definitely be better. But this very dedicated individual must have sifted through quite a lot of our code. But it's a Mystical Ninja Starring Geomon was the game that they worked on. Another one of those - I'm sure it has a very loyal following. But not necessarily well-known game. I think those are in some ways the more interesting games to see. Because they're the kind of games where even an official court is very unlikely to ever be seen of those games. 

Obviously, there's a lot of big-name games that I would be excited to see too. The Banjo-Kazooie is a game I really like. But it is interesting to see other people take it and go on games that are not popular but they really like. Yeah, Rocket is the one for me that I will hopefully at some point release the recompilation of. It's a very unpolished state. I didn't want to put it out there yet. 

[00:59:50] D: Yeah. But asking Wiseguy what recompilation he wants is a bit pointless. Because he can just do it himself.

[00:59:57] JN: It sounds like you've all got a lot going on. 

[01:00:01] D: He clearly loves Superman 64 so much. He just - 

[01:00:04] JN: No. I did not know about this game until watching videos about Recomp. And then I was like, "What is this? What have I been introduced to?" 

[01:00:12] W: It's very notoriously bad game. And so, I thought it'd be very funny to secretly do a recompilation of it without telling anyone. Nobody knew about it. And then I just dropped it in the group chat. Here it is. Without saying anything and not even mentioning that I had been working on it. 

[01:00:27] JN: And just the N64's ET, is that kind of what we're talking about here? 

[01:00:30] W: Basically. Yeah. And I hope if any of the developers of that game ever hear this by any chance, I immensely respect what you've done. Because it's not easy to make an N64 game regardless of how it comes out. 

[01:00:41] JN: They should play your port.

[01:00:43] W: Well, it's not out yet. Maybe one day I'll properly do a release for it. Who knows? Sort of a joke to do the project. 

[01:00:51] D: But to answer your question, I'm actually asking for a Snowboard Kids recomp. That's the one - 

[01:00:56] JN: Which one? Oh, Snowboard Kids.

[01:00:57] D: Snowboard Kids. It's not that obscure. But I do love the game. 

[01:01:02] JN: Oh, yeah. I did play Snowboard Kids. Oh, God. That game is amazing. 

[01:01:06] D: See? 

[01:01:06] JN: I had completely forgotten this game existed.

[01:01:09] D: Imagine that with online multiplayer. Wouldn't it be amazing? 

[01:01:11] JN: That would be really good. Yeah, I'd be down for that. 

[01:01:15] D: There's two of us Wiseguy. Now you know. 

[01:01:19] JN: Perfect. Alrighty. Well, thank you both so much. I have enjoyed this immensely. And, again, I'm going to get absolutely nerd sniped into getting to N64 Recomp at some point through these wonderful episodes. Thank you both for joining us today. 

[01:01:33] W: Yeah. Thank you for having us again. It's been a pleasure.

[01:01:35] D: Thank you very much.

[END]