EPISODE 1890

[INTRODUCTION]

[0:00:00] ANNOUNCER: WebAssembly has grown from a low-level compilation target for C and C++ into one of the most influential technologies in modern computing. It now powers browser applications, edge compute platforms, embedded systems, and a growing ecosystem of languages targeting a portable and secure execution model. Andreas Rossberg is a programming languages researcher and former member of the V8 team at Google. Andreas helped architect WebAssembly from its earliest concepts through its most recent milestone releases, including the groundbreaking 3.0 spec that introduces garbage collection, richer reference types, and major steps towards multi-language interoperability. In this episode, Andreas joins Kevin Ball to explore the history of WebAssembly, the constraints that shaped its earliest design, the major turning points in versions 1, 2, and 3, and what's coming next for WebAssembly. 

Kevin Ball, or KBall, is the Vice President of engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action Discussion Group through latent space. Check out the show notes to follow KBall on Twitter or LinkedIn, or visit his website, kball.llc.

[INTERVIEW]

[0:01:33] KB: Andreas, welcome to the show.

[0:01:35] AR: Oh, thank you. Thanks for having me.

[0:01:37] KB: Yeah, I'm excited to have this conversation. Let's start a little bit with you. Can you give us a bit of your background and how you got involved with WebAssembly and what took you to where we are today, having this conversation?

[0:01:50] AR: Yeah, it's been a bit of a journey. I'm a person who's on both sides of the aisle in terms of academic and industrial work. I used to be more a researcher in programming language, both hardcore theory stuff, but also implementation stuff. At some point, I switched over to industry working for Google, working on the V8 team, and that's one side where the whole WebAssembly thing started. That's how I got involved in that.

[0:02:17] KB: Let's dive in there. Actually, I didn't realize you'd done a lot of academic work on theory, so I'm curious maybe as we go along to explore how that has influenced development of WebAssembly. Let's maybe take us on a quick journey of the history of the WebAssembly. I think, if we have web programmers here, they're probably familiar, but not everybody is. Let's go through, what was the inspiration and the different stages we've been at coming to today.

[0:02:41] AR: Okay, so a bit of history was that before WebAssembly on the web, we basically only had JavaScript, right? It was always clear and common complaint by many people that this is not good enough, and JavaScript has problems, as we all know. If you want to use other languages, you have to compile to JavaScript, and that's far less than ideal as a compilation target. It also can be slow, or at least very unpredictable in terms of performance. Before WebAssembly, there were already two technologies that tried to bring more native code to the web, and one was a native client. That was a Google technology that allowed you to do initially at least x86 code to run that in the browser in some form of sandbox.

Then a bit later, the other thing that occurred was asm.js, which grew out of emscript, which is this tool that was able to translate C to a very star subset of JavaScript that engines could compile more efficiently. That was really a very clever ingenious hack. It was clear that this would not scale very well, right? At some point then, it was mainly people working on the JavaScript engines at Google and at Mozilla. Luke Wagner and Ben Titzer in particular got together and started the idea of like, let's do it the right way. That's how it evolved then. That was back in 2015, I think.

[0:04:11] KB: What would you say are the constraints unique to trying to do this inside of a browser environment the way that it was? You mentioned like, okay, to get fast, we had to trim down the subset. Google tried this native x86 approach. What makes compiling to a browser environment different than just running something on my laptop or server?

[0:04:35] AR: The biggest constraint, of course, is that it has to be safe and secure, right? You're running untrusted code, essentially, on your own machine. You can't afford any undefined behavior, or any other non-safe behavior. That is the top most constraint that hovers above everything else. Then, of course, you also have constraints like, the web is a very particular environment. You don't have access to most of the services that you're used to from regular operating systems, right? There's no file system, or anything like that. You can emulate some of that stuff, but it's not really physically there and various other OS services.

Various programming language implementations assume the presence of some of these services. So, when you compile, at least to the web, then some of these things have to be implemented differently. You have to be aware of these environmental constraints.

[0:05:38] KB: That definitely is a somewhat different environment to have to target. Let's then maybe look at what the big steps have been leading up to this 3.0 release. The first version of WebAssembly, what was in the box?

[0:05:50] AR: The first version is essentially, was targeting customers we already had with asm.js in particular. There were already applications that just compiled a pre-existing C, C++ code bases over to the web using asm.js. We wanted to be able to switch them over as soon as possible, so that we can retire that technology and have something more forward-facing. The 1.0 release was totally targeted at low-level languages, like C, C++ and Rust. The feature set was really only you have this linear byte array as your memory model, and you have the four basic numeric data types and that was all there was to it. Not much more than that. You could define functions, could compute terms of arithmetic and you had your memory. You had a way of emulating function pointers. That was it. Very basic virtual CPU, if you want.

[0:06:52] KB: Yeah. Well, and I remember hearing after that came out, there was somehow, oh, this is super limited. Then people were like, "Hey, look. I rewrote this CPU-intensive library that we used to do in JavaScript. I rewrote it in Rust, or I rewrote it in C. Now it's 30X faster." These stories of improvements. Now, those may be isolated cases and actually, that might be worth coming into. If you have a JavaScript application, what types of benefits do you see compiling down to WebAssembly over JavaScript?

[0:07:25] AR: Yeah. I mean, that depends a lot on what your domain is and what kind of computation you're doing and how dynamic it is. If you're really just doing low-level numeric stuff and you can basically, ahead of time, compile all that to a very efficient code, then that's where WebAssembly really shines, right? You basically get next to native code performance.

If you have more complicated language run time things, especially in 1.0, you need a garbage collector, or anything like that, then you will have a much less pleasant time at that point, because you have to emulate all that and you are fighting against some of the abstractions that WebAssembly imposes, mostly for portability and security reasons. Right, that's why for more high-level languages, for example, WebAssembly 1.0 was not a particularly attractive compilation target. 

Also, in terms of performance, yeah, again, it depends a lot of what you're doing. There are certainly things where JavaScript might even beat WebAssembly. The reason being that it's a way more dynamic system. Because it has to be anything close to fast, it does dynamic profiling, dynamic recompilation and really, really crazy things behind your back at runtime, which, of course, all have a certain cost to them as well, right? You're just not seeing that usually. Of course, you can easily construct cases where this is creating much better runtime than WebAssembly ever could, because it doesn't try to be as smart, right?

This has been a traditionally also fun thing with benchmarks, because if you're not careful with benchmarks, then JavaScript engines could easily optimize away half of your code, what you're trying to measure. Yeah.

[0:09:18] KB: Yeah. I came up in the high-performance computing world, 20, 25 years ago, and benchmarks were, and every compiler was trying to figure out, okay, how can I game this benchmark, but in a way that's not obvious that it's gamed?

[0:09:31] AR: Yeah, that was the time I was on the V8 team in the tense, mid-tense, early, mid-tense, that the benchmark war was all on, right? It was completely ridiculous. I still feel a bit traumatized from that time and basically trust no benchmarks at all.

[0:09:51] KB: We're revisiting that now in the LLM era, and everybody's trying to figure out benchmarks and they're not necessarily representative. Let's keep moving forward. You mentioned, all right, 1.0 targeting very low-level languages, languages that probably aren't having to deal with garbage collecting, aren't having to deal with these other things. Moving forward then, 2.0, was how many years later?

[0:10:12] AR: Ah, depending on what you count, right? There are the proposals and when they are done, and then there's this official process about releasing the spec, which in this case took years after everybody already shipped all the features in it. Technically, I think it was released in 22. Practically, it was more or less done in 2019 or something. Maybe 2020. I don't remember exactly. That's when all the browsers implemented it and the features were completely done, essentially. The main big feature in 2.0 was the support for vector instructions. That was, again, take more advantage of hardware capabilities, really compute-intense stuff and allow applications to take benefit of what modern hardware provides in that regard.

Yeah, so that was a big feature. Then another, somewhat, there were very smaller features. One other interesting feature that was showing a new direction was the addition of what we call reference types. In addition to just these four numeric bit pattern types, you now have an opaque type of references. That was two reasons at that point. One was to have more first-class handling of function pointers, which we can't treat as transparent reference, or transparent pointers, because that would be completely unsafe. To have a way to pass them around, you have to have a type that ensures that you can't peek at their bit patterns, or anything.

The other reference type with external reference and that gave you a way to basically pass through host values. On the web, that would be JavaScript values through WebAssembly code. You can't do anything with them, but you can hold them and round-trip them. That offered a much more convenient interop story with the web in particular, where previously you had to maintain bijections on the boundary that was really ugly and brittle to do.

[0:12:21] KB: Yeah. I do remember that being a big thing of, if you were going to port something over to a WebAssembly for a library or whatever, really having to be careful about what's the interface and keeping it as slim as possible, because it was hard to move anything around.

[0:12:35] AR: Right, right. Yeah. External references, at least in principle, made that easier. Also, these were the inroads to what followed later with garbage collections of a much richer system of reference types.

[0:12:50] KB: Yeah. Well, and it feels like in a lot of ways, 3.0, which is what brings us together here, was just a mammoth step forward in capabilities. I was reading somewhere that some of these things had been in development for seven, eight years, which is just a mind-blowing timeframe for anyone in the tech industry how much has changed in the last seven or eight years. Let's talk about the big release, 3.0, what was in the box and what's it looked like.

[0:13:17] AR: The biggest feature, arguably, is the addition of, yeah, garbage collection, right? That's what I just mentioned with taking reference types to the next level. You now have support for defining your own reference types and a memory layout for them. Then the WebAssembly engine takes care of allocating and deallocating them. It's noteworthy, though, that we try to make sure - so, this has been a somewhat controversial feature, because it's like, okay, that is no longer just abstracting the CPU feature, right? It's a bit more high level. We were very careful to stay within the spirit of WebAssembly of being a low-level language. I always phrase it as low-level as possible, but no lower.

This is a very low-level GC feature, in the sense that all you get is, essentially, structs and arrays. You don't get any objects, or anything like that built in, like you would have in other VMs, like the Java VM, or something. It's really low-level. What the types are doing, they describe to the engine what memory layout you want, but you still have to map all your runtime data structure from your language to that low-level representation, like you would do a native code, right? The only difference being that you tell the engine to allocate the stuff for you, or more particularly, deallocate it.

The complexity there is that, yeah, this type descriptions, if you want to avoid a lot of runtime checks, then you need a more clever type system that always knows what it is you're referencing without having to do any casts, or at least a minimum amount of casts. That's where the type system then becomes significantly more complicated. Yeah, that was one big thing. Let me quickly look at the list.

[0:15:13] KB: Actually, before we go further on that, so since you're targeting not only languages with garbage collection, but still those original languages, is that opt-in? How does one interact with it?

[0:15:25] AR: It's basically completely separate from this linear byte array that you had before. You can still use that. You can use both together. You can completely ignore the instruction set, the part of the instruction set that deals with GC. Then everything will be as before, right? You just don't have to use it. Some people have been worried about this creating extra cost in the engine and actually know, because in web engines in particular, all it's doing, it's giving you, essentially, access to the garbage collector that was already there for JavaScript, right? Now you can allocate something in there from WebAssembly code, and that's all that changes. They have one combined GC. That obviously was there before, so nothing, no extra cost there if you completely ignore this stuff. It's a completely separate memory area that does not overlap at all with the linear memories.

[0:16:26] KB: Yeah. I think that's key. It reminds me of Rust has this principle of cost-free abstractions, right? Anytime they can implement something that doesn't have a runtime cost, right? In this case, maybe it has a little bit of a runtime cost, but it's opt-in. If you don't use it, it doesn't layer any cost to the other pieces.

[0:16:42] AR: Exactly, yeah. Yeah, we also have that philosophy. I think C always had that philosophy, or C++, they always call it pay-as-you-go, or something, or maybe that was somewhere else, but, yeah, there are various names for that. That's definitely a philosophy WebAssembly is also following.

[0:16:59] KB: Yeah, I was looking at the feature list just before this, so I can bring some of them. Another one that I saw that I'd love a little bit of a dive into what it means is this concept of having multiple memories.

[0:17:09] AR: Right. This is more actually a technicality, because contrary to popular belief, you already were able to have multiple memories before. You just were not able to access multiple memories within a single module. In WebAssembly before, you could have multiple modules that talk to each other and each of them defines their own memory. That was totally okay. There was just this weird gap that you could not bring them together in a way that you can directly talk and move values between multiple memories. That was both a performance cliff for some things that want to do that. It was also a gap in the design in terms of modularity, so there are these tools that are basically acting like a static linker, so they take a set of modules and merge them together in one bigger module.

That worked with everything, except memories, because you could not have multiple memories in one module, right? In general, this would fail if more than one module defined one. Now, that was one of the intentional gaps we left in the initial design of WebAssembly 1.0 and always were meant to fill in later. It was more like, okay, we need to get something out of the door quick, so let's defer that. Now, we've added that. It was basically one of the last intentional gaps that we left initially that we filled in now.

[0:18:36] KB: It might actually be worth, just from a design perspective, because I think building a whole, I guess, in this way, case compilation target, a whole assembly level language that is pretty low level, that's a programming domain not many people have explored. What types of intentional gaps did you leave in that 1.0? How did you think about that? What were the trade-offs evaluated?

[0:19:01] AR: There were three main gaps in terms of, oh, okay, here we only allow one of these, when really in general in the future, we want to allow multiple of these. The first thing was multiple values, right? Function, or instruction initially could only return one result. That was silly in terms of a stack machine and constraining. We lifted that, and that was already in the original paper we published that already had that worked out.

Then the other two were multiple tables and multiple memories. Multiple tables kind of sneaked in in 2.0 as part of the reference types extension, because that meant that previously, you only had function references, so you could just do everything with one table. But now you have multiple reference types and we use tables in a more general way, so you need tables of different type, right, that naturally requires you to be able to define multiple ones. Then the multiple memories was the remaining one we added later. There has been talk about having multiple start functions but that is not, right? You can easily work around that.

Yeah, so these were gaps we left in there, where all the instruction set and binary format already anticipated that we will add them. Other than that, it was more like, okay, let's for now stick to the feature set of things that are already in asm.js, so we have to cover everything in there. Then, also, what is the intersection of all common CPUs at that point? So, that everything that has a one-to-one translation in terms of instruction mapping to actual hardware, that is an easy one.
I think we didn't leave that many gaps there. There are more gaps now with SIMD for example, because SIMD is a much sadder story in terms of hardware support, where hardware vendors really can't get their shit together and agree on what the instructions are and what their corner cases are and what the feature matrices are. There are random gaps in there that I really personally hate. Where just one hardware would not allow an efficient mapping of this particular case, so we leave it out and maybe at some point we add that.

Basically, we always have this MVP concept in a more general fashion which is, okay, let's always start with something that makes sense. We already know we want to extend it later, but have something that works and that we can ship and people start using it and we get feedback and experience and have a clearer picture of where we should take it from there. WebAssembly1.0 as a whole was an MVP with garbage collection, like what we added now is the garbage collection MVP. There's a whole list of post-MVP features that we have in mind that we might want to add at some point, but that are not totally essential, right? You can get by without them.

This is more the usual thing we're doing, where we try to keep the broader picture in our head, at least we have a rough idea how that could be integrated later, but focus on the things that we can solve quickly enough. Even though it still might take eight years, I don't know.

[0:22:23] KB: Yeah. I'm curious, actually. One, as you're getting that feedback from each version of MVP, which I think is that is the approach that we all at least idealize as an approach. It's not always easy to follow. Is that feedback from browser implementations? Is that feedback from compiler implementers? What were the different things that ended up shaping from 1.0 to 2.0 to 3.0?

[0:22:49] AR: Yeah. It's often like, sure, compiler writers, engine writers, framework writers. Sometimes it's CPU vendors that want to push their like, this is a bit of what happened with SIMD. There was a very strong push from CPU vendors to have support for their ultra cool and fast feature. At that point, it might be somewhat political. But yeah, usually it's for the bigger features. It's a broad understanding of, okay, we have this low-level language and there clearly are things we can't handle yet, but what would that mean in terms of a feature? Then we design these bigger features.

GC was a discussion from the beginning. Exactions is another thing that's in 3.0 now, right? Which, also, was in the making for very long. Threats is another one that - go, although it's also already implemented everywhere and has been for years. The biggest gap now is still missing is something like, some continuation feature. Maybe we want to talk about that later. There's always things like, we know that there's some language features you can't compile currently, because there's some functionality missing, right? That is one big thing. Usually, as engine people and compiler people, we know what these things are, and we already have plans to some degree for attacking these at some point. Modable priorities.

Then, sometimes, there are smaller things that just come from clients, users, third-party compiler writers that notice, "Oh, you know this is a thing that would be good to have here, because I can't do it easily the other way," and then they propose something as well. Yeah, like that. I mean, this is a process. I don't know how aware people are. This is completely out in the open. WebAssembly is totally an open-source project in a way. The committee is open for everybody, right? In theory, everybody can join there and propose a feature. But there is a very elaborate process you have to go through to push that feature over the edge in the end. In particular, you have to convince the rest of the committee that it's worthwhile. That has happened a number of times, so -

[0:25:04] KB: Well, actually, I wanted to ask, are there any features that have made it out in one of the 2.0, or 3.0 releases that you did not anticipate, that we're not already in the back of your head of like, "Oh, yeah. We'll want to do that eventually"?

[0:25:19] AR: I think more of those are landing now, now that we have gotten over the hump of the big features that we want to do from the beginning. Maybe one thing I didn't anticipate was this relaxed SIMD extension that we have in 3.0 now, which is also, to be honest, not a feature I'm particularly happy about, but that's a different story. Other than that, most of these are kind of, yeah, have been in the making for a long time and always with a trajectory to them.

I don't know when we started talking about 64-bit address spaces, but I'm pretty sure that also was from the beginning that eventually, we will probably support larger address space. Yeah, it's a good question. But actually, in the releases so far, there aren't that many features that came up much later. Maybe just being a function of how long it takes to add a feature.

[0:26:15] KB: Well, and in a lot of ways, I think this is well trodden ground in terms of supporting all of these different languages, and so you're catching up to the cutting edge in a lot of ways. So, then at some point, you catch up to that frontier and then there's what pushes forward. Speaking of that, I think one of the things that I saw that I thought was interesting was 3.0 added tail calls as well, which I think is a very important functionality in terms of enabling efficient use of more functional languages. I'm curious, if you look at the spectrum of languages, what does support for compiling to WebAssembly look like now? Are there any languages that are not actually supporting this yet?

[0:27:00] AR: Well, I mean, that's up to the languages, but I would think that most languages that have a certain user base have some form of support for WebAssembly at that point. 3.0, especially with garbage collection, also tail calls was an enabler for a whole new set of languages targeting it now, like functional languages, like you mentioned, but also other GC languages, like Java and C#. They're using that instruction set as well.

Yeah. I mean, there are so many languages out there that I don't have anything close to complete knowledge of who has a WebAssembly backend and who hasn't, but most of the languages that I come across, usually they already have it in some form with varying maturity, of course. Yeah, it's very much a thing these days in your compiler ecosystem to at least consider it.

[0:27:55] KB: Looking at that now and looking at how much easier it is to navigate across the JavaScript to WebAssembly border now with these new features that you have this wide language support, can we revisit the question of when does it make sense to use a language targeting WebAssembly versus JavaScript, or TypeScript? I think the calculus has probably changed a little bit since 1.0.

[0:28:18] AR: Yeah, absolutely. Right. So, more high-level languages, especially GC languages is much more reasonable to compile into WebAssembly now. I would think that actually, it's reasonable for most applications. The only thing, if you're really building something that all what it's doing is DOM interaction and manipulating the DOM, then you probably won't get much value out of it, right? Definitely not in terms of performance. It won't necessarily be slow either, but you might have to jump through extra hoops with all the glue code, and so on and so forth, which is much quicker if you can just do it in JavaScript directly. That is the main domain where I would say, okay, it's probably still better to use JavaScript. Or, potentially, if you're compiling a language that is very closely integrating with JavaScript by design, then that might also be better to still target in JavaScript itself.

[0:29:18] KB: That's actually an interesting thing to discuss. When I think about building an application in JavaScript and particularly for the web, you have JavaScript, the language, all the runtime and libraries, but you also have all of these web-specific APIs that are available in JavaScript. Are those exposed directly in WebAssembly as well if you're building for the browser, or do you have to hop over to JavaScript to access the API and then hop back?

[0:29:43] AR: Yeah. Yeah, that's a famous question and complaint of, give me DOM access, right? The thing is, yeah, right now you don't have that. That is actually a very important part of the whole WebAssembly security model that it's completely sandboxed, right? You have no ambient capabilities to have any access to your environment, whatsoever. Anything you can do, or interaction you have with your environment has to come through imports and that's core to its very tight sandboxing functionality.

What you have though is this very tight integration with JavaScript in the sense that a WebAssembly function you import can just be supplied as a JavaScript function providing the signatures match up sufficiently well. Basically, you can just import JavaScript functions and then just call them as if they were WebAssembly functions. The problem, of course, is that all the web APIs are really totally designed for JavaScript and they don't really make a lot of - or many of them don't make a lot of sense in terms of a more moderate object system language. You usually end up having to write some, or may have to do some glue code there. But this is more like a tooling problem.

People often make the assumption that it would be way faster if they could access, say, the DOM from WebAssembly directly, but that's not actually necessarily the case. In particular, since JavaScript engines already go to way more length to optimize all sorts of things about accessing objects and JavaScript objects and DOM objects, and that would be ton, and a mega ton of stuff that you would have to replicate in the WebAssembly code generator, and I don't think anybody is willing to do that, because it's silly, right? You wouldn't even, probably not even get much performance out of that.

Because the overhead of calling from WebAssembly into JavaScript is actually mostly dominated by the actual work done in the DOM itself, right, with most of these calls. The assumption that it would be way faster or something is not actually very accurate. That being said, there is some work now. One thing you would like to have is at least have some kind of interfaces for accessing the DOM, which people agree upon. That is something that is a separate layer from WebAssembly core itself, right? People have now started looking into this in the context of this component model work. That might be something maybe similar to these Wasm libraries, but for the web, so we start defining interfaces and they grow organically.

Maybe at some point when they're mature enough, we've iterated enough on them, then we might even standardize some of that. Until then, it might just be a library, or a tool chain that just does this for you, and you don't have to worry about it anymore. Because that's the other thing, right? The Web API is A, gigantic and B, a moving target. Standardizing anything around that is a challenge, right?

[0:33:06] KB: You were more generous than I was. I was going to say, a fool's errand.

[0:33:10] AR: Yeah. You have to go through the pain of trying to find a nice mapping to something that makes sense in WebAssembly and possibly for more than just one language that wants to interface with it.

[0:33:22] KB: That was the thing that I was going to ask us is, now that you have a relatively flexible type system, is there actually a substantial difference in terms of the interfaces per language? WebAssembly leaves a lot of that to the language compiler to it.

[0:33:39] AR: Yeah. There's still no automatic interoperability between languages, right? That is very intentional, because that in practice never works. Because even the smallest impedance mismatches, basically, usually break everything. It was very much an axiom in WebAssembly's design that we not try to solve this problem, because it's essentially unsolvable, at least on that level. Trying to solve, this is what the component model tries to do, right? That defines a somewhat more high-level language agnostic type layer, which is more high level as well than GC types we have in WebAssembly now, because I mentioned that these are very low level, right? They don't know what an object is, or anything like that. They just talk about all the representations really.

The component model has more high-level types and they are particularly designed for multiple language to be able to map to them and talk through these with each other, right? That would be the preferred way to also define maybe a Web API for WebAssembly. On this level of type system that everybody knows already, at least if they want to implement against the component model, they already know how to talk to and have mapped to their language.

[0:34:54] KB: Let's maybe step back a moment and describe the component model. This is another WebAssembly spec, or what exactly is it?

[0:35:03] AR: This is ongoing work still, and at some point, it might be a different spec, but we're not quite there yet. This is trying to define a separate layer on top of WebAssembly itself, which is a more advanced module system, if you want. That's one view of it. Plus, a more high-level language agnostic type language to describe interfaces, right? With that, you can basically specify how you bundle WebAssembly modules. You can define how they instantiate each other probably, and things like that. Then there is this closely connected to that, this Wasm effort, the WebAssembly systems interface which tries to define a whole set of "standard" interfaces with respect to this component model, so that operating system abstractions, for example, so you can use them.

There are different ways to implement these interfaces, either in WebAssembly engine implements natively, so you have direct access to your host environment, or they could be virtualized in other ways. The component model is pretty rich in allowing various ways of implementing interfaces and plugging things together for virtualization and stuff like that. Yeah, so that is ongoing work still. It's a very complicated problem to solve, as you can imagine, and many people try to solve that problem before, and there have been various technologies that don't have the best reputation historically.

I'm not much involved in that, other than talking to the people doing most of the work. But we are very careful to not try to repeat the mistakes, have a more constrained system and take advantage of some of the inherent design advantages that WebAssembly has the strong sandboxing and all that, and the strict portability. It tries not to solve all the problems in the world at once. It makes some informed, opinionated choices, like when modules of different languages talk to each other, it's all a shared nothing concurrency model. So, you can't really pass pointers across the boundary, because that introduces all sorts of extra problems that you want to avoid.

That might also mean that it doesn't solve everybody's problem, but at least it solves a certain set of problems in that space, and from there, we can maybe get more experience about other points in the space after that.

[0:37:35] KB: Got it. To play back to make sure that I understand, right now in the WebAssembly world, if you had different language implementations of different modules, there's no guarantee at all that they're going to be able to talk to that. That's basically on the developer to say, here's the defined interface that you have to match and go. Probably, most people are developing these things as a pluggable piece, either all built in one language, or they're owning the whole system that's interacting. What the component model is understanding correctly is saying, let's define a higher-level interface that is true across languages, so that you could build, essentially, libraries, or modules that can interface with each other, without having to know the whole system front to end.

[0:38:18] AR: Yeah. I mean, the situation with bare WebAssembly is like with hardware, right? You have your hardware types and that's it. Then if you want to have different compilers be able to talk to each other, or to talk to an operating system, or some other kind of environment thing, you need an ABI, right? An application binary interface. That is something you can well define on top of WebAssembly. So far nobody has done that, although a few weeks ago when we had the last face-to-face meeting, I was actually talking about defining a CABI for WebAssembly, which is the main one that everybody wants, so you at least know how to talk to a CFFI, or something and that is standardized across multiple compilers.

The component model you can view as an ABI on steroids, if you want. Because there's a kind of, it's not a fixed ABI. You have a glue layer in between. You can define how you interface to that. But yeah, other than that, that's exactly what it is.

[0:39:17] KB: Well, and this starts to, I think, take you out of the world of WebAssembly is a way to build singular web applications, and into a world where you say, okay, WebAssembly might be a generalized operating system, or virtual machine that you can target for different reasons. Let's maybe actually divert over here. We've talked a lot about web use cases for WebAssembly. What are some of the non-web use cases that people are wanting to use this for?

[0:39:46] AR: Oh, there are many web use cases at this point. Actually, the web is just one among many at this point, and which we hoped for from the beginning, and it worked out better than we expected. I mean, we made very sure that the WebAssembly design and also the spec is not at all tied to the web, or JavaScript for that matter, right? The web in the name was more like a marketing thing, if you want. It's really intended to be like this, maybe a bit hyperbole universal VM, if you want, right?

What are the use cases? People are using it for edge computing a lot. Fastly is the biggest company doing that. They add everything on WebAssembly. There are lots of use cases in embedded space, which confused me for a while, because why would they take this overhead when they have small CPUs anyway? Well, it turns out, the big feature for them is portability. We have people from Siemens, for example, in the working group who explain that we have all these embedded devices and there are new processors every year, or month, or whatever and it's a pain in the ass every time you have to rebuild your entire tool chain.

Now, they don't want to do that anymore. They want to have a WebAssembly tool chain, then basically just switch out the back end forth their CPU directly. They have this very different use case from the web, because first of all, they have much smaller systems, so they need a much leaner runtime, ideally zero size, if you want. They also want to do ahead-of-time compilation. They don't want to do JIT compilation, like you do on the web, right? They have this whole tool chain, but in the end, they compile to the actual hardware they want.

Of course, WebAssembly was designed with AOT in mind as well. We always wanted that to be an option. We also took a lot of care to not shut the door on that. There are other kind of optimizers that take benefit of that, but yeah, actually seeing that happening for real is also interesting. There are more fancy use cases like, in AI, where you use WebAssembly to control pruning in some search, or whatever, so you make some things programmable, but in a portable way, right? With a very universal language, like you script your pruning algorithm that way. I think Microsoft did something like that.

There are blockchains who use it as their execution platform and one of the features they are particularly interested in besides the sandboxing is the well-defined deterministic semantics, because if that's the code you execute on a blockchain, I mean, blockchain is a whole hype word, right? But what matters there technically is you do replicated computation and then you rely on consensus between the different replica to deliver the same result. Of course, that won't fly if you have under-defined, under-specified behavior all over the place, because then, everybody will disagree all the time, right?

WebAssembly is to a large extent, fully specified and fully deterministic. With 3.0, we even have a deterministic profile now, which is specifying if you really want to be fully deterministic, this is what it means, so everybody has a clear picture of what that means and can depend on that. So, you always get the same result.

[0:43:19] KB: Let me actually dig in a little bit on this, because I think naively, I tend to think of most traditional programming environments as being deterministic. As contrasted to now, we do all this stuff with machine learning and LLMs and that's where I think of, oh, that's non-deterministic.

[0:43:33] AR: Yeah, okay. I should explain here that deterministic is used here in more an abstract way. We say that language semantics is non-deterministic if it is maybe specified, but allows multiple different behaviors. Then, any particular execution, or any particular implementation, you can pick one of these. But the point being, if you run on different hardware, or maybe under different hardware constraints, then you could get these different behaviors, and that's what you don't want. The way you model that semantically is usually as non-deterministic. So, you get a random pick of the allowed -

[0:44:13] KB: Got it. Got it. Got it. This is places where essentially, the language spec does not fully pin everything down. So, different implementations of that on hardware, or just different compilers, or whatever, can make different choices. It may be deterministic within a single environment, but across environments, it's non-deterministic.

[0:44:29] AR: Yeah. Even within a single one, it sometimes might not be, you never really know. WebAssembly 1.0 was very neat in that regard. There was only one little source of non-determinism, which was with floating point arithmetic. If they generate NANs, then unfortunately, the IEEE standard for floats failed to define what the bit patterns are for NANs, and all the hardware vendors completely disagree on what they do, and even how they interpret these, right? There are funny things like, IEEE defining a quiet bit, or a signaling bit on NANs, but it doesn't define what way it's to be interpreted, whether one is signaling, or zero. Surely, hardware disagrees on that choice, right? Silly things like that.

Initially, we try to even lock that down and say, okay, if your hardware's behavior diverges from what we specify, you have to normalize all your NANs. It turns out, that is very expensive. You don't really want to pay the cost, because then all float computations get more expensive. Although, that's what, for example, blockchain implementations do exactly, right? They do that. It's not that bad. But you pay a certain price for that. That is the thing why we have non-determinism on the spec level.

Of course, you also have natural non-determinism once you start talking about things like threats, because that's essential in the concurrency model that you have non-determinism. That is more intentional non-determinism. That's really core to what the feature is about. But all the other are accidental non-determinisms that are just really under specifications, because we can do better. The less we have of that, the better.

[0:46:15] KB: Yeah. Well, and this is a level of care, or if you're on the other side, like pedanticism, that most developers never have to worry about. But if you're a language designer, all the way down. Okay, so we talked about some of those use cases. We talked about how hardware embedded folks might be doing this to simplify their tool chain, blockchain you talked about determinism. What are some of the other drivers for non-web use cases to adopt WebAssembly as their target?

[0:46:42] AR: I think it's usually two things. One is portability, and the other is the sandboxing model. Of course, the fact that it's pretty universal is as well, right? It's like Turing complete. You can compile everything into it. It's not constrained in any particular way. Those are the three reasons why it's a good fit for various places where you want to embed some form of compute, without tying yourself to whatever the specific scripting language, or open security holds, or whatever, right? That's how you make use of it. Yeah. It's all fad, which sometimes is a bonus as well.

[0:47:21] KB: Yeah. I think the example that comes to mind for the closest to that universal virtual machine model that anyone's come before, I think, is the JVM and all the different JVM targeting languages. What are the factors that might cause somebody to move from a JVM environment to a WebAssembly environment? Are they exactly what we just said? Is there some other nuance we're missing?

[0:47:42] AR: Yeah. I mean, the JVM is interesting, because that is really a VM that was designed for Java, right? It's basically an abstract syntax for Java, if you want. It has all the Java object model built in, which is pretty heavyweight, including all the reflection stuff, and so on. There are other things that doesn't have built in, right? You can't do. When you compile to JVM and many languages have done that, usually, if you don't happen to be very much exactly like Java, then it's not going to be very fast. There are things like, so for example, with the GC extension now, one thing we have that other VMs don't have is we have tagged pointers. I don't know if what tagged pointers are, but if you have a garbage collection runtime, then usually, the garbage collector has to know what a pointer is, and sometimes you want to unbox integers, so that you don't have to put everything into a box and allocate. You usually use pointer tagging, which is you take a bit of every pointer that marks it as an integer.

Then you can just use an integer as if it was a pointer and the GC knows about this bit and just follows the real points. That's something we actually provide directly as a WebAssembly feature. Whereas in other VMs, you don't have that. There are many language runtimes that make heavy use of such a feature to avoid boxing all their small types, right? Yeah, do things like polymorphism much more efficiently. With a JVM, yeah, you pay the price for the heavyweight object model. So, if you're compiling a language that, for example, has lots of very small allocations, which is, for example, the characteristics of many functional language, then that would not be ideal for you, because, yeah, your characteristics run against what it's optimized for.

Then, there are other things like, I don't know, if you want to do closures and stuff that's also very costly in the JVM, because it's not native to it and you have to encode it in heavyweight ways and all these things, right? You don't have access to the low level as much as you have on WebAssembly. You have to jump through more hoops in a way.

[0:50:04] KB: Let's now look forward a little bit. What does the next generation in WebAssembly look like? What can't WebAssembly do well right now? We mentioned a little bit the component model. That is a big area of exploration and focus. You also had a throwaway comment about continuations. Maybe we can dive into that. Yeah, what's coming down the road?

[0:50:26] AR: Right. In terms of bigger features, I mean, there are very small features in the making, but the two big features I would say that are still missing from core WebAssembly, so not the component model, is threads. It's the one which is close to the finishing line, finally, I hope. The other is what is used to be known as stack switching, or which turns out, we model it as this form of delimited continuations. The reason this feature is important, because this is what allows you to compile at least efficiently all these control abstractions that modern languages have. Control abstractions are things like, I don't know, async await, generators, green threads, you name it, right? All these things where you have non-local control flow.

Usually, these are implemented, or often they are implemented some way or the other by having multiple stacks that you switch between. When you're in a system like WebAssembly where you have to be type safe and where also, the actual stacks are encapsulated, you can't really get to them for safety security reasons. Then it turns out, that there is a neat abstraction that is already well known in the programming language field for decades, which are continuations, or delimited continuations, so that is how we model them.

Basically, continuation is a new stack if you want. You have a way of creating a new one, and then you can resume their computation there, which basically switches over to that stack. Then that computation can suspend, which switches you back to the point where you resumed, very roughly speaking. You can have as many of those as you want.

[0:52:14] KB: I assume that since those control flow features that you mentioned are in a number of languages, including some that I believe already target WebAssembly, how are they supported today? What will this take you forward to?

[0:52:27] AR: Yeah. There are many ways to do that. The most powerful way where you can do all that is you do a global program transformation called CPS. Continuation passing style, where for every function, you have an additional parameter, which tells it where to continue, instead of returning. That is also a well-known technique, but you can imagine that is pretty expensive. Some languages actually do that. Others use tricks like, I don't know, trampolining through JavaScript, for example. That is, by the way, also how C++ implemented exceptions before WebAssembly 3.0. It had to call into JavaScript to create an exception handler, then call back into WebAssembly to actually run its body, and so on and so forth, which is also crazy. Of course, it only works when you have JavaScript. That's one technique.

Yeah. So, once you have exceptions, you can do trampolining, which is you just build up your call stack, and eventually, you throw back to a scheduler that then somehow has an idea of how to build up the call stack again, if it has to, and there are various techniques like that. They all have in common that there are tend to be brittle and costly to some extent, and not very modular. One of the features our proposal has, which is something that came out of academia in the last 15, 20 years, it's based on this idea of effect handlers, which is essentially, the simplest way to understand is the generalization of exception handlers that allow you to resume at the point where you threw. You can resume at any later time.

The thing, the continuation is essentially, your object that allows you to resume. What effect handlers give you, as opposed to these other techniques is that you can compose these handlers in a nice way, and without them interfering with each other. Whereas, if you have to manually do these transformations, and you have more than one of these control effects in your language, then you have to do all of them together. With effect handlers, you can do them in your modular ways separate from each other. They don't have to know much about each other. You can just suspend over another handler, essentially. That is very nice. For example, if you want to do both threads, so green threads, virtual threads, and say, async await, or generators, then one is in as much smaller scope, but you still want to be able to suspend the entire thread that runs the generator directly going to the outer one, without the generator logic having to know anything about that.

[0:55:11] KB: I think it's really interesting. We've had a number of conversations with different folks about advances in the programming language world. We have these newer languages that are building on these capabilities that have really just happened in the last 10, 15 years. If you look forward even further, so that's what's on the horizon now, what else is in the back of your head of, "We're going to want to get there eventually. We're going to need to get there." What do you want to enable with WebAssembly?

[0:55:43] AR: Well, yeah. I mean, we already got quite far, as far as I'm concerned. One thing I would like to see is WebAssembly having more built-in support in standard operating systems, for example. It's in every browser, but why does, I don't know, macOS, or Linux, or Windows not support it out of the box, so that you can actually run just desktop applications using WebAssembly, so it becomes even more of a portable format.

The extreme point of that would be to put it into hardware. I keep hearing people talking about that and considering the ideas so far, as far as I'm aware, nobody has actually tried to do that for real. There are some hurdles you have to get over. In principle, it should be possible, and that might also be very interesting. Because it's a virtual CPU ultimately, and at least some feature subset of it should be possible to implement more or less directly in hardware.

[0:56:44] KB: What you're saying is the next ARM spec, or something like that is directly targeting WebAssembly?

[0:56:51] AR: Well, not the ARM spec, but there might be a Wasm processor, a Wasm CPU of kinds, and maybe a custom ship, or whatever, or have PGA, or something that people who want to run that in certain environments can take advantage of. I don't know. Something like that.

[0:57:08] KB: Love it. Well, we're coming close to the end of our time here. Is there anything we haven't talked about yet that you think would be good to leave our listeners with?

[0:57:17] AR: Yeah. I mean, one thing I could briefly mention is because I'm heavily involved with that side is the specification side of things with WebAssembly. One thing we're particularly proud of is that not just don't we have undefined behavior at all, right? We have everything formally, mathematically specified and verified. We have machine verified proofs that we don't have undefined behavior. There's a lot of work going into that and extending the infrastructure for that now with various research teams that hook it up to various theory improvements to do even more fancy things of theorem proving reasoning about programs and things like that, which I think is a whole new level for an industrial programming language that nobody has done before either. That's something I'm particularly interested in as well, because that ties back to my academic self, half of myself that likes that stuff, too.

I think it's very valuable. I mean, the reason that WebAssembly has such a clean and fairly safe design is because we design, or one of the reasons at least, I would claim is that we design it with formalization hand in hand, right? We didn't just build this artifact and then try to make sense of it after the fact. We really did it at the same time. That's when you have to be honest with yourself, when you're designing something and you have to formalize it and the formalization becomes hard, then probably the design isn't quite right. It's a nice feedback loop in the similar way that implementation is a nice feedback loop. You want to do all these things in parallel and feedback into the design, so both for semantics and performance and these things, which I love people to explore more also, for other languages.

That would give us also, the other nice thing about that is that this gives a way of actually doing verification in depth, right? Because we now have verified hardware, we have a verified engine like WebAssembly and maybe we can now build more verified compilers, and then you can actually verify the entire software stack at some point. I would love to see that kind of thing, because frankly, the state of the art with our craft is still like, these things to be desired.

[0:59:42] KB: Not to be too much on the bubble area, but if we're using non-deterministic machines to generate code, which is what more and more people are doing with LLM-based programming, having a way to deterministically validate and verify it after the fact is going to be ever more important.

[1:00:00] AR: Right. Yeah, absolutely. Yeah. The more code we let be generated by dubious sources.

[1:00:09] KB: I mean, humans are pretty dubious as well, I'll say. I've read a lot of human written code that feels pretty dubious to me.

[1:00:17] AR: Fair enough. Yeah. Yeah. But for some reason, people are less willing to trust humans when writing code than they are willing to trust AI when it's writing code. There was just a study coming out. I just read about today. That is a bit scary. When you're pair programming with a human partner, then you usually question everything he's doing. When you do that with an AI, not so much, although it's at least as failable.

[1:00:45] KB: Formal validation of the outputs.

[1:00:47] AR: Yeah.

[1:00:47] KB: Definitely would be nice.

[1:00:49] AR: But that's very much an open research area as well, because it's a very hard problem in general.

[1:00:55] KB: Awesome. Well, thank you, and let's wrap.

[1:00:58] AR: Thank you very much.

[END]