EPISODE 1771

[INTRODUCTION]

[0:00:00] ANNOUNCER: CodeSandbox was founded in 2017 and provides cloud-based development environments along with other features. It's quickly become one of the most prominent cloud development platforms. Ives van Hoorne is a Co-Founder at CodeSandbox. He joins the show to talk about the platform.

This episode is hosted by Josh Goldberg, an independent full-time open-source developer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ES Slint, the tooling that enables ES Slint and Prettier to run on TypeScript code. Josh is also the author of the O'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and a live code streamer on Twitch. Find Josh on Blue Sky, Mastodon, Twitter, Twitch, YouTube, and .com as Joshua K. Goldberg.

[INTERVIEW]

[0:01:01] JG: Ives, welcome to Software Engineering Daily. How's it going?

[0:01:04] IvH: It's going great. Thanks for having me. 

[0:01:06] JG: Well, thanks for coming on. We're really excited to have you. I personally used your product quite a lot from job interview, sandboxes, to demos. Could you give us a brief introduction to what is CodeSandbox?

[0:01:16] IvH: Yeah. CodeSandbox simply said, is an online development environment. You can start a new web development project in CodeSandbox, and you can completely work in your browser. I tend to compare it with Google Docs and Microsoft Word, where if you're writing a document in Microsoft Word and you write something on your computer, but if you want to share it, that becomes harder. And so, people have been using Google Docs more, where they can just share a link to a website, where they write together in an environment. We wanted to build the same thing with CodeSandbox.

[0:01:50] JG: Fantastic. Before we dive into that in the future of code sharing, I want to dial back a little bit and talk about you as a developer, as a person. How did you first get into programming?

[0:02:00] IvH: That's a long, long time ago. I initially started with coding, not because I wanted to learn coding, but because of - I had a need. I think I was about 10-years-old, or 11. I'm not sure how old I was, but a friend of mine and I, we have a secret language. We tended to just write secret letters to each other in the class,, so we would write secret characters and then the other would have to translate it. That was a lot of fun. But I wanted to go faster. That is when I started looking into whether it would be possible to create a program. That's my first program, a Visual Basic program, where you could - It was really just two text boxes, where if you put something in the left text box, it would translate it and put the solution, or the translation in the right text box. It would also work the other way around.

We had a Dutch Facebook at the time, and we would send public messages to each other with this secret language. That was my first interaction with coding. It was quite challenging. To be honest, after that, I didn't code for a long time. Only when I started to look into gaming and mods, I started to do coding again. But that was my first interaction.

[0:03:17] JG: There's a humor point that your first form of coding was some input on the left, and some output on the right, and you're still doing that decades later.

[0:03:28] IvH: Yeah, that is ultimately, it's the same thing, kind of. Yeah, you have something on the left, and it transforms it and puts it on the right. I guess, you could say the same thing about CodeSandbox, which is very funny. We have a lot of fancy things in CodeSandbox, but the core functionality and the most important functionality remains that you have a code editor on the left and a preview on the right that shows what the code is doing. That's the core functionality.

[0:03:55] JG: How did that core functionality come to be, or how did you come to create the product in the first place?

[0:04:00] IvH: I stopped coding for a while. But later on, when I was, I think, 17, I had done a lot of graphic design for a company, and I started to realize that I liked graphic design, but I didn't like doing graphic design for other people, because they wanted me to design things that I didn't agree with. Things like, "Oh, can you make this fire yellow on this purple background?" I was 17 and a bit naive, and I thought, well, graphic design is not for me, because it's so subjective. I started to move to coding thinking, oh, it's like solving puzzles, and there's only one solution to a puzzle. There's no discussion, or there is no subjectiveness in coding. But I was wrong in hindsight, but I still do still enjoy it. But I started to learn web development.

Initially, I created a portfolio website, and later on, I read in a newspaper, a local newspaper of my little village in the Netherlands, that there was a very cool new startup that was growing extremely fast. I was thinking, I want to join this startup, because there was not much happening in this village. After high school, I asked if I could join them for vacation work, so that I would work over the holidays for them. They essentially said - well, initially they ignored me. They didn't want to. Well, the recruiter was a bit confused, an 18-year-old person would just ask to work there as a developer. But later on, they called me back after I called them a couple of times. They said, "Yeah, you can work here, but you have to work at least for a year. Would it be possible for you to take a gap year?"

I started working there, and everything was in Ruby on Rails. Around that time, a new technology called React became more and more popular. I was intrigued by React. I started building more little pages in React, and I was thinking, "Wow, our Ruby on Rails front end feels a bit antiquated compared to how fast a single page application of React is." I guess, again, I was a bit naive, and I started to convert more and more Ruby on Rails pages into React. It's funny, because when I did that conversion, at some point, we wanted to test it and we put it live. Then the marketing department came to us, because they were distraught. Suddenly, 50% of their analytics were gone, because I didn't think of all those other business things implementing analytics.

Anyway, I'm going on a bit of a tangent, but at some point, I realized that when I was working with my co-workers on React code, it was very hard to share work with each other. It was like, whenever I was on vacation at some point, and I got questions from my co-workers about a piece of code in React Router, and they just sent me snippets on Slack, and I had to decipher from my phone what was going on and what was going wrong, and what the error was. I didn't have a JavaScript interpreter in my own head, so that's when I started thinking that it would be very cool if they could just send me running codes.

At the time, Figma became more popular, Google Docs became the default for a lot of people. I started to think, would it be possible to have a code environment in the browser? At the time, I was just writing down all my ideas for if I ever want to start a startup in the future. I wrote this idea down, didn't do much with it, started going to university after my gap year. Then at the university, initially, I had a lot of fun drinking beer, but that got boring after a while. We started to get lectures about objects-oriented programming in Java, and I was already fully in the React world by then. I was like, why do I have to go through all this again? That's when I started to work on a side project, and I just looked at my list of ideas, pick the top one, the latest one, which was the online web editor, and started to make the first design and sketch.

Then my friend, Buzz, joined, and we started working on it more and more and more. Then in April 2nd, not April 1st, we released the first version of CodeSandbox. That was initially why it got started.

[0:08:17] JG: This was April 2nd, 2017?

[0:08:19] IvH: Yeah, that's right.

[0:08:21] JG: That's been over seven years. What was the tech platform that you used in React land at first back in 2017?

[0:08:29] IvH: Oof, very different. Everything was in JavaScript, not even in TypeScript. We did use flow types to type things with comments and everything. That was very, very interesting, too. The base application was in Create React then. The backend was written in Elixir, using the Phoenix framework. Because Elixir was this cool, new thing that looks a lot like Ruby and Ruby on Rails, but was fully functional. That was an interesting learning project. It turns out that it scales really, really well. Then for the database, we used PostgreSQL. For some caches, we used Redis.

Ultimately, the stack was Create React the front-end, Elixir backend, PostgreSQL database, Redis second in memory database. I deployed all of this to a VPS on Volter, like $20 per month VPS, 2 gigabytes of RAM. It was quite surprising how well that's scaled. It's scaled for a year, I would say. At some point, we got to the 500,000 monthly users. It was still running on this $20 VPS in Volter. After that, we did move to Kubernetes, and we moved the deployment to GCP, to Google Cloud platform. Because the idea was that we can also do easier migrations that way, and easier scaling.

It was intriguing to think that a solution as simple as this works so well. I think the simplicity ultimately helped with scaling it. I remember when building the first version of CodeSandbox, that I was very worried about saving files. If someone creates a sandbox and they press fork, for example, and they get their own version, then the other person gets their own version of a sandbox. How would we scale all of that with all those files? Should we use something like S3? Should we use something like Dropbox? There was a lot of questions about how to store those files.

Ultimately, after a month of very, very advanced thinking of should we use Git and all those things, I decided in that moment to store everything in PostgreSQL. Every file would just be a row in a PostgreSQL table. When you press fork, we would just do a bunch of selects and a bunch of inserts in PostgreSQL to copy all the files. No deduplication, or anything like that. The funny thing is we still use that today, that system. It never hit limits. We now have, I think, we have over 80 million sandboxes, over 500 million files stored in a PostgreSQL database. This query still is within 20 milliseconds, a query, a sandbox can be loaded within 100 milliseconds, just doing a bunch of selects.

Only thing that we don't store in the database are binary files. That was a bit too far. Those are stored in a Google Cloud bucket. Then we store a link inside the database to that file. It was a reminder of me that often, the simple solution works the best, either because it's just so simple that there are less race conditions, or there's less things that can go wrong, but also, because it's much easier to understand how it works. Yeah, that was the initial.

[0:11:54] JG: Before we dive into more on how that works, I want to take a moment to emphasize that point that you have half of a billion file entries in your PostgreSQL database and you're still able to load the core part of your site that involves potentially many file queries in a tenth of a second. That's an incredible scaling and performance feat, no?

[0:12:13] IvH: Yeah. But I would attribute it all to PostgreSQL. PostgreSQL has exceeded my expectations time and time again. If you have a good index on multiple columns, then the performance is incredible and the scaling is also incredible. I would choose PostgreSQL for any database right now. The only exception would be for things like, storing a tremendous amount of data that is inherently tied to timing, so time series data. Then I would look at something like, Clickhouse. But PostgreSQL is for everything else, an incredible solution.

[0:12:54] JG: We'll see if we can get that quote on their homepage. Let's continue that journey. On the very bottom, or back of a CodeSandbox, you have files stored in a PostgreSQL database, and it also links to binary large objects stored in Google Cloud. What is on top of that? How are those retrieved, or what's the system around them?

[0:13:13] IvH: Yeah. I think, the easiest way to describe how everything is retrieved if we look at the oldest version of CodeSandbox, like the initial version of CodeSandbox, because otherwise we need to think about all the permissions, billing and stuff that comes now in between. The simplest, the first version of CodeSandbox, whenever you would retrieve a sandbox, you would call an API call on our Elixir server. The Elixir server would go through a couple of checks, it would check if you have access to that certain sandbox, it would get your user from the cookie. Then it would run a query on the database and that query is huge, and it has 20 joins. That's one of the only queries where it's handwritten, where it's not the query that is generated by an ORM.

That query, it looks at our module stable, that is where all the files are stored, and it looks at our directory stable, which is how the files are linked to the different directories. It looks at our sandbox table. It gets from the sandbox table, it gets the sandbox, then it gets from the directory stable, it gets all the directories that are related to the sandbox, and then from the module stable, it gets all the files that have the same sandbox ID. Then based on all that information, it generates a JSON blurb that contains all the files of the sandbox itself and returns that. You could argue that that's unscalable in the sense of, it wouldn't work for huge projects and that is completely right.

CodeSandbox was specifically built for creating prototypes, creating small projects. That was the initial use case. There was a limit of 500 files for a single sandbox. If there's a limit of 500 files, then it's fine to return the whole contents of the sandbox. Now at this point with CodeSandbox, we also have a second type of project, which we call Devboxes. We have sandboxes for prototyping and Devboxes for development. For Devboxes, we have a more, I would say, sophisticated way of retrieving files. You can lazily retrieve files, so that you don't have to download the whole sandbox just to see what's going on. hat is in a nutshell how it works.

Then we have Redis in between for simple, small things, like caches, but also tracking page views, so that if you access a sandbox twice in the same hour that it's seen as one page view instead of two. But yeah, that's how we retrieve sandboxes.

[0:15:36] JG: That Devboxes concept, that wasn't there in the first versions of CodeSandbox. When did that get added in?

[0:15:42] IvH: Devboxes are now, I would say, approximately two and a half years old, maybe three years old even. When CodeSandbox launched, it was initially not very popular. I put it on Twitter. I had 60 followers on Twitter. Most of my followers were high school friends, so I got three likes. We started to become much more proactive with talking about CodeSandbox. I started to read blog posts about how it worked, started to directly talk to people if they created an account to just get feedback. Or with the idea at some point, Paul Graham, he said that it's better to have 100 fans than 100,000 people that like you. With that idea, we've tried to get to 100 fans by doing a lot of unscalable things.

CodeSandbox started growing and growing and growing, and people started to use it for things that we didn't imagine in the first place. It was initially built for the specific case that I had with at work, where you wanted to ask a question and you wanted to have a live example for that question. People started to use it for other things, things like job interviews, bug reports, documentation, also workshops, where people learn how to code. People started to use it to build new projects. They started to work on a new website, a portfolio website, for example, or they started to work on, for example, a new blog, or even some people started a new startup.

For example, there is one that I've always found funny to - well, funny, it's more like, proud. There is this whiteboarding tool called Excalidraw, I use it a lot. The interesting thing is the initial version of Excalidraw, it was built on CodeSandbox when it was called Excaliber, and it was sandboxes shared over Twitter. People wanted to build real things with CodeSandbox. They wanted to build their portfolio website, things that they ultimately wanted to deploy. While CodeSandbox worked really well for the things that were small, like job interviews, or examples, it didn't really work for the big projects, because of our 500 file limits. That's when we started to look into, if we could create the same experience that we have with CodeSandbox for the smaller projects, but then for big projects.

Still, you should have the capability to share a link with someone and they can see the running code, they can see everything, how it works, and they can press fork to get their own version. That is what Devboxes has become. It was a rewrite of CodeSandbox, because the core system, the file system changed underneath it. We normally, sandboxes all run in the browser, because they can run small projects, but Devboxes, they run on the server. We built a version of CodeSandbox that was really meant for full development.

[0:18:25] JG: How is it different, just thinking in the database context, how is it different that Devboxes retrieve stuff versus the original layout?

[0:18:35] IvH: In the case of Devboxes, we run a VM to run that project, a virtual machine. Essentially, a small server that runs the project itself. Inside that server, we run a process that can read from the file system. Now, when you open a devbox, you don't connect to our API server to get the files. Instead, you connect via a WebSocket connection to a little server that runs inside that VM. Then the editor can ask things. It can ask like, can you give me the contents of the file under the file path/project/hello.txt. Then it will return it. The whole API server was still there, but it was only there for validation, for authentication. But ultimately, the connection for getting the files and understanding what's going on within the project would be done by directly connecting to the server, to the VM itself.

[0:19:31] JG: That's not as scalable as just the original sandboxes. Everything runs on the client. We're just a file and API lookup.

[0:19:39] IvH: Yeah. It's not as scalable. Starting a VM for every user, for every project, it's a real challenge. The last, at least the last four years, I've been really, really deep in learning how to build efficient infrastructure. We went through multiple iterations of finding ways to run VMs efficiently. Initially, we tried Kubernetes, we tried Docker containers. We felt like that was too slow. In 2021, I found a project called Firecracker. It was created by the Amazon team, by AWS team, because they use it to run AWS Lambda and AWS Fargate. Really interesting thing about Firecracker is that it's a VM that can run code, but at any point in time, you can say, pause this VM and it will literally just halt. It will not do any execution anymore. Then you can say, write your memory now to disk.

Then later on, a day later, for example, you can say, create a new VM exactly from this memory that you wrote to disk and it will continue exactly where it left off. It could be in the middle of an operation where it's calculating Fibonacci sequence, for example, and it will just continue. It doesn't matter. That was so interesting. I would say, it's very similar to, if you would close your laptop and you would open it a day later, it will also just continue. Even if you have a NES server running and it's in the middle of a compilation, you can close your laptop and a day later you can open it and it will continue exactly where it left off.

The interesting thing about this is one of the things that people felt with our initial version of CodeSandbox with the server was that it was slow. Because when you would open a project which hasn't opened in a long time, then you would have to wait for the create React step server to start. Maybe you need to run MVM install. It could take a long time before you can actually see a preview. Also, when you press fork, then we would have to create a copy of that file system and we would have to do that same process again. With this approach, we solved both of those problems at the same time. Because whenever someone would go away from a VM, we would just pause it and we would save the memory to disk.

Then when someone later on, like a few days later would open that VM, we would be able to resume the VM exactly from where it stopped, from when it was paused, and it would resume in one second. That was problem one solved. That also helped a ton with scaling, because we now have a rule, like if someone hasn't looked at a VM for five minutes, then we already hibernated, because people won't notice if we hibernate it, because the resuming is so seamless. The second thing that's done now is when someone presses fork, we also create a snapshot of the original VM and we use that snapshot to resume the new VM that was created. When you press fork, you can, we create an exact copy because it will continue exactly where the last VM left off.

Later on, we did optimizations where VMs even share a memory. If you have a VM that started from a snapshot, and someone presses fork, then that new VM will share the memory of the old VM. If two VMs 2 gigabytes of memory, it could be that the total usage of memory is 2 gigabytes, because they refer to the same shared memory. Those little tricks made it possible to scale VMs.

It is the most challenging thing I've worked on, because it's much more challenging than sandboxes, because with sandboxes, we would run everything in the browser, so we would not have to run servers to run the code of the user. The only thing that we had to provide for files and all the execution was on the user part to do. In this case, we had to create a fast service that can run code, but also secure, because people, they shouldn't be able to break out of that environment and were literally giving remote code execution as a service with CodeSandbox.

[0:23:43] JG: That's such a hard problem. When I was at Code Academy, we had issues with people doing incredible amounts of compute and we had to have all these hacks and cool gotchas around say, crypto Bitcoin mining. You have not only that, you also have intentionally the ability for people to call out to the network on the server. How on earth do you make your boxes secure?

[0:24:02] IvH: Yeah, that is challenging. The boxes themselves, they are pretty secure in the sense of, they are very secure. They ultimately, they use the same techniques that AWS, Lambda uses, and AWS Fargate uses. Every VM has its own Unix user. We use a jailer to make sure that everything is in its own environment, but people can still abuse. For example, someone could run a crypto miner on server. Someone could create an account CodeSandbox, create 20 VMs and that will go fast. We can spin up those 20 VMs very quickly, which makes it very easy for them to do, and they can start mining crypto. Crypto miners are the most frustrating people. They are very creative. They have a lot of time on their hands.

The thing that we do right now is we have a detection heuristic that runs every minute inside of VM to detect if a VM is running. I have to say right now, it's a lot of if statements and else statements based on existing crypto miners and their behavior that we've seen before. We're now experimenting with training a little neural network that automatically can detect crypto behavior based on network calls and the process behavior. It's a mouse game. The other problem that we've had a lot was phishing. People were using CodeSandbox a lot for phishing. They were using it to create fake bank sign-in pages. They were using it to create fake Microsoft login pages. We were fighting that for a long time.

We also saw disruption in CodeSandbox as a service, because of it, because for example, Google could for example, block the whole CSV of app, which is our preview domain. They could just block the whole domain, because of an automated check that said, "Oh, there were two phishing sites on this domain. That whole domain should be blocked." Then we had a downtime of three hours because of that. Then we had to fall back to other domains. Or there is still an ongoing issue where Turkey, some ISPs in Turkey, they have blocked CodeSandbox. That's the preview domain. Just because they saw a couple of phishing pages.

Initially, we also applied AI to this actually, to we created screenshots of all these public sandboxes and tried to determine whether it was a phishing sandbox or not. Then we would show warnings to the user, or we would proactively block these sandboxes. Nowadays, we have given up the fight in the sense of whenever someone visits a CodeSandbox preview for the first time, and it's a standalone preview, so they open it not from within the editor, but they open it from, for example, an email. Then we will show initially a big interstitial saying, "Watch out. This is a CodeSandbox preview. It's not a bank login page, or it's used for development purposes." Then they have to press a button saying, "I understand," and then they get to the preview.

It does affect the experience of CodeSandbox. When you share a preview with someone to a real thing, then they still have to go through this interstitial before they can actually access the page. After deploying this, the amount of phishing pages on CodeSandbox has gone down tremendously. Probably, the phishers, they're looking for new pastures. They're looking for places that don't have something like this. We do still sometimes get emails from these services that detects phishing pages. We even automatically handle those now. We scan all our emails and if it's from a domain, an email that we know is a phishing detector, we would automatically ban a sandbox when they put a link in there.

[0:27:44] JG: Yeah, that's tough. There's no right answer here, right? What if I'm a, say, a bootcamp and I'm teaching my students how to write a full page in whatever front-end language. The example app is a login page, one that happens to look like, say, Google's login. How do you know that that's legit, versus the equivalent scammer?

[0:28:02] IvH: Yeah, those were the hardest cases. That's why we prefer to show a warning, instead of proactively banning sandboxes. At this point, it's much easier, because now we just show this interstitial for everyone. If you trust the person who has sent you that page, then you can open it. If you get this page from, I don't know, a random text message, and then you get this interstitial saying, "Don't trust any sign-in form on this page." Then that covers a lot of cases.

[0:28:30] JG: This is yet another example of you've tried the complicated approach. Say, the AI scanning to detect scams. Then the simple scalable cheap one actually, is pretty effective.

[0:28:40] IvH: Yeah, I didn't reflect on that, but you're right. This simple approach, it solved all of it. It is also the simplest approach. No fancy AI, or detection heuristics.

[0:28:51] JG: Before we move on to the client code runs the entire CodeSandbox portion of the DevStack, I have one last question on the back end. At Code Academy, I always wanted to build something where if we did detect someone who's crypto mining, we would let them and then steal their crypto coins and give them something fake in response, just to really stick it to them. Ever tried something like that that you're willing to talk about?

[0:29:15] IvH: I have fantasized about this as well. Yeah, I did think about it. Never did it. There was a case where people did not just use it for crypto, but they were also, and still are sometimes using it to watch advertisements and get money from it. It's very interesting. They start a browser inside a VM and then they have a VNC connection to it, so you can see the browser. Then they start on 20 VMs all at once, they start going to different pages where they watch advertisements and then they get money back for watching those advertisements. It's very interesting. I found this. I found one of those VMs and the VM was just public so I could open it. I saw the VNC window open in the preview.

I just started doing a lot of things inside that you can see, I opened notes back and I sent a message. I could see them at some point, looking at the sandbox and getting very confused and closing all the windows, because they felt they were crossed. That was funny.

[0:30:14] JG: Yeah, the backend team for us, I hear they used to do things to mess with people to make it not worth their while to abuse the platform. It's a great way to stop people.

[0:30:24] IvH: Similarly, when we detected crypto miners, for a while, we just throttled their sandboxes, their devboxes. Then they would feel like they would mine crypto and everything would still work, everything would still run, but they would only be running at 5% speed of what the full VM could run. That's another way to confuse them, I guess.

[0:30:46] JG: I love it. Okay, let's move closer to the front end. Let's stick with the original CodeSandboxes. For now, your PostgreSQL database is continuing to scale wonderfully. You served up the file contents to the user. What happens in the user's browser now?

[0:31:02] IvH: Yeah. When we started at CodeSanbox, we had a very low budget. Using a server was out of the question to run the code. We were students. We had a budget of $100 a month. That was the max that we could have. That was all from student loans in any case. I started to look into whether I could run Webpack in the browser. Webpack was by far the most popular bundler at the time. Initially, I got Webpack to run, but the bundle size of Webpack itself was huge. It was 8 megabytes, which was even more for 2017, that was huge. Then I started to look into more how these different pieces of code are executed. I started to try to execute the code myself, try to build a very simple version of Webpack that would run in the browser to execute the code.

In essence, it all revolves around eval. Like the JavaScript function eval, where you can give it a string and it will evaluate that code. Everyone says that you should never use eval, but that's the core functionality of CodeSandbox. That's what we build the startup around. What happens is we receive all the codes, and the first thing we do is we parse all the codes. We try to understand which files are used and which files import which other files. This results in a dependency graph, as they call it. With this graph, we transpile all the codes. You could have, for example, your source code could be TypeScript, or it could be JavaScript, but not suitable to run in the browser. We would transpile that code to a code that would be possible to run in the browser. Then later on, once everything is transpiled, we would execute that code by using eval.

These pieces of codes, they could import other files. They would do that using a require function call, the common JS way of running code. What we would do is whenever we would eval the code, we would wrap everything in a function where we provide the require as a function. We would override the require function with our own function. Whenever it would require call require, we would resolve that code and we would run eval on that code, creating this inflow loop, if you have infinite imports. That is ultimately how it works.

The logic itself is not extremely complicated. I also have given a talk, and I think I've done a blog post about how this works. I also have a sandbox that has a mini-bumper implemented. That is in a nutshell how it works. It became more advanced, because there are bigger, also big challenges. For example, how are you going to support node modules? Installing dependencies alone, everyone makes a joke that node modules are bigger than the universe. How do you make that efficient? Which is a challenge in and of itself. But the core functionality is essentially, this loop of creating the dependency graph, transpiling all those files in that dependency graph and then call them eval with the required override on the code.

[0:34:08] JG: CodeSandboxes are known for having beautiful visuals, right? How does that interact with the DOM, or the HTML page?

[0:34:16] IvH: Okay. CodeSandbox, if you are looking at the editor page, you're actually looking at two applications. You're looking at the editor itself, that's a Create React application. But the preview on the right, that's a completely different application. That is rendered inside an iframe and that iframe refers to a different entry point, so to say. That entry point is the bundler. When we, the editor downloads all the code from our PostgreSQL API server, then the editor sends that code to the iframe of the preview iframe. It would call post message to send all the files to it. The reason that we have to run everything in an iframe is twofold. One is it isolates the user application completely from our editor, so they cannot mess with our editor, but the other reason is security, because we ultimately are running user code, so they shouldn't be able to access our cookies, or our local storage, those kind of things.

The editor would call post message on the iframe, send the user code to that preview. Inside the preview, we would have the bundler sitting idle and listen for any message that comes in. When it would receive a message, it would then do this loop of executing the code and that would then ultimately fill up the preview. Whenever the user would change code, we would just send the whole bundler - like, everything from the editor back to the preview iframe again, and the preview iframe would then, the bundler in that would make a diff. It would look at the previous, the current version of the code and the new version of the code, and it would compare everything. Then for every file that changed, it would re-evaluate those files. It would just run eval on those files and its parent.

[0:36:02] JG: You're using, let me get this straight, eval iframes and window.post message.

[0:36:09] IvH: Yeah. Yeah. That's the core functionality. Yeah, it sounds counterintuitive, right?

[0:36:15] JG: Yes. Then on the inside, you're also building up a full dependency graph and doing dynamic reloads of the graph based on impact that changes from the editor. You have this incredible mixture of pre-2017 era tech with very lovely computer science concepts for efficient redeploys.

[0:36:33] IvH: Yeah. That's the interesting part. Because you're building, essentially, two things. You're building a, I would not say hardcore, but a pure developer tool like Webpack. But you're also building a UI around it, a developer experience around it. That opens up some unique opportunities, because you have the whole stack. You control the whole stack. A simple thing that we did, for example, is whenever we would get an error inside the bundler, that it could not resolve a dependency. Let's say, you start, you import lodash, but you have not installed lodash yet. We would create a specialized error message in the bundler. There would be a button saying, "Oh, we could not resolve lodash." Then there would be a suggestions that you could click, which says, install lodash. When you click that button, the bundler would post message back to the editor saying, "We need to install lodash." Then the editor would create, would show the UI of installing lodash, it would install lodash and call re-evaluate on the bundler. That is just one of those examples, where because you control the whole experience, you can create a pretty nice experience.

Another one was someone in our Discord was frustrated, because they were using CodeSandbox for a job interview. They hadn't capitalized their components, their React components. The React component was not working for some - I think it was back in the day when React components had to be capitalized to work. Because of that, they filled their interview. As a response, I created a small detection heuristic on that too, where if we detect that there's an error that's about a custom component that's not a default HTML element and React would flow that error, then we would show a suggestion like, "Have you capitalized your component?" Then we would have a button, capitalize component, and you would click it, we would update the code to capitalize the components, to catch those things. That was the most exciting to me, because it's a mix of the core developer tooling, building a bundler, but there is also a developer experience and user experience that you can refine based on what comes back from that bundler.

[0:38:44] JG: Did the person ever respond that they got a job afterwards, or did that help them?

[0:38:49] IvH: No correspondence after. I didn't implement any analytics to know how often that button was pressed. Maybe no one has seen it afterwards. Who knows?

[0:39:01] JG: Just one clarification point for technical areas. When you say install, what is installing in the client?

[0:39:07] IvH: Yeah. Installing dependencies, it has changed over time. The biggest challenge that was there was installing NPM dependencies, because NPM dependencies tend to be pretty big.  Sometimes a library, an NPM dependency could be 2 megabytes. That's not a big problem in of itself. That dependency could also say, if I have 20 other dependencies, and those dependencies could also be 2 megabytes, so only you have 80 megabytes of dependencies you have to download.

We added support for NPM dependencies by creating a separate service, an AWS Lambda service. That Lambda service, we would say, I want lodash, for example, and Lambda would install lodash, and then it would look at what files have a high probability of being required to run the dependency. It would look at the entry point. For example, lodash would say, "Oh, my entry point is in source/index." We would then look at that file and we would look at all those imports in that file. We would, again, go through the same process of creating a dependency graph. We would make sure to only include the files that were required to run the main dependency. We would send out also back as adjacent to the bundler itself.

If the user then would require a file that was not included in this bundle, then we would manually download that single file from a service called unpackaged, which hosts the files for NPM dependencies. When I say, installing a dependency, the only thing the editor does is adds, for example, lodash to the list of dependencies. Ultimately, the bundler, still the bundler is the one that calls a service to download the files required for that dependency.

[0:40:58] JG: This is all cached under Redis.

[0:41:01] IvH: This is cached in S3. Yeah, it's something that came later. It's very interesting how it works, because working with dependencies itself is a very interesting challenge. Because what if, for example, if you install dependency A and it requires lodash version 2, and then you install also dependency B and it requires lodash version 3, how can you make that work? Or if you install dependency A, it requires lodash version, any version that's in 2, and then you install dependency B and it requires lodash specifically version 2.5. How are you going to make sure that both versions get lodash 2.5 to be most efficient in the download? There are some very interesting challenges with building up this dynamic NPM install service. We don't just cache singular dependencies. We also cache combinations of dependencies. For example, React and React DOM, we cache that combines that merged bundle, but we also cache React, React DOM, lodash, and React icons, for example, as combined bundle, so that we don't have to recompute these combinations every time. Yeah, that bucket is huge. It's a couple terabytes, I think.

[0:42:19] JG: There's a benefit then to users using common combinations of packages. If I'm using React and React DOM, the same way everyone else's, that's already pre-cached before I make that installation request.

[0:42:28] IvH: Yeah. For sure. Very often, people will use, I think, React and React DOM is the most common combination. All of this, this whole S3 bucket is cached, is Cloudfare is in front of it. Because of that, we also didn't have a lot of requests to S3. Dependency installations, it's very interesting if you try to install, if you want to run a sandbox with a dependency combination that has been used by another user before, then installation will be within a second. It would just be a matter of downloading the file from the S3 bucket. It would be already pre-bundled. Sometimes we even do pre-transplition on the source files to make it also faster to run, so that the bundler doesn't have to do transpilation on it anymore. It would even store information for every file, like what to require. The dependency graph, we would also embed the dependency graph into dependencies to know which file imports which file.

The bundler would also save work. The bundler wouldn't have to do the parsing of all those files, that to run the code. That is the interesting thing. You could see it as a super-fast NPM, because the dependency combinations, you are most likely using a dependency combination that has been used by someone else before. In that case, you would just have to download that file from an S3 bucket.

[0:43:50] JG: Okay. We've covered most of the stack. We're nearing the end. We've got database and correlated S3 and Redis caches around it. When I make an installation request, you've got a whole bunch of heuristics and strategies there. That all gets sent to the clients in some transpiled bundle form. Then you have the two apps on the client, the editor and the preview, or viewer. Whenever the editor causes a file that we changed, we post message over to the iframe to email the code as necessary.

I think the last bit I want to touch on is the editor itself, because that on its own is an entirely separately complex and interesting application. The code editor that works in the browser, how does that work?

[0:44:29] IvH: I would argue, that's the most complicated part of that specific stack. The first version of the editor was very simple. It was all React components. It was like a file explorer where React components. The editor itself, the code editor was CodeMirror, which is a library created by a Dutch person. Later on, we would move our code editor to Monaco. Monaco is a little piece of VS Code. VS Code has an incredible code base, one of the best code bases I have ever seen. It has really good composition. It has really good APIs. You can already see that it's so good, because they were able to extract the core editor from VS Code and make it a library and keep it in sync with VS Code relatively easily. Monaco is that. It's essentially the VS Code editor, but then exposed as a library.

It doesn't have all the fancy things from VS Code, like extension support, or a file explorer, or all those. It just has the editing part, but it works and it works really well. We moved to Monaco, because it had better, well, I had a more familiar experience for everyone using VS Code. When CodeSandbox started, Atom was the most popular editor, but later on, VS Code became the most popular editor.

That was always a case where people were wondering if they could do the same thing as VS Code. Code Editor is an incredibly complicated piece of technology. You have things like the command split, you have key bindings, you have feeds, because developers, they want feeds. You have settings for all the little things, like line hides, font size, font itself, how the editors should behave when you press alt and move your cursor around, how should multi-cursor work, all of these, there were a ton of settings, and then you have extensions where extensions alter the editor behavior, or add new editor behavior.

Code Editor is one of the most ambitious UIs to ever work on and it's an incredible thing to work on. You learn a ton. Oh, and performance. Just making it performant when you click on a file that it can open that file within 10 milliseconds, incredibly hard. Initially, we were doing everything with Monaco. We had our own UI built around it and Monaco was the core editor, but we had our own UI. Because of that, we were able to create a lot of custom experiences. Later on, I started to become more and more interested in VS Code. In 2018, during my Facebook internship, I was looking at a way to make VS Code run in the browser. VS Code didn't run in the browser back then.

Initially, VS Code was built to run in the browser, but it was not performant enough, so they made VS Code an electron application. It used web technologies to render the UI of VS Code, but it did not run in the browser, because it did use things like, node file system, a lot of node APIs. I was looking at, we could use VS Code samples, because that would save a lot of work. It would create familiarity for developers, their own custom key bindings, those things. We would start to allow extensions. It would be faster, because ultimately, VS Code is much faster than React, mostly because VS Code is imperative instead of declarative. It would have a ton of benefits.

Making VS Code run in the browser ultimately came down to emulating nodes in the browser, because node ultimately is JavaScript with native node modules. For example, file system, net, HTTP, OS. All of these node modules, if you create a browser equivalent version of that that would run in the browser, then essentially, you're implementing nodes and you're tricking VS Code in thinking that it's running on your local computer, but it's running in the browser. That's initially what I did to make VS Code run in the browser.

Later on, the VS Code team actually ported VS Code back to the browser. Now with CodeSandbox, we're running the browser version of VS Code. We're still emulating some node components to run extensions sometimes in the browser. But, for example, a VM, then we use VS Code server, so that you have the native VS Code experience. But we do have a little add-on that we put on VS Code that allows us to render React components inside the VS Code UI, and that allows us to create custom experiences still using React, but within the context of VS Code.

[0:49:00] JG: How does that work with type information? Let's say, I want to hover my mouse over lodash, import it from lodash and see what are the methods available under it?

[0:49:09] IvH: Yeah, that's a very interesting question. If we are looking at the DevBox version, it's quite simple, because VS Code server gets the files from the file system. Well, in fact, the extension, the TypeScript extension, runs on the server itself, and it would read the files. That's a solved problem. But for sandboxes that completely run in the browser, we have a bit of an interesting solution. We run VS Code extensions in the browser. Those VS Code extensions expect to run in a node environment they expect to have. When they do require FS, they expect to have the FS module available. They want to be able to run read file, read file, read dir, write file, all those things.

The first thing that we did was implement file system, but then to run in the browser. We can run VS code extensions in the browser. They will think that they are in a node environment, but they're running in the browser. The second thing is whenever you install a dependency like lodash, we do the same thing that we essentially did for the bundler. We have a service that calls NPM install for lodash. Then it looks at all the type files that are installed, and it will create again, a JSON bundle that just only includes all the type files, so only the .ts files, or the .ts files. Then in the browser, we would download this bundle and we would write all of those files into our in-memory file system, as we call it, for this fake file system that we've created in the browser. Then when the TypeScript extension would run, it would, again, think that it's in a node environment and it would read files from node modules and those files, those are all coming from this in-browser file system that was populated from this dependency package here.

[0:50:59] JG: Is that a different service, just to clarify, from the runtime.js file service?

[0:51:05] IvH: Yeah. Yeah. But it shares a lot of code. I essentially copied the code from the dependency package here and then changed some things to make it more fitting for type fetching. I have to say, that's how it works for the first six years. Now recently, I've made a change that we actually download the DAR file directly from NPM. Those have become better and better. At this point, unpacking a Gzip DAR file is really fast in the browser. So, for a lot of dependencies, we don't go to our own servers anymore. We literally download from the NPM registry directly the DAR file and we unzip it in memory and then we save it. It's a bit less efficient, but we had to do it, because half of our bandwidth usage was just from this type fetcher. Cloudflare told us that we had to go on a more expensive plan.

I was thinking, well, what if we then just download that actually from the NPM registry then we don't pay for the bandwidth of those bundles. The funny thing is, because of that, we significantly reduced our Cloudflare bill. But Cloudflare backs the NPM registry. Cloudflare itself doesn't know there's a difference in bandwidth. In fact, the bandwidth has increased, because now DAR files, they don't create this perfect bundles of typings, but at least we don't get billed for that bandwidth.

[0:52:27] JG: Microsoft owns NPM. If they want to reduce that, they can always add a feature to NPM to help you out.

[0:52:33] IvH: Yeah, that's true. That's true. It was an interesting thing. Unpacking a Gzipped DAR file definitely would be super inefficient back in 2017, or back in 2018. Nowadays, you have, I think it's a browser feature called decompression stream, or something, which makes it super-fast, because ultimately, it uses the native decompression of the computer.

[0:52:54] JG: It is really beautiful how the browsers have made things like this possible. More and more over time, more and more of your stack is becoming built in.

[0:53:01] IvH: True. I would even say that if I would rebuild CodeSandbox today, I would probably not build the bundler as it is today, because right now everything is based on common JS with this require override. I think if I would build it today, I would use the ES module capabilities of the browser and then perhaps, use a service worker that acts similar to a feed dev server. Whenever you try to download a file, a JavaScript file, our service worker would maybe transpile that file and give it back. Then the execution and bundling is done by the browser. It's not done by us anymore. We would not even be using eval. Yeah, essentially, it would be fully ESM.

It would be a Vite. It would literally be an in-browser version of Vite. If I would rewrite everything, I would do it that way, because it's much more native to the browser and the world is moving to ESM in any case.

[0:53:56] JG: Eventually.

[0:53:57] IvH: Right, eventually. It'll take a while.

[0:54:00] JG: Do you think you'll ever have time to try out, say in a few years, the fully browser bundle and transpiler for CodeSandbox?

[0:54:07] IvH: Yeah. If no one else has done it. That's also the big question. Because I think that's a big opportunity and I can understand if people would explore it. Right now, the bundler as it works today works really well. I'm a bit worried that if I would rewrite it completely using this, that we would introduce bugs. It would still be an interesting experiment. Yeah, it is something that I would want to explore. I had it in my mind for a while. But I'm not sure when I will explore it.

[0:54:37] JG: What's another long-term exploration that you would love to get to, if and only if you had time?

[0:54:45] IvH: What I find very interesting about this whole Firecracker stack that we now have for development environments is I think the fact that we can clone a VM within a second, that we can resume a VM within a second, that's incredibly powerful technology that is not just applicable to development environments, but also CI/CD systems, even to deployments. Yeah, I think there's a whole plethora of different use cases for that technology. I would love to generalize this infrastructure, so that people can use it for other purposes that go beyond the cloud development environments. Yeah, I think it's incredibly powerful. It could be used for so much more. That's something that I would find very interesting. Making it so that the infrastructure that we have today, we can open source it, that we can make it generic enough that people can use it for their own use cases.

[0:55:39] JG: A lot of companies, a lot of projects have CI/CD times where you have whatever, let's say, a half dozen tasks, each of those tasks spends 20 seconds in NPM, or PNPM, or yarn install, and then three seconds in the task itself. It's very frustrating. It's very wasteful.

[0:55:55] IvH: Yeah. This technology, I was thinking, you could, for example, one thing that you could do is you could do all the preparation that needs to be done for CI/CD and then create a snapshot. Whenever you need to do a CI/CD run, it will continue exactly from that snapshot. It will just then pull the latest code and then run the tests. Also, for parallelization, if you, for example, want to run a test suite across 12 different workers, what you could do is you could create a snapshot. You could do all the preparation work on one VM. Then after that, you clone this VM 12 times and all these 12 VMs, they run a different part of the test suite in parallel.

One of the things that we even support, or we have never deployed it, but it's something, it's an experiment that I built is that you can do cloning of VMs over the network even. You have a VM running on one machine and then another machine starts a clone of that VM and they use the network to sync the memory between the VMs. That's also a capability. Then you could really think, if you have a CI/CD cluster that is, I don't know, 20 servers big, you could do the preparation on one machine and then clone that machine to all the other 20 machines and they run all one-twentieth of the test stack using the result of the preparation of the single server has done.

[0:57:13] JG: It's actually worth it, both time and compute to do it that way.

[0:57:19] IvH: Yeah. I would say so. It's just an idea. It's like, you probably will find things that don't work in practice, but that always happens when you try things. I also wonder, for example, there's now a ton of research being done in how to efficiently train AI models using hundreds of servers at the same time. Because ultimately, that's the biggest challenge. How can you scale so much, so much compute and data transfer in a data center? I think those challenges that they have, they are not the same, but they are in a similar vein, which is, it's an interesting space, at least.

[0:57:59] JG: How soon will I be running my CI jobs in CodeSandbox then, given all your caching?

[0:58:07] IvH: I think not soon. Well, it would be good to explore different use cases and work on the different capabilities for the stack itself. In fact, you could run your CICD today already, it's just that the platform has not been built for it, but we can expose an API, or you can even create a devbox that runs CI/CD. Let's say, just next year. Then next year, you can ask me, can I run CI/CD? Then I will say, yes, we have this API. Then you can give it a Docker image and it will run your code.

[0:58:38] JG: We'll follow up in a year. Same with the AI models. This has already run longer than our typical SED interviews, because you've got so much fascinating tech to talk about. I want to end on a less technical, perhaps, still fascinating note on your personal points. Can you tell us a little bit about volleyball and why it's such a fantastic game to play?

[0:58:59] IvH: Yeah. I mentioned this before, in my free time, I do a lot of volleyball. volleyball is something that I started doing when I was, I don't know, nine-years-old or something, and did it a lot. During high school, I did it an extremely amount of lot, if that would be worked. I think I trained six times in a week. I got a bit burned out on it during high school. I was a bit too, I guess, a bit too competitive, so I stopped for a while and didn't do much volleyball. But I recently picked it up again, I think a year ago. Yeah, about a year ago. It's so incredibly good to play volleyball, because it's when during the training, or during the match, I completely forget everything, everything surrounding. I could be worrying about something, but when I play volleyball, I don't worry about anything. I just worry about the ball. It's a very good from psychological vantage point, but also, I'm much fitter now and the day after I feel much more energized, because of the sports. Yeah, I'm a big fan of volleyball. It's a reminder for me that it's important to not just do coding, or meetings the whole day. Ultimately, it's also important to stay healthy and exercise.

[1:00:16] JG: I think a lot of people maybe did high school volleyball, just hanging around in gym, maybe have done some casual beach volleyball with friends. What is different about your areas and brands of volleyball that were not just standing around awkwardly waiting for the ball to come to your ass?

[1:00:30] IvH: Oh, so you mean, what makes it fast?

[1:00:34] JG: What makes it a good workout? What are you doing?

[1:00:37] IvH: When I play volleyball, I'm mostly found on the ground, I think. There's a lot of diving going on. If you want to get to the ball and it's just out of reach, and you just jump. Recently, I started to also track my heart rate during volleyball to see how intense it is. I've now done it two times. My heart rate goes up to 195 when I play volleyball. Especially at the net, it's also, you jump a ton. Because, for example, when we play volleyball, if I play mid, the thing you do is whenever the setter has the volleyball, whenever the ball goes to the setter, you have to jump and fake an attack. You always jump when the setter has the ball, because that way you confuse the opponent about where the ball will be attacked, so that you confuse the blockers.

In just one rally, you can jump two or three times and then you have 40 rallies and that's four times, 160 times, yeah, you jump 350 times or something in a single match. It's very intense in that sense. It's very explosive. There's no long running, but it requires a lot of short bursts of energy.

[1:01:54] JG: That sounds very intense. It's also very psychological. You're not turning your brain off. You're still thinking.

[1:01:59] IvH: Yeah, it's true. There's a lot of prediction going on. You can confuse your opponents. They can confuse you. What I like the most is the competitive aspect of it, like winning matches, or losing matches, too. That makes it a lot of fun, compared to, for example, running. I also do a lot of running nowadays. But for me, something is only - when it comes to sports, I find something mostly exciting when it's a competition that keeps me in it, I guess.

[1:02:24] JG: Great. Well, this has been an absolutely phenomenal interview. Ives, thank you so much for talking us through the startup CodeSandbox, the backend, the database, the caching layers, the front-end, the other. This is a lot of stuff. If folks want to find out more about you and CodeSandbox on the Internet, where would you direct them?

[1:02:41] IvH: My Twitter is where I'm most active, I'd say. That's @ComputeIves, C-O-M-P-U-I-V-E-S. That's my Twitter. CodeSandbox can be found on codesandbox.io, or codesandbox.com, or codesandbox.org. That's a small tidbit. We went with codesandbox.io, because the domain was only $30, and codesandbox.com was $3,000. Then later on, a lawyer who learned how to code using CodeSandbox gifted the codesandbox.com domain to us. That's how we ultimately got the codesandbox.com domain.

[1:03:20] JG: Well, all that being said, this has been Josh Goldberg, and Ives van Hoorne with Software Engineering Daily. Thanks all. Cheers.

[1:03:25] IvH: Thanks.

[END]