EPISODE 1906

[INTRODUCTION]

[0:00:00] ANNOUNCER: Developer tooling shapes how software gets written day-to-day, but the best tools often disappear into the background once they succeed. Formatting, linting, and build systems can either create friction and endless debate, or quietly remove entire classes of problems from a team's workflow. Over the past decade, the JavaScript ecosystem has wrestled with both extremes, as it scaled rapidly and accumulated complexity. Prettier emerged as a response to the surprisingly human problem of engineers spending too much time debating code style, instead of building software. It offers a deterministic, opinionated formatter that helped normalize automation as part of everyday development.

James Long is a design and product engineer who has worked at Mozilla and Stripe, and he's the creator of Prettier. He joins the show with Josh Goldberg to talk about the origins of Prettier, why formatting debates are so emotionally charged, the technical challenges of building formatters, the realities of maintaining popular open-source tools, and how the JavaScript tooling ecosystem continues to evolve.

This episode is hosted by Josh Goldberg, an independent full-time open-source developer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ESLint, a powerful static analysis toolset for JavaScript and TypeScript. He is also the author of the O'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and the co-founder of SquiggleConf, a conference for excellent web developer tooling. Find Josh on Bluesky, Fosstodon and dotcom as @JoshuaKGoldberg.

[INTERVIEW]

[0:01:53] JG: With me today is James Long, a design and product engineer who has worked at companies such as Mozilla and Stripe, created Prettier, and created an open-source to the actual finance app. James, welcome to Software Engineering Daily.

[0:02:04] JL: Thank you, Josh. Very happy to be here.

[0:02:06] JG: So excited to talk to you. As a user of Prettier and advocate for, this is a dream come true. Before we dive into Prettier and formatting and all these fun JavaScript and TypeScript dev tools, can you tell us who are you and how did you get into coding?

[0:02:18] JL: Sure. So, I'm James. I've been coding, really, since - I've been in the industry for over two decades, but I was the typical cliche, 90s kid that was hacking on computers. Just always, we happen to have, I think it was an Apple II in our house when I was in middle school. I just instantly gravitated towards that, just a lot of nerdy kids did in that age, right? I remember programming in, was it Q basic? There was the one that was just like, you had the 10, 20, 30, 40 with the line numbers for some reason. It wasn't one, two, three, four, I'm pretty sure. I think the thing that really drawn me to it also is that I could draw things to the screen. I remember drawing lines, and it was just that green terminal look was the only color that it had, but you could draw lines and draw shapes. It was really cool as you could actually play notes, too. It had a command, like play A flat, so you could draw little lines and have these things animated and it would play music. It just instantly drew me to it.

Then later in high school, 3D games was something that just really attracted me to. It just blew my mind that you could do that with a computer. It took me a couple of years to figure out how that even worked, but I bought some books around the direct 3D programming and things like that. I studied it in college. Then I got a job and just has always just - it's just always been that thing that has satisfied a creative itch for me.

[0:03:33] JG: We're going to talk more about that creative itch as we continue. How did you get from 3D programming and notes and such into working on web development, or even web development tooling?

[0:03:44] JL: It was easier to get a job for sure. I went to a small school in Chattanooga, Tennessee. There was a local web development shop, and there weren't any game development shops around that. It was also, I felt even though I had tinkered around with game dev programming, it's just a harder industry to get into. That was most people were using C, or C++ back then. I wasn't as strong in that. The web was a little bit more - it was just more accessible. The web was interesting too, because it was also a visual thing. It was a good blend of practicality, but the front-end part of it was still satisfied that creative itch a little bit, where I could still do visual things, still work with designers and still come up with visual things.

The games industry is such a different industry. The first couple years, I was doing the web stuff with my eye on game development. But the more I was able to get in touch with people and learn about that industry, it just became clear that it was just such a grueling industry, super low level, super low pay, just seemed very grueling and just not really fun. I knew that it would become just a - the game stuff would be just a side project to me. The pay would be coming from web development stuff.

[0:04:51] JG: There's something fun about web development, how you can just get a page locally, throw some JavaScript and CSS on there and instantly see the changes as you edit them, right?

[0:04:59] JL: Yeah. Also, just the ability to just share it with a friend instantly. Because the game development stuff, a lot of the studios and the engines were getting really good, where you could have hot reloading and do a lot of visual stuff. You could be in something where you could select elements and move them around and then patch, script elements to them. You can't share it. You can't just give somebody a URL, right? Of course, eventually once WebGL and all of that got good enough, then you can do that, too. Yes, the web had a very instant gratification for creativity.

[0:05:28] JG: Let's talk about the web. You have been working in web dev tooling for a while. The most notable project folks associate you with is Prettier. First of all, what on earth is prettier? Why do we need something like this?

[0:05:39] JL: Prettier is a JavaScript formatter. It takes your code, and then you give it basically one parameter. It's got a couple other options, but mostly the philosophy of it was to be very opinionated, to not take options, you just give it one parameter, which is the print width. You say, I want there to be 80 characters, or 100 characters, I want all of my code to fit within this size. Then it formats it based off of that size. It's a deterministic formatter. Whatever the input is, no matter how many spaces, new lines, however, whoever was super opinionated about how they wrote the code, you'll get the exact same format out.

The reason why you need this is just, well, the original reason was mainly to just get rid of these fights and little nits between teams, right? Or even specific team members that were really opinionated about it. I hated the PR review. I was working back at Mozilla before I created Prettier. I just felt such a waste of time to be having these PR reviews go back and forth about formatting nits. It was a weirdly emotional thing that just to me just did not matter at all. It means zero difference on the users that you were using. It was like this thing that tended to fall into some of our OCD, or perfectionism traits that maybe a lot of people just couldn't help but care about the formatting. This is a tool that is automating a way, so that you can relieve that care about it and have a perfectly consistent code base across all of your teams, and it's just something that you can accept.

Now the thing that was surprising to me is that we very quickly saw how it relieved some of the pain and annoyance of actually writing code. Now when you have this format or embedded into your editor as you're writing code, you can write code in a very stupid way. You can write it all on one line and then just press one keystroke and it just formats it in the production ready format that you could commit. That was a surprisingly effective thing that actually, I actually was not building it for and actually was something that was unexpected.

[0:07:35] JG: Well, Prettier I came around in around 2017, right?

[0:07:38] JL: Yes. I think at the end of 2017 was when it started. It actually launched January 2018, if I am remembering that right.

[0:07:46] JG: Well, at that time, we already had tools like ESLint and TSLint. Now just ESLint and TypeScript ESLint that can format code. There are stylistic and formatting rules in linters like ESLint. Why would I want something like Prettier as second additional tool, when I can just use ESLint?

[0:08:03] JL: I think there's two reasons. So, we did that at Mozilla. We use ESLint. I think it would try to automatically fix your code based off of those things. However, if you looked at our ESLint config, I swear it was it was a hundred rules. These weren't even including the actual semantic rules. These were just the formatting rules. It didn't solve the team dynamics of the you wanted to have control over the formatting. I forget exactly why we would still have the problem of PR nits. Actually, I think that leads me to my second point. I do remember why. It was because there was those - even though ESLint had multiple, multiple, I think over a hundred, I don't know exactly how many rules based off of the formatting, it still didn't cover everything.

There are things about how you and JavaScript that are very dynamic, or that are very hard to express as a lint rule, such as if you call a function, give it three or four arguments, and then the last argument is a function. You know how JavaScript, we tend to do the first three arguments that are numbers, all on the same line as the call to the function. But then, the anonymous inline function that we're passing to it, we will break that on the opening bracket and then write that inline indented, right? You just can't express that with ESLint's rule-based system. You need a much more expressive dynamic, like a holistic formatter that can reason about where that function is, what's the parent of that function? I think those are the two things. One is that it allowed different people in different teams to have too much control over the formatting, because they could customize all of these hundreds of rules, and it just didn't cover enough.

[0:09:40] JG: I remember trying to roll out Prettier on a team recognized at Microsoft, and an actual common piece of pushback we got was, I like the idea of faster code authoring. However, I put so much intent and meaning behind the formatting choices, such as when you have a class, or with many properties, putting the equals all aligns. I don't like how Prettier removes my ability to do that. What would you say to someone giving you that pushback when you're trying to advocate for Prettier?

[0:10:06] JL: At this point right now, I don't really hear that much anymore. I think that argument has been proven false to where most people use it now, and the benefits of it are worth it. I think that was an attachment to the code that you wrote that that argument just wasn't correct. There was not meaning there. Once you start using it, you see that you're still able to understand the code just as much. It was harder to make the argument in 2018, 2019, when people were pushing back this a little bit more at the beginning. I think what we would try to tell people is what I just said, but in a more gentle way. It's like, "Hey, you should just try this. You should give it five minutes. You should give it a little bit of time. See if there actually is meaning in that. You will find that in a lot of places that you thought there was meaning that you can actually just understand the code relatively the same."

If there is something that's an embedded matrix that you really do want aligned really well, then we have the ability to ignore, like you can say, Prettier ignore in a comment above that expression and Prettier will not format it. There are rare cases where you can care about it. To say that you need to care about 100%, or even you need to care about 20% to 30% of your code, that the formatting matters that much, I think is just not an argument that we were ever convinced by. We would just try to tell people that. If they didn't agree, then they didn't have to use Prettier.

I think, eventually, they saw the social proof as more and more people started using it, then they would try it out and see that. I think you might need to care about 1% and 0.1% of your code, these few places that you really are having something really specifically formatted. That's what we would try to say.

[0:11:41] JG: It's interesting also that this came out relatively recently in the JavaScript lifetime. This was not a thing in the 90s, not a thing in the early 2000s. It took until closer to 2020 than 2010. What do you think took so long for the community to create and then adopt something like Prettier?

[0:11:56] JL: This is a good time to also talk about Christopher Chedeau. who I will actually consider a co-creator of Prettier. I think he and I both individually say that we created Prettier and I have no problem with that. I technically created the first implementation. He came in very quickly and I think we can talk more about the history of this later. I say this, because he has a great post on his blog that I think is called The Birth of Prettier. I forget his exact blog. I think, hopefully, we can link to it. Or if you just search Christpher Chedeau, birth of prettier, then that post will come up.

He has a good explanation about like, there were several formators that were attempted and also, ESLint, I think, was probably a lot of people use ESLint with the formatting rules. a lot of people just thought that was good enough. That was one thing that people just didn't really see why you needed a holistic formatter, like a whole different tool. A lot of people didn't see that. There were some other efforts that tried it. I think the thing that they got wrong is that they would try to follow the philosophy of some other formatters, like Go format. The difference between Go format and Prettier is that Go format is it's very - I'm trying to think about the right word. It's even more opinionated than Prettier, basically. It doesn't support some of the sophisticated breaking patterns that Prettier does. It's like, it will always format a struct, or things, and it will always break it across a line. It doesn't have a concept of a print width, right?

The thing about the print width that was interesting about Prettier is that it sounds like, why wouldn't you just hard code a print width, or something like that? It's interesting about this because with Prettier, you need the ability to, in JavaScript specifically, compared to Go, because you do a lot of nesting of inline JavaScript functions, and there's all these just really weird patterns, you cannot have a deterministic, or not a deterministic. You can't have a simplistic approach to formatting, like the Go format does, which is like, hey, if you have a struct, we're going to spread it across all of the lines. Because what if you pass - like in JavaScript, if you had the inline function case, well, sometimes we format it differently if it's at the top level, or if you pass it as an argument to a function.

In Go format, it just didn't support that thing at all. Whether it was a function argument, or top level, it was broken up the exact same. I think some of the other JavaScript formatters had taken that approach. It just didn't work, because it just failed way too quickly on all kinds of real-world patterns. That looked too ugly. For this project to succeed, it needed to pass, even though I say I don't care about the formatting, we all do care a little bit, right? It can't look horrendous. It needed to meet a bar that I don't think other formatters had quite hit yet. I think it just happened to be the right timing, where people were geared up for this, but they hadn't quite experienced the tool that did it well enough yet.

I did, for whatever reason, I don't know if I got lucky, or I think I'm decent at marketing, too. I might have just gotten lucky with my marketing. Part of these things are always luck, right? That I just might have happened to be at the right time at the right place. I remember specifically, my tweet that went viral that kicked everything off. Sometimes you just get lucky like that.

[0:15:07] JG: When you think about it too, at the time, that was right about when promises were getting ratified, async away was getting ratified, rolling out to browsers. The JavaScript of 2015, 16, 17 had a lot more callback hells and pyramids of doom than the code we write today, and especially compared to other languages.

[0:15:23] JL: That's a great point, actually. That is very much true.

[0:15:25] JG: All right. You've got Prettier. Christopher Chedeau, who is helping out your rolling things out to the community. What was the initial reaction to Prettier like?

[0:15:36] JL: It was a mixed reaction, for sure. I mean, there was a certain amount of virality to it to where I was surprised for sure at how quickly a percentage of the JavaScript user base adopted it, relatively quickly. It seemed like there was a decent amount of people that just use it for a couple minutes and then totally got it and got Prettier peeled, I guess, for the lack of a better word. There was a decent amount of pushback for sure. This is the thing that's like, we're not trying to sell something here. There's no money made. I think that is another interesting conversation, especially in the context of recent tailwind drama, whatnot. It's not really drama, but the news about open-source funding and things like that.

At the time, it was in our interest to sell people, just because we thought it was better. It's like, if you don't like this, then you don't have to use it. We were happy with whoever wanted to come onboard, and a lot of people did. It was enough to start the momentum of it growing pretty quickly. I think, also, I think probably the hardest thing is that there was a lot of opinions that needed to be worked through of what the formatting should be. I think that's the most interesting thing to talk about in terms of the reaction, which is just the pages and pages of GitHub issues about people being like, "I don't like how it formats this, or we haven't quite solved how it formats comments when you have an inline comment and a ternary. Then there's a long, or ternary specifically with something that took years." There was different formats that were tried that were switched back and forth. There's a lot of opinion.

Again, people do care about formatting, so there's a lot of responses and opinions to that kind of a thing. That is something that was difficult, especially for me around that time for that first year, because you can do this for months and months at a time, and it seems like those kinds of discussions don't really ever stop. It feels like you're just pushing a boulder up the hill forever. I'm very grateful to the maintainers who kept working through all of those issues and helped just the whole Prettier community that helped work through a lot of those issues.

Eventually, you just have to make a decision, right? Eventually, it's like, this is the issue. We're going to decide this formatting for this issue. We're going to close the issue, and we just got to move on, because you can't just keep talking.

[0:17:43] JG: One must imagine the Prettier maintainers happy. It is interesting, though, that open source, especially in the tooling space is so notorious for burning people out and then money being difficult. I think the drama you're alluding to is that the Tailwind Company over the last few years has had to lay off quite a few employees with the rise of AI and difficulty and showing off, or selling their paid products now. Does the Prettier project have financial concerns? Is that something that's come up?

[0:18:09] JL: It is something for sure that I have seen discussed. To be totally honest, I'm not the best person to talk about this. I have not been really involved in this project at all, even for a couple years. I wasn't very involved for the first nine months in 2017. Around that time, I was also trying to build my - I left Mozilla and was trying to do self-employed stuff. I did contracting, and I was trying to build a product to see if I could scale up that product, to see if that could cover enough of my expenses, just to be a lifestyle business. I had a lot of other things going on that was really hard to do, kind of this free open-source work.

I scaled back a little bit and was involved in decisions for a while, but especially in the last couple of years, I haven't really been super involved.  This is what Christopher Chedeau, I think he deserves a ton of credit and totally deserves. I think for the entire sum of the life of Prettier, Christopher has put in way more work than I have. I think he deserves not only - he definitely was a co-creator, plus he stuck around to help structure some of these, how this money comes in and how it gets dispersed to the maintainers.

I know they have a structure there. I know there's a decent monthly payment that goes towards the contributors. As far as I know, nobody's full-time working on Prettier, which is what is different from Tailwind, which they officially formed a business around Tailwind. Prettier, there's no business. As far as I know, there's no business of Prettier. There's no one or two people who's trying to make this work and actually make this a full-time thing. It's just more of the typical open-source thing, where there's a couple of people doing it part-time and they should absolutely be compensated for that, and there is some compensation there.

It's not enough. I think for open-source stuff, they thrive on sponsors and donations. That's never enough. I think that the amount of work that the maintainers do, they probably deserve more money. It's not the thing that there's not a business about the falter, because there's not money coming in. It's just, hey, if Prettier doesn't get enough donations, then there might be a year or two where there's just not a ton of work done on it. That is what it is. I can't tell you currently exactly how bad things are. I think as far as I know, it's okay. I think there should be more money coming in, basically, is always the case.

[0:20:19] JG: For a tool that's existed for about a decade, looking on opencollector.com/prettier, the total raised money across those nine plus years is just about $243,000.

[0:20:31] JL: Yeah. That is crazy.

[0:20:32] JG: Consider the average dev salary in America today.

[0:20:36] JL: Yeah. It's not enough. It's not enough. Actually, it's funny, because when I started Prettier and it was investing a lot of time into it for those in 2018, my wife is like, "Okay. So, you're going to make money somehow, right? How is this going to make money? Why aren't you making money right now? Because you're putting all this time into it." I was like, "You don't understand. It's open source. It's awesome. We're just working on stuff together." The reality is, yes, we spend time on things and we care about things. At a certain point, it needs to translate into a reality where it becomes something that people are putting their weekly hours into something that's just maintaining. It's not a joyful, fun thing that they're just doing to be creative anymore. It's an actual job. So, yes. It's a hard thing to figure out.

[0:21:16] JG: You touched on a really interesting point that I wanted to drill into before moving on to higher level tooling discussions. There is a big difference, as you just alluded to, between the initial explorations, the so to speak, fun exploratory part of open source and product creation. Then the perhaps, less fun for some and more for others, maintenance phase of it. You'll see often in these open-source projects where you have a certain set of people join at the beginning, or create the thing. Then over the time, they transition to a different group of people partially, folks who are more excited about maintenance. What do you think is so different about the two forms, the difference between creating, or exploratory phases, versus maintaining or contributing phases?

[0:21:55] JG: They're very different. They are still I think, can be mixed together, because I think - I guess, for example, one thing that comes in mind is now, and this is something we could talk about more, too. There is the Rust rewrites of Prettier, which is Oxfmt. That's a creative place to be, right? There can be a mixture. If you look at a whole timeline of our project, it's not just at the very beginning is this creative fun part, and then it just tails off into this totally boring maintenance thing for 10 years. Usually, there's a really bump of the fun part of this is a cool idea. Then it goes down into maintenance. Then it's like, hey, we should do a plug-in system. Then that becomes a fun, creative thing, where it's like, you are doing something new again, so that's another bump.

It's a mixture of things. I do think there is people who seek out and love to work on ambiguous ideas, or just see things in the ether and really enjoy bringing those into reality. there's certain people that find that really hard and just it's just not something that gives them their right energy. they really do enjoy taking something that's existing and just making it one step better. Doing that week by week and by the end of the year, you've made something way better than it was at the beginning of the year.

I think there's, I don't know if it's different personalities, or it's just - I think there's different sources of energy there. Now, I do say that I think everybody would enjoy to do what they consider the fun work more, right? There's probably a very small amount of people who actually find the actual real boring parts, like going and just really digging into those obscure bugs. There's even people that do find that fun. I think it's best for a project to have complimentary people who some people find the things fun about how do you design this API and how do you create this new thing and how you bring that into the world and how do you execute it?

Plus, the people who really care about the currency, the things improving it day-to-day, talking to users and fixing bugs. I found that there's always people, once you create something that's exciting, there's always these couple people that come in. This happened with actual, my personal finance app. I open-sourced it. Now, there's this one guy who spends a ton of time maintaining it. He's not getting much money from it. I think there is a similar level of contributions, but he just seems to really love maintaining this project. He loves watching users come in and download the product and help them getting it running, and it solves part of their daily life. I think people get different sources of joy, I guess. It's really interesting to see this in action.

[0:24:25] JG: Let's say that you had infinite time and infinite money, and the only stipulation was that you had to spend that time working on something for Prettier. What would you work on?

[0:24:35] JL: That is a great question. Because I think what's interesting about Prettier is that unlike projects, I guess, Tailwind comes to mind, just because of recent events. Tailwind must evolve with the browser. The browser and CSS is always evolving, right? There's always new features added to it. Prettier feels like a project more to me that has a better chance of being completed in a way. I mean, there's new JavaScript syntax, but it's relatively rare to get a new JavaScript syntax that is fundamental new syntax that you need to rethink large parts of the system.

If I had to do anything, if I had infinite time and money, I think it's an interesting question, because Prettier is a thing that I use every single day. It covers all the things that I use in JavaScript. To me, it is completed. To be totally honest, I don't know, because I understand the Rust rewrites. I understand people run Prettier on their entire codebase and they want that to be fast. I don't really even resonate with that performance need. You're rewriting an entire project in Rust. I respect it. I think you should do it. I think Oxfmt is a great tool. People want to spend their time on it. I run Prettier in my buffer, in my Emacs buffer, it always runs fast enough. It just feels like a completed project from my perspective. I'm sure there's all sorts of use.

I don't really use it to format markdown, or format all these other things. I'm also not exposing myself to this entire world of plugins. I embedded Prettier into Obsidian. As I'm working in a code block, I can run a keystroke and it runs Prettier on just that code block. I never want to run it on the rest of the markdown file. It just run it onto that code block. I don't know if I have anything. I'm sorry to disappoint you. To me, it's perfect. To me, it's completed. I'll happily switch to Oxfmt if it works just as well and it's a tiny bit faster.

[0:26:30] JG: That's interesting on two points. One, you have to love open-source maintainers saying that the thing is done. You don't get that in corporate life as much. Two, you don't use Prettier on markdown. I would have thought for the same reasons it's useful in JavaScript, consistency, ease of writing code, not having to manually format. Is that not a thing for you?

[0:26:48] JL: I use Prettier in a weird way in markdown. I personally don't find it as useful to format my markdown code, or my markdown narrative content. That is the content that I am writing paragraphs on and typing out paragraphs. I do a little bit of markdown syntax. There's not nearly as much structure there as code. Now, for the code blocks, though, I run Prettier on the code blocks. The way I do it is in Obsidian. As I'm writing code in the code block, and it could be 100 lines, I have a keystroke that runs Prettier on just that code block, right? I do run Prettier on the code in my markdown files, but I don't run it on the entire markdown file. I don't just reform at the entire markdown file every time. As I'm writing code in markdown, I reformat the code block that I'm working on, because that's the only thing I'm interested in.

Maybe I don't hit these perf problems, because I do write massive posts. I want to be able to write 20-page posts. The formatting of my code block is because it's always only applied to the code. It's always instant. It's always super-fast. I'm not needing to reformat the entire markdown file every time, if that makes sense. It's a little bit of a maybe esoteric setup.

[0:28:06] JG: That does make sense. When you talk to companies like 4.0, or organizations like Biome, oftentimes they describe the user need, their targeting is one step higher, or larger in scale than the common open-source project, where a company might have thousands, tens of thousands more files. If they, let's say, update a dependency like Prettier, then they need to run on all thousands upon thousands of files, and then they get into performance issues. What do you think of that user pain, or that solution of rewriting in Rust? Does that feel reasonable to you?

[0:28:38] JL: I think it's totally reasonable. I think it's a question of resources. If there's the right number of people that are going to invest in this, and it's not going to be a - If it's managed well and resource well, I think there is a couple part-time maintainers of Prettier. If a user was going to come in and say, "Hey, you all need to rewrite this in Rust." It's like, for the current state of things, it's better for the current maintainers who just work on the current JavaScript code base and make that better. If there is a whole separate group of people working on those Oxfmt tools and Biome, and they have funding to pay developers to work on this full time for a year or two, I think it's amazing. I think it does solve a pain point for sure. I'm fully supportive of those tools.

Again, I'm also not super involved in Prettier right now, so this might not be surprising, but I don't have any super emotional investment in Prettier. It's Prettier was to go away and say, Oxfmt was to completely replace it, as long as it meets the similar needs and works a similar way, and it formats the code in a similar fashion, then I'm all for it and I fully support those tools.

[0:29:42] JG: It's very pragmatic. I want to take a step back and talk about JavaScript and web tooling as a whole. There's a criticism sometimes targeted towards the ecosystem that we have a million tools. It's hard to keep track of them. They all do different things. It's hard to set up. On the one hand, Prettier has been stable for a very long time. Although we just discussed modern-ish, competitor-ish to Prettier, that it's for the most part just what people use. That being said, there are a lot of tools that has to integrate with. Well, how do you feel, or what is your reaction to people criticizing the fluid ever changing form of JavaScript dev tooling?

[0:30:16] JL: Yeah. It's a mixed bag, because I think that there was a time, I remember when we were tweaking our Webpack configs and just hyper-optimizing things and the ability to compose together a unique system that only applies, maybe if you're at a company, Stripe is a super complex company. They use Webpack. They're moving to, I think, they're moving to VNOW, actually. The ability to compose your own system that meets your needs if your company has specific needs that something out of the box, I think there's a higher risk of something out of the box that doesn't - it's supposed to do everything together, when you can't use that because it works a certain way, right? There's a benefit for things being composable and you're going to hack them together.

I would say that those complaints though are well, I think they are grounded in something real, like a real pain point. We've also seen tools like button, which just, they literally just work. You don't have to do anything. When I was using node, you'd run it into all these issues where an import was trying to import a file with a wrong file extension. I just don't hit those things in button. There is a significant user experience improvement with tools that do just work, where you don't have to duct tape things together. Vite, I think is another great example of this. I think Vite is just a really great tool, because it does allow you to configure things to a certain level. That's why a company like Stripe is actually able to adopt Vite.

However, Vite has that out of box feeling, right? You can take Vite. It doesn't take hardly anything to configure it. You can get an index8.html file that loads a JavaScript file and it renders something to the page very, very easily. We've grown a lot. I think those complaints were valid five years ago. I think since then, the state of the tool right now is much better. I think that those complaints are either out of date and they just haven't used some of these newer tools, but I think we've certainly grown a lot out of that pain. I don't think there's that as much pain anymore as there used to be.

[0:32:16] JG: It's funny. People still complain about how CSS, the joke is it's difficult to center. That's been a part of Flex for over a decade now. What do you see as the next big pain point to be solved for web tooling then?

[0:32:28] JL: I have to be honest that I'm really happy with tooling right now. A lot of my thought goes more on the web itself and the runtime of the web. I don't have a grand - I hope to not disappoint you again. I don't have a great answer. I think, Bun is amazing. I think Bun has solved the backend environment and the ability to run scripts very well. I think Vite has solved the bundling problem very, very well. I am not a huge fan of React server components. I'm just going to say that right now. I think that the NextJS and that world, I am of a lot of hesitation around that stuff. I think that has introduced a lot of pain points, which you require tooling to see what's going on in those things. I think that the solution is not better tooling for those things. I just don't agree with that fundamental approach, generally speaking, which is maybe a different topic.

I think, I'm thinking more along those lines, which is like, is React server components really the right thing here? How can the client and the server communicate better? What are better strategies? What is Remix doing? Those are where most of my thoughts are right now. I think we need better ways to do server-side rendering with hydration. I think we should be exploring more, more things with that, which will probably require specific types of tooling to be able to control those environments more.

[0:33:47] JG: Is there a stack out there that you think is closest to your target end state for this type of area?

[0:33:52] JL: I've been thinking about this. I left Stripe back in October, so I've had a couple months to reevaluate things. That's right. I just didn't really have to have that much time. I'm still evaluating my opinions. I've been looking at SolidJS a lot for the front-end part. I really do like what they're doing in SolidJS a lot. I looked at Astro, which was interesting. I wasn't a huge fan of some of the way that they do the syntax and how you actually have the Astro files, but the actual islands concept and things like that was pretty compelling. I think Remix 3. I'm decent friends with Michael and Ryan, the creators of Remix, and this new version that's going to be coming out soon. That potentially might match some of my more recent thoughts the most, because I think there's some solid starter kits which do some server-side rendering with stuff. I'm not super familiar with that, but I would probably say, Remix 3 is a very interesting take on this and probably is something that I'm going to keep a close eye on the most.

[0:34:53] JG: We'll have to look forward to having those folks on the show at some point. Is there anything else before we start talking about wrap-up topics that you want to bring up about Prettier, tooling around it, so on?

[0:35:03] JL: One thing that might be more, maybe to go dig into it a little bit more technically is, and maybe to go back to your question about what I would work on in Prettier is, the reason why duct-taping these types of tools together gets really slow is because they have to parse the JavaScript into whatever they're doing and then they generate JavaScript out of that, and every single tool in that step does that. That's why I never, to be totally honest, I never liked the ESLint Prettier plugin stuff, because it's so slow.

Basically, if you use that setup, that is the setup that you use to format your file, that is so much slower than just invoking Prettier directly. Don't do that. If you're going to format your files, just call Prettier. Don't call ESLint and tell it to call Prettier, because there is a significant overhead there. Maybe they've solved that. I'm not entirely sure, but at one point at least, you solved it.

[0:36:01] JG: We did not. We just told people to stop doing that. I'm so happy to hear you say this. There are so many reasons why you shouldn't be running Prettier in ESLint. Primarily, what you just referred to of the performance of it directly, but also, we have a lot of slow rules now, rules that use type information, or end-point analysis across file info. When you have auto-fixers from those rules and auto-fixers from Prettier rules, you end up running all of them all the time constantly. ESLint rule re-run rules up to 10 times when there are fixes that conflict with each other. You take sometimes multiple seconds after each safe, where it should have just taken maybe half a second, which is obscene as a user experience.

[0:36:40] JL: Yes. I think this is where, going back to the fragmentation of tooling also, which is powerful, because you can take these different tools and put them together in different ways, but there is a really compelling motivation for bundling things together into a single tool. I honestly haven't looked super closely at what the Oxfmt and other - I think the interesting thing here, I think, is that it is a suite of tools. This is what Biome, I think, was trying to aim to be also, but the reason why this is so compelling, and actually, I should have talked about this when I talked about my support of these tools, what probably does excite me most about these tools is that it solves this core problem, where they have a standard workflow to work with ASTs, and they reduce the amount of time that they're parsing JavaScript and generating JavaScript, and now they can parse JavaScript once, work with the ASTs, get the format to format the AST, and it stays in AST, and then they pass the AST to a linter and then they can lint against the AST, and it's a whole standardized toolchain.

I'm guessing here, I'm not super close to how all of this is working. A lot of times, a JavaScript to Rust rewrite isn't faster, because it's in Rust. I mean, it probably is faster. But a lot of times it's faster, because you've had a chance to rework all of the algorithms and how the entire thing works altogether. The fact that they're able to do that, and it probably is faster because it's Rust, but I think there are cases where, there's been a rewrite from some front-end job, or a JavaScript thing to Rust and they use WebAssembly, and they say, it's 10 times faster. You go and look at it, and it's just - they use a completely different algorithm, right? If they had rewritten it in JavaScript, it might have been just about the same.

I don't think that's the case for these tooling, but I do think it's a fun - It's an addition that not only will Rust be faster, it's an additional reason to also optimize all of this toolchain stuff, too. I'm very excited about that. I think that was the core problem with Prettier and how it integrates with the rest of the ecosystem.

[0:38:36] JG: That is exciting. I've got two more topics to bring up with you, a technical and a non-technical. For the technical topic, a lot of people might be listening to this and thinking, "Wow, this is really cool stuff." Or, "Wow, this is really useful stuff." Ideally, both, and they might want to get involved. I know a lot of developers get a little intimidated when you start throwing around terms like parsing and ASTs. Let's say, you were talking to a relatively new developer who's not very confident in these areas, how would you explain what file parsing, ASTs, printing out, what that all means, and what it means for tools like Prettier and ESLint?

[0:39:06] JL: I would explain it as if you have code in a file, it's a string. You need to be able to work with that. The code that you're writing is not English, right? At least now with AI, most of the time, we are writing English, but if you're going to talk about just code, it is a structured formatted thing. We have a specific grammar that you are abiding by when you write code. You have a function, and you have the name of the function, and you have the arguments, and then you have a bracket. To work with that in the compiler, or whatever is running your code, or doing whatever, it needs to be able to work with that somehow, right? It's just a way of parsing that into a data structure. That's all that a compiler is. The AST is a data structure, and it just is a thing, it's a tree, and it represents the different pieces of the code, and it just gives semantic meaning to what you visually see on the screen, which is the code, which looks like it's a readable piece of thing.

It's really a semantic tree of information. That's all that a parser is, is it takes that and creates that AST, which is a tree of semantic information. Then the compiler can take that AST, because it has all the semantic information embedded into it. It can do interesting things to it. It can say, "Oh, this is a JavaScript function, and let me compile that out to a Go function," if you're working on a JavaScript to Go compiler, or something like that. I think that would be the most basic explanation. Does that make sense?

[0:40:31] JG: That does make sense. But it brings up another question of, okay, so Prettier as an example of a tool that works with abstract syntax trees and parsing, takes in your text, your string, turns it into an AST, then what?

[0:40:43] JL: Prettier is a unique one in that it generates another layer, so there's an intermediate representation. Most compilers, what they do is they take in an AST, and then they will generate another AST, and then they'll generate the code from that AST. This is a very simplistic way to put it, but if you're compiling C code to assembly, you have one thing in, which is the AST for the C code, and then something out. There's different ways to do that out part. Prettier is another version of that out part, and the way that it does that out part is it converts the AST into its own intermediate representation grammar, which is another data structure. It's not an AST. I'm not going to go into why. It's more like a flat array of description, which is little pieces of text with additional information about where to break the code. It has this necessary of arrays with the snippets of the text, plus these break informations, which is where we encode how to format the code, and then there's one more pass, which takes that data structure and generates a string, and that's how it works the whole thing.

Prettier is basically a compiler. It's a job ship to job ship compiler, right? You have source code to AST to the compiler, to whatever intermediate output format is, and then something which generates the string from that intermediate output, and then you have the string in and string out.

[0:42:11] JG: Leading question. That sounds real easy. Why is it not easy?

[0:42:16] JL: Yeah. Compilers are really hard. The biggest thing is just performance. You can be really naive and write actually pretty simple compilers. You need a lot of caching to make things really performant. There are a lot, especially for JavaScript, it's a very complex language. An AST is a tree of nodes, but there could be 100 different types of nodes, right? You need to, if we're talking about Prettier specifically, it needs to be able to know how to handle every single one of those nodes. There's like, could be one big compiler file that's thousands of lines long, that is a big switch statement that has switched on the node type. You have to implement every single one of those nodes, right? Then on the output of it, you need to also do similar things also.

Something else that comes in mind that a lot of people don't think of is error handling. When something goes wrong, you need to know where you were in that original source string. You can say, "Oh, I failed on this open curly bracket node," or that's not actually a node, like a switch statement node. "This is wrong right here. And so, I need to throw an error." Well, you can't just tell the error, "Hey, this switch statement is not correct." You have to tell the error, "Oh, this switch statement is not correct. Here is the line of code that is not correct on. Let me display you the piece of code and point to exactly where it's wrong." That is really hard. It's actually really easy. You can sit down and write a compiler. I think there's simple, tiny compiler, instructions of how to do that out there right now that you could probably do pretty easily, but how to trace the source information through that compiler is a really hard problem. There's whole areas of research about how to do that. It's the kind of thing that to make this workable in reality, it has a lot of little practical problems that make it hard.

[0:43:54] JG: I think this will be the last leading technical problem. What you just described sounds really cool, but also a fairly large matter work. I'd rather do something more straightforward and simple, like Prettier. Printing out code is pretty easy, right?

[0:44:07] JL: Here's a perfect example about why it's really hard, actually, is comments. Yes, you can take something and print it out easily, but how do you handle comments? Specifically, this is a unique thing about a formatter, which is unlike a compiler, which you're compiling C code to assembly, you just get rid of all the comments and you just output the assembly code. If for a formatter, you have to maintain all of those comments. What's weird about it is because you changed the code, you got to figure out where to put the comment. That's a really weird problem, because comments are not semantically meaningful, right? I think that was the hardest thing in Prettier is how do we handle comments?

Comments are not an AST node either. When you parse something into an AST, it's not part of the AST. We had to extend ASTs to have a - I think this is how it worked. We could tag AST nodes with like, this was the comment and this is where it was around this node. Then when that was formatted, we could decide where to put the comment in the output. There's so many weird edge cases where you can't just put a comment here, because we've changed it. Now that we've collapsed, we've collapsed it on one line. We can't put the comment in there, because the comment will comment out the entire rest of the line, right? We have to move the comment up and it's a whole bucket of edge cases. It's hard.

[0:45:24] JG: Lovely. All right, last question for you. You have a fantastic personal site with a bunch of explorations. Can you tell us about things like the circle of fits and Lydian scale? What is cool about sound and noise?

[0:45:36] JL: Yeah. This is something that I've been recently getting into is music theory. I'm going to sound like a dork talking about this and also, probably be very wrong, because I'm very new to this. I think when I was learning piano years ago, it was like, to me, the simple conceptual way to think about it was that you just have to learn all of the keys. As long as you play within those keys, then that's how you play music.

What I've learned two years ago when I started picking this back up is that is just so completely wrong. There's these things called harmonics, which if you play a certain note, like a C, I'm not going to be able to say this right, but G is harmonic with C, right? That's a fifth. I do have a little synthesizer on my site, because I was exploring this. What are the harmonics about this? What's weird is that the circle of fifths are - I'm not going to be able to explain this well, but it's frequencies that sound harmonic, but the way that the harmonics work with music theory is that if you keep walking up through the circle of fifths, you'll get into notes that sound harmonic, they're actually not in the same key.

In music, the thing that is fascinating to me about this, and it's not that it sounds perfectly in tune, but what's interesting is that it's a little bit dissonant, but it's a good dissonance. It's a dissonance that gets you to expect something else in the next note. I was just learning about this. There's seventh chords, right? Well, there's actually major seventh and minor seventh chords. If you play C, a C seventh, right? That's C, E, G, and B. That is the C major seventh. But the C seventh, I think the default seventh is a minor seventh. A C seven is actually a B flat. It actually is playing a C minor seventh, which is the B flat is not in the key of C. That is getting you to it. It's a little bit dissonant, but it's a beautiful dissonance, and it's getting you to expect it to resolve either to a full C chord, or something else.

Music is this beautiful movement about a little bit of dissonance and moving up and through the keys. It's something that's really fascinating to me. I'm not explaining it well. But if anybody is here listening to this, that is in music theory, you know exactly what I'm talking about.

[0:47:47] JG: You're learning to play the piano again?

[0:47:49] JL: I am learning. Yes.

[0:47:52] JG: That's a beautiful instrument.

[0:47:53] JL: Yes, the piano is great. I try to learn the guitar for about a year, and it just doesn't map to the way - guitar is crazy hard. I don't know how people play guitar. On the piano, you can see the entire chord and everything laid out perfectly, right? With guitar, you have to learn patterns. Then the way that the strings relate to each other is really weird, and you're having to do these weird shapes. Piano is way easier for me, for sure.

[0:48:17] JG: Well, thank you for all this. This is great. I'm so excited to process everything you talked about, not just with musical theory, of course, but also Prettier and your thoughts in toolings. We talked about how Prettier came to be, some of the problems that it solves for users, some of the pushback that users previously gave and tend to no longer. James, if people wanted to find you online and learn more about you, or your work, where would you direct them to go?

[0:48:40] JL: There's jlongster.com, which is my website, but then I'm also on Twitter/X @JLongster. That's the name that I use everywhere.

[0:48:47] JG: Well, for Software Engineering Daily, this has been James Long and Josh Goldberg. Thank you for listening, everyone. Cheers.

[END]