EPISODE 1618

[EPISODE]

[0:00:00] ANNOUNCER: Mastodon is an open-source, decentralized social network. Eugen Rochko started building Mastodon in response to his dissatisfaction with centralized social networks like Facebook and Twitter. In the Mastodon model, users can run their own nodes and other users can connect to them. You can follow users whose accounts reside in other nodes. Eugen joins the show today to talk about the engineering behind the project and more.

This episode is hosted by Josh Goldberg, an independent full-time open source developer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ESLint, the tooling that enables ESLint and prettier to run on TypeScript code. Josh is also the author of the O'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and a live code streamer on Twitch. Find Josh on Bluesky, Mastodon, Twitter, Twitch, YouTube and dot com as Joshua K. Goldberg.

[INTERVIEW]

[0:01:09] JG: Eugen, welcome to Software Engineering Daily.

[0:01:11] ER: Hi.

[0:01:11] JG: How's it going?

[0:01:12] ER: It's going good.

[0:01:14] JG: So, you are the CEO of Mastodon, the company which is the primary steward and contributor and maintainer of Mastodon, the open source technology. Is that accurate?

[0:01:23] ER: That is correct, yes.

[0:01:24] JG: Cool. Could you give us for those who haven't touched the project much, a quick overview of what that is and how it came to be?

[0:01:31] ER: Well, Mastodon is a decentralized social network or platform. It is actually a piece of software that you can install on your own infrastructure to run essentially your own version of a social media platform with short text messages, polls, videos, images. You follow people, and they follow you back, and then you see their posts in your home feed. You can add them to lists and then you have a separate feed for specifically those people. There is a trending tab where stuff that is popular is being displayed. So, everything that you might expect from a social media platform.

The main difference being that anyone can run this on their own infrastructure. When you do that, you're not isolated, so your own silo, but you can still participate in a global decentralized social network with everyone else who's running their own version as well.

[0:02:20] JG: At first glance, that started to sound like a more popular social network, such as Google Plus or Yammer. But then you started talking about decentralization, which is quite different from how traditional networks work. Yes?

[0:02:32] ER: Yes. Pretty much. I mean, it's a completely different situation. I mean, we're kind of used to there being a platform owner who decides everything, and who runs everything and has to make money to keep doing that. This is different because it's basically a network that can exist, basically, as long as anyone has the motivation to continue to run it, and it doesn't cost as much to run because basically, you can run a node for your friends and family, and it costs very little to do so. If you have a domain name and some Raspberry Pi nodes or something, you can just basically do it for essentially free. As long as you have the technical know-how, of course.

For everybody else who doesn't have technical know-how, there are the large servers that are operated by organizations and high-profile individuals, I suppose. So, one of those organizations is us, Mastodon gGmbH. gGmbH is German for nonprofit LLC, basically. And we operate what most well-known node in the Mastodon space, mastodon.social. We have, I think, over 200,000 monthly active users just on that node. Yes, it's kind of what everybody thinks of when they hear Mastodon.

[0:03:45] JG: I personally found the Fastodon network node, which is free open-source software, which in fact, is what Mastodon is. It is free open-source software, which means you have to have some kind of revenue involved just to be able to run the large servers, right?

[0:03:59] ER: That's true. But the revenue that we have is basically donations from crowd funding. It's not like strictly speaking revenue in the commercial sense.

[0:04:07] JG: How has that process been? Trying to get folks to pay for something that is inherently free?

[0:04:11] ER: Well, how do I say. For the longest time, I haven't been very aggressive about trying to crowdfund and convince people to donate. It's been sort of in the background. I had a Patreon since the very beginning, started working on Mastodon 2016. Back then, it was just basically a hobby project while I was in my last year at uni, and I didn't know that was going to be a thing that I was going to be doing full time for seven more years to come.

So, after I graduated, I opened a Patreon for it, and had like $0 per month for a while, and then I had $5 per month and I was happy. Then, there was the big break and it rose to $200 per month. I was like, "Oh, hell yes, I can do this", because it wasn't a situation where even that amount of money was allowing me to basically pay my bills and continue working on it. Nowadays, I think it's about $20,000 per month. It might be 30,000, if you, buy a year because some of the Patreon supporters are on an annual plan and not a monthly one. So, it's kind of hard to calculate. But something around that point, I think, a yearly budget of half a million or so. Nowadays, it's no longer like a thing I do on my own from my parents' bedroom. It's a nonprofit that's been founded in Germany, and a couple years back. There's three full-time employees, and there is, I think, a total of 12 people working on various parts of Mastodon, most of them contractors, some volunteers, and again, three full-time employees. So, money may sound like a lot to a listener who has never had to hire people before. But having even just a few full-time employees quickly eats into that whole amount.

[0:05:57] JG: Yes. And the nature of open source is that unless you have dedicated staffing, it can be very unreliable to rely, or to try to rely on people for long-term efforts. Right?

[0:06:05] ER: Exactly. Some projects can survive on volunteer contributions, although many struggle with this, and projects just go abandoned, or they don't improve as fast as they need to for the users. But Mastodon is a user-facing product. It's not a programmer library. So, there is kind of different layers of people who can contribute and people who depend on the software, not everybody who uses it can contribute. Of course, there's a higher bar for how good it has to be, and how cohesive it has to be as a product. Because with the programming library, or even, I guess, the Linux kernel, it doesn't really matter if there is like an optional feature somewhere that doesn't make sense. Because if you don't want to use it, you don't use it.

But if there is an optional feature somewhere that sort of interrupts the flow of the user experience, that becomes confusing, that is actually a problem. So, we try not to rely too much on volunteer contributions, that again, somebody can just contribute, but then they're gone, and somebody else has to maintain it in the long term. So, most of our work is done by people who work on that full-time. 

[0:07:13] JG: There's an interesting counterpoint that can be made using the Mastodon model, which is a lot of corporations fear going open source, because then they can't rely on the trust, safety, security of the contributions. But Mastodon is inherently built to be very freedom of speech, very security-focused, and you are fully open source. So, is there a way that you've been able to make sure that folks don't accidentally inject a logger or some sort of violation of the project goals into the source code?

[0:07:42] ER: I feel like it's not a question of proprietary versus open source. In a sense, I believe that open source is definitely more secure than proprietary just because you have more eyes on the code, more auditability. With a proprietary program, I have no idea and I have no way of knowing what's in there, how it's written, if there is a bug in there. Only the people who work at that company can know that, and I don't know if they have the incentives and the time to look for security vulnerabilities or do anything of that sort.

I mean, some have, I suppose bug bounties. But besides that. With an open-source project, the code is out in the open, lots of people can look through it, lots of people can find bugs, submit bug reports, and so on and so forth. Of course, the other side of the coin is that if there is a bug, obviously, somebody else can find it first and exploit it first. Of course, you kind of rely on people reporting it instead of exploiting it, or more than one person finding it and one of those people reporting it. And you can incentivize people to report by doing bug bounty programs as well. That's not unique to proprietary programs, but also to open source. Although, of course, you need money for this. You need more funding to fund the bug bounty programs. We've had some in the past. Our most successful bug bounty program was actually a grant from the European Commission. They ran a program where they gave a bunch of open source projects, I think, including liberal office and us, basically, I think it was $20,000 or something for a bug bounty program and they ran it through a platform called Integrity. We received a lot of very good reports through that based on those bounties, and we improved the software a lot.

One other thing that can improve open source programs security is security audits. Obviously, that's not unique to open-source software either. But if there is a person interested in mastering being more secure, they can basically hire somebody to do a security audit, submit it to us, and then we work out the issues. There was actually recently, Mozilla, has sponsored the security audit for Mastodon, so we improved the project through that as well.

And as to how you can prevent people from injecting vulnerabilities into your code on purpose, code reviews, and basically quality control on the software. It's not like anybody can just write code and put it into the Mastodon repository. It has to go through a pull requests that one of the core contributors who work for Mastodon gGmbH have to review and approve.

[0:10:11] JG: So, it's not the Wikipedia model of software. It's the actual open source that GitHub or similar kind of pull request model of making sure everything is reviewed.

[0:10:19] ER: Exactly. But I'd say, even Wikipedia, has kind of outgrown that. I know that reputation that anyone can put anything on Wikipedia. But I think they have a very rigorous system now where they review the edits. If somebody is trolling or putting false information on there, they get banned pretty quickly.

[0:10:35] JG: Let's talk a little bit about the code itself, because I'd love to dive into how Mastodon actually works. If I were to set up a server, what actually is happening by that process? And how does that interact with the rest of the mastodon network?

[0:10:49] ER: So, we are using a protocol called ActivityPub, which is a W3C standard, that defines a common language between servers using essentially JSON. It's a form of JSON called JSON-LD or JSON-linked data. So, it is JSON with some extra stuff in it, for making it extensible, and making it verifiable. There are ways to modify to bring it in a standard form that can be compared. There's name spacing and stuff like that. That's not relevant for this particular question. But this ActivityPub standard defines how people can send each other messages and what they can do with each other.

So, it's not strictly server to server, it's actor to actor, as the activity pub name suggests. It's about activities and actors who perform those activities. So, every Mastodon user is an actor who has an inbox and an outbox, very similar to email. The outbox is the stuff that you write and publish, and the inbox is stuff that people write to you. All the Mastodon posts and pictures are activities. For example, if you write a post on Mastodon, it is a note object, and you do the create activity of the note object, and if it has a picture, that is an attachment, and if it has hashtags, it's a tag. Everything is defined in this common language. Then, it is distributed to your followers. If you want to follow somebody, you create a follow activity, where the target is the person you want to follow. And then you deliver that follow activity to their inbox. And then they record you as one of the followers. When they publish a new activity, they send it to you, because now they know you're a follower.

It's sort of in rough terms. There is more complexity there to allow for all the different features and security models and everything. But that's the rough idea behind it. There's other protocols that help this, for example, to be able to convert the username that you share around and that you remember. For example, my username is Gargron@mastodon.social. To convert that into an inbox that you can send an activity to, we use a protocol called WebFinger, which is basically a standard URL on a domain, which answers queries as to where can I find information about this user. So, when you mention somebody in a message, your server makes a request that standard URL with the username that you've typed, gets the link in there to the inbox, and then that way they can communicate.

[0:13:32] JG: Got it. So, what's stopping me from creating a bad actor server, getting a lot of followers or, I forget the terminology, exactly, actors working with that server, and then starting to post bad messages, like saying that you Gargron, sent a message to me, or I started following some abhorrent person. Is there anything in the network or protocol that stops fake or bad actor messages?

[0:13:54] ER: There are a lot of security measures. For example, that actors can't fake messages from other people. There's cryptography involved and messages are signed, deliveries are signed. The host names have to match between activities and actors, so you can't just author something that looks like somebody else's message from your own account, and so on and so forth. That's just on the protocol level security.

There's also the moderation level. If you start a server that participates in malicious activity that sends spam or abusive messages, then you as the server owner, can block messages from that server completely, and you will not be bothered by it.

[0:14:32] JG: That's part of the advantage of being decentralized, right? That if there's a bad actor, then you can move to some other server and kind of shepherd them away from the community?

[0:14:40] ER: Exactly. The bad actors tend to assemble on specific servers that they share with other people with similar interests or levels of maliciousness. And then, they're kind of easier to block because you block that server and they're all gone. Instead of being like interspersed with everybody else on the single space, where you're forced to be together all the time like a commercial centralized social network.

[0:15:05] JG: Sure. I'd like to follow up on some of the questions and answers that were brought up in your last interview with Software Engineering Daily. Back in 2018, the concept, the topic of blockchain was brought up and you very intelligently predicted that it would not be a major part of Mastodon in the years to come. Since then, blockchain had its peak, modern crypto, Web 3.0 and so on had a very big popular peak, and has fallen down since then. Has your stance on blockchain in any way shifted over the last few years when it comes to Macedon and ActivityPub?

[0:15:38] ER: No, absolutely not. I said it was stupid back then and I'm still saying it now. It's a solution in search of a problem. I have never, in my years, working on the software or before that, seen a situation where a problem would be solved by a blockchain and not by - I mean most of the products that have a blockchain integrated in them, they could have just as easily just been using a central database. There's no real benefit to it except hype, and trying to make money.

[0:16:07] JG: This brings me so much joy to hear. Thank you. You also mentioned that one of the big tech debt areas in the Mastodon codebase, back then was Webpack. That because it's a little bit older of a web project started at around 2016, 2017, you will still rely on some older Webpack scenarios which have since been more solid in the public space, such as with Vite. Has that situation changed at all?

[0:16:31] ER: I'm sorry. Is that how you pronounce Vite? Because I've been saying Vite.

[0:16:35] JG: I've been saying Vite but I think it might have evolved into a GIF/GIF situation. I'm not -

[0:16:40] ER: Okay. All right. Well keep that in mind. Yes. So, it's kind of strange to me to hear Webpack being described as all technology because I remember when JavaScript heard stuff like Gulp, and stuff like that, and Webpack was the new shiny thing that was so much better. But yes, if I have to talk about Webpack, it's been a pain for years because it's basically the biggest resource hog in Mastodon, and Mastodon is written in Ruby. A lot of people think that because it's written in Ruby, it is a resource hog. I would actually counter that claim and say that as far as Ruby applications go, Mastodon is pretty efficient. In any production-level application, you're going to hit the bottleneck of your database sooner, and then you hit the bottleneck of your programming language.

But Webpack has been a pain because on smaller setups, let's say, people using the Raspberry Pi's or a small VPS with only one gigabyte of RAM, it's been just the deal breaker. Because it needs like multiple gigabytes of RAM just to run once. The only reason we make people run Webpack at all in production environments is just so they can compile the assets from the source code, right? It's actually becoming less relevant now, because we're switching our focus a lot more on containers and providing Docker containers and Helm charts, so people can deploy mastered on using Kubernetes and stuff like that.

We're no longer insisting that people go and install all of the dependencies raw on their Linux install. So, in a container, the Webpack step actually runs on our machines that build the images and upload them, so you don't have to do that yourself anymore. But for years, it has been a very big source of anguish for admins, and for us, hearing that Mastodon is such a resource while because the Webpack process runs out of memory.

[0:18:32] JG: We can consider this an open call that for contributors. If anyone listening is a particularly savvy Webpack, [inaudible 0:18:39], or Vite, or similar user, perhaps this is a good opportunity for them to get involved with the project.

[0:18:43] ER: Yes. The thing is, that over so many years, we've had quite a few customizations and specific workflows integrated into Mastodon for generating the code, that have been a little bit harder to untangle and replace with something else like Vite or Vite. For example, I think we have some custom code for generating all of the emoji-related stuff in the front end. So, Mastodon has this. Obviously, it has support for emoji, and it has an emoji picker, where you can click on an emoji in the compose box, and it opens a list of all the emojis and you can click on them. But also, you can type a short code, and it will suggest you an emoji and then you can just tap it, and it will insert it very simple stuff. And then it also has to - when it's rendering all the posts, it has to replace Unicode with the appropriate graphic from the two emoji pack that we use for consistency, because some platforms, like Windows, don't have very good-looking emoji yet.


So, for that reason, we actually need this huge list of all the Unicode emojis and all of the graphics paths that they correspond to, and all the short codes that they correspond to, and that is a massive, basically database from a JavaScript package. So, the customization that we have was generating a sort of smaller list of that, only stuff that we need from a big database that is a package on npm. And this was done using pre-eval, in Webpack, which basically runs the JavaScript code during compilation instead of in runtime, and then saves the result of that instead of the source code.

That has been like one of the parts that's been kind of hard to replace. It's kind of an effect of when you have such a project, people come and they bring optimizations, right? Then, that's really good, because it makes stuff faster. But if you ever need to change it again, you have to undo the optimization, and that can be harder than just starting from scratch.

[0:20:41] JG: Yes. That's wild. You're one of the rare people I get to interview who in one sentence are talking about Helm charts and Kubernetes, and the next sentence talking about Unicode and emoji on the front end. Do you ever feel stretched across that giant spectrum of technology that it can be hard to focus on one area?

[0:21:00] ER: I do feel a little bit stretched, but it's kind of my role. I mean, I am in technical terms, the CEO of the company. So, I still write code for Mastodon, but we have one other developer who writes code probably more often than I do nowadays. But my responsibilities now include managing other people and across different teams. So indeed, we have a DevOps engineer who is running the deployment of Mastodon Social, and we're using Kubernetes, and Helm for that. And of course, all the technologies involved in Mastodon itself, that's basically what I grew up on, what I started with, and wrote myself. Then, of course, we have two teams. One working on the Mastodon for Android app, and one Mastodon for iOS app. Those are using different technologies as well, because both are native apps. One Java, one Swift, so I have to manage all of that.

[0:21:52] JG: That's quite a big spectrum. And then to add on to that tech requirements, the difficulty of maintenance, your platform, as part of its nature, supports folks swapping in and out different front ends. For example, I personally use the very excellent Elk Project for my Mastodon clients. Do you find that having to support folks doing wild and wacky things in community space adds additional cost to running?

[0:22:15] ER: Actually, no, I did not. Because all of these projects, they're using the same API that our web client is using. That was one of my original goals in making Mastodon that it was going to be API-centric, that it was going to have a very sensible and clean API that anybody could use to make their own client. That has kind of been a success, and there is not a function in the web app that isn't available through the API. So, it's really totally not a problem that projects like Elk or Fan Pi, use the same API to provide different interfaces. In fact, it's good. Because if somebody is unhappy with Mastodon using the standard interface, and they're happy using Elk, that's a win for everybody.

[0:22:55] JG: Yes. Let's talk about that. What are the things that you want to work out over the next year or three for the standard interface or underlying platform?

[0:23:03] ER: Well, what we're doing right now is a little modernizing of the underlying technology. So, it's going to be - that might sound boring to people who are not programmers, but this is a programming podcast, so I think I'll allow myself to like focus on that. As mentioned, we need to change from Webpack to Vite, to just make every operation related to compiling the code, and reloading the code faster. But we're also slowly replacing the actual front-end code. Because it's a mix. It's a mix of everything. There's different programming styles in there. There's different approaches to state in React in there. Because basically, it has existed as a code base for seven years.

When I started, the landscape of React, programs was quite different to what it is now, and Redux was still quite new. I chose that technology, because I realized that - well, actually, Mastodon front end at first was a multi-column application like TweetDeck. That was like my thing that I wanted to do. So, I realized that the same data would be displayed in multiple places at the same time, so it would have to be normalized. And so Redux with global state was the tool for that.

But even within Redux, so much has changed over the years and what they recommend is a best practice, the style of JavaScript itself has changed, like all of these extensions that have been added all the code features that have been added and removed over the years through Babel plugins, and presets. So, some of the oldest code is using like, basically, it's not using the shorthand function definition syntax, which I now quite like, and it's pretty quite standard in JavaScript project.

You find some places that have like this really old-looking code, and then you'll find places that have the new-looking code. But then, even more importantly, we're switching out class-based React components to functional components and hooks. I'm actually quite a fan of the hook system. I don't know why. I mean, honestly, I actually don't know if it has any tangible benefits from using classes. But when I write functional components with hooks, it just feels better for some reason. Like I'm doing something cool. So, we're switching out that. And then of course, there is a bit of a move towards TypeScript in the code base. Our code base is huge. So, you can do it in one go. But one by one, we're replacing the components. I'm personally not 100% convinced that this is actually having any benefits, because I'm not really a TypeScript person. But nevertheless, that's what we're doing.

[0:25:31] JG: I have a theory that one of the many benefits of the function components over class components in React, whether it's JavaScript, or TypeScript, is that the code becomes a little more succinct and predictable to read. I hope that that works out well for you.

[0:25:43] ER: Yes. I suppose so. I mean, perhaps it's just an adjustment period. But there are some components that even functionally, they have a sort of big area in the front, where it's just definitions of state, and all those different hooks, which can look a lot quite succinct is what how I describe it. Nevertheless, that's one part on the JavaScript side.

Then, of course, there's cleaning up to do both visually, and in the way that components are styled, and the way that style sheets are written, has to be cleaned up. I want to move towards using CSS variables, because what we've been using so far are SCSS. That's Sass, right? That's the name of the technology, Sass.

[0:26:24] JG: Sassy CSS.

[0:26:25] ER: Exactly. We've been using Sassy CSS, and we've been using their variables, which are obviously not actual variables in CSS afterwards. And the way that I wrote the design initially involved a lot of modifying the different colors by using lighten, darken to create sort of levels and distinctions in the different UI elements. That's not been very good for being able to theme it and change it dynamically. And also, for maintenance of all the different colors. If you want to change them around, you'll find afterwards that because they were relying on the color being darker or lighter, that it doesn't look as good with a different color than it was originally.

So, that's another change. Changing to CSS variables. Then, of course, we also want to integrate with some of the newer CSS features like nowadays, you can check if somebody prefers the dark theme, or if they prefer higher contrast, or if they prefer reduced motion in the stylesheet. But when we were writing the code that didn't exist, so all those things were just settings in the Mastodon code itself that were then used as classes.

[0:27:36] JG: Especially, if you're switching from, let's say, only having one light mode to supporting multiple dark mode, light mode, even high contrast mode. It can be very difficult to massage the CSS into say darkening instead of lightening. Right?

[0:27:48] ER: Exactly. And there's been an additional challenge that years back, we added a theming system and the light and the dark and the high contrast styles of Mastodon are separate themes. So, this idea of making them support your browser preferences means that they have to somehow be united into one again, or made a special case, visa vie, like completely different themes. Like when somebody styles Mastodon as Windows 98 or something.

[0:28:15] JG: Do people style Mastodon is Windows 98? Is that something that I can opt into?

[0:28:20] ER: Oh, yes, that theme exists somewhere. Yes. I think it originated on the server called Cyberspace, or something like that. I'd have to look it up. I think it closed down a few years back. But the theme remains somewhere. So, yes, I'll give you a link.

[0:28:35] JG: Oh, that's beautiful. Are there any other wild and wacky projects you'd like to shout out such as the Windows 98 theme?

[0:28:41] ER: Well, perhaps not wacky. But there is a person working on another theme. I'm aware of two different themes for Mastodon. One of them tries to make it look as close to Twitter as possible. I think it's called Bird theme or something like that. The server of the person who makes it is called Mementomori.social. That is an easy-to-remember name. But the name of the theme itself, I'm kind of blanking on right now. Then there's another one that has a kind of orange color to it, but honestly, I forgot what it is actually called. But they are quite popular, those two themes for people using Mastodon.

[0:29:15] JG: We can include links in the show notes, I believe. That's very exciting. That's honestly one of my favorite features of Mastodon that you can have all these fun things made by the community that they're directly supported by the platform.

[0:29:25] ER: Yes, of course. Another thing that we're modernizing is the visual look of Mastodon itself. I think I've made quite some strides in the recent two versions, 4.0 and 4.2, where we've really cleaned up the UI and made it look more modern. And in the upcoming version, we're switching our icons from Font Awesome 4.7, which is like probably a decade old by this point, to the most recent material icons, which would be consistent with our Android app. And so, it already looks more modern with that.

I've been doing like a lot of cleanup of input elements, making stuff more consistent. There's lots of stuff to redesign. There are screens that have not had received as much love as they should have. One cool thing is that because we have professional designers working on the iOS and Android apps, whenever they touch a feature, and they create mock-ups for it for the app, I can work from that and integrate that back into the web app and make that look more polished and professional. So, probably going to redo all of the list-related parts in the web UI. Because that is a very underdeveloped part in the UI.

[0:30:37] JG: Do you have a formal design system?

[0:30:39] ER: Not in the web app now. In the new apps, yes. But that's because we were able - basically, both the Android and the iOS apps were started from scratch about two years back with professional designers straight from the start. So, of course, when you get the chance to do it from the beginning, from a blank slate, you develop a formal design system, and it works great. But Mastodon started as a hobby project, and I've been started developing it for years and years. So, no, it does not have a formal design system. The most formal design system is in my head of, this is how it's supposed to look. This is all the different elements. But even in the style sheets themselves, it's kind of a mess, if I'm honest, from a code perspective, in terms of reusability of the different classes and elements and stuff like that. So, there's a lot of work that can be done on consolidating the different elements and looks in the UI into a more sensible cell sheet.

[0:31:33] JG: I think it speaks positively to your character that on a podcast, you admit, some of the code that I wrote as a hobby project might not have scaled and aged over the last seven years. I want to bring up a few of the most commonly requested issues or topics for Mastodon. I have an issue query up on your issue tracker. On GitHub, the most commented issue is 12423 support post-migration. I'm guessing you've talked about this in the past. Could you tell us what that is and why it's so contentious?

[0:32:03] ER: So, the thing about Mastodon being decentralized is that you kind of rely on either hosting it yourself or relying on the third party to host it for you. And the third party might not provide the service for as long as you want it to. So, in other words, servers shut down sometimes.

So, one thing that you can't do in Mastodon is you can migrate to another account. You can just say, "This is my new account." And Mastodon supports moving all of your followers from the old account to the new account. That's actually pretty amazing. In many ways, I think that is sufficient for it to not be a big deal that a server shuts down. I mean, there are definitely edge cases or scenarios where it's really bad because, what if you don't get a chance to move it in time, and then you lose your account. That would be horrible. That's why it's also important for the providers to be reliable and trustworthy parties. 

I mean, just to give some examples, there's a server operated by Mozilla. Mozilla.social and there is one operated by Vivaldi, vivaldi.social. Of course, mastodon.social. They are big organizations with a reputation and no name. They're probably not going to disappear overnight. But if you choose the random server hosted by an individual from somewhere who has like 10 users, there is a chance that this will not stay for years to come. So, that's just the preamble to that. You can move your account and you can move your followers, because Mastodon is mostly a microblogging network. Most messages ephemeral.

So, it doesn't matter too much that you lose, like your old posts, because they're probably not relevant anymore. But of course, there are like people want to keep them as well. That's why that is one of the most commented-on feature requests. They want to move to a new account and they want to also keep their posts and not just their followers.

I have talked about this at length about the post-migration, and I honestly don't want to shoot it down. I think that it's something that we want to work on, and have it working. But people need to understand that it's not as simple as it sounds, and mostly not just from the technical perspective, but also from basically a security perspective. Because basically, what you want is you take your entire posting history from potentially years, including media files that could be like gigabytes in size and total. You could have thousands of videos or images, and you want to take that and you want to move it from one provider to another provider. That other provider has to agree to this. They have to be like, "Okay, oh, a new user that comes with like 100 gigabytes of data with them. That is a tough pill to swallow for providers that people can just come in and bring all of that stuff with them."

So, just from a technical standpoint, transferring large amounts of data is a bit of a problem. From a second standpoint, what if it's not a person and doing it, but what if it's a spammer? What if it's a spam bot, and they're bringing all of their pre-written spam posts straight away in this convenient format? That is, of course, another consideration. Then, the third consideration, okay, it's not a spammer. It is a person, but they are malicious, and the post that they bring violate the rules. This can be a problem both in terms of bringing rule-violating posts into a server, but also can kind of frame the other server for hosting that sort of content.

Like, "Oh, look, they're hosting this person for" like, they have 70,000 posts on this account. This must be a badly moderated server, where in fact, they just migrated. So, those are the challenges, both technical and in terms of security and sort of moderation that this feature comes with, which is why it's not something that we've just done in 10 minutes, years back. But it's something that we want to look into. I mean, we were looking into it, and that's something that we'd want to get done eventually. 

Because yes, it's one of the - when people talk about Bluesky, for example, that's one of the - at least from when technical people talk about Bluesky, that's one of the things they mentioned. Like, "Oh, data portability, like account portability, that you can bring your posts from one to the other." I think all of those challenges that I mentioned, are actually definitely going to be a problem for Bluesky as well in that regard.

[0:36:24] JG: What is Bluesky?

[0:36:24] ER: So, Bluesky, it's a social media platform that is also developing its own decentralized social media protocol. Except that last I looked, they were not actually decentralized yet. They were just theoretically considering it in the future. That might have changed in the meantime, I'm not sure. But that was basically the situation for years. It started as basically Jack Dorsey's project, where he was like, let's make Twitter decentralized. Let's invite a bunch of people from different projects that deal with decentralization into a chat room and have them figure out which one's the best. And then we'll adopt that, right?

So, for a whole year, there was like, I don't know more, probably more than 20 people in a chat room in matrix discussing with each other, like which social media protocol is the best. So, I was there representing ActivityPub. There's a couple of other people representing activity pub, I believe. There are people from a project called GUN, or G-U-N, which is an easy-to-remember name, but I don't remember super much like what their protocol was about.

There were others. There's people from Matrix, and so on, and so forth. Everybody was making their pitch. For a year, there was kind of very little movement from the sides of Twitter. There was like one person from Twitter in the chat room, and they barely ever talked. Then, I sort of moved out of that, because I didn't feel like it was a very productive use of time. But eventually, it did make some movements. They started a Discord server and moved everybody from Matrix to Discord. They gave more details to the public. They made a document where they described all the different protocols, and with the comments, and then eventually, they said, "Basically, we don't like any of these, we're going to make a new one."

So, that was the birth of what they call AT Proto, their own protocol. Then, they hired somebody to be the leader of the project and a bunch of other people to do the work, and they started a public benefit corporation, or public benefit LLC. Honestly, I don't remember what their exact form is, I just remember it was public benefit, and they received some funding. I think, 8 million or so. And they started Bluesky. And then the only thing is that Jack Dorsey left Twitter by that point. So, he was no longer in a position to make Twitter actually use this protocol, so it became something slightly different, basically in the same position as Mastodon in terms of their chance of becoming the de facto protocol.

[0:38:58] JG: So, Bluesky, in theory could eventually integrate with ActivityPub. AT and AP could talk to each other in the future?

[0:39:07] ER: Well, no. The thing is, I doubt it. I mean, different protocols don't really talk to each other. You either use one protocol or the other. Honestly, I wish they'd used ActivityPub, because everything that they wanted to do on paper, they could have done with ActivityPub, or by extending ActivityPub, or building on top of it. It's a very extensible protocol, and the players in this ecosystem are very open to growth and new features, and just getting stuff to work better. But they chose to do something completely separate. Besides using bridges, I don't really see the protocols themselves talking to each other in the future. So yes, it's really a situation where now there's two competing standards. But one of them is you know, a W3C-approved standard.

[0:39:52] JG: Was that a little humblebrag right there? A little jab?

[0:39:56] ER: Well, it isn't a brag. I didn't make ActivityPub. I was just sitting on some of the sessions, giving feedback. But yes, I'm definitely in the ActivityPub camp, if that's what you're asking.

[0:40:05] JG: Sure. That makes sense. I want to, actually, before we start talking about the broader ecosystem. Go back to a couple other features or UX areas in Mastodon. DMs, direct messages. I cannot currently slide into anyone's DMs as the youth say, because DMs work quite differently on Mastodon than say Twitter or Facebook. Is that something you are willing able to comment on right now?

[0:40:28] ER: Yes, I can comment on that. It's kind of my fault, really. Going a little bit back into Mastodon history, right? Mastodon actually didn't start on ActivityPub. It started on a preceding protocol called [inaudible 0:40:40]. That was the dominant, I mean, I say dominant. It was the dominant protocol at the time, except the dominant was it was like, very few servers, and very few users actually using it. But it was still the one protocol that was known and the route, except it wasn't a protocol, it was a draft, it was never published formally anywhere. But okay, that's just details. Basically, that protocol was not using JSON, it was using Atom feeds, basically, like news feeds that you can subscribe to. But for accounts that had some extensions, very similar, like activity streams, kind of related technologies, but with the XML. It had no concept of privacy, or private posts, or anything of that sort.

So, Mastodon was originally built on that, to integrate with the existing platform at the time, called new social. When Mastodon started getting attention, started getting big, when actual people started using it, they started asking for privacy features. Let me lock down my account so that people can just follow me automatically so that I can approve them. Let me post updates that only my followers can see. Let me do DMs. I was not part of the protocol that existed.

So, there were some shenanigans with trying to extend it at first. But then I was very happy when ActivityPub came along, because that article actually had all of that stuff figured out and formally defined like privacy and audience levels and targeted audiences, and so on, so forth. But the thing is, that it did not have a - no, in fact, it goes back to a different decision that I made, which was, I had to decide, do I want to make a separate model for DMs, and redo all of the image attachments, video attachments, polls, and everything else that you can do in a message, again, for a different model in the code, or do I just want to reuse the status model, and just make it have like privacy settings. So, you can decide who sees it, right?

I made the choice for the latter, and that is the source of the problems that you're mentioning now, essentially. Because even when the protocol changes came about, and you could formally define in the protocol who the intended audience is, and so on, and so forth. The real fact is that in Mastodon, there's just no difference in the models of the status and the direct message. That definitely has been a source of some user experience issues in the past. Because on one hand, sometimes you have people not realizing that you can do this, and that Mastodon sort of has a DM feature. But on the other hand, you have people unhappy that the DM feature works as it does, because it does not actually work intuitively as DMs on another platform.

The main difference being that DMs are basically part of a thread. They're just normal posts that are visible only to the intended recipients. But otherwise, they act like normal posts. So, what you can do is you can have a conversation, a public conversation, and then you can take it private from there. But it's still part of the thread. And if you mentioned more people in the post, then they can see that post and so on. That can be confusing. There are definitely benefits to it as well, because it's kind of like a way to take somebody aside in a conversation and like, "Okay, this does not concern other people anymore. Let me just tell it to you specifically", and that's good. But then there's the bad, is the unexpected behaviors, and the fact that you can just say I want to DM this person, and just want to have like a one-on-one chat, like you would expect in Telegram or Discord or whatever. That's definitely been a source of contention.

[0:44:12] JG: That's fascinating. So, in a sense, DMs or the equivalent on Mastodon are much more fully featured than other platforms. They integrate with the rest of the platform, at the cost of a lot of people have no idea how to use them, is an immature rephrasing. 

[0:44:27] ER: Yes, exactly. At the cost of basically - the main problem I'd say is the threading, because let's compare to like Twitter DMs, right? At least how they used to work. I haven't used Twitter in like a year probably by now. You would define like a private conversation, right? And you'd have like the recipient, and then you just have like - no matter what you wrote, or you don't have to think about threading replies with each other. You just write a reply and it's like a chat, like a one-on-one chat. Then, if you add more people to it, they can see all the past messages, but you can never like add somebody to the conversation accidentally either, right? So, that's how people expect them to work.

On Mastodon, there are posts, so they have to be threaded for them to show up as replies. Anybody who's mentioned is the intended recipient. So, if you mentioned more people, there'll be able to see that specific posts. But if they're not mentioned in the previous ones, they cannot see them either. That is the main difference.

[0:45:24] JG: Speaking broadly, one could interpret there being an unfair responsibility on Mastodon shoulders that you are the premiere, not Twitter, social network right now. So, when there's a user experience pain such as the difficulty of understanding or using DM, the thing that comes to mind for folks when they compare you to Twitter. Do you feel that that responsibility exists and is fair? Or is that an inaccurate way to phrase how folks view Mastodon?

[0:45:53] ER: It's probably an accurate way of describing how people view Mastodon for sure. I mean, people compare Mastodon to its competitors all the time. But I wouldn't say that being the primary alternative to Twitter is like a burden. I mean, in some ways, it is, because all of these other people who run social media networks, like Mark Zuckerberg, or Musk, they're billionaires, and they're making money off of it, and I'm just some guy, who is just making a modest living out of it, and some of the responsibilities that fall on me from the attention that Mastodon gets, as the primary alternative there. They don't seem fair sometimes and it can be tough.

But on the other hand, it's kind of a nice problem to have, because you've created something that people want to use, and it brings me joy when people use it, and when it gets popular, right? So, there's that, of course. But I'd say, if we're talking unfairness, there are some parts that can be described as unfair. Because if you go into it from a technical standpoint, Mastodon being open source free software, and being based on this open protocol, means that if you don't like something about how it works, you can just change it. If you don't like Mastodon specifically, or if you find some issue with how it does things, there are other projects in the ecosystem, in this decentralized social web, that do it differently, like, that are using the same protocol.

There are platforms like Misskey, and Firefish, and Pleroma, and Pixelfed that do different approaches to the same idea. They do it differently. They look different, and they have made different decisions about certain features, or certain visual approaches. So, from a technical standpoint, you could say, you don't like something specifically about Mastodon, you could try something else that's still in this ecosystem, and you don't necessarily have to exert the pressure on Mastodon itself to change something just for you, when you have all this freedom.

But that unfairness only exists on this technical level, because in practical terms, you can't expect people to know all of these details, or to have the skills to change the code, or to have the skills to run their own server, or to have the knowledge that these other platforms in the same ecosystem exist, and that they can be better. Some of those projects are not as good or as easy to use for a new user, as Mastodon is. So, there's a lot of - you have to apply a lot of understanding to this sort of unfairness, if you put it that way.

[0:48:34] JG: That makes a lot of sense. There's a sweet irony there that Twitter and similar have a tendency to boil issues down, so I can fit in a 280-character tweet, but this issue cannot be boiled down to that. It's actually quite nuanced and complex, that we're trying to build a decentralized alternative to the traditional billionaire-backed social media platform. There's going to be nuanced difficulties along the road.

[0:48:54] ER: Yes.

[0:48:55] JG: That's great. I'd like to end our last few minutes together with a few less heavy questions. For example, toot. What is your opinion of that word when it comes to Mastodon?

[0:49:05] ER: There is a funny story behind that. Originally, there was no special word for posts on Mastodon. In fact, in the code, they're called statuses, because they're status updates. All of this microblogging stuff actually comes from status updates on instant messaging applications. I remember when I was using Pidgin, with Jabber, you would set your status to listening to, I don't know, some band that you're listening to. Or out for a walk, and that updates you could - on some clients, you could even see like the previous ones that you've had, and I'm pretty sure that's where like Twitter itself came from. It was just an extension of that, let's do that, but without the instant messaging. Let's just have those status updates.

So anyway, posts in Mastodon were called statuses in the code and originally. But then when people started using it in late 2016, there was a guy a YouTuber, [inaudible 0:49:56] who approached me and he basically said, I'll support your Patreon forever if you change the button to say, "Toot". I'm actually skipping ahead a little bit, because there was like a lot of conversations going after that point about what the post should be called. Because that was basically one of the first things that people discussed on the platform. Like, "Okay, we're here. What are they called? They're not tweets. What are they going to be called?" So, people have competing ideas.

I remember, I think the first community that came on Mastodon was like the Dutch community, and they had like their Dutch word for it, that was also very similar to what the German word nowadays would be like. I think it was called [inaudible 0:50:32]. It's very similar to the German term, which is basically the toot, but in German and Dutch respectively.

Anyway, so essentially, there was some kind of community consensus on it, and [inaudible 0:50:42] kind of approached me and said, kind of, like jokingly dared me to change it. I just did. I just changed it. We've had it for years, up to I think last year, is when we changed it back to post and publish. Over the years, we've received a lot of complaints about it being toot. There are a lot of people who find it endearing. I mean, if you just think of it as the noise that often makes, it is. Unfortunately, in some slang, it also means to fart, which is worse. There have been a lot of opinion pieces by big journalists or political pundits. They were like, "It can never take off if they're called toots." So, eventually, I just got tired of just it being that silly, and I would just wanted something neutral. Okay, let them be posts. Let the buttons they publish. And if people want to call them toot to each other, they can do that. But the software is not going to enforce that. That way, we get rid of all of these complaints about being not serious enough, and too silly, and fart jokes, and whatever. So, that is my opinion.

[0:51:42] JG: Got it. Yes, I think Mastodon, it does not have a lack of fun things such as the aforementioned Windows 98. Last question I have for you. Mastodon is famously named after some band or song in progressive rock. Prog rock has a lot of interesting philosophies about freedom of expression and individualism. Do you think that those philosophies Jive well with the ideas of free speech and expression in Mastodon itself, the software and the company?

[0:52:09] ER: Gosh. I'm not even sure how to answer that. Because I don't think I've engaged with progressive music for its values on free speech or expression or anything like that. If it sounds beautiful, that's why I like it, and I wouldn't even have found that if you didn't put it that way. Mastodon is indeed named - I mean, technically, Mastodon is named after the animal that existed. It was like a predecessor to mammals. Mastodon was also a bully kind of elephant with a shorter trunk.

But the reason that I'm aware of that animal is because of the band Mastodon, which are - I don't think they're actually technically progressive metal. They're just metal or maybe sludge. But some of their albums and songs definitely probably go into the progressive area, for example. I would recommend the album The Czar. That's one of their best, in my opinion. Anyway. Sorry. I don't know how to answer the question about Mastodon and progressive music. I just like progressive metal and rock.

[0:53:07] JG: In that case, let's narrow the question. What do you like about say Mastodon or that Czar album?

[0:53:15] ER: I don't know. No, it just sounds good. I mean, I'm not a musical analyst. I can't like, I mean, I probably could, if I sat down, and I worked out all the little details that I like, but I just think it sounds good. That being said, Mastodon is not - it's not my favorite band. It's just a band that I like. So, I mean, if we went to the bands that were my favorite, I'd say, I listened to Leprous a lot. That is actually progressive metal, straight up, not that sludge or anything. Unlike Haken, unlike The Deer Hunter, that is not metal. That is more like progressive rock or theater rock. And of course, Devin Townsend.

[0:53:49] JG: Well, that's great. That's all the questions I have for you. But thank you so much for coming on. Is there anything else you'd like to leave the listeners with before we head out?

[0:53:57] ER: Thank you very much for having me on this interview, and I hope everybody enjoys using Mastodon.

[0:54:01] JG: Great. Thanks so much, Eugen. This has been an absolute pleasure. Have a great day.

[0:54:05] ER: Thank you. Cheers.

[END]