EPISODE 1972

[0:00:12] GV: Hello, and welcome to SED News. This is the monthly installment of SE Daily with a slight twist, which is that we go through the main tech headlines, we then dive into a deeper topic in the middle, and then we take a spin around our favorite Hacker News, things of the month in Hacker News highlights. As usual, it's myself, Gregor Vand, and with me is Sean Falconer. 

[0:00:34] SF: Hey, Gregor. Hey, everyone out there. 

[0:00:37] GV: Yeah, so I'm going to be seeing you soon, Sean, I think. I'm coming in. I'm actually heading off on a flight after this to SF, going to our Supabase Conference on Friday. But I think you're going to come along for some of that. 

[0:00:50] SF: Yes, I will be there at least for some of the day on Friday. And I guess you'll take off and then, I don't know, 30 hours later, you'll be in San Francisco or something. 

[0:00:59] GV: Yeah, it's one of the funny ones where you leave on a Tuesday afternoon and arrive on a Tuesday evening, even though you just spent close to 30 hours traveling. Yeah, it's going to be a good week ahead. Obviously, that's sort of a lot of my week looking forward is, yeah, going to SF. Given my role at Subbase, I also have a lot of people to meet. All the main companies we talk to, people like Vercel and OpenAI. And yeah, it's going to be fun on the Friday. We've got Steve Wozniak as one of the headline speakers. So that's a bit of a pinch me moment to be in a room with Steve Wozniak. But yeah, what about yourself, Sean? What have you been up to? 

[0:01:36] SF: I mean, luckily, no work travel for me for a little bit. I do have a personal trip doing a pilgrimage to the Great White North to visit my family here for Canadian Thanksgiving in a couple of weeks. Primarily just heads down on kind of Q4, 2027 planning. We also have our annual conference coming up in November. So we're all kind of busy working on that to pull all those things together for the launches and big announcements that we need to make. So people listening that are interested, stay tuned for all that stuff. 

[0:02:06] GV: Yeah, nice. But yeah, otherwise on my side, yeah, the pollution in Singapore has been a bit crazy actually lately. Basically, there's a tech angle to this in a second. Yeah, so it's basically a lot of haze from forest fires coming from Indonesia. It passes over Singapore. But unfortunately, yeah, it's been sticking around for quite a while. But it did prompt me to buy an air purifier. And I'm sure some people have them. I'm a bit late to the party maybe on that one. But yeah, super happy. I bought a Dyson air purifier. I think it's just amazing. The tech is like - I don't know. It looks cool. The app is really nice. Yeah, I don't know. It's one of these like new tech moments. I didn't expect to be quite so attached to an air purifier, but I really do love it. My wife's always just like, "Go. Just move around, whichever room you want that thing in, it's yours." Yeah, I don't know. Do you have one in SF or probably not needed, right? 

[0:03:02] SF: I don't think so. No. I mean, I probably would have came across it, but it does feel like something that my wife would invest in. 

[0:03:07] GV: Yeah, I think some people get them, especially when they have like early infant kids, they sort of get a small one to stick in the kids' room. Certainly, first time for me. Because of the app, you can look at your air quality indoors. I'm in the office and I'm checking. When it gets a bit high, I just stick it on. Things that you can get excited about when there's pollution. 

[0:03:29] SF: Yeah. I mean, we do get fire. There's fires usually in Northern California that impact the air quality. You can send me your tips for air purification. 

[0:03:39] GV: Cool. Yeah, I should do that. I say cool. I mean, it's not exactly. 

[0:03:44] SF: Yeah, awesome. 

[0:03:46] GV: Awesome. Yeah, fire is awesome. But yeah, at least there is tech out there to help with this. Yeah, I guess diving into the headlines, we've seen a couple of kind of large product drops, if you want to call that, in the last week or so. First one is Jev. So this is from a company, TypeSafe AI. The company was actually founded in 2024. But, I mean, I don't think many people really knew what they were up to until about September 15th, when basically they released Jev, which they pitch it as effectively a model where you don't - the outputs are not chats. They are TypeSafe'd responses with a score against them as to like how accurate it thinks that structured response is. 

And the whole point is this thing is not meant to be really used by a human. It's meant to be sort of layer one. They call it like system one layer for structured decisions, like tool calls on routing and classification. But the big thing here is it made such a huge splash, if you want to - Ii don't know what other word to use in the tech world. And it's kind of being like touted as like is this the next revolution on LLMs. It was founded by ex-ChatGPT engineers amongst others. It's got some pretty sort of big heft behind it in terms of who's behind it. But yeah, on the counter, there isn't like a lot that we can really dig into though, in terms of open weights or anything like that. Is that right, Sean? 

[0:05:17] SF: Yeah. I mean, there is, there has been recently, there is open Jev, which is now on Hugging Face, which is a full size 27 billion open weight alternative with near parity. Came out two weeks essentially since the launch. But that was something that I think that was a distilled model to kind of mimic the Jev experience. It's not official. I don't think it's officially from the owners or the creators of Jev. 

But it is really interesting what they've done. It makes a ton of sense. Because what's happened with large language models is that people use them for a lot of times stuff that they're not necessarily this most efficient model to do, like classification, even choosing a tool and things like that. These are more structured response. And you have to do a lot of work to kind of force them into this behavior of accepting structure and generating structure. And generally, a lot of times it's not that reliable. And it's inefficient in terms of token costs, inputs and outputs and things like that. 

So to have a model that's using the same kind of transformer approach, technology and general-purpose, but specific for solving these kind of more structured things where you expect a structured response around classification solves is like a much better way of essentially solving that problem than trying to jam that into something that's accepting of large language models. Because those were really designed to kind of function as a human might function in human modes of conversation. But those are very inefficient for a lot of these types of tasks, whereas Jev is really built for like machine-to-machine communication, where we're kind of trying to jam an LLM into, but it's just not a great use of tokens. It's expensive and it's also costly in terms of just the time to compute in the inference cost. 

So if you focus on that, you make a much more efficient model, both on token costs and then also the time essentially that it takes to process it. So it makes a ton of sense. And I think that's why people are excited about it. It fit this gap in the market essentially between what you might do with traditional purpose-built models for things like classification or routing with the nice thing of the generality essentially that you get from the large language models in the transformer technology. 

[0:07:26] GV: Yeah. I mean, I think it's just one of these huge step changes, which is kind of nice to see where we've spent I don't know how many months talking about the next frontier model or open weight model. But they're all kind of trying to achieve roughly the same thing if you want to broad brush that. Whereas Jev has come along knowing that there is like another way to kind of approach this. 

I like the fact that, yeah, Jev does - the name does come from Jevons. Paradox. So that idea that the more efficient something becomes, that the more we actually consume it and then the efficiency is kind of lost completely. I.e., like light bulbs that we have these super efficient light bulbs now but we end up just leaving them on. Or I have 10 light bulbs, where we used to have one. 

[0:08:12] SF: Yeah. It's like when was the last time you turned off your phone or your computer and like that, right? You just leave these things on perpetually basically. 

[0:08:19] GV: Exactly. Yeah, I like the naming of that. Yeah, we might actually have an episode coming up with the founder. So that will be exciting if, in a couple of weeks, hopefully that's going to happen. So look out for that on our more regular programming. 

[0:08:32] SF: Yeah. I think so far, some of the pushback that you see is some of the benchmarks that they've published, they're kind of all self-published. There's no paper calibration and no sort of independent studies of these yet. I don't know. I mean, of course you would expect that. It's like they just launched it. 

But I think what's really interesting too is how people have gotten creative with it. You see people using it to solve like games and things like that, where it is like you have a player in a game, you think of like Mario or something like that. It's like, well, Mario is a fixed set of things that he can do at any given moment. Let's use a model essentially to figure out what is the next thing to do? Do I jump? Do a duck? Do I shoot a fireball or whatever it might be? So people are using those, and that's basically a routing problem. And we've had systems for solving those problems for years, but now you have this really dynamic system that can do that for - you don't have to go train a net new model essentially every time that you want to apply it like that. 

[0:09:28] GV: Yeah. Funny you mentioned Mario. I saw you can watch Jev playing Pokemon in real time. Yeah, there's like a website where it's live playing Pokemon. 

[0:09:38] SF: Yeah, I did find a GitHub repo of someone doing this for Doom as well just to be on team. 

[0:09:46] GV: Yeah, I've seen Pokemon at least as like, okay, that at least looks creative and different. But yeah, nice. 

[0:09:53] SF: Yeah. I mean, I think the interesting thing is if you look at a lot of compound AI systems today for doing things like model routing or other types of decisions, a lot of times they're pairing a traditional ML classification model with a large language model. Or sometimes they're doing what I said, where they use an inexpensive large language model to force it into being a classification model. Basically, Jev is the perfect solution to that kind of problem solving. 

[0:10:19] GV: Yeah. Yeah, I mean, definitely, when I say give it a try, it is difficult to get on the - it has a waitlist, I think, at the moment? 

[0:10:28] SF: I know a bunch of my colleagues have been playing with it, so I don't know how hard it is to actually get off the waitlist and try it out. I think they cleared a ton of people, too, if you look at the data. I forget the exact numbers, but they cleared tens of thousands of people off the wait list within a very short period of time. 

[0:10:45] GV: Nice. Okay. Yeah, I saw some of my colleagues talking about it, and it looked like some hadn't managed to get in, whereas one that got early access, so long as they had a credit card on it, then they could invite people to their org or something like that. But yeah, very interested to see how this shakes out, and I would love to play around with it. I don't have access yet, even though some of my colleagues do. So maybe I will on the episode, if we record it, I will hopefully ask for some access as well. 

Yeah, moving on. Another big product drop was Meta's Muse. So quite a different product in the sense that it's not aimed at developers per se. It's much more in keeping with, I guess, AI products of late, which is a personal assistant. We've kind of covered this area in many SED News episodes, but it's the fact that Muse actually seems to be solving the problems fast and well, I think is the best way to put it. 

They've launched this. It's an app. You can download it on Mac and on devices. But they are actually going to be also releasing hardware to go along with this called a Muse Charm, which is a sort of like keychain device with a little screen. Think of like a sort of Tamagotchi-looking thing. But I guess you'll just be able to talk to that as well. And I reckon they're kind of cruising on the whole Labubu. I don't know if people have heard of this, but, yeah, Labubu in Asia and stuff. These little charms that people put on their bags, and it's like ridiculously popular. And I suspect they're playing off because they look very similar to be frank. I'm thinking - 

[0:12:21] SF: Yeah, my kids have some. 

[0:12:22] GV: Yeah, okay. So the inspiration has probably come from something along those lines, but the idea that you can show people that you effectively use Muse. And you can kind of give it a name. Even if you're just using it in the app itself, I think you can give your Muse. Because I mean, Muse, the noun, you can give your Muse a name and so on. 

And yeah, I mean, I guess I've been reading and hearing people talking about different examples. Again, I can't access it right now because in Singapore, it's on a wait list. So I look forward to flying to the U.S. in 30 hours and hopefully trying to get it onto a device. But what I've at least picked up is that it is able to solve problems such as booking a restaurant 10 times as fast as Claude, or say 10 times as fast as it actually completes it, whereas something like a Claude gets pretty far and then it sort of says, "Oh, sorry. I couldn't actually book your restaurant for you for some reason." But yeah, what's your take on this one, Sean? 

[0:13:18] SF: Yeah. I mean, definitely, as we've covered before, I think this AI assistant market is going to be like a huge, huge market eventually. And I think like OpenClaw was kind of the first real breakthrough where people realize that, "Oh, maybe there's this more comfort level with AI taking actions on my behalf than we thought we potentially realized." And it kind of shotgunned this. I totally understand the investment. 

I think where Meta is going to potentially run into challenges, and maybe they'll overcome this, is just that because Meta is so strongly associated with Facebook and the history there in terms of using people's data for advertising and stuff, and then you have this device that potentially you're going to use on your behalf to manage part of your life. How much am I willing to kind of trust my personal information with that company? And does it line up to ads and so forth? 

Supposedly, all this runs within a secure VM that's supposed to create a boundary between your Muse system and these other systems. But just kind of looking at the history of some of the moves that Meta has made, I can understand some level of distrust there from the consumer market as well. 

[0:14:31] GV: Yeah. It's interesting, last month, we talked about a company, Instinct. And this was kind of their play as well. And just as we're talking, I just thought I'd just go back to their site. Because I just thought, well, hang on, we were talking about Instinct last month. And Muse is literally in this space. And what are they doing at all to sort of recognize this? 

Interestingly, if anyone listened to last month, we talked about Instinct's website, which looked like it was from 1992 and had Times New Roman. And it was trying to be all kind of back to the future kind of thing. 

[0:15:01] SF: Right, yeah. 

[0:15:02] GV: They've actually updated the site, and now it's not Times New Roman. It is a Sans Serif font. I.e., more modern. It has a little logo now, a little stick man logo thing. And interestingly, at the bottom it just says text instinct to get started. And I'm curious, I haven't used Wayback Machine or anything, but I'm curious if that was a bit of a - before, it was very like, "Oh, you can't really get access unless you're special." I wonder if that's being prompted. 

But anyway, I'm sidebarring here, but I just thought it was kind of interesting given that there's this just a kind of explosion in the personal assistant space. I mean, yeah, I think it's super interesting the fact that they actually even talk about the architecture here, the Muse secure VM architecture. I think that's something that people are quite concerned about with any on device and especially in this case where this thing is only valuable if you give it access to all the files. Or i say all the files, but files and especially data that you have, like your calendar, which would often be through some cloud service. 

I think the sort of paradox here is people probably have given a lot of data to Meta already, and it's not. So either you pull out the ecosystem altogether, or you just lean into it and say, "Well, you probably already have a lot of data on me, whether it's from having been on Facebook for a while, and then Instagram." And I don't know exactly what they would pull from WhatsApp, but at least WhatsApp is, I imagine, able to come into this in some way as well. Yeah, it's a very smart play from Meta, I guess, especially given, again, through the last few months, you and I, Sean, we've not been able to really pinpoint where Meta was going exactly. 

[0:16:40] SF: Yeah. It also feels aligned with where Meta has been successful as a company. This is much more of a B2C AI play than going after agentic coding harnesses or something like that, or even going after building models. I think this aligns with where they've been successful with Facebook and Instagram and these types of vets. So the personal assistant could be essentially the killer consumer app for AI beyond where we started with things like ChatGPT. 

[0:17:12] GV: Yeah. Have you used it or would you use it, do you think? 

[0:17:16] SF: I don't think I would. The hardware part of it doesn't really interest me. I don't really like the idea of even Meta Glasses and stuff like that. I don't want to advertise my use of things that could be perceived as monitoring other people and things like that. I just feel like there's a certain uncomfortableness that comes with that. It makes me uncomfortable. I don't want to make people uncomfortable with it. That part of it doesn't interest me. 

The idea of a personal assistant, AI personal assistant, certainly would be great. I just don't know if this is the one that I would personally put my faith in. I don't know if I'm ready to give it my banking information. 

[0:17:52] GV: I agree. I'm curious to try it. I mean, I think for some of these more - 

[0:17:57] SF: I guess like hotel bookings. I mean, clearly it needs some way of paying for those things. So I'm sure there's some credit card transaction part of it. 

[0:18:06] GV: Yeah, I think that's going to be something - I mean, I'm aware. Obviously, Stripe have been on this for a while, but I think they're finally getting the recognition that they want on machines paying for things. They've got this MPP, machine payment protocol. But I think people have maybe been struggling to kind of really utilize it because you need a machine that you trust to go and do the payment in the first place. So yeah, perhaps something like Muse is kind of the first step to that, where it's at least an interface that people are going to use more. 

And yeah, I'm curious how much I would use it like day to day. But as a novelty, I'm definitely curious just to give it a try, like book me a restaurant in SF or something. But yeah, let's come back next month if I've managed to get access, and we'll see if anything vaguely useful came out of trying to use. But yeah, very interesting. 

Finally on the headlines. So we did talk about - it's now the third time running we're going to be talking about Nscale, which is not Anyscale, and it's not Scale AI. It's Nscale. And Nscale is basically an inference data center company. But crucially, they sit in geographic areas that are different. For example, they have a center in a very picturesque looking part of Norway, for example. But they're about to go public. And so by going public, they fell, I think, on S1. And we get to sort of see a bit more into the business. Also, interestingly, this is founded by an Australian person. 

[0:19:40] SF: It used to be a crypto mining company. 

[0:19:43] GV: Yeah, exactly. So it was founded by a guy, Josh Payne, who's actually an Australian. It's interesting, the lineage of this. He was a miner, like an actual miner in Australian mines. And then, yeah, as you mentioned, Sean, the company, Nscale, that he then eventually founded was a crypto mining company, effectively. But then, yeah, has pivoted into inference for running LLMs and so on for large companies. 

In the S1, for their IPO, This is where it's sort of interesting where at least the FT's reported that ByteDance, i.e., the owner of TikTok, was 73% of 2025 revenue. But interestingly, they just don't name ByteDance at all on the S1, which I think is quite interesting. So it feels like an arbitrage company here where they pick up companies that couldn't go and use inference data centers, say, sitting in the U.S. So they are instead picking up any of that traffic where there's kind of trade bans at the moment. 

[0:20:46] SF: Yeah, I think also the fact that a fair amount of their revenue is coming from a small number of customers. I think we'll probably see this more and more with the AI companies that end up publishing in S1. And part of it's just because I think if you do close an OpenAI or an Anthropic or something like that, the volume of their use is just so high that it easily skews your overall usage numbers and the amount of revenue they're going to get at. So I think we'll see that more and more, which of course is a potential risk to the business depending on the longevity of those companies and so forth. But yeah, I mean, there's an interesting journey, miner to crypto miner to neocloud. 

[0:21:25] GV: Yeah, exactly. Yeah, neocloud, of course, that's the term here. Yeah, just a very smart business guy. I think it's interesting that the UK has tried to sort of claim it as one of its own. It's a UK company. Well, it is. But it is an Australian founder. And the problem now is that they're actually going to look to IPO in the US on the NYSE, which is a pretty well-trodden path now that companies that maybe start in the UK as tech don't end up listing in the UK because they think. Although the liquidity and better valuations and so on all happened in the U.S., which is pretty much true. And I don't know what it will take to kind of break that cycle, but it's unfortunate that we've got another one of those here. 

Interestingly, he is almost the - if they were to list in the UK, he would have actually been effectively one of the top paid CEOs, like almost the same as AstraZeneca, AstraZeneca's CEO, which maybe does also suggest maybe the numbers are a bit crazy here at the moment on valuation pay. It's kind of just like a, yeah, very opportunistic punt on pivoting this company from crypto mining. Yeah, curious to see how this shakes out or whether this is, yeah, just trying to sort of draw in people that just like the term neocloud and think that that's where they should be investing. 

[0:22:47] SF: Yeah. I mean, that's a really, I guess, hot area in term that's kind of come up over the last year is the neocloud. I think Nscale, are they the first to kind of file an S1? At least the first one I can think of. 

[0:23:02] GV: Yeah, yeah, that's true. Yeah, it could be like an interesting canary in the coal mine, huh? 

[0:23:07] SF: Yeah, exactly. 

[0:23:09] GV: Yeah, but especially being the first. Perhaps, again, there's like a sense of is there a "crash" coming? Should we get this kind of one and done before any other problems? So yeah, we'll follow along. We'll see how that one shakes out. 

Moving on to the main topic for this week. So we're kind of, I guess, subtitling this AI risk frontier lab slowdowns and conspiracy theories. So it's really that there's been a sort of series of events, especially over the last few months that don't quite kind of add up maybe in some people's minds as to are the large frontier model creators doing what they say they're doing or more when they are talking about things like, "Oh, we should be slowing down. We should be being more careful." But then at the same time, shipping new models and saying that these are the most catastrophically smart, but dangerous models we've ever produced. Is that not some kind of paradox that's kind of going on? Yeah. I mean, where do we kind of start with this with this kind of story, Sean? 

[0:24:17] SF: Yeah. I mean, we've covered parts of this before, like even going back to Mythos, we kind of touched on. The sort of byline from Anthropic was like, "Oh, we have this like super powerful model. It's too dangerous to give to everybody. So we're only going to give it out to like a few people." 

And we had talked months ago when this came up of like, well, there's definitely like a marketing angle to that. It's like the art of scarcity. It's like you talk about the thing that is so amazing and incredible. But no, you can't have it. And then eventually it is available. And then, of course, everybody wants it. 

But kind of tracing back what's happened over the last little while. There was the Hugging Face intrusion that happened back in July where roughly 700 OpenAI agents that were running some sort of internal cyber benchmark with the hacking guardrail switched off compromised Hugging Face production infrastructure. And we talked about that. 

And then there's been employees. We've also seen this even going back. There's employees in July that had said that - sort of whistleblowers of like, "Oh, these models are dangerous. It's a risk to human life even." And then there was a bunch of employees from the Frontier Labs back in July that were urging the government to deliver a pace automated AI development. 

And I think the big thing was on September 12th, Dario, the CEO and one of the founders of Anthropic, came out and published his essay, 'Pace the Frontier', which I recommend people check out. And then Sam Altman, Elon Musk, Demis from Google all essentially publicly agreed in the same day, I think on X, that they agreed with Dario and that we should pace it. And of course, immediately there was conspiracy theories around this. 

But then on top of that, like 10 days later, Anthropic ships Opus 5.5, OpenAI ships GPT-6 essentially the same day. So there's this, I don't know, somewhat - I think you referred to it as a paradox, but they're saying slow down. But at the same time, they're releasing models really, really fast, which sends kind of a strange signal to people. 

But I would say to Dario's credit is that he's been consistent in, I think, always kind of warning about the risk of AI. So it's not like he just, in the last month, suddenly started talking about this. He has been talking about it for a long time. But some of the things around the conspiracy theories and why people are concerned about how genuine this is, is that one is if you take all the top models in the space and all these companies kind of collude to say like, "Hey, we should slow down the pace, we should add regulation here," then in some ways it's a way to protect their position in the market. Because it's much harder for a smaller company, a newer player, to potentially catch up because they already have a head start and they have a huge amount of financing and the people to navigate whatever the regulations say in terms of what you need to do as a model provider. That's one thing. 

And then one of the other conspiracies is that it's related to the IPO and margin protection because the open-weight models coming out of China are really threatening how much people are willing to pay for tokens. All the frontier labs see the writing on the wall that their companies are going to be devalued. And this is a way for them to explain why they're not doing certain things or slow down the launch of the IPO or even protect against later of like, "Why isn't the latest, greatest model out?" It's like, "Well, you know, we have to adhere for the regulations or we're not sure of the safety." So they can kind of control the narrative around performance is another one of the conspiracies. 

But I think the other big part of this, too, has been President Trump's reaction, as well as Jensen Huang from NVIDIA. They've kind of come out on the opposite side, where Trump said that it's all a hoax, and Jensen spoke out against that they shouldn't pace AI, and it's unrealistic, and that these threats aren't real. And of course, NVIDIA also has a strong incentive to say those things because the faster AI moves, the more GPUs you need to buy. And you're probably going to be buying those from NVIDIA and it helps their company. So there's all this going on. It's a lot to try to parse and dig into. But I don't know, Gregor, what are your thoughts on this? 

[0:28:39] SF: Yeah. Well, I think one thing that stuck out is I think a lot of people, whether news outlets or otherwise, kind of have interpreted it as, "Oh, it was this call to pause." And I don't think that that wasn't really the word. The essay that you referenced did actually use the word pace. I know we're sort of splitting hairs potentially on words here, but they are quite different. Pacing is saying, "Hey," as it sounds, "let's be clear about how fast we should be developing models. Because if we kind of all just let loose, then who knows what could happen? That's quite different saying pause, which is, as it would sound, let's just stop producing new models until we've figured out tighter guardrails or whatever it was that they would have done if they'd said pause. 

So pacing, I think that's the problem though with pacing. Because pacing can then - sounds to me from a business angle could almost sound like colluding. And colluding is kind of when you have like a cartel. A cartel is when you have multiple companies who all basically behind the scenes agree with each other to do to sort of what I would call sandbagging. I.e., you know you can do more, you know you could jack a price up, you know you could produce something more powerful, but it's all in your collective interest not to do that because you're kind of trying to eke out the most amount of performance and especially financial performance. You've kind of modeled it over like three years and gone, "Well, hang on. If we jump to our best possible model tomorrow -" when I say best possible, what we know we could achieve. Ultimately, if we jump to that tomorrow, well, where's all the step changes we can keep charging people for over the next two years? 

And I think that that's kind of this sort of some people saying, "Oh, it's a cartel with a safety label." I got to say, it's hard to ignore that at times because we've got all these IPOs. I say all these IPOs. We've got likely OpenAI and Anthropic both wanting to IPO in the next. I mean, they keep saying they're kind of gearing up for it. I don't know. In the next six months, one of them will do it. So they're both on the same path and they both have like the same incentives to kind of - I would say to collude on some ground on this one. And again, trying to eke out as much financial performance as they can. That's definitely where it sits with me at the moment. 

[0:30:57] SF: What do you think in terms of the risk, though? Some of the risks that people are calling out, either the hacking risk, and then there's, of course, people saying that some people within the labs or the whistleblowers are saying that there's a 3% up to 10% chance of human extinction by 2030. How do you react to those things? 

[0:31:19] GV: Yeah, that's a good question. Even my mother asked me the same question. 

[0:31:24] SF: Yeah, my wife has been asking me.

[0:31:25] GV: When I spoke to my parents a couple of weekends ago, they don't often ask me about work or AI stuff, but they'd seen these doom headlines and they even said Anthropic, somebody from Anthropic. I slightly think it is hyped up. The problem is I can't put my finger on why. Why would someone say all this stuff and then leave the company? 

I think what I'm getting at is it is absolutely possible. These doom scenarios, of course they are possible. I think it's just where are we seeing the real rubber hits the road on the doom stuff. It's like, okay, we've seen the Hugging Face hack was pretty sort of eye-opening because it kind of went on for a while. But at the same time, I keep saying it, you can put guardrails on this stuff. The bit I don't understand - I'm sure there's someone shouting at their earphones right now that works in one of the frontier labs. But you can put guardrails on this stuff. And I'm curious why it's not more of we can still develop models, but without them actually needing to go off and do this kind of hacking and so on. Yeah, I don't know. That's like the layman's take, given I don't work in the frontier labs themselves. I feel like a layman when then thinking about these concepts. 

[0:32:47] SF: Yeah. I mean, if you look though at the history of technology, all technologies kind of have light and dark aspects. Any new technology can be used for greatness or good things and can also be abused either accidentally or on purpose. If you look at things like electricity, in the 1890s, electricity caused tons of fires and the government was kind of too slow to react to it. So what it ended up doing was the insurers actually ended up funding their own kind of private testing to deal with that. 

Drug design is another example where drugs do amazing things. They also do potentially bad things. But there's checks and balances to medical drug design. It's slow process, slow process by design. Even stuff like the printing press, it brought mass literacy to the world, but it was also used for mass printing propaganda. And social media is a good example of that where it's like there's been good things, I'm sure, that have come into social media, but it's also a propaganda machine that people abuse. There's all these things. 

And I think AI is similar. I think the challenge, of course, is that is it moving so fast that the abuse scenarios sort of take over before you are able to recognize the truly transformative things that you can get out of AI? And truly transformative, I see it as being able to use models that solve diseases that we haven't been able to solve. Or really crack into things that have been too complicated for humans to address previously. We're probably quite a ways away from that. But the danger, of course, is that we move so fast to try to get there that we end up hurting ourselves in the process. I think that's like the balance that you're trying to carve out right now. 

And I think the truth in all the conspiracy theories versus what the intention is behind Anthropic and the others that are involved, I'm sure it's somewhere in the middle. It's probably not extreme, like this is all faithfully, they realize they're just doing this for the better of everybody involved. But then I also think that extreme conspiracy where four or five companies got together, like we're going to go and trick the world and manipulate the public markets. It's probably a little extreme as well. So the truth is somewhere in the middle, I guess. 

[0:35:07] GV: Yeah. No. I think those are really good examples. I mean, yeah, as you say, we've seen this through history. I think, yeah, I mean, you touched on it, like encryption being classed as a munition before it was like seen as like a technology that was actually a force for good, quite frankly. Again, in most cases, of course, encryption can be used for nefarious purposes. But hiding information that would have otherwise perhaps incriminated someone that probably should have gone to jail, for example. But we like to think that actually encrypting the majority of sensitive information in the world has only been a good thing, even though the ability to crack it is getting almost easier and easier, except for quantum. 

The printing press, I think, is the interesting one. Because, again, I think that's closer to what we're talking about here, because it plays on fear. People feared that, oh, people that can't read right now, let's keep them down. Because a printing press will mean that more and more people have access to this crazy thing called words on a page. And they will potentially be able to learn how to read and then read information that we don't want them to know about. And that really feels kind of at the moment we maybe have people in the frontier labs. That's why I kind of like that I don't sit inside one because I can kind of postulate like, "Well, here I am, not sitting inside one of these labs. What do I think?" 

And yeah, I do think perhaps that what they see is sort of, oh, the normal people outside the labs can't possibly have access to this. That would be a terrible thing. I'm not sure I agree that just companies founded not that long ago in relative terms should necessarily be able to decide where this technology now goes. I mean, they are now one of hundreds and hundreds of companies that can produce a model. And that is a technology that is now available. 

On that point, I mean, it seemed that Google, OpenAI, and Anthropic, they're wanting to set up something called the Standards Authority for Frontier AI, which is modeled explicitly on FINRA, which is the body that polices Wall Street brokers, but on behalf of the industry itself. It's not a government agency. Do you think that's like a step in the right direction? 

[0:37:21] SF: I mean, I guess at least in some sense, they're trying to put more than just words behind this. Because Dario's essay essentially lays out three different strategies for pacing AI. One is embedding some sort of third party within the frontier labs that can actually evaluate how safe the models are before they launch the models. Then there's a government level regulations involved. And then there's ideally a worldwide. 

It's similar to how we have controls around the development of nuclear weapons and things like that, right? So they're trying to do that. But, of course, it's one thing to write it down, I guess, is another thing to actually make it happen. To try to get everybody to cooperate on it, I think, is very challenging. You already have the president of the United States having a completely opposite end-of-spectrum reaction to it. And then the other challenge is just even if the U.S. could decide that the labs here are going to slow down, doesn't mean that the labs outside of the US do as well. And then that potentially becomes a threat to the technology and how the technology is used and are the people in the US okay with it. It's overall like a very complicated issue. 

I remember even hearing an interview with Dario last year where he talked about how obviously his views maybe changed, but he said that he didn't feel like they could afford to slow down or pace things at that time because they had a head start on, say, China and other areas of the world, but it wasn't that big a head start. So if they were multiple years ahead, then they could perhaps slow down to make sure things were safer overall. But because there's this external threat, then they can't slow down the pace, essentially. 

So I guess things have changed somewhat. Maybe they've leapt ahead far enough, or maybe the threat is big enough or real enough. But all of this has also been sort of self-reported when you look at the stuff that the labs have come out with in terms of where there's been a breach or someone has declared some sort of issue. If that happens in things like airlines, there's some sort of incident, there's third parties today that go and investigate what actually happened. We just don't have any of those kind of checks and balances. So it feels like something is necessary. 

So I guess overall, I would say it's kind of at least a start to something that probably needs to be done. I don't know enough about how some of these other things work in other industries to say like is this the right approach or not. But it feels like it definitely feels like something needs to be done. Because you look at almost any other area where safety, public safety is a potential issue. Whether that's making a vehicle or flying a plane, designing a drug. There's there is multiple checks and balances before you're able to do any of those things. I can't just go create a car and sell it on my own and people will come after me. There's things that prevent you from just doing that. 

[0:40:23] GV: Yeah, that's a good analogy. It's always like where's the tipping point between something that until - yeah. Well, in many areas, you can just create software and sell it, and no one's like, "Oh, but what if that software does X or Y?" Unless it is something that clearly crosses a legal boundary, it could still be doing something quite terrible as software or like cause damage, but it's kind of encapsulated in like, "Well, if a company causes damage, that's like a liability and that's a civil thing." 

I mean, a different example, I guess, then is when crypto all took off and basically financial regulation had to come in because they thought that could cause damage to the average person from like a financial standpoint. If you're being misled as to what you are doing. And yeah, so now - but now in this case, it's like, should there be regulation on top of just developing a model because of all the ways a model can be used? Yeah, sort of the car, the airplane example is interesting, where you can't just - I mean, you can design it, you can build it, but you can't sell it to anyone. 

[0:41:32] SF: Yeah. I can't just start offering commercial flights. 

[0:41:35] GV: Exactly. Yeah. Well, I think that's a really good place to leave it actually. Hopefully, that's been kind of helpful a bit more. I don't want to say theoretical, but yeah, it's obviously just looking through why has any of this even come about and where are things going in the last, I guess, few weeks, few months in terms of where the Frontier Labs especially seem to be positioning themselves on this. 

[0:41:58] SF: Yeah. I do want to say I'm firmly on the side that we've already opened the Pandora's box. There's no putting it back in at this point that the innovations are going to come. It's like this tidal wave is coming. You can either try to swim the shore and escape the tidal wave, get the high ground, or you can try to grab a surfboard and surf that tidal wave. And I'm much more on the surfboard side of like, "Let's figure this out." There's, I think, a tremendous amount of potential upside, like the light side of the technology that there's too much upside to ignore. But let's figure out a way to also make sure that the dark side doesn't end up taking over the upside that we can look forward to. 

[0:42:41] GV: Yeah, absolutely. Well, talking of upsides, Hacker News highlights. 

[0:42:47] SF: Yes, absolutely. 

[0:42:48] GV: Good time to move on to, yeah, usually our sort of favorite part of the show. Do you want to kick us off, Sean? What did you come across as one of your favorites in the last week? 

[0:42:58] SF: Yeah. There was this one that I really like the title of it. It's called Pirating the Pirates, which was posted by piotrgrabowski, I think. So essentially there's this issue where film preservation is just like sort of piracy problem hidden inside of it, where studios end up releasing a film but then they manipulate it in some way. I mean, the classic example is with Star Wars, where George Lucas keeps re-editing the existing footage. It's really, really difficult to actually find the original Star Wars footage because it's been re-edited by George Lucas over and over again. Or something like MGM put 18 minutes back into The Good, The Bad and the Ugly with dubbed dialogue and sound alike replacing a dead actor's voice. And a lot of people who enjoyed the originals are frustrated essentially by this. 

So there's this effort where these boutique kind of restoration sort of combination of like amateurs and somewhat professionals take stuff like the DVDs, the LaserDiscs, the VHS, and various international cuts, and they kind of like try to reverse engineer back to a correct original cut, which is really, really amazing. It's just like this labor of love to try to restore the original version. But it takes all this work. And this article is about someone's firsthand experience doing this, working on something for several years with The Good, The Bad and the Ugly. 

[0:44:26] GV: Yeah, that's really awesome. Just having a quick look at it. And yeah, like lots of screenshots kind of showing like four different versions effectively of that film, depending on the medium and when it was released, and like who was behind that release. And the colors just are all over the place, like four completely different color palettes, if you want to call it that. Super interesting. Yeah, I'm going to dig into that more. I've got lots of flying time, so I think that'll be one that's on my list. 

On my end, a little bit more technical, I guess, but it could be interesting. I mean, the name just got me. It's called Fakecloud. So fakecloud.dev. And this was posted by - and this might lead into some interesting commentary here. It was posted by an user called The Anonymous One. And what was interesting was - so Fakecloud is meant to be an AWS sort of testing harness effectively. And I say they, because, again, it's quite anonymous who's behind this. They say, "Fakecloud gives you a local AWS environment that behaves like infrastructure, not a mock." So your app can use the regular AWS SDK, CLI, and IAC tools. And it then says, "Unlike today's local stack community setup, you do not need an account, auth token, or paid plan just to keep core dev flows local." 

Yeah, super interesting. Obviously, a lot of people that develop against AWS can be pretty costly to just be developing against it on a - because AWS have no concept of a free testing environment, really. We used to go test against their APIs. But interestingly, in the comments, people were a little bit concerned that maybe, first of all, who's behind this? Second of all, I love this, it just looks vibe-coded. I'm not really sure if that's really a reason to distrust something these days, because so many things are, and front-end isn't often the strong suit of some developers, so that's fine. But they have also suggested ministack.org as an alternative. So yeah, go check those out, Fakecloud or Ministack. I thought that was pretty cool. 

We are bumping at time. So yeah, we'll need to maybe save our predictions for next time. But yeah, looking forward to catching you in person again, Sean. And I'm sure we'll have a lot to digest on the next SED News when all things are done with conferences and so on. 

[0:46:45] SF: Yeah, absolutely. And I'll see you later this week. And thanks everyone for listening. 

[0:46:49] GV: Yeah, thanks everyone. See you. 

[END]