EPISODE 1965

[0:00:12] GV: Hello and welcome to SED News. This is the monthly format of Software Engineering Daily where myself, Gregor, and we also have Sean Falconer. 

[0:00:22] SF: Hey, Gregor. 

[0:00:23] GV: Yeah, we have the two of us taking you through a spin of the main tech headlines. We then dive into a deeper topic in the middle and then we finish up with Hacker News highlights where we pick some of our favorites from the last couple of weeks as well. But yeah, both Sean and I have been super busy so that's why this release is going out I think about a week later than it should. So apologies to anyone that was hoping to see one last week. But yeah. Yeah, Sean, on your end, what have you been up to? 

[0:00:52] SF: Other than the usual road warrior travel schedule, school started again for my kids. So my kids are back in that routine. So we're kind of back in the thick of it. And I'm a little bit nervous because my summer has been – there's been so much travel involved and we haven't even hit like conference season yet. I'm like, "Oh my gosh, I don't know what's going to happen at that point." But it has been a great time this summer. And I'm glad that we always figure out a way to make these sessions work, which I really enjoy. So despite us living on opposite sides of the world and sometimes having busy work schedules, we're able to find time to do this. 

[0:01:23] GV: Yeah, absolutely. For anyone interested, yeah, Sean's 4pm right now, my 7am over in Singapore. Yeah. But this seems to be the slot that works. 

[0:01:34] SF: Yeah, exactly. 

[0:01:35] GV: Yeah, no, on my side, yeah, talking of conferences, yes, we've got the Supabase Conference coming up the 2nd of October. So yeah, this is just crunch month, especially for that. Anyone that's maybe familiar inside a tech company that has their own conference, it becomes a pretty huge deal. Sort of all eyes on that. Unfortunately, though, there are always other things and other things that come up. 

[0:01:56] SF: Yeah, it's not like your day job stops. 

[0:01:59] GV: No, exactly. It's like conference crunch plus all the other things. 

[0:02:02] SF: Yeah, ours is coming up in two months too. So people are heads down on that right now as well. And they have been for months probably. 

[0:02:08] GV: Yeah, exactly. Exactly. Yeah. So yeah, I did actually go to Stripe do their kind of – as a few companies do, they kind of have their headline conference usually in the Bay Area or the US at least, which Stripe did in May. And then they do like a sort of world tour where they hit some of the bigger cities around the world, like London, for example. But Singapore does get them, which is great. Singapore tends to be the hub for Asia for these kind of world tour conferences. So went to Stripe's conference here and caught up with the folks there, which was pretty super interesting. And some of our other team were like on the stage talking about things. Yeah, exciting times. 

But yeah, let's roll into the headlines. We're doing a slightly different one this month, which is sort of an end of summer special. Yeah, I also feel like it's definitely end of summer, just the sort of feeling of end of summer, even though it's still very hot in a lot of parts of the world and etc. But end of summer special of sort of M&A that's been going on. So mergers and acquisitions. There has been a lot. There's been a lot of activity. We didn't cover a ton of M&A through the summer, actually. No, that wasn't happening. We just didn't focus on it too much. Yeah, this is kind of fun. 

[0:03:14] SF: Yeah, I think we talked about Anyscale, Nscale. 

[0:03:17] GV: Oh, yeah. Anyscale, Nscale, Scale. 

[0:03:20] SF: Scale AI. 

[0:03:23] GV: But no, I don't think we're going to touch on them today. We've got far more things that we can touch on, which is great. Yeah, I think I was going to bed last night, yeah, the news was dropping that NVIDIA were definitely – definite as it can be, but definitely buying Hugging Face. This this had been rumored i think last week. But yeah, it's been sort of announced for about 12.9 billion. Yeah, this is kind of interesting. 

I did just actually jump straight to the Hacker News comments on this one to begin with. And of course, a lot of – we'll get into in a second what is Hugging Face. I mean, for anyone that doesn't know, et cetera. But I think a lot of the community comments were, "Oh, they're doing this so that they can piss off the big frontier model people, basically." But I think that was a bit of a hot take from some of the Hacker News. 

[0:04:10] SF: Yeah, I don't understand why NVIDIA would want to do that because they make a lot of money from those people. In my mind, it's really – I mean, I guess if you don't know what Hugging Face is, it's become really like the default hosting and distribution layer for sort of not open AI, but open sort of AI models. And they have a lot of developers. There's almost like 20 million developers there. And there's over 3 million models and there's data sets and all kinds of stuff on there. And actually, funny enough, I had my first ever blog post published on Hugging Face this week. So in some ways, I give myself credit for this acquisition going through. 

[0:04:44] GV: You got over the line. That's basically what you're saying. 

[0:04:47] SF: Yeah. I'm waiting to see my share of the $12.9 billion. But yeah, I mean, I think that potentially this is a way that they can own essentially distribution of the hardware that all these models are going to run on. And then they also are sort of owning the community aware. 

And I would think if you think of a company like Google, especially in the early days, you know, a lot of the moves that they made where sometimes they would give away particular pieces of software or they gave away like Android operating some stuff. It's because it was in their best interest for more people to be on the internet. Because the more people that are going to be on the internet, then being like the front door of the internet at that time, it's more search, more ads, more revenue, and it's like flywheel spin. So they could afford to give away stuff because in a lot of ways it's like a customer acquisition channel for them. And I think you might be able to make a similar argument here where the more people who are building models, using models, running models is ultimately great for NVIDIA's business. Because the more that happens, essentially, the more money they make. So why not? 

[0:05:50] GV: Yeah, absolutely. And yeah, I mean, when I said sort of pissing off the big frontier models, I mean, that is a very hot take. It's more sort of a hedge, really, which could be construed as pissing off. But basically saying, "Hey, we don't –" that's what I think people are also looking at if they get too much completely interlocked with the circular financing and so on. But there is a whole world out there that has nothing to do with the frontier model creators. And that basically all – most of it lives on Hugging Face. So then it does make a lot of sense for NVIDIA to be very much there and make sure it doesn't get, I don't know, bought by somebody else. Because clearly, they were open to acquisition. So who else could have bought this? Probably quite a lot of people. 

[0:06:30] SF: I mean, it's probably a great landing for Hugging Face because, I don't know, I think the acquisition was really about buying essentially a community of users than it was buying a net new revenue stream or something like that, right? 

[0:06:43] GV: Yeah, yeah, that also makes sense. Yeah, we did actually have Hugging Face on the podcast on an episode. I think it was last year sometime. So go check that out as well. But yeah, very, very interesting. Huge, huge move. And slightly unexpected, I think. I think when it was rumored last week, it definitely came as a surprise to people. But yeah, hopefully NVIDIA is a good home for them as opposed to any other company that could have come along and done this. 

Yeah. Okay. So moving on to another acquisition, Arize. Is that how you say it? 

[0:07:11] SF: Yeah, that's right. 

[0:07:12] GV: Nice. Okay. So yeah, Arize. I think you've been covering this one, Sean. 

[0:07:17] SF: Yeah. Dynatrace obviously signed an agreement and acquired Arize for a little bit under a billion dollars, so $915 million. And if you're unfamiliar with Arize, which probably a lot of people are, although I see their billboards around the San Francisco Bay Area, they are in the AI observability and evaluation platform space. So if there's other companies in the space like Fiddler and Galileo, there's a whole host of them. But really some of the core capabilities around tracing, LLM apps and agents, what they're actually doing at runtime, detecting hallucinations, scoring that output, monitoring model agent behavior production. This is a huge space. It's very hard, especially when you're running agents or multi-agent systems at scale and enterprise just having visibility into what they're doing. And then if you're a regulated industry, you also need to know what they're doing to apply with certain regulations and so forth. 

Essentially they're in that space, but the acquisition makes a lot of sense. I don't think there's a world where agent observability is like a separate category of product over traditional application observability. Eventually these things are just going to merge. Because a business and enterprise doesn't want to have two different observability platforms that have to run for the software anyway. They'd much rather deal with one vendor. Dynatrace is one of the well-known players in observability for applications and infrastructure services. So adding this to their tool belt makes a lot of sense. 

And I think one of the things that will be interesting is part of Arize's differentiation was being kind of vendor neutral and being able to plug into any model or framework. And I think one of the concerns I've seen people raise is will they continue to be neutral now that they're part of this larger entity of Dynatrace? And will that mean that they'll bring some of that stuff, be less essentially neutral in terms of the way that they work? I have no idea, but that's kind of like one of the concerns that was brought up. But I think we're going to see a lot more of this consolidation. 

Obviously, there's a huge amount of M&A activity going on right now, but I think there's a lot of sort of moment in time AI companies right now or genres of AI companies where they're solving an important problem. But it's hard to create a really, truly large business on a specific problem. So either they have to grow into something bigger, like a bigger platform, or they'll get absorbed essentially by existing platforms from adjacent areas. 

[0:09:41] GV: How old did you say Arize was? 

[0:09:43] SF: They're pretty young. I think their last fundraise was in 2025. They did a Series C for like $70 million or something like that. So they haven't raised that much money. So pretty great. I'm sure everybody that works there is probably pretty happy because they – I don't know exactly when they started, but I would imagine they're too old. I could probably look it up. But they probably haven't sunk that much time into something and they haven't raised that much money to get almost a billion-dollar exit. Yeah, so they were officially founded in 2020. 

[0:10:09] GV: Yeah, especially in this space. Yeah, it probably would have been hard for them to kind of end up in some crazy multiple in the billions. Yeah, this does this does feel like a nice landing for them. 

[0:10:21] SF: Yeah, and I'm sure they were probably at a place where – I mean, I don't know, I'm just kind of guessing. But given that their fundraise was over a year ago, they were probably at a place where, "Do we go for another funding round?" which would probably need to be a fairly large round. And then you're kind of cutting off an exit like this at that point. And you're signing up for potentially another six years on this journey to get to like something really big. So you're cutting off a lot of the potential exits and landings. 

And then on top of that, you're signing up for another half decade commitment to the company. And sometimes that just maybe doesn't make sense. It might be overall like a better outcome for everybody. And I'm sure they'll still be able to continue to work on the vision of what they're doing in the company just within a larger organization that can better support it. 

[0:11:05] GV: Yeah. So yeah, next one is Temporal. So again, you're covering that one, Sean. 

[0:11:12] SF: Yeah, so Temporal, they've also been on SED. I interviewed their CTO a couple of years ago. They might have been on multiple times even at this point. So this is not them being acquired, but right now the rumor is that they are looking to raise $500 million at at least a $12 billion valuation. So nothing's closed yet. It could all, of course, change, but that's the rumor that's out there. And that's more than double the $5 billion valuation they had just six months ago where they raised $300 million Series C. 

So if you don't know Temporal, they are an open source, durable execution platform. They also have a cloud managed offering for this. But it lets essentially developers write long-running multi-step workflows that automatically can survive crashes and retries and timeouts and failures without you having to do state management and retry logic all yourself. 

If you look at a lot of companies, they'll have some either hand-rolled or some version of durable execution. Sometimes it's Temporal, something else. But they're originally really popular for things like payment processing or fulfillment pipelines where you don't want that thing to fail. But AI has been this huge tailwind in this space because AI agents are essentially inherently long-running multi-step processes that are also very failure prone. And you don't want to, every time it fails, have to start it all over again and then repay for those tokens. 

So having something like a Temporal to be able to pick up basically where you left off and retry and handle all that logic for you without you having to do it yourself really makes a lot of sense. I think this concept of durable agents has been a real tailwind for Temporal. And in a lot of ways, they've been at it for quite a long time. It's like one of those companies where it's like overnight success, but 10 years in the making. They basically created, I think, this category of durable execution. They put in a lot of hard work to kind of grind. 

They did a really good job of getting into some of the big Bay Area digital native companies. And I think that they were successful there. And essentially, developers would leave and they would bring that technology to the next company. And eventually they got enough word of mouth network effects where they just now actually almost getting this viral growth from it because they have so many network effects of people being really happy with the product and taking it with them. 

[0:13:29] GV: Yeah, they seem to have also this year – I mean, obviously, I've been through this with Supabase now this year. But they're sort of going from C to D to E in one year basically. They did series D. Temporal did series D in February of this year, I think. And so this would be not really even six months later they're looking at their next round at double – I believe that's double the valuation. Five billion on the last round and then – or more than double. And 12 billion on this round. Yeah. And I think that context is really interesting of, yeah, how Temporal has been around a long time basically replacing state machines and this kind of thing. 

[0:14:07] SF: Step functions and all kinds of stuff. 

[0:14:08] GV: Yeah, just retry logic effectively that used to be just hand-rolled, I think, by every company. Yeah, and then now it's only this huge tailwind with AI for all the reasons you just mentioned. Yeah, I think super interesting. 

Yeah, next one that crossed the radar was InstantDB. It's definitely more of an acqui-hire than acquisition, at least that's how it's being reported. But yeah, the team of InstantDB joining OpenAI. Yeah, InstantDB is, yeah, open source, basically, backend. You got a database auth storage layer. But the key thing here is this is a very useful thing for agents basically being able to spin up that layer of infrastructure. And yeah, clearly, OpenAI don't seem to be doing a lot of the building of that themselves at least not that we can see in public. Acquisitions seem to be – or acqui-hires again, in this case, again, seem to be how they're doing it. 

This is interesting and definitely one where there's a cloud offering. They had a cloud offering, which is, I believe, going to be mixed by OpenAI. I think this is when developers get a little bit angsty when there's a product that they use and, "Okay, the team is going to go in-house with OpenAI, but ultimately this product that they – I believe, they said they were handling 400,000 apps. I think this is a bit of a loss, I guess, for the ecosystem at the moment, perhaps, if it just gets, by getting just sort of subsumed inside OpenAI. But we shall see, see if there's a better version of this. 

[0:15:37] SF: Yeah, I mean, it seems like with OpenAI, it might be – with some of these moves, who knows what their actual intention is. But it seems like they're kind of trying to build their own agent infrastructure stack with some of these pieces that they're kind of assembling. And they've been making all kinds of different moves and agents for two years now. And I think part of that is how do you capture more of the AI value chain outside of the model itself. And even outside of what they were originally really well known for, which was like the consumer interface of ChatGPT around their models. 

[0:16:11] GV: Yeah. One of their investors was Jeff Dean. So pretty big name. And yeah, we've got also some just sort of, I think, general news on Jeff Dean. Is that right? 

[0:16:21] SF: Yeah. So I mean, the big news over the last month was that a little less than a month ago, at the beginning of August, Jeff Dean announced he was leaving Google after 27 years. He's employee number 30. He was most recently chief scientist. He's probably one of the most famous and world-renowned engineers in the world. You could probably make a fair argument that Google might not exist, at least in the form that it is today if Jeff Dean hadn't been there with all the contributions he made to MapReduce, Bigtable, Spanner, TensorFlow, Google Brain. He's just this godlike figure. 

The fun Jeff Dean facts that are like these like Chuck Norris memes and stuff like that, that I remember it was only Jeff Dean knows the final number of pi and crazy things like that. Jeff Dean doesn't get compiler errors. He tells the compiler that it's wrong and fixes it. And just stuff like that. 

So it is quite the thing. I never thought that he would leave. I figured he would just eventually retire. But it looks like he's going off to start his own company. And he took three really heavyweight people with him as well. And they're now starting this company called Discovery Loop, which is – I don't think there's a ton of information out there, but they say it's a public benefit corporation aimed at using AI to automate and massively scale scientific experimentation. Definitely something interesting to pay attention to, see where they go. 

I would imagine them raising money is a very easy conversation. They probably just, "Here's my resume," and slide across the blank check and have people just fill it out. But it'll be interesting to see what happens at Google coming out of this. Clearly, that's a pretty big loss. And Google did invest in Jeff Dean's company. And maybe there's a certain marketing angle to that to kind of save face and all this when you have such a high-profile person leaving. But what do you think, Gregor? 

[0:18:09] GV: Yeah, no, I was sort of thinking, okay, what is the – I love tipping points, right? What was the tipping point for Jeff Dean to leave Google? And I guess it's just like there has been no better time for someone of his stature and mind, I guess, where you are going to get the backed – if you actually want it, you can probably have an amicable conversation and be backed still by Google and even bring people with you and not become some kind of rupture. We've seen this a few times. And yeah, I think that's it really. There has been no better time that he could now step away from the mothership and to do his own thing. But I'm sure it will be very closely intertwined with things that Google do anyway. Yeah, I think that's definitely – life goals right there. If you can give such a contribution through one company, but then get like all the backing to then go and do your own thing from that company as well and be able to take some of the amazing talent that Google still has, yeah, amazing. 

[0:19:08] SF: Yeah. Just one more fun Jeff Dean fact before we go off this is Jeff Dean proved that P equals NP when he solved all NP problems in polynomial time. 

[0:19:20] GV: Yeah, there we go. Back to just – I guess we called it M&A, and it's more just A&R, acquisitions and raises. 

[0:19:29] SF: So back to A&R. Yeah, this one is a little bit maybe for some off the radar exactly, but that's why I wanted to bring it in is a company, HiddenLayer, who are a security startup and they make tools to protect AI models, agents and workflows. And obviously we have covered so many of these attacks that have happened involving some form of agent or whether it was intended or not. 

So it's very interesting that, yeah, they've managed to raise 100 million, which actually now for a security startup, that's pretty impressive. There definitely seemed to be some, I would say, decline in the security company raising space. I haven't seen a lot of big raises from security companies, strangely, even though we keep hitting all these security problems. It just seems that unless the company is basically something AI, and then they maybe weave security into it, but this is very much – they've been in security the whole time. They've managed to, I think, pivot that story and the products well into the AI space. 

And they claim that they haven't given exact ARR numbers. But they say that they're now in – the ARR is now the tens of millions. And 90 of that was driven by new customers signing in the past year. Again, that's tens of millions in ARR for a security company of their age. Impressive as well. It is still pretty challenging to get companies to pay for a "new security company". 

Well, I say nobody got fired for buying CrowdStrike. Maybe somebody did at some point. But honestly, I still think nobody gets fired for buying CrowdStrike. That's just a classic case of, yeah, they had a bad month a couple years ago. But then nobody seems to remember that anymore. 

[0:21:09] SF: I think it's steep in security it's like a steep hill to climb, because unless you're doing some sort of maybe consumer related security. But most of this is going to be like B2B. And how do you get like a truly large enterprise to trust some smaller player in the market that's relatively new with something like security? But I do think that given that trust and governance and security around models is like so top of mind for businesses right now, I do think that there's gonna be a lot of companies that kind of emerge out of this. New companies that have raises and are addressing real problems in the space. Because that's a big part I think of what needs to be there to unlock B2B enterprise AI, which is significantly wagging behind other areas of AI right now. 

[0:21:55] GV: Yeah. And I think it's interesting. They want to be known as like an EDR solution, which is endpoint detection response, but specifically for AI. They do things like they scan models that you are using. They actually scan the files to make sure that you are using the model that you think you're using. Again, having spoken about Hugging Face. And actually, our main topic is a lot about the open-weight/"open source" ecosystem. 

And so Provenance, which is do you know where this thing came from. This very much weaves into that, which is do you know the model. Is the model the one that you really think that you're using? Obviously, we've seen all sorts of like package manager-based attacks where someone can take over a name or something. On Hugging Face, I must say I'm not fully – I don't go into Hugging Face a whole ton these days. But I remember when using it. It was still kind of confusing, at least back, I don't know, a year and a half ago, two years, it was still maybe confusing. Is this a model made by anyone reputable with data that I can trust or not? 

Enterprise has to be super careful now when they're pushing their developers to use AI and especially open the small amount of actually open source models. Yeah, great. But do you know what's in them? So, anyway. I think that's why we've seen this from HiddenLayer. They've managed 100 million at series B, which is, yeah, super impressive for a security-based startup. 

Yeah, just moving us along. This one is super buzzy startup. Got coverage in Wall Street Journal. But instinct.ai. I think buzzy, I want to keep touching on that, because this was a 250 million series B at 2.5 billion valuation. Honestly, I'm sure if there's anyone listening out there knows this product is great. I'm still a bit confused what the why here or why this feels more like the team. You've got Noah Shin is, I believe, the – at least one of the co-founders. He was at Sierra. And yeah, it's like youngish team. And it's sort of like an OpenClaw but polished. It's kind of my read on it. Yeah, I'm curious why the crazy valuation on such a young company. But yeah, I don't know if this crossed your radar as a company, Sean, or not. 

[0:24:05] SF: No, I hadn't seen the news around this. But it's basically what I could tell kind of going after the AI personal assistant market and unless of like an open source completely vibe coded thing by a single individual and more of a company approach. I mean, I don't know. I'm assuming that they were able to get that valuation in part because of maybe who the founders were, and then also in combination with whatever growth they might be seeing. But I also would think that the total addressable market is massive. Maybe that helps justify some of it. But that does seem quite a large or a very high valuation. And also, for $250 million series B. 

[0:24:45] GV: Yeah. So I realized in our notes, I put the wrong URL. There's instinct.ai, which is not them. What's fun though, if you go to instinct.co, which is them, it is just a website that looks like it's from 1992. 

[0:25:00] SF: Yes, it does. 

[0:25:01] GV: It's a white background. It's Times New Roman. It just says instinct. And then it says contact. And then you've got San Francisco. And the comms, it's all written. There's no there's no science comms at instinct.co. Yeah, I'll just do a quick read. It then just has basically one about paragraph, and it just says, "Instinct is the personal assistant that understands what you're working on and what's important to you. It connects to your applications and devices, email, messaging screen, audio location, and more. The interface is simple. There are no new interfaces. It's trained to use a phone and a computer." I'll pause there. There's more. But yeah, very sort of OpenClaw-ish. But I mean, this is the ultimate – I'll just say this is the ultimate flex right here. You know, "Hey, we have a website that is just a white Times New Roman and we just raised 250 million, 2.5." Yeah, signing the times perhaps, but – 

[0:25:50] SF: they're going for agent readability on the website, right? 

[0:25:53] GV: Yes, that's definitely what they're doing. Agent doesn't need to know anything more than this, basically. But yeah, you know what? I'm going to, if I can in Singapore – sometimes things are not available here yet. I'll try it out. I was never an OpenClaw person, but maybe this can – I definitely need some form of personal assistant, executive assistant. So I will try this out. 

All right. And then just a final one. I just popped this in. It was kind of interesting. It's called owner.com. And they basically just do kind of like backend software for businesses. It sounds super boring. Unsexy effectively. It's an eight-year-old company. But I think it's kind of cool. They've raised 240 million series D at 2.3 billion. Actually, kind of similar numbers, just different round, to Instinct. But yeah, I think this is cool. It's good to see how companies, if they really do keep moving themselves forward and taking advantage of what AI can do, then you can still be raising on these unsexy business models. Kind of almost the anti to Instinct in this case. Yeah. 

[0:26:53] SF: That's a hard market to like selling into the long tail of restaurants. OpenTable was able to do it through a lot of just sheer will of beating the street with sales reps and stuff like that. And back then when OpenTable did it, they were actually putting physical computers into the restaurants and things like that. 

But I think the thing is, is like if you can do it, because it's hard, it's very sticky. And then you have this mass amount of customers that probably part of it's like, "Okay. Well, I've captured the interest of this customer. Chat are the other things I could sell into that same customer that helped them that also helped me by generating more revenue?" 

[0:27:29] GV: Yeah, as you say, it's very restaurant back office. Or if we say back office, yeah, it's like they cover everything. Point of sale, a branded restaurant app, marketing campaigns, online menu. When you say online menu, literally a restaurant menu, reviews engine, all this stuff. So they're kind of trying to cover every single base that a restaurant or any kind of food service, I guess, would be needing. Yeah, super verticalized in that sense. But nice to see just something that doesn't – of course, AI comes into it. But it's not just an AI company claiming to sort of solve all the world's problems. It's very specific for something. I think we can all relate to restaurants. 

[0:28:04] SF: Yes, it's encouraging for all those founders up there that might be listening that don't have a pure AI company. Yes, you can actually still build a business and raise money despite what maybe all the headlines tell you. 

[0:28:15] GV: Yeah, if you want to raise money. Not all do. But if you do want to raise money for something that seemed boring, definitely still possible. Yeah, nice. 

Yeah. I mean, I guess just sort of we're going to get into our main topic in a couple minutes. In terms of other news, I think it's just that we've seen an absolute deluge of new models coming in. My night last night was top of Hacker News was Meta Muse Spark 1.3 and Gemini 3.8 Flash. And then now woken up this morning. What do we have this morning, Sean? 

[0:28:47] SF: GPT 6 Astra. Hot off the presses. 

[0:28:50] GV: Yeah. Very exciting. So we have admittedly not had time to really dig into all the things that Astra claims to be able to do. But certainly, from the pretty – I would say they've gone up a level from the launch website perspective for Astra. Again, IPO looming perhaps. But yeah, they're really going all out, pitching it against Fable 5 and 5.1 and showing lots of nice charts that it's all the benchmarks are far above. Yeah, we'll probably get into that one in more detail next month. But yeah, any hot takes? 

[0:29:24] SF: I mean, I think it's just limited release right now. But I mean, some of the things that i was able to dig into with the press was they talked about how they had to spend kind of extra time shipping safeguards and enhancing the safeguards around it so that the model couldn't find and weaponize zero-day on its own. 

And I think part of this is trying to really address some of the criticism that we've seen over the last couple of months when it comes to the frontier models. There was, of course, what happened. And we covered previously with the Mythos, Fable of it all, where it was around for a week and it was pulled. I guess, who knows? We're only less than a day into Astra, so maybe they'll get pulled off the market later this week. But I think they're trying to address some of the concerns there. 

[0:30:09] GV: Yeah. So we have spent quite a bit of time on sort of when we normally this is like the headline section and we've obviously gone through a lot of tech A&R as we now call it. The main topic this week, it's actually kind of running on a little bit from last month's where we dived into – we call it the Kimi moment last month. Kimi, this open-weight model that literally all the weights you could find on Hugging Face and was really starting to show that an open-weight model can rival the closed source frontier models. Something that then popped up in my – actually just from a friend, sending me an article about what's called the transfer station market in China. And that's kind of framing what we're talking about today. It's sort of like the last couple of weeks is really like models. There's been a couple of weeks where the model map, if you like, has been rewritten in some ways. But I think just diving into how we got here is interesting. We didn't really touch on that so much last month. 

Kicking off, yeah, even since we talked about Kimi last month, we've seen like a bit of a proliferation and open rate releases. Some of these names might be new to people. So Z.ai released GLM 5.3 Flash. We've had Qwen dropping a whole bunch of new models and all sorts of new models coming out from all these Chinese foundries or however they like to be called. But what we perhaps didn't appreciate was that for a while, there's been this transfer station economy, which is basically where, although you cannot access frontier models from OpenAI and Anthropic in China, there has been a huge economy where basically you are able to access them for very low cost. So I'll kind of go through the three ways they've been able to do it. And then we'll just have a little pause there. 

Basically, the first way – apparently, I love all these kind of naming conventions and they come out of Chinese proverbs and stuff. But apparently, it's called the one fish three meals. And this is from this article. I should definitely give a shout out. The blog is called China Talk. It's unattributed because, for obvious reasons, I think there's far too much being talked about here that if someone is named on this, that could be problematic for them. I totally understand that. 

The one fish, three meals, what is that? So basically, they talk about either you can – meal number one from the fish is you can bulk register accounts to farm free credits, reselling unused quota, corporate discount arbitrage and API maxing. One $200 max plan gets basically carved up to multiple users. Meal two is what's called model swapping. So users select Claude Opus, but the proxy would silently root to Sonnet, Haiku, or in a sort of "worst case", GLM or Qwen, and fraudulently relabels the output. Researchers audited 17 API proxies and found that widespread model swapping, basically. Proxy access to "Gemini 2.5" achieved only 30% on a medical benchmark, which is way off the actual official 83%. 

[0:33:16] SF: What are they doing with the model swapping? Are they essentially capturing the value of Claude somewhere else? Essentially, someone thinks that they're using Claude Opus. They're paying that amount. They get some cheaper basically version of that in response. And then the labs are using the delta between that to do something for themselves. 

[0:33:38] GV: Yeah, I think that's exactly it. Yeah, user thinks they're using, let's just say, Opus. Yeah, which was the example given here. They think they're using Opus. And basically, yeah, lab goes off and charges them, because they say, "Hey, I'm a proxy." I'm trying to think of a good example. I don't want to muddy the waters with names, but I think an example here that at least gets people thinking is imagine Cursor. It's not Cursor, just to be super clear. It's not Cursor. But imagine Cursor was to tell you, "Hey, would you like to use Opus or Sonnet?" And you pick Opus. And then I get a response back and I think that Opus created that. And I pay for Opus amounts. I instead get a Sonnet response back, and I go off and keep doing my work. However, lab keeps the money that I paid them for Opus and then goes and actually uses Opus to – yeah. And we'll get to sort of why are they doing this at all. But yeah, that was a good sort of question. 

[0:34:36] SF: Yeah. They're basically using the excess sort of capacity that's there because they're not actually responding with the high value model to use it for their own purposes.

[0:34:44] GV: Exactly. Yeah, meal number three is the logs are the products effectively. So every request. Passing through a proxy, like a prompt, or response, or tool calls, or iterations, that all does sit on this proxy server. And if it wasn't clear, yeah, these users in China are – they're using these. They pay a proxy to get access to these models. The proxy tends to go through Singapore. Currently, Singapore data centers are like some of the most heavily used in the – or VPNs heavily used in the world. 

Yeah, the logs sit on the proxy server. So then for AI coding agents, they contain all the reasoning chains, engineering decisions, like repository context, human-verified correct outputs as well. They're basically capturing what a proxy literally is. It sits in the middle. It captures basically all the ins and the outs as if you were Anthropic or OpenAi. Yeah. 

[0:35:34] SF: But without the concerns of user privacy or GDPR or anything like that. Farm this user data however we find necessary essentially. 

[0:35:44] GV: Exactly. Yeah. And apparently, yeah, certainly that last one, like logs that seems to be, I guess, "the margin" of these proxy products. I think they see that as the highest value basically. You capture users through – 

[0:35:59] SF: It's real human behavior. Also, those form essentially memory paths. It's like, "I'm trying to accomplish this task." And the traces give you a graph for how to accomplish that. And it's probably not always going to be that efficient. So then if you mine them, you can take that into training and synthesize essentially sort of better neural pathways for similar types of memories. 

[0:36:25] GV: Yeah, exactly. I think this is like just – I got to say, I had missed this certainly. And so when I was hanging out with a friend the other day, who's a programmer, CTO, and he spends a lot of time in China. He's German originally, but a lot of time in China. And so he's always the guy who just knows what's going on and why and so on. Yeah, he was talking to me about this and I was like, "What are you talking about?" And he's like, "Okay, I'll send you this article." Yeah. 

And why does that even kind of play into especially what we talked about last month with Kimi and just this open-weight surge, if you like? Well, it doesn't mean this is not the entire reason this has been possible. That would be undermining. But distillation, this idea that you can capture the inputs and likely responses from these models, you basically are able to then take that and then train models, open-weight models, or create open-weight models. That sounds a bit like, "Oh, but how on earth would you do this? Actuallly, that's super slow." 

I think the volume here is what's always hard to like fathom. The Chinese population and just, I think, the sheer scale of what has been going on here is absolutely unreal. Yeah, I mean, when people talked about distillation, I don't think it was fully understood. How do you even do distillation? Well, this is a sort of a very quick and interesting look into how that even works. 

And then, yeah, that is sort of a huge reason. I think that we have seen just a massive proliferation of open-weight models and the offer quality that we're seeing. Again, lots of very, very smart people in China creating these models. But these proxies do exist. This stuff is happening. And it's a bit hard to see why that would be happening without – yeah, one seems to have led to the other at least in some form. 

[0:38:12] SF: Yeah. I mean, in order for you to – the part of the process of distillation is essentially using a larger model or like a teacher model to generate the final outputs, the answers or confidence scores. And then you use that to essentially teach the other model. So you want to make a really good open-weight model and you have access to the best frontier model. And you're not even paying for it because you're using the margin that you're saving from deviating routing to cheaper models. 

And then on top of that, you also have human trace behavior for how they're interacting with agents. Then that's all data that you can use for essentially distillation of reinforcement learning and fine tuning to make these open-weight models really, really good without having needing access to like – this is a way essentially you can get access to the models and all the data that you need to do this process. 

[0:39:07] GV: Yeah. It makes the idea of models are banned. It's very interesting. Like if a model is banned by, say, the Chinese government, but then equally these proxies sound like pretty prolific. And you wonder actually, are these actually supported by the country as opposed to like is it a blind eye or is it actively supported? And it makes the story of why are models banned even more interesting. Yeah, you wouldn't really be able to do this on the proxy level. Or the proxies wouldn't have any teeth for a user to go and use them if you didn't need to go do this. I think it's absolutely genius if I’m just being super honest. I think it is super smart because you always got to think 10 steps ahead of like, "Well, we don't want –" at least that's probably I guess the way the Chinese government thinks, like, "Oh, we don't want users to have access to this. But clearly, they are very powerful and very helpful. But what's the way we could sort of harness the output of this?" It's super interesting. Yeah, hopefully, that gives a little bit of backstory to where we are at the moment with the open-weight side of things. 

[0:40:11] SF: Yeah, it does make you think like if they're able to do these types of things that it'd be hard to get away with this in the US, I would say. 

[0:40:21] GV: You'd hope it'd be hard to get away with it, yeah.

[0:40:23] SF: Yeah. What's that mean for – how does like an open-weight model stay competitive with essentially this market that is kind of stopping it, nothing to be able to make better and better models. If you're willing to essentially circumvent people's privacy, circumvent certain laws to train these models, and then you're a legit company not doing that, it makes it pretty hard to have an equal footing in terms of competition. 

[0:40:48] GV: Yeah. Well, I think that's a good segue into then – I was also just for this week generally sort of wanting to maybe look at within the main topic what is Meta doing. Because that's a topic we haven't really – we haven't touched on Meta in a big way in a while. And thinking about it last week, I realized I don't actually even know what Meta are up to, which says a lot, because it just means they're not terribly interested from a pure personal coding level. But it is very interesting just to look at them from the landscape point of view. I hadn't even appreciated that they did actually shelf the Llama family of models back in April. Everything's moved to Muse. 

Actually, it was a closed model Muse code. That was starting to show a bit of a reset. And that was under you know Alexandr Wang who had come over from scale.ai. But now they're moving back to sort of open-weight models. There's Muse Glimmer, which was under Apache 2.0. 

So then to your point, yeah, Sean, I think they can compete in the sense that open-weight models now have a lot more reputation in the sense of quality of output. Open-weight doesn't mean less quality for certain tasks perhaps like coding. You can actually get very, very far on these "cheaper models". And Meta is not an AI company. It's still a social platform company. It's very difficult, I think, for them to shift that image of themselves. 

[0:42:07] SF: I don't really know what's their go-to-market around their models. It's just not the natural – 

[0:42:12] GV: I think it's compute, basically. Yeah, which has come out here. So it's like, "Okay, we can still release an open-weight model, but it's on Meta compute. And so we're just going to go down the traditional –"

[0:42:23] SF: Yeah, I mean, that's how they monetize it. So it makes sense to basically give the model away or give the weights away, essentially, because it's almost like a customer acquisition channel, but it's how you market it. But that is probably not the natural location that people, they're thinking about – But even if I think more and more companies are going to have essentially a hybrid model strategy where they have some closed frontier models from the top labs but then they also have open-weight models. And probably a lot of them are not going to necessarily run those open-weight models themselves. They'll go through an inference provider like Fireworks, or Base10, or whoever it might be to do that. But then they're gonna want to do that so they have kind of a model agnostic strategy. And they can direct certain workloads to different models depending on what the use case and the workload is. 

But is Meta a consideration in there? I wouldn't think it would be the natural location that a lot of these companies are going to go. I guess maybe Meta's approach is like, "Hey, well, if we can continue to prove that our model is the best in the open-weight market, then naturally people will want to use our model because it's the best. But that's going to be tough to do, like we said, given that all the moves that the Chinese companies are doing. 

[0:43:38] GV: Yeah, absolutely. So we are going to have to leave it there. I think hopefully that's been like a little insight. I know we like to try and do a deep dive. I hope that's been deep enough this week. We had so much to cover in the first segment. But yeah, go check out if you're interested. Yeah, it's the article. It's on this blog called China Talk. It's called How to Buy Cheap Cloud Tokens in China. Very straightforward, isn't it? 

[0:44:02] SF: Yeah, very straightforward. 

[0:44:03] GV: Yeah, that was published in May. Sorry, there is a name against it. But yeah, kind of who overalls behind this blog is a little bit less. But this person, they are a research associate at Oxford China Policy Lab, and they hold a master's degree from University of Oxford. I do feel like a lot of the research here is pretty well backed up. So go check that out. 

Yeah, we're probably going to have time for maybe one each on the Hacker News highlights. But yeah, favorite part of the show. Let's get into it. Sean, what was your highlight from Hacker News? 

[0:44:35] SF: Yeah. I grabbed one. It's kind of still on theme of some of the stuff that we're covering around LLMs and inference and cost. But there was a recent – there's a blog post. It's on openteams.com. Intelligence versus cost. And it was posted by the anonymous one on Hacker News. But essentially, it's someone's personal deep dive into – first, they're critiquing some of the existing charts that you see that people use to sort of compare intelligence versus the cost for LLMs. 

And one of the things they point out is that the chart uses a log scale, which visually ends up flattening the actual massive price gaps. Their argument is that's kind of misleading and it doesn't really show the true story. And then what they do in the article is they redraw that on a linear scale and show what the real world pricing and the gap is between the open-weight model and like a Fable 5.1. 

And essentially their argument is that Fable's seven and a half times more expensive, but you're only going to get a small sort of intelligence bump that most people wouldn't even notice. Does it really make sense if you could take an open-weight model, run it even on a last generation GPU for 1400 bucks a month or something like that, unlimited. And you're getting kind of 90% of what actually people need is kind of the argument. Now, I will say that this is someone's personal journey and it's not necessarily scientifically based and things like that, but just interesting kind of food for thought, I thought. 

[0:46:04] GV: Yeah, no, that's super interesting. Yeah, on my end. I'm trying to think which one to pick. I'll go with – I mean, it's on Hacker News, and I think that's what's cool, but it's not super tech related, but AI does kind of come into it as well. But there's a great article called Invisible Companies. And this was posted by Itononro. 

So what is an invisible company? Well, it's basically the argument that people can actually make good money from finding companies that just are very good in their sector, but are just kind of like completely not the buzzy company to buy. And so if you're a VC and you want a slightly alternative strategy, then you can go and find companies that very much are outliers in their segment for how much money they're able to produce. 

But the reason that it's called invisible companies is you actually also don't want them to become visible. You don't want them to kind of get into become a segment that it becomes hot and people now want to buy companies in that segment. Because the whole point is the aim isn't to sell these companies on necessarily, it's to hold them and to make money from them. AI comes into it in the sense of like, well, you can definitely do a lot more research much faster to find these companies. But does that make them more or less visible? That's part of what the article discusses. 

Yeah, I don't know. I'm sure we've got a lot of listeners out there who are just generally entrepreneurs themselves. I definitely found this one very interesting as well and just completely off tech topic. Sometimes it's nice just to read something that's not related to models or AI models or that kind of thing. 

[0:47:42] SF: Absolutely. Awesome. 

[0:47:44] GV: Yeah. And no Doom this week, unfortunately. No Doom running on something, but I hope, I'm sure we'll find something for next month. But yeah, I guess just any quick thoughts on what we might see in the few weeks ahead, Sean? 

[0:47:57] SF: I mean, it's not a real hot take, but given all the movement that we – all the M&A. What did you describe it? A&R? 

[0:48:05] GV: A&R. Yeah, yeah, yeah. 

[0:48:05] SF: Yeah. I think that's really just the beginning. I think we're going to see a lot going on through the end of the year of consolidation of companies and also large fundraisers from variety of different companies. Because I would suspect a lot of the companies that raised fairly large rounds in 2022, that when the market was really hot and were sort of overvalued and to a point where it was hard to look at the value, some collection of those companies are probably in a place where they're running out of money and maybe don't have the growth to raise. And then some collection of them have probably done really well and they're going to raise money and so forth. We'll probably see a lot of that. 

[0:48:42] GV: Yeah. I was just double checking. My prediction last month was that we're going to already see Kimi 3.5. Sadly, no. But definitely, as we've covered a lot of stuff in the last – a lot of open-weight in the last months, I think my prediction for the next few weeks is I really think we're going to start to see some cost inflection point. I think there's going to be some kind of not backlash exactly. But I do think there's going to be some moment where some big company somewhere perhaps is like, We're not going to use Anthropic anymore. It's too expensive. We're using Kimi or we're using something." I'd kind of like to see it because I think I would like to see the cost being driven down to some kind of equilibrium. Yeah. 

[0:49:23] SF: I would suspect too one other thing is that I think we'll see a big model announcement from Google sooner rather than later too. Because they've been way too quiet other than like the Jeff Dean news. 

[0:49:33] GV: Yeah. Cool. Well, yeah. Thanks everyone for tuning in. And we shall catch you next month. 

[0:49:39] SF: Thanks everyone. Cheers