EPISODE 1835

[INTRODUCTION]

[0:00:00] ANNOUNCER: AI tools are transforming how developers write code. Although it's difficult to pinpoint how much code is now AI-generated code, estimates suggest it's between 20% and 40%, and this figure is poised to grow in the coming years. This evolution has given rise to a new coding paradigm in which developers act as directors, guiding and refining AI-generated solutions rather than manually writing every line of code. This approach was recently termed vibe coding by Andrej Karpathy, and it shifts the programmer's role from detailed coding to overseeing and enhancing AI-produced code. It emphasizes collaborative interaction with AI, blending human creativity with machine efficiency to solve complex problems. 

Vish Abrams is the Chief Architect at Heroku and previously worked at Oracle and NASA, among other organizations. In this episode, Vish joins the show with Kevin Ball for a wide-ranging conversation about the state of AI-based tools, whether there are limits to vibe coding, AI tools for individuals versus AI tools for teams, the model context protocol, Heroku's managed inference service, and much more. 

Kevin Ball, or Kball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow Kball on Twitter or LinkedIn, or visit his website, kball.llc. 

[INTERVIEW]

[0:01:47] KB: Vish, welcome back to the show. 

[0:01:49] VA: Thank you.

[0:01:50] KB: I say welcome back because you were on not long ago, as I understand talking about Heroku and the 12-factor application. But let's have you reintroduce yourself really quickly, so folks who missed that episode know who you are, and then let's get rolling. 

[0:02:02] VA: Awesome. Yes. My name is Vish Abrams. I'm the Chief Architect of Heroku. I've been with Heroku and Salesforce, as Heroku is a subsidiary of Salesforce for about six years now. Before that, I built Cloud at Oracle, and I was very involved in the OpenStack project, which we initially created at NASA. Some people may be familiar with that when it was a big open source project back in the day before Kubernetes and the CNCF took over. 

[0:02:26] KB: Awesome. You wanted to come back on the show to talk about something that in my mind is very different from what you were last on, right? You were talking about this 12-factor application, which is like how do you structure and take this principled approach to app development. Then we wanted to then have a conversation now about vibe coding, which is this hot new thing. It's very, very trendy right now. But maybe let's start. When you think the word vibe coding, what does that mean to you? 

[0:02:52] VA: I think Karpathy was the one who originally coined the term, and it's taken on a life of its own. I saw a post from someone where they were talking about how when there's a need for a new term to arise, the first term that fits that mold just gets adopted by everyone. It feels like that happened with vibe coding. I think when he was talking about it, he was talking about it being in a flow state as you code and how the AI tools for coding can get you into that state where you're not really deep in the code thinking about other details, but you're looking down on what's happening from above and just directing the flow of the code in a new direction, which I think is a cool way to think about it. 

But it seems to be more associated with the concept of can you build an application, at least the first few iterations of an application without ever having to write any code yourself. Sort of like you have a team of engineers. Maybe they're in another country, and you're giving them instructions, and then they come back the next day with an answer. There's a little bit of distance between you and the code, and you're seeing where you get to. Is that your take as well? 

[0:04:04] KB: Yes. I mean, I think there is this sense of riding the wave and just letting things flow and letting go of control, which, I mean, to me, as a long-time software engineer, it's a little uncomfortable but also neat to see. 

[0:04:17] VA: Yes. The control factor is definitely, I think, where people get on different sides of the issue, right? I've seen some reactions that say, "Oh, this is just a fad. It's never going to happen this way. Once you start doing anything real, it doesn't go anywhere," and other people saying, "No, this is the future. All coding is going to be this way." I guess we're on this journey together to find out exactly how deep the rabbit hole goes, maybe. 

[0:04:42] KB: Yes. I mean, I think there's something really interesting to explore there, which is for a long time, there's been this trend in the industry towards no-code tools and things. They all kind of over time have gotten better and allow you to do more and more without having to think about the code. At some point, you tend to hit some sort of complexity cutoff where you're like, "Okay, I want to do this thing now, and my no-code tool does not give me an abstraction to handle that. Now, I've got to bring a developer into it." Do you feel like vibe coding has that complexity cutoff? 

[0:05:12] VA: I think it does. I am surprised consistently by how that complexity cutoff is being pushed forward almost on a daily basis. I mean, when I look at all the different areas where generative AI is coming into the picture and video production, image production, coding, it seems to be moving faster towards something real than at any point I ever expect. I'm always like, "Ah, it'll be a couple of years before you can do anything real." Then in a month, there's another step forward, right? 

I hesitate to make any kind of firm opinion about how far along that complexity curve we will eventually get. But my sense of it is that there is an asymptote that we're going to hit that kind of like we did with self-driving where it seemed like it was always right around the corner, right around the corner. But that last 10% is super hard, right? I feel like we're going to hit one of those, but I keep being surprised that it hasn't happened yet. So who knows? 

[0:06:08] KB: Yes. Well, and I think there's something interesting here that feels a little bit different than the no-code example or something like that, which is you can use some of the same techniques that go into vibe coding to gain productivity even in a more structured environment. I think one of the things that we might look at here is, first off, obvious use case for these things is prototyping. 

[0:06:30] VA: A hundred percent. Yes.

[0:06:31] KB: There's a lot of interesting stuff we can do there, and maybe you can go all vibe, right? I saw a guy at a discussion group coding a Doom clone, and he was literally like, "I want a Doom clone. Go. Here's the screenshot. Feed it back into cursor. It's not working. Fix it," right? It was at that level of abstraction. I won't say that he got to a great Doom clone, but it was surprising to me how far he got. 

[0:06:52] VA: Right. One of the things that's been super fascinating to me is that the vibe coding skill set, meaning there's some way of thinking and understanding how systems work that I think makes vibe coding easier for someone who's actually coded before. Not to say that people who haven't coded can't do anything. My wife actually recently was doing some real estate research and managed to get Google Maps to have pins of some houses she was thinking about investing in. She got help from ChatGPT to go through the whole thing. It's something she never could have done on her own. She's just not technical enough to build that without the help. It definitely enables some people who have no coding ability to get really, really far. 

But I'm impressed with the people that do have coding ability and are able to prompt and ask the right questions and think through things differently in their planning seem to get a heck of a lot farther with it. There is definitely a skill set that you still need in order to be successful in this new world. I think with the right technical skill set, we are going to take it pretty far. I think it will go beyond prototyping. As it stands right now, there is definitely a wall. When I think about most business software and the complexity involved and think about maintaining it over time, QA, those kinds of things, it's where you really start hitting the wall on some of this vibe coding-esque style of programming. 

[0:08:17] KB: Well, and I think that's where we get into some really interesting territory, which is, how do you transition from vibe coding to a production environment? Are there productivity boosts that you can get from this? You called it vibe coding style. I remember, I think, Steve Yegge at Sourcegraph coined the term like CHOP, like chat-oriented programming. This new way of interacting with a code base combined with some set of tooling and engineering principles can still work in a production environment. 

[0:08:47] VA: Yes. I think it can. This is an area that I've been thinking a lot about because we're thinking about vibe coding right now as the zero to one. Can you get from nothing to a prototype or a concept very, very quickly? It's obviously super valuable there, but how do we bridge that gap from day one to day two? I mean, maybe there's a new term we need. Maybe we needed to get vibe operations or something. But how do you get that same productivity boost when you're thinking about maintaining an application over time, dealing with security updates, productionizing it, scaling it, all of those bits? 

I think some of the recent advancements or protocols that have been proposed like model context protocol to give agents a little bit more accessibility to operational tools is the right direction to push there. But I don't know that we've explored that space enough to see how effective they can be. If you give your assistant the ability to start reading the logs out of your production application and reading telemetry, does it start helping you debug production issues, right? That's an interesting space that I don't feel like has been explored yet. 

[0:09:57] KB: I think there is some hints in that direction that I think are interesting, right? I'm using Cursor, which is one of the tools that is really big in this space for product development. I will tell it, "Hey, write specs for this. Write tests." Now, run those, and look at the output. It will use those to debug at least in development what's going on. Or like if you have your linter set up well, it'll run a thing in the agent mode, and it'll be like, "Oh, this isn't meeting the lints. What do I need to do? I need to change it this way and this way and this way." That feedback loop can put it more on the rails. But, yes, it's how far does that go? How far can you get there? 

[0:10:33] VA: I think there's some really interesting space around examining the limitations. I've read some really nice write-ups of the places where this type of coding starts to break down. Like when is the LLM failing or when is it bad, and analyzing where that happens. The first thing that I discovered was that if I'm writing code in Python or Go, and it's fairly basic using the standard library things that have been around for a long time, it tends to be pretty darn good. If I ask it to write some Rust using a new library that came out in the past year, it just starts making things up, right? There are definite areas where you're on the beaten path, and it works really well, and there are areas where it starts to break down.

One that I haven't seen personally that I read about is this, since you speak of unit tests, cases where the LLM will actually start rewriting the code to match the test instead of fixing the test or vice versa, right? It analyzes. It finds the problem in the wrong place, which is it feels very similar to a lot of new programmer problems. When you have a junior programmer, the same types of mistakes you see there, you often see in these tools. 

[0:11:39] KB: To your point, this pushes towards having a technical background being really helpful, right? A technique I've used there is I'll be like, "All right, I'm getting this error. Analyze the code and the tests, and tell me where you think the bug is. Is it in the tests, or is it in the code?" It'll go. Then I as a senior engineer can look at it and be like, "Ah, that doesn't seem so good. Go off in this direction." You need to have a human in the loop to keep it steering. 

[0:12:05] VA: Yes. I think that's where I've come to as well is there is this value, but it does mean that your skill set is a little different from how it would be if you're writing the code, right? You now have to think of - there's this awesome interview with Jensen Huang on Huge If True, and he talked about how he sees humans changing the way that they do things when agents are on the picture. I think this applies to coding specifically. It's suddenly you don't just need a deep technical skill set. You actually need more of a management skill set as well because you're directing an entity, not a person, but an agent to do work. The way that you successfully direct an agent to do work is similar to how you would successfully direct a human to do work. 

[0:12:47] KB: A hundred percent, a hundred percent. I have an example I can bring. Last Friday, we had an interesting thing we wanted to try. Read a research paper, interesting new technique. How do we go about building a prototype? Well, I had a conversation with an agent, and I said, "Write me a spec for this based on this paper," and it wrote a spec. The spec was so-so. It wasn't great. I was like, "Okay, you need to look at this document, this document, and cut these things out." I'm doing this feedback loop as if I'm an architect or a manager. Once it gets to a place that I like, maybe I do some hand editing. I say, "Okay. Now, go and build it." 

Then once again, there's a review cycle when it does that. It very much is that tech lead or manager mindset to get it to implement but on a much faster pace. We went from finished reading the paper to testing it in dev in 20 minutes. 

[0:13:34] VA: That's the area that I found incredible productivity boost in my own work, where even to the point of I just need to understand something and normally I would have to find some examples, go through a bunch of documentation. I'm building a prototype of how to deploy something in Kubernetes that I have in some edge case of something that I haven't used before. Whereas before that cycle of learning and trying and prototyping would take probably a day, I can have deep research, for example, go out, and collect a bunch of information, and give me a first pass at it that's 90% of the way you're there in five minutes, which is just a huge difference. 

I do see people talking about how they tried out AI or agents, and they don't seem to be getting a lot out of it. There does seem to be a skill set that you need to develop. It's like you need to know how to ask the right question and bring in the right information in order to get value out of it. If you don't ask the question the right way or bring in the right information, then it just seems like a waste, right? 

[0:14:34] KB: Yes. I think you're totally right. There's a mindset aspect too, right? It's do I expect this thing to be a magic solvent for me box. Or do I expect a back and forth and a steering and an experimenting and a probabilistic molding of this thing? 

[0:14:51] VA: Yes. One other comparison that I've made is it feels like when I'm working with an agent at this point, it feels like pair programming where I don't have my hands on the keyboard, right? It's like the other person is doing the typing, and I'm saying, "No, no. Maybe you should change that thing." But I'm still, and I'm curious if you feel like is that vibe coding. Or is vibe coding this other world where you're even more hands off, and you're just not even really looking at the code that's produced? Or is there a spectrum? 

[0:15:15] KB: Well, I think this is the key question, right? To me, this is actually how you go from vibe coding to production is you add validation. Instead of turning your brain off, you are paying attention. You're finding ways to validate. You're looking for formal ways to validate, which might be tests or type checking, and you are guiding the process. I think pure vibe coding is like, "Whoa, I don't care about any of that. Just make this thing happen," which I think as you highlight, it'll get you somewhere. But that's going to plateau very quickly. 

[0:15:45] VA: I mean, I think I'm on the same page with that one. That's my sense, but I'm following with bated breath just to see what the next new big innovation change. It's an exciting time to be in the industry because there is so much going on. I remember six months ago, a year ago, the whole story was around are we going to replace programmers. We're not going to need programmers in the next year or two. We're going to have AGI in next year, all of these things. My position early on and actually Heroku's position as well was that these kind of tooling was going to actually increase the number of developers because it makes it more accessible to more people.

It may be that there are less people with software developer in their title, but they're going to be a lot more people doing software development using these tools than there ever happened before, which means you need a lot more people to operate or a lot more places to operate those things, operation of software that more and more people more people are writing. I mean, if you have hundreds of people building personal apps that they just - because it's so easy. They just built it for themselves. It solves a good problem for them. They're going to need a place to run that app at some point. That's worked for us so far. 

[0:16:53] KB: Let's maybe dive down. There's two interesting things to explore there. One is you touched on industry implications. I think there's a lot of people feeling a lot of uncertainty about what does this mean for the job. We can kind of go down like what does the software industry look like as this plays out. But the other that I think maybe we should touch on even first is what sort of tooling needs to be created to enable this boom in individuals vibe coding themselves personal apps. 

[0:17:22] VA: Yes. I think it's interesting, and I actually know some people in this space that are very heavily trying to solve that particular problem. It's like because you need a different set of infrastructure in order to make that experience a real thing, where you can build a tool that's a personal app. How do you give it your credit card to buy something? There's a lot of trust issues when you start thinking about that level of personal assistance and apps that you could build. 

But I think one thing that's interesting, and you mentioned at the beginning of us talking that this seems very different from what we were talking about with 12-factor, right? But one of the things that I think has become more obvious to me over time is that at the base level, you still have an application running. Even if a person isn't writing code, that code is eventually getting written and then having to execute. The underlying infrastructure, what makes a good application, what makes a manageable application, all of those things are really the same. They're just maybe a little bit more under the surface. 

I think this is true of what we're seeing in the protocol and API development space, too. The thing that makes agents able to use existing functionality, billing plugins, is having a good API, having it well-documented. If it's easy to use for a human, it's going to be easy to use for an agent, right? You have to solve all of the problems you had to solve for humans, for the agents as well. I almost feel like there's going to be this big loop where everyone goes, "Oh, wow. So we actually have to continue doing what we were doing anyway," which is funny. 

[0:18:58] KB: You are spot on with this. One of the things that it reminds me of is one of the things I've discovered as I use LLMs to code more and more and more is the same practices that are good software development practice for humans make the LLM do a much better job. Did you decompose your concerns well so that it's clear what does what and how things work? Well, the LLM is going to be able to generate code a lot easier if they're well-decomposed. Did you document it? Do you have tests? All these different pieces support LLM-driven coding in a production environment, which is not really vibes only. It's got those layers of validation, but that's how you get to value. 

[0:19:36] VA: I think there's an interesting tool set, too, in terms of improving. It's along the lines of what you're talking about of setting up the right structure that an LLM needs to live within to build good software. Early on, what I noticed, and this is harkening back to the problem of I was trying to write Rust code, and it was having a big challenge with it. I think in many cases, in order for the LLM to be successful, you need a set of information to feed to the LLM. You said use Cursor. It might be in your Cursor rules file that helps it do a better job. 

I think in many cases, in order for the LLM to be successful, you need a set of information to feed to the LLM. And you said you use Cursor, it might be in your Cursor rules file, that helps it do a better job. I think one of the things that we haven't really solved yet as an industry is how do we get that information into the LLMs in the most effective way? I don't know that it's just throwing a bunch of extra stuff into the prompt and saying, "Make sure to DRY and use these libraries and stuff like that." I think there might be something more to that. 

One of the things I've been pushing for is leaning on the open source framework communities to try and build some of these things. I imagined I called it like a framework knowledge base. Ember.js might have a set of information that could be used either in RAG for the LLM. It could be used to fine-tune LLMs. But as a community, if we can start building up these repositories of information of how to do good coding in a particular language and framework, then the LLMs are going to get better and then we can start now and maybe a year from now we'll figure out, "Oh, there's a new way to put memory into the LLM and it'll happen a different way," but we need that knowledge base. It's like we need better data is kind of what I'm getting to. 

[0:21:09] KB: 100%. I think one of the things that I've seen is you want to integrate a new library. You go to their docs. But docs written for humans, while good, are not actually optimal for our LLM context. And so we have actually a process at my company where we'll take a set of docs and then we will pull it down and we'll say, "Okay, write a tutorial based on this with a bunch of code examples for this, this and that." And then that becomes our context, anytime we're referencing it, we reference. 

[0:21:34] VA: Oh, that's fascinating. I mean, I wrote an article about this, gosh, almost a year ago now, and it didn't ever really get enough attention. A couple people went, "Yeah, it sounds like a good idea. But things take time to gain momentum." But I feel like there's got to be some best practices happening inside of companies around this thing. And could we do this as a community? Could we get open source behind sharing these things and having them out there so that everybody benefits? 

[0:21:59] KB: Totally agree. I feel like every docs page should have a link to the LLM docs for example, which are going to be heavier on code examples and a few other things that help the LLM do well. 

[0:22:10] VA: Yeah, that seems like an obvious direction to go. It's just there's so much frothiness in the space right now, it's hard to get anybody to focus on the same things and collaborate. 

[0:22:20] KB: Well, and I think another interesting area to go into, and I'm curious if this is something you're thinking about at Heroku. It feels like a lot of the tools right now are focused on the individual coder, right? Cursor, or Lovable, or all these other different things is like one person is going to come and they're going to prompt this thing and create an output. But I feel like there's also a whole layer of team-level challenges, right? How do you deal with sharing? What are useful prompts? What are useful things? If the code is no longer the starting source of truth, there's a chat that is the starting source of truth, how does that get represented? How do we collaborate as teams in this? 

[0:23:00] VA: That is a fascinating question. And to be honest, I haven't thought about that one in detail. Ideas are bubbling up as you say it about what is the optimal way to collaborate with multiple people. I mean, the direction I was initially thinking of is do you use shared deployments of applications as a way to start having that shared state? You have the repository, which is shared. You have the deployed application. Maybe you have a staging environment that's shared. And can you start feeding information from those places? 

But also, I recognize there is an interesting question around the team collaboration and how you get the - where do you store the artifacts that you use to produce what you produced? Because that can be useful. I think about all the times I go continue a conversation that I had last week with an LLM because there's a bunch of relevant context, next question I want to ask that are kind of sitting there, right? And that gets lost if it's just a person sitting on their laptop doing it. Fascinating idea. 

I think one interesting place where this gap is becoming a bit obvious. I mentioned earlier model context protocol. It's an area that we've been looking at a lot because of all the interest. When Anthropic released it initially, there was this big question in the industry. It's like, "It seems sort of like this proprietary thing that they did and it could be useful to have a standard," but everybody was waiting for everyone else to kind of adopt it and use it. And then suddenly, everybody went, "Oh, this is the standard. Now everybody's doing it. We're seeing all these releases about it. 

But the thing that is most fascinating to me is how heavily the standard was focused on a use case of running on an individual's computer and using standard I/O to expose the tools. The protocol was clearly written for that use case, and almost all of the open source tools that you see out there are focused just on the standard I/O implementation. And coming from a distributed systems background and thinking about deployments in clouds, I'm like, "This is very strange," because I think about a tool as something that's remote and that is used like an API. 

And even the spec itself is a little light on that use. It didn't have authentication built in. It uses JSON RPC, which is a pretty flexible protocol, but it seems it was clearly built off of the idea of having a stateful request response on a single person's machine instead of a remote stateless protocol, which would be more appropriate for a web service. And that's an area we've been digging into a little bit, is can we make this a little bit easier for people to use in the more remote context? 

[0:25:24] KB: I would love to hear more about that because we had an example of that, once again, coming from my workspace where we prototyped something using MCP. We're like, "This is great. We should integrate this into our agent," which runs in a cloud-based distributed environment. And it was like, "Oh, now there's a whole slew of things we have to handle that MCP doesn't give you out of the box."

[0:25:41] VA: Yeah, I think one of the interesting choices was to lean on the SSE HTTP protocol for the remote part of the spec, which has been around for a while. And it supports sort of streaming and notifications, which I think is one of the things that we're trying to achieve, but it's not very used in the industry. It's much more common in industry to do something a little bit more like what OpenAPI was doing with their tool calling. 

And I think there was actually - initially, a long time ago, the plugin model had a JSON OpenAPI spec that you kind of put alongside your plugin in a well-known location. And that feels a little bit more like what you would see in a remote request response because most APIs are already built that way. So it's very easy for someone to throw that additional endpoint in there with a little JSON schema saying what the API does and just do tool calling that way. This direction leans a lot more on the stateful side, but there have been some changes. 

I don't know if you've been keeping up with the changes going into model context protocol, but there was just an extension, a new protocol that focuses a little bit more on stateless and allows you to upgrade into a stateful connection if you want to, which I think gives some more room for those type of use cases. It still uses JSON RPC because that's sort of what the underlying specification uses. 

And there is also some work, I don't know if it's merged yet around supporting OAuth as an authentication mechanism, which I think is a huge thing. And now that we've added Windsurf , Cursor, all of the tooling has added model context protocol support, if OAuth gets in the spec, then your tool will be able to use a remote server using OAuth and you'll have a pretty simple flow where you'll just say, "There's a server over there, click a button, authenticate." 

It doesn't necessarily solve the service-to-service authentication issue. But, definitely, a user-to-surface authentication is much easier in that world. And I think we'll get to service-to-service and more standardized way of authenticating and connecting these things over the next few months just because so much is going into this right now. And it will all be easier, is my guess. 

[0:27:44] KB: Well, and my associations with Heroku are back from like 2010-ish or whatever. And it was like that was the principle, is suddenly this is easy. This is easy to do. I'm kind of curious, are there other big gaps you're seeing people trying to deploy agentic things or all this stuff where it's not easy now, but you see a route to making it easy? 

[0:28:04] VA: I think that's the general space that we're trying to be in. We're working right now on exposing the Heroku platform as a model context protocol service so that we might be able to give LLMs access to deployment concerns instead of just development concerns, which I think is a really interesting space. And I'm sure we'll uncover some interesting issues and learnings from doing that. But then we're also focusing on just trying to make it super easy for people to deploy model context protocol servers on Heroku. And it does involve solving some of these remote stateful problems. 

I mean, at the very least, if model context protocol servers are always stateful, then they actually don't follow the 12-factor app principles, because the 12-factor app is supposed to be stateless. You're kind of painting yourself into a hard to maintain corner there. Then we're trying to make that easier by building some bridges. And, essentially, where there's rough edges in the spec, just building in maybe some new ideas and some polyfill mechanisms to help push the spec forward. We're trying to participate in the community and get involved there and I'm excited about where it can go. It's got the attention and I'm super familiar with this kind of development because OpenStack followed this exact same model. 

When we released OpenStack, it didn't work. It was an idea. And it got a ton of interest because there was a moment in the industry where everybody was saying, "Oh no, what do we do about AWS?" Right? We got to compete here. They're way ahead. How do we catch up? Let's all collaborate on an idea. And I feel like the MCP thing is in the same space, where it's not quite there yet. The technology is in process, it's improving, but the idea is sound. And everybody wants to collaborate on the idea and participate in the idea, which is a great recipe for really pushing technology forward quickly. 

[0:29:50] KB: Well, and the fact that they started with something that more or less works in a local environment means that, as an indie hacker, you can go wild because you only care about your local environment. It's when we start taking it more towards production use cases that all of these edge cases become important. 

[0:30:06] VA: I totally agree with that. Having a nice local development story is super important for getting any kind of attention. One of the areas that Kubernetes struggled for a long time was how hard it was to use locally. 

[0:30:20] KB: I have some battle scars from that, yeah. 

[0:30:22] VA: It is a good platform to build off of. Having that as a starting point, it's got the momentum. We just need to get some tweaks in. I mean, as with any protocol, it's not necessarily about having the protocol perfect. It's about everybody using it, right? It doesn't matter how good it is if nobody uses it. And that's the big lesson. 

[0:30:43] KB: We went down one branch of where we had been. Let's rewind back and talk about the other branch of what we of what we had touched on, which is what's going to happen to people who are software developers now? Everybody's nervous around it. And you started to paint what to me sounds like a pretty optimistic picture. 

[0:31:03] VA: I mean, nobody knows the future. But I definitely have people asking me, "Should I stop studying software engineering? It just seems like there's not going to be an industry anymore." I definitely don't think we're anywhere close to that, right? But I do think the skill set is different. I mean, the skill set for writing is going to be different. The skill set for art is going to be different. Everything is kind of changing, but it doesn't mean that artists aren't going to have a purpose anymore or software engineers aren't going to have a purpose, so we're not going to have them. When photography came around, it suddenly wasn't all that impressive to be able to paint a realistic image. But art didn't go away. They just changed what they were focused on. And I think the same thing is true in our industry. It's like the value of being able to hammer out a bunch of code quickly may go down, but the value of building a system that works and is maintainable and operable over time is still going to be there. And the way that you build it just might start changing a little bit. That's my take. 

[0:32:03] KB: Yeah, I think that makes a lot of sense. I also like to do the econ geek thing. At a certain price point, there's some amount of demand. What we're saying is the cost of writing software is going down. Then either we need to have - it's going to support fewer people or there's a lot more demand for software. I think software probably has infinite demand. Software is dreams made reality. I have an idea and suddenly it can be true. I don't know any other place in the world where you can get kind of that level of creativity in terms of what's possible. 

[0:32:41] VA: I think that's true. I have a comparison that I have been making which is - I consume a lot of content on YouTube. I'm so impressed by how you can learn things that just you would have had to go find a mentor in some other country to go learn a skill set. And now you can just pick it up immediately by watching some YouTube videos. I think it's fascinating. 

And what I've seen over the past, say, 10 years of consuming that content is the minimum bar on the quality of content has just continually been going up. But it doesn't mean there's less content creators, it just means they're producing much better stuff, right? And I think it's similar in what you're saying is there is sort of an endless amount of consuming of that kind of value, or at least we haven't saturated in it yet. 

And so if everyone is able to produce apps, the minimum quality of an app is going to go up, but there's still going to be a lot of apps out there. I mean, it just seems sort of obvious that that would be the case. It's not going to go away. You're going to have analysts, and business people, and marketing people building apps. You're going to have developers building apps. The floor is higher. They're all going to be better, right? Are there going to be less of them? Is there going to be less people doing it? I don't really buy that part of the story. It seems like there's going to be an equivalent amount or more. What do you think? 

[0:34:07] KB: To me, I feel like there are so many both personal and business use cases out there that don't have good software right now because it was too expensive to create it, right? 

[0:34:18] VA: Oh, yeah, yeah. 

[0:34:20] KB: Essentially, every local store could benefit from a system customized to their inventory management or other things like that. And maybe it's not totally custom software. Maybe it's built on top of software, but it's some amount that's customized to their workflow. But that never happens because software engineers are expensive. And to build something that's solid, you got to hire a team of five people for six weeks or whatever it is to build it and get it solid. 

But if suddenly that same level of quality, as you highlight, the quality of what an individual can produce goes up, instead of six people for five weeks, you can hire one person for two weeks and they work with an LLM to iterate and get you that piece of software, maybe that's now in your budget. 

[0:35:02] VA: Right. And so you're expanding the market significantly. Yeah, I think that's the key point. I'm thinking about in my early days of programming when I was doing some work with nonprofits and I was building access databases and just throwing together the most janky interface for them to run their business or nonprofit off of and how, with that same amount of effort, I could build something much, much better for them that makes their jobs all much easier, because now they can afford the better experience. I think that's totally right on. 

[0:35:36] KB: The question to me, though, is what happens to teams, right? I feel like the industry for a long time has had this fundamental unit of a team size that I think is popularized by the Amazon two pizza team, right? And it's like maybe you have five or six developers, you've got a product manager, you've got a designer, a subject matter expert, something like that, this set of eight or nine people, and that's kind of a fundamental unit for a lot of at least product development. Do you need five or six developers if they're able to ride the vibe coding wave and produce 10 times as much in half the time? I don't know if you do. 

[0:36:14] VA: That is a very interesting question. I have an alternative challenge that rises to my mind from that question. And it's very personal because I've recently taken over managing the team of Heroku engineers, which is there's about 220 people on that team. And what I've noticed at a certain scale, all of the problems are the interfaces between the different groups, as opposed to within the team themselves, right? 

You might be able to say, "Well, suddenly, my 220 people only need two people on each team, but now you have 110 teams." How do you collaborate and deliver across groups in a situation like that? Because most large pieces of software end up needing that kind of interaction from people building different components. The interesting question is, does solving our technology problems make our people problems worse. And are the LLMs going to be able to help us with that part of the problem? 

[0:37:11] KB: Yeah, it's a real question. I can see a couple of different ways you could go, right? Maybe your 200-person team, which is made up of - I'm just going to round to maybe 20 teams of 10. Maybe you still only have 20 fundamental units, and those are now teams of two. And so you have fewer people, fewer connections, it's not a problem. Or maybe, as you highlight, you have a flowering. You still need 200 people or you have 200 people because that's great. But now instead of 20 teams, you've got 100 teams. That's a whole other layer of challenge. 

[0:37:43] VA: And it feels like, in some ways, it's kind of an economics problem. And maybe this leans back into the story that you were telling about how there's an unlimited demand in this particular case. I've never been on a software team anywhere that ran out of things to do, right? It's like you can always find - if it suddenly costs one-fifth, teams of 10 to teams of 2 in development cost to build something, you've just released four-fifths, 80 % of new capacity that, if it's that much cheaper, you're going to find something for them to do to make the business money, because suddenly, the profit is higher. The margins are better, right? You're going to find more work and expand your business. 

And I think what I've noticed is that economics problems are fundamentally super counterintuitive, right? As humans, we tend to think of everything as zero-sum, and almost all actual economic situations really are not zero-sum. And I think that's an interesting takeaway that I haven't thought about in depth, but the way that you put it about the unlimited demand, I think, really is the key factor of that equation. There is a demand for software that kind of will just keep - they say it eats the world, right? It's eating the world because the demand doesn't stop. And so it does seem like the value and the productivity will increase. 

I mean, obviously, there's going to be some shakeups, and there's going to be some challenges. A skill set that was valuable 10 years ago may not be the same skill set that's going to be valued 10 years in the future. But there are whole new areas of industry that will open up because of that economic shift. 

[0:39:14] KB: Talking about that skill set and bringing us back to vibe coding, what skills do you think people need to be developing in a vibe coding-driven world? 

[0:39:26] VA: The biggest one that seems to be relevant in my own experience working is the problem-solving skill set is still strongly there. And I've seen a wide range of people who are good at debugging issues. You have those developers on your team that are really good at sort of jumping into a production problem and getting right to what's going wrong super quickly. And that skill set is actually pretty rare. Not a lot of people are good at that particular part of debugging. And that part seems to be super relevant in this space. Because at least right now, the LLMs get into very weird corners a lot, right? And being able to quickly isolate where it's going wrong and how to fix it I think is going to be super important because it amplifies your productivity. You can be stuck. It's kind of like that classic example of you spend all day debugging something and it turns out you missed a semicolon. The smallest problem can cost you so much time if you're looking in the wrong place, right? And I think the thing is going to be true as we start using these agents as well. 

One skill set I think is the debugging problem-solving skill set, being able to sort of break problems into smaller pieces and really isolate what the source of the problem is, is going to still be super useful. And then being able to clearly communicate and specify what you need seems to also be very, very important. When you ask the right question, you get the right answer. When you ask the wrong question, you get the wrong answer. I mean, it's a lot of the same skill sets you need when talking to a person. If you're telling the person the wrong thing to do, you're not going to get what you want, right? Clear communication. Being able to understand and clearly specify what you need. LLMs seem to do a lot better when - the English language is very ambiguous. And being able to cut through that and make ambiguity and be very clear about what you need and how you need it to be done means the LLM can jump ahead much faster than if you just give it a very vague direction forward. Cutting through ambiguity seems another one. I'm sure there are more. Those are the ones that come to mind. 

[0:41:31] KB: Have you seen a good way to teach people debugging? Because I too have seen this like very wide discrepancy in debugging skills among people that may be very competent in other parts of engineering. 

[0:41:45] VA: Yeah. I did read a book that I felt super concisely explained the way that I think about debugging. And I have leaned on sort of that understanding. Sometimes you need to read something to know how to explain it to someone else. I've leaned on some of the understanding I gained from that book when I'm helping people. But I can't say that I've solved the Rubik's Cube of conveying that skill. 

I think some people, their brains naturally work that way a little bit more. But the concise way - I can find the book. I don't remember the name offhand. It's been a while since I read it. But it was talking about problem-solving not just in a programming space, but in like, "If you're needing to fix your car, here's how you go about figuring out what to do." Right? And And the concise version, it involves seeing the whole problem space, finding somewhere in the middle, cut the problem space in half, and determine which side of the line your problem is on, and then continuing to do that. It's bisect. That was the gist of what it was talking about. And when I read it, I was like, "This is what I do. This is why I can do this." I don't think I could have explained it to someone before I read the description, but it was super helpful in sort of understanding how I developed the skill. 

[0:42:59] KB: Yeah. No, there is actually something there. I think one of the things that I've observed in folks who are better at debugging is they spend a lot of time ruling things out, even things that poorer debuggers assume, essentially. Right? And so they'll do it like, "What assumptions do I have? How do I validate them? How do I rule out this side of things? How do I narrow in or bisect towards where the problem likely lies?" 

[0:43:22] VA: I think that's the essential question, is how do you throw out half the options? Most people are looking for the thing that's going wrong. They're looking to find the needle instead of removing half of the hay, I guess. If you just keep removing half of the hay, that's how you actually get there. 

[0:43:40] KB: That makes a ton of sense. Also, I suspect that's why LLMs are not that good at debugging because they're very confirmation-focused. They're constantly looking for a thing, predicting what is the next thing. 

[0:43:51] VA: Fascinating. Yeah, that's good. I hadn't put my finger on that one yet. That's really insightful. 

[0:43:56] KB: I hadn't thought of that until the way you described it. Yeah, that's interesting. Okay, so that's debugging. I think communication is another one where, I mean, of course we know all technical people are great at straightforward human communication. 

[0:44:09] VA: Oh, yeah. It's definitely a broad skill set in the software engineering community. 

[0:44:14] KB: But it is very learnable, right? Because it's a very common challenge. And there's both courses, there are groups, if you want to learn to speak better, like Toastmasters and things like that, but they very much will work on that clear communication. I wonder if we will see a renaissance and like, "Hey, actually, now our coders talk to the rest of the company."

[0:44:34] VA: That's interesting. That ties in with the team topic you were discussing earlier, because it seems like it could go in a couple of different directions. One of them could be, well, you just can get so much more done with one person. So maybe you just give that super introverted engineer an LLM and say, "Go solve this whole problem by yourself," or it could be that you end up with a lot more team bottlenecks between teams where you actually need to focus on that side a little bit more. That's fascinating. 

[0:45:04] KB: I've been thinking about it in terms of constraints and sort of the law of constraints and how those things work great. Historically, a lot of software development was constrained on writing code. That was the slowest piece. I think this moves the constraint around. And very often, now the constraint is on decision-making. What is it we want to build? What is it we need to validate? What is it we need to test? And to me, that elevates one level of technical problem solving. How do we want to approach this? But it also really elevates constraints in product and user research and design. Those become a much more limiting factor in how fast you can go. 

[0:45:41] VA: But what about when agents are the ones doing the purchasing instead of the people? 

[0:45:46] KB: That's moving a step or two down the road. How did their decisions work? 

[0:45:51] VA: I think we're a little bit far from that. But I think you're right, there are new bottlenecks. I mean, anytime new technology comes in, it moves the challenge area around a little bit. And we're having a really, really big movement here. And I don't know that we have even understood where the new places to optimize are going to be. As you say, if we suddenly have much more actual code output capacity, do the bottlenecks move into validation? Do they move into QA also? That's another area that could be a little bit harder to automate. Yeah. 

[0:46:21] KB: It's a fascinating time. Well, we're getting close to the end of our time. Do you have anything that we haven't talked about yet that you would really like to bring forward? 

[0:46:28] VA: I hinted on this a little bit, but I think one area that I'm really interested in, and it's something that we've been exploring a little bit, is what are the tool sets? What is the information that you need to provide to an agent to have it help with the areas around the main code production part of software, right? And I think we've hinted at it as in team collaboration and things like that. How do we get teams more involved? 

But I also think about it is like - I mean, we've been through many phases, ops, devops, then Google SRE, how you manage software over time has evolved and changed over the past couple of decades. But if you think about if your goal is not just to make the code author better, but your goal is to make the SRA team better, or the operations team better, or even if it's the developer who's operating the code, if you're making day two operations better, what actual information do you need to get to the LLM? And what are the constraints and guidelines you need to put around that? 

I mean, I think everybody's worried about having an agent scale up and down their production deployment of something because suddenly there's a lot of dollars on the line that you don't want to necessarily offload to a human, but you need to automate some parts of that or else you're just increasing your operations costs. And so where do we draw the line? Do we start feeding logs and metrics into the agent and let it make decisions? Is there a human in the loop there to approve or deny? Those types of questions are I think areas that are going to become super relevant over the next few months as people start moving farther along in the deployment cycle with their agent-developed code bases. 

[0:48:05] KB: Yeah, that is a fascinating domain. Are you all tackling any of those particular pieces right now? 

[0:48:10] VA: Yeah. I mean, right now, we're just exploring. For example, we give an agent access to the Heroku API, that means that the agent can say, "Scale my dynos up." And it could say, "Let me check the logs to see if there's a problem." There are capabilities that the agent suddenly has because it has access to the deployment system. We haven't done enough investigation yet to figure out where the right guardrails need to be in that system, right? We're in the more exploration, "Let's give people the tools." Kind of like you're saying, you give people standard IO, they can play with it locally and figure out what's going on. I think we're going to take the same approach. It's like let's just get the tools out there. Let's let people start trying out different things with their seven or eight different coding agents that are out there, and then see what we learn. See what areas we need to improve our API. 

I mean, we're thinking about, is the API - in many cases, as you're saying, the API that you give to a human might be the right thing. Agents work a lot like humans do. Well-documented API, for example, is probably still valuable for an agent. But I think you made a really key point when you talked about the information about your documentation even isn't necessarily the right thing that you need to give to an agent. 

And I think the same thing is true in the operations world. You want to give a human the ability to tail their log so they can look at something live. That's probably not the way you want to offer that thing to an agent because you probably don't want to stream logs in and have it be responding in real time to the logs. There's probably a different interface that the agent needs to see that information. Maybe you expose an API that's a RAG endpoint for logs, just as a complete flyer of an idea, right? There's going to be different APIs that you need to actually make the agent successful, and we're only going to discover those by getting the tooling into the hands of people to start playing with and then learning with our software development community. 

[0:49:59] KB: Yeah, it's fascinating. I mean, I think one of the things that I've seen both in terms of having agents write code and call functions is the more semantically distinct you can make your functions the better the LLM does edit. Fetched logs with a bunch of parameters is going to do a lot worse than fetched last day's logs, fetched last week's logs, and kind of exploding out the options into semantically distinct things that communicate in their names what it is that the agent would want to do with them. 

[0:50:31] VA: Right, because ultimately you're providing - there's two parts. There's the actual tool and then there's the decision making, the reasoning that needs to happen about which tool to use. 

[0:50:40] KB: And what parameters to pass it and all that other stuff. Yeah, yeah, yeah. 

[0:50:43] VA: Yeah, that's really cool. I'm also fascinated to see the MCP space is not just about tools. They have a few different other endpoints that they're exposing. And I feel like there's a little bit more definition in the same respect. There's a difference between resource and a tool. And when you're exposing data, you could make a tool call that exposes the data or you can make a resource that exposes the data. But I think the advantage of clearly separating some of these things and having well-defined categories is the same problem of it makes it easier for the agent to make decisions. And I think we're just at the beginning of that. 

For example, I don't believe the MCP protocol standardizes the types of responses that comes. It standardizes the request, but not the response. And so when you start thinking about chaining LLM calls together or agents using agents, that response format actually becomes kind of important. And if it's just free text, then it's a little hard to use the tool effectively. I think we're going to start to see maybe we'll have a standard web search schema for a tool call, right? And you might decide, "I'm going to use Brave web search, or I'm going to use Perplexity, or I'm going to use Google web search," but the way that the communication happens is in a standard way so that the tools can be optimized in their training set or in their fine tuning to make those calls in the same way. And I think that's going to make all of the tool calling and agent use better. 

But until we have sort of - it's so early, we haven't gotten to that point where we've all agreed on, "This is probably the best interface for this thing." We're all exploring and trying different things. I do imagine we'll start getting into a little bit more well-defined interfaces around types of tools, calling conventions, and things like that that'll make the agents a lot more powerful in the future. 

[0:52:27] KB: Absolutely. Well, and at the end of the day, we'll let you do more things just on vibes. 

[0:52:32] VA: Yeah, vibe coding all the way. 

[0:52:35] KB: The vibes are good? 

[0:52:36] VA: Good vibes over here for sure. 

[END]