[00:00:00] JMC: Hi, Deepak. Welcome to Software Engineering Daily.

[00:00:03] DC: Jordi, nice to be here.

[00:00:04] JMC: It's your second time in the show. We were talking about a minute ago about the fact that you had recorded an interview with Jeff back in 2016. We will comment on that later. But your last tweet is months ago and I'm not here to question why you've abandoned my favorite social media platform. You and several million others, which is quite obvious. But you mentioned, it's actually, you last week there's a morning of Jeff Beck, if I'm not wrong. Correct me if I'm wrong. So, what do you like about Jeff Beck's music and specifically, I guess, guitarists on guitars. Are you a guitar player yourself?

[00:00:44] DC: I am really bad guitar player. The best way I play guitar is playing it through a really long reverb and echo loop chain. I'm actually more of a – I'm a synthesizer person, and I almost treat guitars as string oscillators. But since I was a child, like early teenager, even maybe before that. I've loved guitar playing. And Jeff Beck's been always been one of my probably three favorite guitar players.

[00:01:13] JMC: Who would be your top three then? Jeff Beck, we've got one. Rest in peace.

[00:01:17] DC: Steve Vai has always been my favorite. In fact, yes, when I first moved to the US, in 1996, I think three days after I landed up in university, I caught a bus to a Steve Vai concert. Basically, I've never done anything else on the US at that time. I've seen a couple more since then. It's tough to say who else I think. Jeff Beck and Steve Vai are probably my top two. But I'm going to call it, Jimmy Page. I've always been a huge fan of John McLaughlin. They’re the bunch of other, definitely to go on, there's so many good. I love good guitar players. It's nice to pick one.

[00:01:56] JMC: It's the same, but we can’t keep talking about music, which would be a great time to topic to spend time on it. But no, you actually are a longtime AWS employee. In fact, when you were interviewed by Jeff back in 2016, you already were, and you probably talked about technologies that were hot at that time in AWS. But you have been, right now or very recently, named a really fancy title, which you can explain in a minute. So, walk us through your career. Mention what you talked with Jeff back in the day, if you remember. I'll probably link the show, the episode in the show notes. Most importantly, talk about how your thrilling remit and charter right now at AWS.

[00:02:49] DC: Yes. I talked to Jeff a long time ago. So, I've been at AWS for a little over 15 years now. For most of the time, I've been involved in the compute side, which is what I talked to Jeff about. I started off on EC2, really low level on HPC and performance, and how to build high performance networks and big instances on the product side.

But then, increasingly, I got more interested in how software is packaged and delivered. So, in 2014, I started a containers organization that I ran for nine years. That was a lot of what we talked about then, container orchestration and container-based delivery. For a while, for the last year and a half or so, I also ran a serverless organizations that I think like Lambda and EventBridge, as well as things like CloudFormation and CDK. It's a good start. In general, I started getting super interested in the software deployment, software delivery pipeline, and I also picked up our IDE organization. That's the org that does CloudShell, the AWS toolkit for things like Visual Studio. Visual Studio code, JetBrains, and things like CloudShell.

I continued to move up the stack of the like to say, and about three months ago, we started a new organization, and our charter is in some ways, somewhat bombastic, but also very simple. My org is called Next Generation Developer Experience. As the name suggests, we consider us a developer experience organization. We are a bunch of folks who love making it easy for people to build and deploy and run applications in the end, especially the way they interact with AWS. That's how I like to think about it. But we all have a conviction that generative AI is going to change the way any of us interacts with software, and with AWS, specifically.

That's a sort of thing we set out to do as an organization. It didn't come out of nowhere. There's a bunch of things that have already started. There's a few things that are already in there. We've talked a little bit before I think this recording started about CodeWhisperer. That's one of the teams in my organization. It predates me be taking over. But it made complete sense combining the IDE or because CodeWhisperer is delivered through our toolkits and et cetera, along with CodeWhisperer and then things like CodeCatalyst, at some of our early efforts in delivering software in ways that are less about writing code, and other ways of doing it, that's what historically would be considered low code and no code. I'm not always a biggest fan of those words. But the general idea is there's a bunch of people who aren't just sitting in an IDE and writing code and deploying it that way. They do it in different ways. And with Gen AI, I think those ways actually are going to get very, very tractable. So, super interested in that as well.

So, the exciting part is, and for me, personally, this is the most excited I've been about how foundational change, I think, we have seen since I first did EC2 run instances in 2006, is I think we're going to change as an industry, just how people interact with software, with goal, with applications. Our goal is to do that for AWS.

[00:06:08] JMC: Okay, makes sense. So, what is it then that makes code or application delivery, or software delivery, so prone to be impacted in a really high way by Gen AI? What is it about Gen AI that makes it such a great technology to provide a revolutionary change to code specifically, or even pipelines as code? So, everything as code, right? What is it about those two elements that are such a great match?

[00:06:43] DC: Yes. I mean, for those people who got introduced to Generative AI, for many folks, it was two things in the creative space, like with photography, with video. We have a customer called Runway that AWS has, that does text a video, that's quite fascinating. There's a bunch. You see the word Festival Diffusion is done on text or images.

The general, if you look at what Generative AI does, is a well-trained model is able to do and infer a lot of intent. And is able to deliver, and give you – I wouldn't say read your mind. But if you are good at writing a prompt, it’s able to take that prompt and convert that intent into something. If it's been trained on code, which things like CodeWhisperer are, so it understands the semantics of code. It's able to convert your intent into well understood code.

Now, if your model is trained well, it's got good corpus behind it. It works really well. But what makes it super interesting over time, and I think this is what makes it so fascinating is I think you can always give these prompts and these models great context. For example, what your account credentials are, which AWS APIs you may want to use, if your prompt is written well. I think that's the part that makes it super fascinating is you take intent and context together, and a well-trained model can then generate code or something else. In this case, you're talking about code. That is a manifestation of that, and how it understands that intent, and that context. What Generative AI has proved and it's super early. I mean, it's kind of fascinating how quickly this timeline has gone is that given good intent and good context, you can get really good results. And as we get better in understanding intent and context, we’re only going to get better. I think that's the part that fascinates a bunch of us and why you're starting to see so much interest in things like CodeWhisperer.

[00:08:46] JMC: Yes, because not everything on a day to day, on the everyday life of a developer, is actually writing code, right? And requesting code completions, and requesting suggestions of code too. But actually, looking up, stuff? Finding – there's so many real-world examples of acquiring new code base. Is it documented? What the hell is this? How do I find my way through? Or even in my own code base, how do I look up an item? What is this module about? What does it do and how should I call it? What are the coding guidelines of the company? I'm a newcomer. I’m a newbie, a new hire, and I need to be onboarded into the codebase, the company, the best practices, and so forth.

There's a ton of development activity, heavy activities that are proud to be revolutionized right now. Co-suggestions have been important in IDEs for a while. They are probably going to be getting much better. But yes, there's a whole range of activities that I think you were hinting at right now. Asking, there's this natural language and a way of interfacing and reacting to things that it's going to be – it’s going to enrich so many different activities in the day to day life of a developer, right?

[00:10:05] DC: Absolutely. I mean, as you've probably heard us say, we are three kilometers into a marathon or a 10k race. Pick your distance. It's early. But you can already see the sign, whether it's the natural language conversational interfaces that you have. Whether it's the fact that there's two or three areas where I think Generative AI is going to make a significant impact. One of them is just eliminating toil. There's a bunch of things that people don't like doing, and the fact that you can get that done very quickly, is going to help you do more.

There's the code completion part, which, as you said, has been around in IDEs for a long time. But I think, in general, the IDE code completion was limited to language semantics. With things like CodeWhisperer, which has deep understanding of the AWS API, for example, it's more than that. You can tell it in natural language, “Hey, I want a method that does X with an S3 bucket.” It's not just about getting the syntax right. It's actually about giving you the code block, and calling the right – if you're writing Java code, the right libraries from the SDK, et cetera, to complete that. This is just the beginning. There's so much more we can do over time around helping people understand what the code is saying. You already mentioned things like learning the codebase, much faster, and so on.

So, I think, what makes it fascinating is, it's already helpful, because it is essentially providing your documentation on the fly without having to actually look at the documentation, and become an expert or look at every method. But at the same time, you can also see where it could go. So, if you combine the two, I think that's where I see a lot of developers getting excited. Because in some ways, today, yes, it's super helpful. But it's an incremental improvement on what they were doing, especially for more experienced developers. But I think where it really starts making an impact is for people who are less experienced, and experienced developers can see that if they commit to this now, there is a path that's going to happen really quickly, that is going to make a huge impact to them as well. I think that’s what, where I see all the excitement.

[00:12:26] JMC: Yes. I think that is going to release a lot of productivity, because, as you say, the need, the amount of oversight, companionship, training, that pay programming that the average junior developer will require, it does require currently from senior developers, and will be taken over by a CodeWhisperer, will obviously make the on ramp. Not obviously. But it seems that it's going to make the on ramp, the onboarding much faster of these junior developers. But it's also releasing senior developers which have, we all know that they make – they are the 10x developers of the company, or the project. It's going to release so much time from them, not only from onboarding new junior developers, but I'm thinking of code reviews, and all the different activities, that it's going to let them contribute to the project much more and so forth. So, I can only agree with you. It's going to be terrific in many areas.

[00:13:29] DC: Yes, I mean, I think one of my favorite sites always has been when new developers joined the company, straight out of college, as senior developer that sit next to them. They lead the bear program with them, or explain certain idioms, et cetera, as they're helping them onboard to the way code is built here. I think what it does is the things that they now are going to talk to them about are going to change to much more high value things. Then, they can focus on hard problems, on educating about how do you think about building a system, things like that, where it's tough to be senior developer, but the AI assistants, as you may call them, essentially, it it's like giving – the way I like to say it, is it's giving the junior developer two partners. One is a senior developer. It’s going to help them through hard problems, and one is the AI assistant that's going to help them onboard quickly, help them get much more product much fast – or write code much faster.

If you've learned anything from the history of technology. It's like, the more junior people, the people who aren't brought to learn in certain mediums, pick up the new ones really quickly, and become so much more effective with them, and I think that's going to happen.

[00:14:47] JMC: In RSA, there's certain aspects that we are now factoring in known in this conversation, when in general, it’s like there are grumpy senior developers. And me as a junior developer, I've been hesitant to tap their shoulder more than twice for maybe the same question because I'm very junior, but I'm learning. You know what, CodeWhisperer does not get upset if you ask him or her or it, the same question every time, or variations like variations of the same question every time, because it's got infinite patience, and the strong compute behind.

So, yes, code suggest, good throughput, it's clear that the amount of volume that – the amount of code that CodeWhisperer is going to generate that is toil, most are boilerplate stuff, is going to improve the throughput of everyone. What about testing? Does it help in any way in test generation? Does it give ideas of test coverage? Yes, what does it do in the realm of testing?

[00:15:51] DC: Yes, not today. I mean, the CodeWhisperer is not automatically – if you're writing a piece of software today, it doesn't automatically create a test for you. What it can do is have you write your tests, because test is also a code. That part is obviously there. But today, CodeWhisperer doesn't like, “Here's my code base, go write a bunch of unit tests, because I hate writing unit tests. Now, I'll have them.” Obviously, as you can imagine, those are the kinds of areas that our customers are talking to us about, just unit test generation, code reviews, everything is on developer lifecycle is pretty much up for grabs for our customers. I think, what they and us are all trying to do together is figure out which are the areas where we can make an impact now, where there's more work required both in the underlying science and the underlying methodologies to make it useful.

I think that's the interesting part about this time is we've already established over the last, I would say, year, that coding companions like CodeWhisperer, are making an impact to helping developers get stuff done. The question now is, how do we take that to the next level?

[00:17:06] JMC: Before I ask you about more the deployment side, which you have already hinted that with the calls to the API in S3 and so forth. But I'd like to elaborate a bit more, because it's probably, I mean, you just mentioned it, that the dev side of things is making a lot of progress, and it's heavily prone to be revolutionized. But what about the proper – before I ask you that, one thing that I don't want to miss about CodeWhisperer, that makes me really happy as someone who involved in the SPDX project is that it behaves as a very nice, and proficient, and has a really high standard for the etiquette of the open source, the management of open source licensing. Because correct me if I'm wrong, and I'd like to elaborate on how it does this. But the code suggestions are tagged in some way with the license of the code base it has been trained on. Could you explain how that works?

[00:18:03] DC: Right from the beginning of CodeWhisperer, and when, I think, from almost the initial press release fact that we wrote as we do, when we are doing product development inside Amazon, the general idea of helping, making sure that because we knew our customers would be often in enterprises, where they really, really care about what the software, where the software comes from. 

[00:18:27] JMC: The concern is, if CodeWhisperer, or any other, but in this case, CodeWhisperer is suggesting the literal code, chunks of code from the training dataset, will that make me the CodeWhisperer client suitable, a target for illegal challenge for misusing the license? That the trained – that’s the clinch, right?

[00:18:54] DC: Yes, all these companies are ready. Today, when developers bring in packages from left, wherever, and are putting in code, they already have a lot of effort which is very manual, or in many ways, are various scanning, et cetera. So, us, a lot of our customers are very aware of this problem. They spent a lot of resources on it. So, it was very – right from the beginning, we knew that is the problem we needed to address.

What CodeWhisperer has for those who don't know, it has a reference tracker. What the reference tracker does is, as you're writing code, it is able to identify, “Hey, here's a piece of code that we believe is similar, or could we interpret it as being similar to a piece of open source code?” You may have picked it up from some repository. What it does, is it'll tell you that, and then point to the repository that it thinks it is that open source, particular piece of code comes from. Then you can go in, evaluate the license, decide if that's the license that a company is allowed to use. If it's okay, what – all the things that you have to do a lot of work to do, to figure out and get to that point. It just gives it to you. And it was right from the rave even built a model and trained them, this was part of the goal.

So, it wasn't something that was added on. I think that's why it works as well as it does.

[00:20:09] JMC: It does. From what I've heard, it really does. I know that many people that appreciate and want to be a good open source citizen, not only approve of it, but endorse it. There's also another aspect about code security, right? That I'm not sure how it actually works and I'd love for you to explain to us. Because it does some kind of a scanning. I'm not sure if it scans the training data for known vulnerabilities or the other way around. If CodeWhisperer in one of – without any filtering suggests code, that may contain any kind vulnerability. Because let's remember, LLMs for code or for anything, are not intelligent. They're really good at predicting the next word, and they make sense, but they're not definitely human and definitely intelligent in a human way.

So, it's necessary to put filters up, but behind the output so that potentially vulnerabilities are filtered out, or any other concerns. Could you describe how CodeWhisperer actually helps in that sense? In what way does it do it?

[00:21:25] DC: Yes. So, there is no – it doesn't take one approach. For good security, you need multiple approaches. For example, you can’t scan for the top tier OWASP, top 10 vulnerability. That's one example. You have script library best practices that is trained not – it understands though. It's been trained on internal AWS security best practices and things like if you're using AWS APIs, it kind of knows what we would recommend and not recommend, from a security perspective, like opening up your bucket, opening your bucket to the old world, is the classic example.

So, there's some things that are part of its training and rule, some things you scan, and then you can suggest remediations to the core vulnerability. I think this comes from, we've had a product for a while called CodeGuru, which is our Generative AI historically. It's an AI rules-based system that looks at code and scans code and tells you how to improve your code to be secure. Some of those practices is what made it into CodeWhisperer. But more recently, as part of this new org, we actually combine the CodeWhisperer and CodeGuru teams into a single team. The idea is pretty simple. In the end, the quality of your code is very important, and quality include security.

Right now, we are doing some somewhat simple stuff, you could argue, where we are taking a bunch of well tried and tested. Take a look at, here are some well-known vulnerabilities. Are these showing up? Here's how you would never use open up an S3 bucket. We can train those or you can add rules on. Now, if you combine it with something like CodeGuru, which you can apply to CodeWhisperer code anyway. But if you make it just part of – I think there's some very interesting ideas that we have there. So, we've put the two teams together so they can start thinking together about how to make code generation and continue to keep it secure.

[00:23:21] JMC: So, how does the deployment side work in CodeWhisperer? I guess, there are three types of ways of interacting with AI assistants, AI companions, wherever we want to call them, which would be requesting that coaches listen via comment, which would be the most traditional way, and maybe an instruction, and finally, a natural language, a plain English in this case, plain language interface in which we treat the thing as a human that understands our language. How would deploying an application in CodeWhisperer work in such fashion? Is it ready for a chat interaction and say, “Hey, deploy this? Hey, what is the performance of my application in Australia or in said region of AWS?” Can you do such nifty and detailed render things?

[00:24:20] DC: Not yet. There's a bunch of – again, there’s an area where – one those areas where, all I will say is watch the space. CodeWhisperer is kind of in the middle of what you said, which is it does the way you add code in CodeWhisperer is either by typing or adding in a comment, but the comment can be written in natural language, and it converts that into code. So, I think that's the interesting part. Once the code is checked in, then, of course, your standard deployment practices come in. But I think we were trying a few other interesting ideas around this. So, if you've ever been to the Lambda console, it has an embed added editor. We have started a new version of the embedded editor that we've started releasing. You can find it in the ECS console, right now, maybe in a few others.

I remember when ECS adopted it, because I was running ECS at the time. But the fun part of this embedded cloud editor that's in the AWS console and it's up to the service teams to pull it in. But you'll see it in many, many places in the console, where it makes sense, is that has CodeWhisperer built into it. So, I'm in the Lambda console and I am going to write my own Lambda function, because you have an editor right there. CodeWhisperer can write the whole function for you, nd then you can deploy the Lambda function from right there.

So, that's an example of where it does make that process a lot easier. The reason we did cloud editor, the way we've done it, is we decided to just – the idea was, “Hey, we're just going to make this simpler in some way, the simple editor, the syntax highlighting, et cetera.” But make it an embeddable widget in any AWS console, and it has CodeWhisperer built in. So, if you have access to CodeWhisperer, it will actually start computing your code for you, and Lambda is a great one, because you’re writing snippets of code, writing functions, and you're just deploying them right there, and that works really, really well for that use case.

[00:26:21] JMC: I can only see CodeWhisperer excelling at GitHub’s declarative models, right? In which your system, and this is something that some of your products actually support. I know for a fact. Yes, if your whole system is written, is using Kubernetes, for example, and it's described in a declarative way, in code in a Git repo, or any kind of repo, then interacting with it with a whole system through CodeWhisperer must be actually something kind of easy, right?

[00:27:00] DC: I don't know if it's easy yet. I think the part that we are all still figuring out, I mean, it works. It works quite well. It does make it easier. I think it actually today, works easier enough in serverless world where you don't even have to think about anything, you just sort of just do it in line. But I think the part you’re poking at, which is super interesting and asking about is, I actually think some of the ways we interact with code are going to evolve. Because the AI is going to do so much of it for you. Or it's going to do assembling some of these things, putting them in place, telling you what – almost anticipating your next step, and making it a lot more seamless is going to be a lot easier.

In the end, once you have an end to end deployment pipeline, the important part is checking in the code. So, if you can get much faster checking in the code, you are making the pipelines much faster. Now, as you saw the things we talked about earlier on testing and reviews, et cetera come in, as those also get incorporated, you’re again finding the next place to make the automation that much better. So, I think those are the areas where you will see the most impact. But right now, I think the biggest impact is you've already invested in automation and pipelines, the fact that you're checking in code much faster, and at a much higher throughput is the part that's going to make a significant impact. We see that, right?

A good example is we have this customer – there's this customer called Persistent System. They’re a service services company, and they're made CodeWhisperer available to, I think, over 16,000 developers go off. Because the general idea is that's how you're working on code and checking it in. So, that's a great example of broad adoption, and the impact it can have.

[00:28:39] JMC: Oh, yes. In fact, I mean, this is a claim that other previous CEOs made in an interview, but definitely your website makes that is you've apparently run an internal survey test, benchmark, bakeoff, in which you claim that 50 – the teams using CodeWhisper will increase, at least, in this test productivity in 57%. If I'm remembering the figure correctly, it's a quite bold claim, and I'm intrigued.

We were discussing before the recording that there's no one size fits all for measuring productivity. But you were hinting right now to a few items like measuring throughput, and then commits, and check ins, and review time cycle, all those things. So, I guess does AWS for your own internal teams has any kind of framework to measure not only productivity, by the way. You are also hinting in this way, in this direction before. Also, in terms of developer experience, and making the experience something pleasurable and reducing attrition and all these things, does AWS have a framework for that? So, in what way have you come up with this astonishing figure?

[00:30:02] DC: YES. We do have a framework and it's evolved quite a bit since those measurements were taken in some ways, because we have a much better insight into how people are using. The interface, the way they're using it has evolved as well. I think a great example of this, and I don't have a number to share. But at the New York Summit, we added CodeWhisperer to Blue Notebooks. So, these are writing ETL pipelines using CodeWhisperer. It's a very simple constrained use case, which by the way, it's similar to your declarative kind of example.

[00:30:36] JMC: But that was a month ago, right? Because the meeting in New York was –

[00:30:40] DC: Yes, just a little three weeks ago. Time flies these days. Three weeks, four weeks ago. 

[00:30:44] JMC: Yes. But it was yesterday.

[00:30:45] DC: Yes. The rate of adoption of using CodeWhisperer for these pipelines is kind of astonishing. I'm waiting for us to talk about it, like the numbers in more quantitative terms. But I'm also not surprised because that's the kind of problem that's really well served by something like CodeWhisperer.

But going back to your original question, the specific numbers we shared, whether CodeWhisperer was helping developers complete tasks at an average of 57% faster. And developers who were using CodeWhisperer were more likely to complete a coding task successfully than those who did it. It was, basically, we wrote before, right? Around the time we launched, we took a bunch of builders and gave them a set of tasks, and we measured, and we did some measurements on those.

The way to think about developing a task faster is not just about how much code you can write, how fast you're typing, and how fast, how much CodeWhisperer was completing for you. If you're writing anything in the kinds of application that people write, you spend your time looking at documentation, seeing what an API does, what its methods do, what the responses are. CodeWhisperer, in particular, in this case, the tasks were AWS friendly, is very good at AWS API. The classic example I like using is, create an S3 bucket, which has X, Y, Z characteristics, or you're putting an object in it. The thing I always ask CodeWhisperer to write when I first time I ever tested it was write a function for me that increments account every time I add an object to an S3 bucket. Something like that.

Things like that get much easier and you don't have to look up any documentation, and the time adds up. So, that's where you get these tasks average. It's not just a time on keyboard, it’s wall time. The wall time includes things like looking at documentation, and coding companions make that process a lot, lot, much, much, much more clean and much faster. You're also less likely to get frustrated and leave, which is what the other metrics are.

[00:32:53] JMC: Exactly more delightful.

[00:32:58] DC: Yes. I can’t find the documentation I'm looking for. I can't figure it out here. It does – 

[00:33:04] JMC: Yes, because I mean, I can see that in an era, in the current era in which interest rates are really high, requiring justifying budget and OpEx increases for your team. For your business unit, it’s way tougher than it was two years ago. So, and at the same time, the goals of the company, especially in our sector in software are incredibly demanding. Those have not gone down. They were already high. That puts a lot of pressure on developers. Attrition is a concern is a concern, right? Developers make good money. I think it's in general, a well-paid function. But yeah, attrition, I think, has grown and making software delivery, the software engineering, the software experience, much less frustrated and even delightful in discovering new things about. It is the most positive thing about Gen AI for software delivery, I think for app delivery, don't you think?

[00:34:07] DC: Yes. I look, in the end with Gen AI making things delightful, is one of the core goal. Making organizations more productive. Of course, the end goal we all have at AWS. We want you to be much more productive with – we've always liked to make our customers more agile. This is the reason why people started using EC2 back in the day and started using the cloud was agility, more than anything else. The definition and our expectation of agility are only going to change over time. We are where we are. We've done a lot, and now we have this new – that's why it excites me so much, is I think you can get that step change in agility and delight in ways that we were – I like making – I ran containers for a long time, but also like making fun of the container world, which is I think we added a lot of complexity into the process and made it less delightful. Even though we were able to a couple more, because of automation that we could build.

But I think it did add a lot of, I don't call it pain, but a lot more overhead to getting things done. Now, we have this Gen AI thing, which is already shown like sort of in its infancy how much fun it can be, but also how much it can make your life easier. And getting the flywheel rolling and seeing, which is where we are right now, our customers are telling us what they want next. So, we are cranking on that flywheel, and you will start seeing some of the work that's come out in the space in general, things like Bedrock, Bedrock agents, CodeWhisperer and the enhancements that you've seen there, are the early signs of how that flywheel is starting to move.

In the end, our goal is very simple. We want you to do a lot more than you are doing today, without having to invest in necessarily, pools of – it takes work to get that much more effective. Generally, it makes you very effective and makes your best people. Best people more effective as well, because they're spending their time on the highest value things. I think that's the part that gets us all super excited.

[00:36:16] JMC: Yes. It’s going to reduce cognitive load. It has already done it. Complexity and for cloud native, which I think is more complex, actually, you just said, but also the way to go. I mean, depends if you're running a system that only requires absolute throughput, then keep it in a mainframe if you wish and isolated and that's okay. But if you're a customer driven organization that needs to move fast in a highly competitive world, you’re most likely would like to have a systems oriented, or microservices architecture that is cloud native in a way.

But in that sense, like CodeWhisperer and others are just making it well, more delightful. I guess, what should we – you’ve already just mentioned a few products. But yes, what should we expect from AWS in the sense? So, you've described the present now. It's limited, exciting. Being iterated with your clients as you just described a few of them quite faster as of now. But what in the near future is, I mean, no scoops, no necessary, no announcements. But in general, what should we expect from AWS in this specific area?

[00:37:32] DC: Yes. I will go back a little bit and give a sort of a slightly broader picture, because I think it helps. In general, we look at AI in layers of the stack as it were, and Generative AI in particular. You Know, your lowest layer is things like intervention and training, the Silicon, because being able to do a lot of training and a lot of influence at a lower cost more efficiently is going to be actually more really, really important. The hardest part about LLMs in general is that it takes a lot of resources to get them trained, takes a lot of resources to do infer it. 

So, there's a lot that we are doing to optimize the lowest layer of the stack. AWS is really good at it. We've learned a lot with Graviton and some of the other things that we've done, with our network, with the superclusters that we're building. So, I think that's one part. The second part is what I would call the stack that the services that are used to build AI applications. My org is a customer of all the services. So, these are things like Bedrock. Things like Bedrock agents that allow application developers to use AI, to LLM.

In fact, in many ways, one of the beauties of Generative AI, especially deep learning was very data centric. Useful for data scientists, but not necessarily as much for app builders, and Generative AI is sure data is always critical, and it allows you to do more with your data. But in the end, Generative AI makes AI something for builders, which is what makes me excited, which is that third layer. Having all these tools underneath you, having the Bedrocks of the world, having the Bedrock agents of the world, what can we start building? How can we fine tune these models for specific applications? How can we do rag to make sure that we are getting the latest and greatest APIs? That allows us to build high level systems like CodeWhisperer, like many others that you can imagine the kinds of things that we're working on. I also talked about sort of low code, no code application building. The fun part about generative AI that's not just X is multimodal.

There's a lot of interesting applications. You've probably seen some other ones in the creative side already. So, I think the future that lies is, as all these layers of the stack evolve, as the Silicon gets better, as these middle services to build AI systems get better, you have access to more models, the ability to fine tune them, methodology to sort of do very, very custom things, then you can start giving more value on top. A lot of our efforts are around what have we learned with CodeWhisperer around what people want to do with code, the feedback they're getting. But code is only one part of this. Whole developer experience challenge.

For example, we also brought in the CodeCatalyst team and the Amplify team into this organization. We have lots of ideas around how to enable end to end delivery and web application developer. So, I think that's making the most up here.

[00:40:40] JMC: Yes. I was thinking right now the only thing that maybe CodeWhisperer, or any other code, AI assistant for software delivery, is not well designed for is helping with prompt generation. I mean, are they able to – so, are they able to tell you, you're asking me something the wrong way. This is the way it should be structuring your – well, maybe have a good companion for prompt generation that helps you with a prompt generation, and then CodeWhisperer, it receives the input and generates the output. I'm not sure. I could be completely wrong.

[00:41:22] DC: Yes, this is personal opinion that I'm going to say now, which is, I think there's a lot of people who asked me the question, are software engineers – is front engineering a new discipline? I will say the answer is no. I think it is a discipline that software engineers will become good at, because it'll be – it's just another tool in their toolkit is how to write prompts. They will teach themselves. I've seen my – the teams in my organization, the engineers we have become really, really good at prompt engineering. I will also add that I think in the end, for the average developer, it may not matter. Because if you're really good at natural – if you're able to express yourself in natural language, the systems, this middle tier will be able to translate that into prompts and convert that into meaningful things.

Sure, there'll be a small class of people will want to go in and edit prompts and be prompt ninjas. That's what I think what will end up being in our end. But for the average developer, I don't think it will be something that need to think about in the fullness of time. In the near term, it is a skill that a lot developers are going to pick up because they are the ones building the high level application. But if you're working a little higher up the stack, you're probably talking in natural language, you're probably clicking and dragging, and the prompts are underneath, and how good you are. If you're really good at converting natural language into really high-quality prompt, that's actually part of the problem that you're trying to solve for your customers.

So, I think the way we look at it is taking the customer view is what can we do to make customers lives easier? And if the answer, and this where the personal opinion comes in. A potential answer is they don't need to become experts or prompt engineers, and we become on their behalf because the smaller number of people can then make them a larger number of people more productive. I think that's something that I personally believe, and I think there's a reason and there's enough data from talking to a customer where I think that has some play.

[00:43:17] JMC: Well, Deepak, that has been really interesting. I presume, I can only assume that if reinvent later this year, and the end of this year, somewhere around, it's always in December, correct?

[00:43:30] DC: It's always the week after Thanksgiving.

[00:43:33] JMC: Exactly, exactly. I'm not American. I always forget that that's a cornerstone of American lifecycle in general –

[00:43:43] DC: As far as I understand it.

[00:43:47] JMC: That's a good way of thinking. I need to have a good Thanksgiving menu once in my life. And I will never forget when it happens and what it's all about, because food anchors everything in one's memory better than any other thing. Maybe love is the only other thing that sticks with more strength to our memories.

So, I presume there's going to be a lot from your team in that event and maybe trickling out before and after. I can only just be quite excited about hearing about those things in a few months. Yes, other than that, I would like to thank you for your time here and wish you the best because, as you said, you're in a very critical point in time, in a good way. I think AWS is definitely well placed for many, many reasons that you explained to deliver a solid developer experience, actual in general.

[00:44:52] DC: Yes. It's super exciting. I said, watch this space. NBS said that every team at Amazon, almost every now is on I was doing something with Generative AI. There's a ton going on. I haven't been this excited in years and I don’t tend to get excited about tech in general. So, it's a great time to be around. It’s such fundamental, foundational game changing tech, and what makes me excited is getting it into the hands of all our customers, and in the coming weeks and months you'll see it done. I'd love to get feedback from them. Thank you for your time. I enjoyed this too.

[00:45:31] JMC: My pleasure. Looking forward to those. Take care.

[00:45:34] DC: Ciao.

[END]