EPISODE 1604

[INTRODUCTION]

[0:00:00] ANNOUNCER: This episode of Software Engineering Daily is part of our on-site coverage of KubeCon 2023, which took place from November 6th through the 9th in Chicago. In today's interview, host Jordi Mon Companys speaks with David DeSanto, who is the Chief Product Officer at GitLab.

This episode of Software Engineering Daily is hosted by Jordi Mon Companys. Check the show notes for more information on Jordi's work and where to find him.

[INTERVIEW]

[0:00:37] JMC: Hi David. Welcome to Software Engineering Daily.

[0:00:39] DD: Thank you for having me.

[0:00:42] JMC: You're the CPO of GitLab. Most of your experience is in security, right? Now you are at the helm of a generalist DevOps, DevSecOps platform, correct?

[0:00:53] DD: Correct.

[0:00:54] JMC: Had that changed your views on product, or your career in general?

[0:00:59] DD: One thing about my past, I was a software developer. I went into security and then eventually, into product management. I had a big appreciation for the developer experience. I also was a former GitLab customer, and my former employer still is a GitLab customer, but I was very familiar with GitLab. GitLab actually reached on 2019. They wanted to add security and compliance to DevOps. I, through the course of talking to who was then CPO and Sid, the Co-Founder of CEO and the team, I realized that you can't truly shift security left, unless you do it as part of DevOps.

That, we said we were shifting security left at my former employer, but we were just giving a security tool a plug-in into CI. It made it a fun challenge to help GitLab truly do a shift left with security and build in compliance controls and all the other things enterprises need today. I look at GitLab's growth as a company, going from what was a developer platform to a DevOps platform, to now a DevSecOps platform and recognizing in the industry as a security player.

[0:02:08] JMC: Yeah, I agree. The fact that it's one platform for everyone in a way, it does - I mean, I'm going to phrase it as it forces all the stakeholders to be in one place and it doesn't force anything. But that's constrain of having all source code, all CI, all CD pipelines, whatever might be managed with GitLab in one place, it does also foster collaboration, and particularly with security people, right? They have to be there to apply policies, or restrictions, or scanners and so forth. I guess, that actually, that constraint might play well into the DevOps philosophy of collaboration across the board.

[0:02:51] DD: Yeah. I look at it through the eyes of our customers. A couple of years ago, when UBS became GitLab customer, they did a use case study about their transformation at their company, and they actually referred to what you're referring to as the taking down of walls, they didn't realize existed in the company. Now, you had a single source of truth within GitLab that was where they were with development, where they were with security, where they were with ops. It fostered collaboration inside the company, because now people are working side by side and not in their silos.

They're just one example. A lot of customers have talked about how their efficiency has gone up. There was a great study that Forrester did a couple of years ago on what is GitLab ultimate and what does it mean from an efficiency standpoint. After they did their interviews of customers, they came back to a 7X increase in efficiency for the company, because you don't have your silos anymore.

I think, if you look at GitLab and GitLab's journey as a company and as a platform, what you're seeing is we were able to get developers that are cloud-rate developers and they're platform engineers, or CI/CD team members. Now it's been also getting that security team members in there as well.

[0:04:05] JMC: We are in Chicago, Illinois, the US 2023 at an event called KubeCon North America. For everyone that doesn't know, this is the cloud native computing foundation's flagship event in the US. When I was here in the previous edition in Detroit 2022, the conversation was mostly about, I would say, EBPF, DevSecOps of course, but now it's taken over by AI. I know for a fact that GitLab has also reacted to this trend. In fact, Sid the CEO mentioned it in a investor's call recently, whenever GitLab, since it's a public company, reports publicly about its numbers. Before we dive into what Sid actually mentioned, can you tell us about what is the GitLab's general impression approach to AI?

[0:04:58] DD: Yeah. We decided to start adding AI into GitLab and the AI-powered workflows within GitLab are known as GitLab Duo. I think that's you're referring to the earnings call where Sid talked about GitLab Duo. We said that we had to define core tenants, or principles as to how we would do that. What a lot of people may not know about GitLab is GitLab is a very transparent company. We're the most transparent publicly traded company in the world. We're also someone who worries about privacy.

When we started looking at how we would do AI, we first said, GitLab covers the entire software development lifecycle. AI should enable everyone in the software development lifecycle, not just a group of team members. You hear a lot about how AI is helping developers out. We had our recent DevSecOps survey that touched on AI. 25% of the SDLC is actually writing code. If you look at that, and you sorted the one part, it's not helping the entire organization. That first tenant was the entire SDLC can sort everyone with AI.

The second one was being privacy and transparency first. GitLab is trusted by more than 50% of the Fortune 100 to help them secure their intellectual property, their source code. The privacy component is, we did not want to train, or fine tune our models using our customer's code. That is their intellectual property. Then the transparency comes in that we want to tell you exactly what we're doing with AI. GitLab is an open core company, we're source available. Also in our docs, we actually list out what models we're using for what features, how they were trained. That way, you get that trust build up, because you can actually see what's happening.

That leads us into the final tenant, or principle. That was best-in-class AI. What that's really about is selecting the right AI model for that use case. Today, GitLab uses around 16 different AI models to power GitLab Duo, because we've selected the right model for the right use case. A great example of that is, our explain is vulnerability feature. We announced at RSA, USA 2023, so back in April of this year. That feature uses a model that's only trained on vulnerability data. It's not going to be able to tell you about how to bake good chocolate chip cookies, or refactor a bunch of code into a different language, or whatever you'd want to ask it. It can answer security questions. That's how we've approached it. I think that's what's a lot of GitLab Duo to continue to grow and help our customers.

[0:07:28] JMC: I might sound like, a fan here, but I couldn't agree more. I couldn't find it more right, correct to this approach. Mostly, not because I say so. But I'm seeing companies that have properly approached, tackled AI in their SDLC. Another place is faster before GitLab did so. I'm thinking of, for example, OpenAI, right? Even GitHub, right? How they are reacting in day two to the consequences of the decisions that they made at the beginning. That relates to, for example, one thing that you mentioned about being very cautious about not training your models with your client's data, right?

Everyone in GitHub was like, "Oh, okay. So, you're using my code to train what someone else, potentially competitor, potentially who knows what, who, the output of the LLM that you have trained with my data is going to be used by someone." I find that approach really, because I find that concern really legitimate. I find the approach that you mentioned quite reasonable, to say the least. Also, the legal consequences of this, right? I don't know that if GitHub in particular has thought - I mean, there are plenty of legal challenges about that. There will be legal outcomes from this that pertain to, and we're not legal experts, nor you or me.

[0:08:44] DD: Nor can I comment on open litigation for a competitor.

[0:08:47] JMC: No worries at all. But, no, no. This will have consequences on fair trade of fair use, apologies and stuff like that. I think that the approach that you mentioned is quite not only cautious, but correct. Also, I appreciate the transparency. I think, this also gets validated, not necessarily the transparency, but the training of models with specific data sets with what, for example, OpenAI announced this very week that they will allow you to use private GPTs, right? I think that's clear.

The data out there says that the output of an LLM is severely, if not completely conditioned by the training data, so why not do it this way? Which actually segues very well with what I mentioned in the beginning of the question, or the previous question is like, Sid mentioned 13, or several use cases. Are you trying to provide a LLM, or training data set for those LLMs that you mentioned, or those models that you mentioned for each one of these specific use cases?

[0:09:46] DD: Yes, we are. The intent was to pick a model, whether it's a model we create ourselves, an open-source model, or it's a commercial model, because we have commercial partnership that meet that use case. The best two examples I can give to you of the power of this is that one, our suggested reviewers feature, it recommends the right code viewer based off the code commits that are in that merge request. It is a small model that we can run an inference for each customer. That way when we're using their data, their commit history to be able to recommend a reviewer, that stays within them and we can't see that.

The other one is code suggestions. Our AI-powered code completion solution. It is using a model for code completion. That's where the cursor is in the middle of the file and it can make recommendations in the next several lines, basically, what it sees. It's separate for code generation, where you're actually writing a comment and you're getting code generated. We're routing to the right model that is better for each of those use cases. You can actually do that the whole way across, all of those features. Yeah, I think at the earnings call, we talk about 13, there's now 14. It depends on whether you're looking at them from the marketing point, where we talk about them as a use case, or the actual feature name and to get web docs.

Yeah, it's everything across the entire software development lifecycle. I think that's where GitLab is taking a slightly different approach. We realize that as a DevSecOps platform, and you're right on the privacy part, we're also an enterprise DevSecOps platform and we're trusted by more than 50% of the Fortune 100. We need to be cautious and careful with customer data. That's just giving us a different approach. That's helping them get through planning, helping to get through code creation, getting through code review, through security audits, getting to code being deployed, and then being able to provide feedback back to them on application usage.

What I would say is that we focus very much on where customers ask us to focus first. It's not as well talked about, but again, we've actually started adding AI in 2021 to our platform. We started with what our customer's pain point was, which is getting through code review. Then we worked our way into security and then into code creation. What I find fascinating about that is when you look at trends and a good example of the trend would be, and that our AI DevSecOps survey, AI was not - top reason people use AI was for getting through code review and generating unit tests. Then it was creating code. I think that, combined with our approach, combined with those principles, or tenets, you can see GitLab's vision for how AI will help software development.

[0:12:18] JMC: Okay, I agree. There's extremely reasonable again. Wait a minute, LLMs are basically guessing, let me play a bit the devil's advocate here, guessing the next word, right? They understand the underlying network of words that compose the human language. Not only words, but syntax and they map it in a vectorial way, and so forth. I find that has a direct application on code suggestion and code generation. I think that's taking all the attention, because there are figures out there that they can create 30% of the code. I do agree with you that you will see, well, an impact on the 25% of the code that is created in the software delivery lifecycle.

My question is, how are LLMs then helping the rest of the activities that happen in the SDLC, like you mentioned? Because if the purpose of LLMs is to discover the next right word, how can this help find code reviewers, or summarize merge requests, or etc., or issues, or whatever?

[0:13:17] DD: It's an interesting question. I think you have to look at it from each of the use cases. In the case of code reviewing and find the right code reviewer, that model was built by GitLab. It was built based off the knowledge you need to have to be able to map that out and then make the right recommendation. In the case of something like our vulnerability, explain this vulnerability, and then working towards futures with that. That is trained only on vulnerability data, so it's able to look at a CVE ID in a line of code. Then to your point, looking at the model is able to start to then extrapolate, well, this is vulnerability is about X. Here are examples of that X.

I think that's where not trying to use a single LLM for everything is really helping out. It cuts down on hallucinations, or confabulations, because the model, to your point, can be trained on headlights of data, or it can be trained on data just for a specific use case. I think that's that nuance there.

[0:14:15] JMC: Looking forward a bit, are there any specific areas in which AI is going to be applied even more so than what you described, or areas that hasn't been applied yet to, and that you plan to incorporate?

[0:14:27] DD: Yeah. As I mentioned earlier this year, we launched explain this vulnerability. We recently renamed it to vulnerability summary. Our intent is to extend that to being able to resolve a vulnerability on behalf of the developer. That's getting into taking what we've been doing with refactoring code, comparing it back to the vulnerability data and being able to propose a resolution that the developer can accept.

Today, GitLab can do auto remediation of vulnerabilities for software composition analysis. That's dependency scanning, or container scanning. This is taking it to the next level, where actually in-line in the code, we can go in and refact that code. What I'll tell you is that the nice thing about that is GitLab is a DevSecOps platform. After we refactor that code, the same security scanners can run again and tell you that now there's a vulnerability resolved, but a new vulnerability wasn't introduced. It's not just taking the AI at its word. There's the right way to put that.

Then using tried and true techniques to validate it. What we'll say is outside of that, to your question about futures, one area that we've started looking at is how to apply AI as part of CI/CD. We've worked on things like generating templates for deploying into Kubernetes, we're at KubeCon after all. Or deploying into Terraform, deploying into hyper-scalers. GitLab is cloud agnostic. We're actually available on the top three hyper-scalers today. We work in all three environments efficiently. Working on how you connect your app there.

We have bigger ambitions of being able to auto resolve pipeline errors, being able to find optimizations in deploying the application, monitoring the application, and making configuration recommendations. What I would say to you is we very much have been focused on DevSec when it comes to AI today, and we want to get that ops part to be just as good as the dev and the sec part.

[0:16:13] JMC: One thing that is also quite popular lately and that I know for a fact that GitLab has a feature supporting, but I'm not sure if AI has been infused there is the ability to spin up development environments that are cloud-based mostly, but that allow for collaboration and even well, maybe pair programming. What's your opinion on that side of the SDLC and what's GitLab's current state and vision on that?

[0:16:39] DD: Yeah. I'll tell you, if there was something that was outside of AI and security that has been a passion of mine, it's probably the remote development offering we have. The main reason why is that a lot of people think about remote development, whether that is the environments themselves, or the developers experiencing that around onboarding. Onboarding is important. We have customers who have talked about going from weeks for a developer to commit lines of code to minutes, because they're now using remote development. For me, it's about the security and the compliance part of it.

When you think about developers, they're cloning projects to their laptop, they're creating branches, they're mirroring branches, they're committing code, they're checking out code. All that is sitting on their laptop, or on their computer and think about the pandemic. How many people had to work from home that were not working from home? Did they have the tools and the environment to do that safely, or where they now just using their work computer that was on the corporate network at home in an unsecure area?

I look at it as like, how you now protect your intellectual property while not impacting the developer. For those who are not familiar, remote development, its concept is that the developer's IDE is actually mirroring an environment that is not on their laptop. I compare it to terminal services, or Windows desktop back years ago. It's that, but within their IDE. Now, they're working in say, VS Code, or JetBrains. It feels local to them, but they're actually, it's running on a container in a Kubernetes cluster that the company manages.

Now, that laptop can all be thrown away and not lose intellectual property. Yes, there's a boost in developer onboarding. Again, one of our FinTech customers talk about when they bought GitLab, it was taking four weeks from the time a developer joined a project to their first code commit, to now, that's being done the first day. They're able to make a code commit, because they joined the project, they opened up their terminal, they point out the remote development environment. It is ready to go. It's a golden image, and they don't have to set up boot - they don't bootstrap the environment, they don't have to figure out their dependencies. It's just ready to go and they can start coding. Imagine that now from that security standpoint, and that's another way it might be the dev part of DevSecOps, but it's also the sec part of DevSecOps.

[0:18:57] JMC: I think it's a beautiful example of good developer experience that not only does not trump, but actually enhances security, right? There's actually, in my view of the world, at least, marries very well with GitOps, because if you add a complete declarative version of the system, or the application, you don't need the system in its entirety, or in a Git repo, in a GitLab repo, then, I mean, I think the combination of those two things is incredibly, incredibly powerful, because you literally just have a data representation of the application in one single place, and you just need to clone it, copy it, and pull it up in the remote environment that you've created. Do you agree with that?

[0:19:39] DD: I do. As you're describing that, I was thinking very much how GitLab's dev file, which defines that developer's remote development environment. Mind you, there's a single file for the entire project, so everyone's getting the exact same image, or an image built for them, based off of that. Is very much like a manifest file that you'd use within Kubernetes, or a Terraform deploy script. It's defining what type of container it is, what are the packages you need to go in there, what are the services, how do they run. It looks a lot the same. I didn't really thought about it from that angle, but there's a lot of overlap.

[0:20:14] JMC: Going back to security, what are the features does GitLab consider fundamental for the DevSecOps lifecycle, and in which one's do you also plan to infuse AI, or make it AI-powered?

[0:20:27] DD: Yeah, great question. Today, GitLab security portfolio, for lack of a better way to put it, we're a single application, single platform, but those components are called secure and govern. Secure is application security scanning. Today, we provide SaaS, secure detection desk, container scanning, dependency scanning, license compliance, fuzz testing, and API security. I feel like, I'm missing one or two. But the entire suite of what you would need from a scanning standpoint. Then govern very much about the compliance, governance and visibility, and that gets into things like setting global, immutable security execution policies that, when certain things happen, a security scan has to be run.

Or if a result comes back and there's a finding, does the security policy fail the build? Does it alert the security team? Does it put up a gate that now needs approval, but the pipeline was successful, but can't be deployed? To us, that is the bringing of software supply chain security to the forefront of how people build software.

Now, to your question about AI, we added that AI feature explaining this vulnerability on top of our vulnerability management component of GitLab, and so developers can click a button and learn a natural language what the vulnerability is. An example of that being exploited in their programming language, an example of how to resolve it. We already talked about how we wanted to extend that to resolving it, but I think that's only just part of it. Part of our code suggestions capability is looking at the project as a whole, not just the source code, or the commits, but pipeline information, deployment information, analytics data.

If we can take that and build what I'll call an X-ray of the project, that is becoming part of code suggestions, but we can use that same thing to improve security scanning. The best way to think about it is SaaS is a necessary UL. You need to run it, but it has a lot of false positives. But what if we could use the knowledge of the project to say, that's a false positive, you don't need to worry about it?

I see AI coming in to optimize scanning and scanning results. I see it coming in to identify anomalous behavior that may look normal at the top, but based off user data and user activity to say, "Hey, I think Jordi's account is compromised and alerts someone." We see it as the same way it's an AI pair programmer for the developer, it's a security analyst counterpart to the security team.

[0:22:53] JMC: We might get called out by the bingo card people that check the buzzword.

[0:23:00] DD: I think we have to say, synergy and a couple of other words to get there.

[0:23:04] JMC: Well, I'm going to mention s-bombs. Exactly. I mean, but mostly because I'm really interested. I used to work for the Linux Foundation in the SPDX project, and Jim Zemlin today mentioned that the growth of that project and in general adoption of S-bombs is critical and it's growing. I've got different opinions about that, but I do know for a fact that I think, correct me if I'm wrong, that GitLab has embraced actually CycloneDX mostly. In general, GitLab is providing S-bombs functionality. Could you elaborate on that?

[0:23:34] DD: Yeah, correct. Yeah. We initially embraced CycloneDX, we now support Salsa. But our focus has been on helping people understand what's in their software. If you think about GitLab's journey, we started as a developer platform, we came to DevOps platform, our DevSecOps platform. That sec has to be about software build materials and software supply chain security. How I describe people our vision for security and compliance, is GitLab is bringing software supply chain security to the forefront of how you build software. That means, we have to be able to do S-bombs, like you're saying. I'm thinking of other words for the bingo card now, just so we can make sure everyone's happy.

It's also not just security scanning, not the controls, it's to your point, an S-bomb build, artifacts that are signed. The chain of custody of an artifact to know it's still the original artifact and who's actually touched it.

[0:24:23] JMC: Time put.

[0:24:24] DD: All of those things, right? Binary authorization gates. The things that are needed so you know the thing that you have is the actual thing that it's supposed to be. I always think back to the NPM vulnerabilities the last couple of years, where people were grabbing NPM packages that looked like they were signed, they looked valid, but they had been manipulated.

We're always asking ourselves, how do you prevent that from happening as part of GitLab and how do you make sure that you are generating artifacts that can be shared safely, securely and be unmolested.

[0:24:53] JMC: Since GitLab is again, a DevSecOps platform, it does provide most of the functionality, if not all that supports that. Talking about, you mentioned just NPM, what package/container image, repositories does GitLab actually incorporate and does it add any trust element, any secure element on top of those?

[0:25:16] DD: Yeah, so we have our own built-in package container in Helm registries. As you're building artifacts, you can store them in those. The nice thing is that when you bring the security scanners into it, we can also run continuous scanning against those in the registry. What that allows us to do is when we publish a new vulnerability to our vulnerability database, we publish updates a couple times a week. If there's now a known vulnerability, we can alert you that something in your registry now has a vulnerability.

What's nice about that is it extends not just to what is actually in the registry, but what's been deployed from the registry. If you have a container that now has a vulnerability in it, that was a newly found vulnerability in the industry, a research team found it, that container originally two years ago at GitLab, you'd have to run another CI pipeline, so the security scanners run. They now run agnostic of that and we're able to alert you and say, "Hey, you've got X amount of containers running in operations that have vulnerabilities in them," and get that chain of custody, that tracking that you'd be able to resolve it.

I think the way I've always thought about it is, and maybe it's an interesting way to think about it as a former security practitioner, researcher, eventually went to the dark side, joined product, but is that everything has to be about security. When you think about it from that point of view, then you realize, one, this will fire up the bingo cards. But developers don't wake up in the morning and say, "I want to write a zero-day vulnerability in my application," right? They want to write secure code. Their security is about enabling them to be better developers.

Obviously, the security teams compliance teams are obvious, but you can say the same thing with operations teams. The one feature in GitLab Duo that I was surprised who adopted it was explain this code. That feature when we first launched it, I really thought it was for developers. But security team members, operations team members, people who don't write code for a living, but now need to test and securely deploy and operate can now understand that code in a way that is different than what they would have had without that knowledge.

If you look at that again, security has to be across the entire SDLC, just like we've talked about AI earlier, because you want to know that you're deploying applications safely and securely and no one in the team, even though sometimes you'll think there's an affairs person in your team, because they're just silly, or the person that always picks on people, they still want to write good code. They still want to operate a secure company. They still want to deploy and operate. 

Everyone wakes up in the middle of the night worried about that vulnerability. It doesn't matter who it is. I think that's the important part when you tie in our conversations so far, right? The KubeCon, agree, which by the way, I could ask you as a former member of the company, but is it KubCon? Is it KubeCon? Because I hear it say both ways.

[0:28:06] JMC: Wait, wait. Can you pronounce it again?

[0:28:09] DD: It's funny. I say KubCon, but some people say KubeCon.

[0:28:12] JMC: I would go for the second one, yeah. I think the actual complete full name is cloud native con/KubeCon. But I would go for KubeCon.

[0:28:22] JMC: KubeCon, like an ice cube with KubeCon?

[0:28:24] JMC: Yeah, yeah, yeah.

[0:28:25] DD: It's interesting. I ask only just because we're talking about it, and I literally can't walk through the showroom floor expo hall without someone saying it one way or the other way. Anyway, we're at this event, right? It's about the operators and the platform engineering teams, but they're still also worried about security, right? You're right. Amsterdam and Detroit last year, they feel like security conferences. Today, this week, it feels very much like an AI conference, but security is still that underlying concern that everyone has.

[0:28:56] JMC: Yeah, I think you're right. I would also would have been surprised by the user persona of that feature being eventually, I think, you said security and operators, instead of developers, who you might have designed for initially. I think it's AI for software delivery, for software engineering, for software generation is such an infant stage that all preconceived ideas can just change radically. Because I've been working with a few companies, and some of the assumptions was that gen AI for co-generation is going to mostly help junior devs onboard a new project faster. But the data eventually turned out to be true initially, and then eventually, not so too true, because senior devs would take over the requests, in this case, it was a chat interface to the LLM, to the inference model, and actually, use it way more than juniors, which felt eventually, which reported that they feel confident asking as many questions as possible, because the LLM does not run out of patience.

But didn't feel secure of accepting the code suggestions, because, well, they are junior. They are fresh out of college, and they don't know. Again, I think that we're all traversing new places, and making assumptions in this world is actually quite difficult, honestly.

[0:30:17] DD: No, it's interesting. One of the things I love about data is that it can be both informative and frustrating at the same time. What I'm seeing as a trend is each company that's applying AI as part of software development is coming back with different stats. Something I was sharing with team members earlier today is that several of our customers are talking about their most experienced developers are getting a better efficiency boost from AI, than their, say, their counterparts that are fresh out of college, as you said. Say, three, six months into their career, and they have less of a boost.

But if you then look at stats from other companies, they may say it's the other way. I think what is telling us is one, generative AI and use of AI and software development is still so nascent that it really depends on the company and the type of companies you work with. We have primarily enterprise customers, heavily regulated customers and government, FinTech, health care, and so forth. Maybe our customer base has a different experience than someone who's supporting maybe e-commerce development, or SMB companies, or whatever the case might be. It'll be very interesting to see over the next year, the two years as general AI and the SDLC matures, what it actually hands out to be.

[0:31:31] JMC: Do you have thoughts on the UX of and the consumption of AI, LLM output suggestions for the end user, right? Because co-completion has been around for a long while. It was initially, I think, statistically driven since code is pretty repetitive and structured and in a way, impotent. It was mostly driven through comments, through /commands and stuff like that. Now, chat has revolutionized this in a way. It changed it dramatically. I don't know if GitLab in general has opinions about when is a chat interface most useful, when is a AI infused via commands more useful, etc., etc. In general, your opinions, your views on the UX. I know you're not designing a per se, but if you have opinions on that.

[0:32:17] DD: Yeah. I find myself being very close to UX. In my career, I started off as a graphic designer and doing the actual work. Actually, as part of my role at GitLab, I don't just have - so product is not just the PMs. It's also our monetization operations team and it's also UX for the company. I'm close to it personally from my life experiences, but also, we work very closely with our designers as part of product.

For us, what we've seen in customer conversations and research is that we're humans and we gravitate towards conversation. Because of that, we put a bigger investment over the last several months into GitLab Duo chat, and we're seeing that become the primary interface that people will gravitate towards. If we go back to the last question about the experience, the developer, and how they engage with AI, what we're seeing is that the more advanced developers want to use our code generation, which is done either via a comment, where you're running a comment and you start describing like, create me a function that does X, new line, like have that function also do Y, you start describing it and it builds it for you, you can also do that in chat.

We're seeing people go over and ask GitLab Duo chat, "Hey, I need to be able to do X, Y, and Z. Then okay, well, can you finally do this? They go back and forth in this conversation, they cut and paste and put it into their editor. Over the course of the next three to six months, I would say, as GitLab Duo chat moves into beta later this month, so our November release will be beta. As it moves towards GA, you're going to see more of those features we talked about in the beginning being available in chat.

The main reason for that is one, again, we're humans, but two, context is so important. If you can ask GitLab Duo chat about the issue that you're working on, so you're in planning like, "Hey, can you summarize the conversation? Can you give me my next tasks?" Then from there, you can start writing software and get guidance on the software development. Then you move into code review, and you just keep on having it follow you and it has that context, it becomes super powerful.

[0:34:20] JMC: The last feature I'd like to comment on is the value stream forecasting. Could you explain what that is?

[0:34:26] DD: Yeah. One of the powers you have as a GitLab user is that we can do value stream management very effectively. We have all the data about your company from planning, to coding, to securing, deploying, and so forth. We're able to leverage things like DORA for metrics and also, GitLab's specific metrics and report on how your teams are doing. It's honestly one of the features that now drive customers and organizations to GitLab Ultimate, because that's where our value stream analytics is the most powerful.

Earlier this year, we launched a feature called value streams dashboard, which gives you an organizational level of roll up of all of that data, and then you can draw down into it. In that process, we asked ourselves what's missing, and it was forecasting. If you go back to our conversation about tenants, we realized you could probably use an LLM and feed it a bunch of data and ask it. Or you could build a purposely driven model that understands the value stream data and ask it to forecast it.

The reason why I share that with you is it gets to a point where we actually were able to do that without needing GPUs and get very accurate reporting. What value stream forecasting does is that you can go into different metrics, and I'll use deployment frequency as an example. You can see your trends over time in the value stream report. Well, now you can toggle on forecasting and it'll start to predict the next three to six months for you.

To give you a great example of it in use, GitLab deploys a 127 times a month to gitlab.com. If you look at our value stream report, and you look at the last three, six, nine months, it's just a flat line. If you look at all of the data from GitLab from 2013 till today, you can see the trends of how we're improving our deployment frequency. It actually said that in the next six to nine months, we will be deploying it a 130 times a month.

Now, is that a big jump in deployment frequency? Maybe not. But could it also tell you where you need to hire and staff and make sure you're ready to go? It absolutely can, because the more you're deploying, the more SRE is important, the more platform engineering is important. For us, it's getting us that guidance and our customers are using it, or seeing that exact same thing. They're finding where trends are going down, and they need to start interceding earlier. Think about it as the, hey, it's not a problem today, but the trend, this is going to be a problem for you in six months. It allows you to make sure everyone's efficient and everyone can contribute effectively.

[0:36:51] JMC: What if that, and to conclude the interview with this in a jokingly fashion, what if that feature receives a merge request from McKinsey to incorporate their vision of, because you've mentioned DORA metrics, and there's a space framework, and other ways of measuring depth productivity? What if the GitLab team receives a merge request from McKinsey guys to incorporate their vision of DevSecOps? They didn't define a framework per se. I'm by the way, making a joke, in reference to the article that spurred a bit of controversy, I think, a couple of months ago. What would you do?

[0:37:26] DD: That's a great question. What I would say is that GitLab is an open core company. We have a very healthy community of contributors. We're trending from the hundreds towards thousands of contributors per release. What I would say is that we would review any merge request, even from McKinsey, and discuss the benefits and impact to what we're doing today. Some of my favorite features are actually community contributions. If they were making a valid contribution and it incorporated their metrics in a way that did not deteriorate from the other metrics, or take away from the other metrics that people rely on, sure.

What I would say is we have a really great developer relations team, who review community contributions with engineering, and I would trust them and the PM for that area of the product to make the right call.

[0:38:17] JMC: Well, thanks so much, David, for being with us. I look forward to experiencing and using all those features that you mentioned that are going to be released and the ones that are already there in GitLab Duo. Yeah, great to have you in the show.

[0:38:30] DD: Yeah. Thank you very much for having me. For everyone who listens to this podcast, GitLab Duo features are still free on gitlab.com, till they become GA. If you've not tried out GitLab Duo, go check it out. It's available on gitlab.com. If you're self-managed, available there, too. A little bit easier if you're gitlab.com and set up for yourself, just toggle. But check it out and Jordi, I love the conversation. Enjoyed our time together, and thank you for having me.

[0:38:55] JMC: I appreciate it.

[END]