EPISODE 1857
 
[INTRODUCTION]
 
[0:00:00] Announcer: Wiz is a cloud security platform that helps organizations identify and remediate risks across their cloud environments. The company's platform scans layers of the cloud stack, including virtual machines, containers, and serverless configurations, to detect vulnerabilities and misconfigurations in context. The Model Context Protocol, or MCP, is emerging as a potential standard for connecting LLM applications to external data sources and tools. It has rapidly gained traction across the industry, with broad backing from companies such as OpenAI, Microsoft, and Google. While the protocol offers great opportunities, it also introduces certain security risks. Rami McCarthy is a Principal Security Researcher at Wiz. He joins the podcast with Gregor Vand to talk about security research, AI, and secrets leakage, MCP security, supply chain attacks, career advice, and more. 
 
Gregor Vand is a CTO and founder, currently working at the intersection of communication, security, and AI, and is based in Singapore. His latest venture, Wyntik.ai, reimagines what email can be in the AI era. For more on Gregor, find him at vand.hk or on LinkedIn. 
 
[INTERVIEW]
 
[0:01:31] GV: Hello. Welcome to Software Engineering Daily. My guest today is Rami McCarthy from Wiz. Hi, Rami. Thanks for coming on. 
 
[0:01:40] RM: Thanks for having me. Excited to be here. 
 
[0:01:41] GV: Yeah, very excited to have you here. And as I sort of just mentioned, you're here effectively behalf of Wiz. You joined fairly recently, but you definitely have quite an illustrious past in the security world. A lot of research that I'm sure a lot of our audience will be familiar with, whether sort of directly or indirectly. I think it'd be kind of just fun if you could just pretty much take us through kind of your kind of journey from whether it's high school or college through to where you are at the moment. 
 
[0:02:14] RM: Awesome. Yeah, you mentioned I joined Wiz recently, and I really feel like that is a logical combination of a lot of what I've done in my career. I studied security in university academically. I feel like I came up at a really good time where those programs were starting to pop up in the US. Specifically, the NSA was funding a lot of universities to have really good cybersecurity programs. And I stumbled my way into security consulting and pen testing, which gave me the chance to see hundreds of different security programs, and also sort of sparked this pattern of research that has pulled forward into now being in a purely research role. 
 
In fact, at NCC group, where I was a consultant, Clint Gibler, who does tl;dr sec, was one of the research directors and was frankly the person who got me going down this route, and is someone I've had the joy to work with consistently since. After consulting, I moved in-house to a small health tech where I got to help build a security program sort of from not day zero, but from day one. And we built the program from 150 people at the company to 800 through a series D, that sort of startup journey. And by the time I left, the team was eight people in security. I was managing three people. I was responsible for everything from cloud security to security operations, to AppSec and beyond. 
 
And I moved from there to Figma, which had an incredibly strong team with a lot of really interesting engineering focus and was able to take what I had learned applying a whole program at a smaller company to really focus on infrastructure and cloud security at a company that was fairly high on the maturity curve, but obviously had a lot of scaling challenges and opportunities as those hyper growth startups tend to. 
 
This journey from consulting, into health tech, into Figma, the threat of research was tied all between there. And I ended up deciding to take a sabbatical after Figma and took a year off, and decided to dip my toe in what research work would look like. And so my sabbatical, very casual, very chill, involved publishing 35 blog posts, doing a few podcasts, advising for startups, really just dipping my hand in this idea of a portfolio of work across the industry. 
 
And I came out of it with conviction that I have a lot of energy for diving deep into problems in security, that when you're in a single team or company, you just don't have the bandwidth to look at. And so the industry in many ways is reliant, some of us being given the time for whatever reason to go a little deeper in these niches and, in my mind, come back with actionable guidance. 
 
And so I built on working in-house and security programs. And now at Wiz, where I'm able to sort of take that practitioner perspective and research experience to ideally work not just for Wiz the product or Wiz customers, who are half of the Fortune 500 or Fortune 100, this broad pool of the industry, but to write publicly, to work publicly, and to hopefully help folks who have nothing to do with Wiz as well. 
 
[0:05:32] GV: Yeah, I think that's a really good kind of call out, where exactly you're able to have this reach, I guess, through Wiz. And as you call out, it doesn't necessarily have to be something sort of to do with Wiz specifically, but it sounds like a great environment to be able to kind of channel this research through. 
 
And before we kind of get onto some specific things that you have researched, I guess, just what is kind of your - you talked about the sabbatical being, sounded about 35 articles. What kind of is the guiding thing for you in terms of where you even wanted to go? Where did you want to investigate and why? How do you even think about that? Because that's almost, I guess, one post per week, basically, that year. 
 
[0:06:20] RM: Yeah. I think the secret to writing 35 posts in a year is don't have full-time employment and have several years of pent-up backlog. I'm constantly tracking articles, ideas, Slack threads that I could turn into blog posts. And there's a few threads that I could group what I write about and talk about in security. And so one big interest for me is real-world discussions of scaling security programs. 
 
If we look in the industry, there's a lot of discussion on the technical side, and there's a lot of abstract, academic, book-like discussion on the programmatic side. But if you look for people who are - there's this term from Will Arsen, who is a very well-known tech writer and operator, who says operators who write, you are actually in the weeds working day-to-day and then using that to come up with content. I think if you work day-to-day, you will never have a lack of ideas. 
 
The first is scaling programs. The second is my niche, if anything, is cloud security. And so, for example, every couple of months, I would try to have something technical to write. So I'm not just often the clouds talking about these, "Now that I don't work anywhere, I can say we should all change how we work." And so that included very minor vulnerability discovery reporting AWS to some extent, and some of their services, and how that impacts customers. 
 
It included a little bit of sort of data diving, experimentation, looking at state of the industry. And then the final thing I'll say is I appreciated the opportunity on sabbatical to write things that are difficult to write when you're employed, because people will ascribe them to your company. And so I, in my personal capacity, occasionally like to plant the stake in the ground in that category of post that is trying to be a little too honest about some of the ways the industry works, some of the challenges you see. 
 
I always want to avoid offending anyone, calling any specific company, organization, individual out. And I want to avoid over-pattern matching. Assuming I've seen something once, everyone in the industry must have this bad practice. But I think it's really important that we find the people and opportunities to sort of talk generally. 
 
One example, not to dive into the specifics here too deeply, but I wrote a post about how to answer security questionnaires. And there's a lot of conference talks on how to answer security questionnaires, but most of them have to be very careful that their customers don't see that talk and take away anything, but we are here in full faith, working ourselves to the bone to best answer your questions to the fullness of our ability. 
 
And so having the distance from any specific employer to sort of share my personal opinion on how the sausage is made on things like that was really important to me and something that I appreciated the opportunity to get out of my system before I'm back in the warm embrace of working somewhere where I want to make sure I represent not just my opinions, but the organization well. 
 
[0:09:25] GV: Yeah, I think that's a very thoughtful way of looking at it. I'm certainly aware of running a company for 10 years, but I even felt an effectively co-founder. And we didn't really have positions, CEO, CTO, but it felt very ascribed that anything I say publicly is going to be, yeah, towards that. And then as soon as I was a free agent, I felt much freer to be able to express, not, I think, anything controversial. But just to me, it was almost slightly liberating being able to say this is Gregor, this is not the company. Yeah, fascinating. Very interesting. 
 
Let's actually get into sort of some of the topics that you've maybe been writing about a little bit more recently. And I guess just from an audience perspective, some maybe top of mind topics. I mean, AI is effectively where we're going to go at the moment. 
 
[0:10:11] RM: How far did we make it here before it came up, right? 
 
[0:10:14] GV: Right. Exactly. 
 
[0:10:14] RM: I think we're doing pretty well. 
 
[0:10:15] GV: Yeah, not bad. Yeah, it's almost 10 minutes that I didn't mention AI. This is good. But we're going to have to go there. And I think just as we'll start kind of light here, which is, I think, you did some research. Sort of can vaguely say that four out of five top secrets in public repos were actually AI-related. If that's sort of true, that's quite a big number. Can you talk to us, yeah, what are you seeing? We're starting light here with sort of vibe coding, and we'll get to sort of beyond that, because I imagine probably the average vibe coder is not maybe touching Wiz and vice versa. But let's just sort of start there, and what have you seen with this whole vibe coding movement, I guess.
 
[0:10:59] RM: Yeah. Let me split track that into secrets and vibe coding. They're deeply related. On the secrets research, I want to give credit to Shy, my co-worker, who did a lot of that work as well. And what we found was specifically we were looking for new methodologies for finding public secrets. There's a lot of work done on sort of secret scanning on GitHub. I actually think the genre is pretty overdone from a research perspective. And so we didn't want to just take low-hanging fruit of like we found some bug bounty style secrets publicly. But instead, are there these sort of sources and patterns that are leading to this problem persisting? And I think that's what we strove for. 
 
And we didn't go in with this conclusion. But after looking at the data and what we saw across GitHub, the answer was the biggest source, the biggest new tailwind in secrets leakage is AI. And so this ties into vibe coding, right? Both we've seen research backed, and the data reflects evidence that, historically, to date, LLMs have tended to generate hard-coded secrets. The patterns they use in development by default, especially with older models, would use that pattern frequently. 
 
Additionally, we've seen that a lot of AI tooling is not necessarily building on best practices of really robust security-oriented developer tools, by which I mean they're using plain text configuration files checked into repos without well-known paths. And so this together leads to AI over-contributing to secret leakage. Also, AI is a new technology with new services, with new types of secrets. And a lot of the secret scanning tools are surprisingly slow on the uptick, I would say, where we saw that there are all sorts of truck-sized gaps you could drive through the secret scanning coverage that's out there. 
 
One fun example I found was China has a whole set of AI tigers of these massive, successful AI platforms. And Western secret scanning organizations, companies, GitHub, the platform, don't have the incentive or awareness to really detect those. And certainly, it has less impact on the average US tech company that there are these Chinese AI secrets leaking. But I think it's an example of how, when a new class of AI secret appears, how do we get better than these string-based matches? 
 
The vibe coding comment you made was that most of the vibe coders aren't at maybe some of these tech companies. And I would agree, most of the vibe coders aren't at these tech companies. However, I think we've also seen - and we at Wiz, we look at a lot of posture data, we're looking a lot at how organizations are using AI for development. And we've seen that there's a lot of known and unknown uses of AI in coding. 
 
And so even if you are not going full and tend vibe coding where you no longer look at the code, or you're doing a no-code, vibe code approach like Lovable or Bolt, maybe there's less of that. But certainly, there's plenty of opportunity for AI-generated code to be introduced to the average corporation. And we want to, I constantly want to think about how we can empower employees and companies to gain the benefits while pragmatically limiting some of the downside risks of secrets leakage, right? Which we talked about. But also raising the waterline for the security level of AI-generated code. 
 
Because one thing that we've seen, I reviewed, I think it was like 25 academic papers, as well as some industry research, AI-generated code tends to have vulnerabilities. I'm not going to put a number on it, but it's something in a quarter or a third of code snippets generated by most models will have some sort of vulnerability if it's sufficiently complex. I'm not saying that that's any worse than the average developer, but I am saying it means we need to treat it like code that requires a level of security, scaffolding, suspenders, guardrail support. And beyond the secrets blog, I also published a blog post on vibe coding where I really take a strong position that step one for the industry is going to lead on adding security context through rules files. 
 
[0:15:16] GV: Yeah, absolutely. And it's not clear what the maybe watershed moment will be when an average, I don't know, call it enterprise, non-typical coding employee is actually using these tools. And suddenly, it is a big problem. It's always good to have effectively gotten ahead on this, even if, as we sort of both agreed, maybe it's not an area that Wiz specifically is sort of going to be seeing things right now. 
 
But one area I think that probably is a bit more pertinent is MCP, and MCP servers, and MCP security. And you put out a very good post recently through Wiz on this. I think we should just kind of break this down. I mean, this is a very interesting topic. Myself and Sean Falconer, one of the other hosts, we do SED News, and it was something that came up during our MCP sort of deep dive. We did touch on this, just there's a sheer lack of security understanding. It was so exciting for everyone, "Oh my God, MCP is awesome. We can agentic everything." And then, of course, a couple of people popped up going, "Hang on. There's a lot of problems here, though, from a security standpoint." Let's kind of dig into those. I think, can you just sort of set the scene for where did you kind of start with this, and what - let's just look at MCP server security, sort of ground zero. 
 
[0:16:35] RM: Yeah. I think, like you said, people are very excited about MCP. And so, from a high-level perspective, how did I come to this is part of working on research at Wiz is working on research that matters to our customers. And so I think there's some belief in the industry that, like, MCP is being pushed. Somehow, there's big MCP out here really trying to voice this on everyone. You'll see this in certain corners of discussion forums where people are, "Why can't we just use APIs?" 
 
I have no stake in that game. What I care about is that Wiz customers were very interested in adopting MCP. And there was very little clear guidance to what degree they should have any concern, what concerns are addressable, what concerns are part of the specification. And so I came to this from just the average CISO does not have any time to do a deep dive into MCP. 
 
From reading the spec to understanding the two blog posts a day being put out full of fud by security vendors, right? How do we cut through this noise and give our customers an understanding? And also, how do we, Wiz, think about MCP security as we think about, "Is this somewhere we can serve our customers through product, through further research?" 
 
MCP security, big topic, but actually I think useful when folks find a way to model it mentally compared to some other system or problem they understand really well. And we can talk about a few different parts of MCP. I think local servers, when MCP first came out, were the biggest deal. There were, I think, 4,000 within that first month after MCP really got big. There were 4,000 servers. These are small, little local binaries that can expose tools for you to call that you download off GitHub and run locally. And this is very comparable from a risk perspective to supply chain for packages and extensions. 
 
It is the idea that it's particularly novel that we should worry about these being malicious, I think, is overwrought. There's nothing specific about MCP that makes a malicious local binary scary. But from that perspective, I actually think the biggest immediate risk is it's just a playground for both attackers and folks who maybe aren't malicious but don't have great practices on shipping local binaries to distribute something across a bunch of your employees. 
 
This was exacerbated because there's no official registry yet. I think it is very imminent. We've seen progress even recently. But as of right now, there's no central discovery mechanism, vetting mechanism. You have startups popping up that are trying to do risk scoring or to provide these registries of MCPs locally. 
 
I think, fundamentally, the answer there is like minimize your use of local MCP servers. It's not a great answer. But I think if I had to give like one practical takeaway, it is have a small list of approved local MCP servers, the more these MCP servers are tied to organizations. You can use the Wiz MCP server if you are a Wiz customer. You can use an MCP server from GitHub for GitHub. You can sort of take that approach with an allow listing within your organization. Now, how you enforce that technically is its whole depth of problem, same with any supply chain. 
 
But I think from a local server perspective, the Pareto on risk is all about limiting the variety and reputation of what your engineers consume, but also making sure that you think really hard as a leader and a security leader about how to loosen some of the pressure around this. If you don't give people access to anything, you're going to have Shadow AI, Shadow MCP. You really have to think about what is the core business driver that this can serve, if any, and how do we expose that. And if people want a model for a regulated organization pushing forward MCP adoption, I would point really strongly towards Block, because Block has goose. They have an active blog. They're very active in this practical application of MCP, despite being an organization that is in FinTech and heavily regulated, and has a lot of requirements. I think there's some cases that show it definitely can be done. 
 
Beyond local servers, the next place to look at is remote servers. And this is where I think things are going. And frankly, once we move to remote servers, a lot of the risk moves to the vendors who are providing you these remote servers. A lot of remote servers are going to be your average SaaS startup is probably working on one right now. You have a vendor risk problem here, right? If you start sending your data off to a fly-by-night remote server because it says it's the GitHub server and it's not posted on the official GitHub domain, you're going to get in trouble. 
 
But basically, if you already have a trusted vendor who you're using their APIs, you as the user mostly only can care about picking those trusted vendors. There's credential management problems, and there is some level of security risk within the server itself. But as the average security team, you probably aren't empowered to care about security risk of your vendor's software, right? You're going to ask for a pen test or an audit report, you're not going to worry about some of the cooler attacks we've seen. 
 
And this is like a place we could rabbit hole, and I think we won't. But there are a lot of really interesting AI native attacks. They're mostly all flavors of prompt injection or flavors of confused deputy. You've seen examples against, I think, both GitHub and GitLab recently have had these sort of clever indirect prompt injection, where if you put the prompt injection in, say, a GitHub issue, it's very easy to see how that gets into the tool context and then causes some sort of unintended side effect. 
 
From a what we can do perspective, I do think that there's a very big difference between running tools and using an AI assistant, or an AI coding assistant, or an MCP client in a modality of human-in-the-loop versus in a way where there is auto approval. And that's a really bright line to me. I think if you're running with auto approval, when these indirect property injections crop up, you're going to be exposed to that risk, and it's just a risk you need to choose. 
 
Some of the more sophisticated MCP clients allow you as an organization to disable auto-approval or GWipe as an enterprise control. And I think that that's really some risk calculus you have to do. But auto-approval simply means that when these injections are happening, you don't even have the chance of a human catching it. And I will say that many of the demos we've seen and proofs of concept, if you were a human clicking yes on each command being run, you would stop and go like, "Why is this opening a GitHub issue with my entire biography?" I think those are some of the initial thoughts I had and tried to summarize in the piece, but happy to take a pause there, too, if we want to go down a specific hole. 
 
[0:23:43] GV: Yeah, I mean, there's lots we could cover there. I think the big stand-out at the beginning there is the fact that, as you call out, these are sort of patterns that we've seen before, is sort of how you put it. And I think this is - I like the fact that you really kind of used the registry as sort of quite an anchoring point to all of this, because I think, okay, now we know that registries, rather, package management supply chain security, is such a big area that things can go wrong. But we only know that now. And things have - there were some pretty big problems that occurred before NPM registry really kind of got its act together in terms of things like personation and type of squatting and things that we might want to like dig into a little bit there just to kind of give some clear concepts. 
 
But I think this is why MCP is interesting, because you've got this technology. One bit of it is, well, you've still got the package management risk there anyway. These are just repositories. We've got to figure that one out. But as you then call out, it's actually the fact that the whole point of MCP is that it's a server. It is running somewhere. And yeah, on the SED News, I gave the example of like, "Well, it's okay if you're running through Claude." You download Claude, and he probably trust Claude to have to have figured out which MCP package server they're using for this. But it gets very messy very quickly when you go and try and find someone else's. 
 
On that episode, I mentioned Smithery. And you've mentioned in your article, Glama, which I think is a really good call out in terms of Glama.ai. Could you maybe just talk about that registry? Because they bring security scoring effectively, and I thought that was quite interesting. 
 
[0:25:25] RM: Yeah. And I'd say that mention is definitely not an official endorsement. And frankly, I think was a bit of a back-hand endorsement because the registry there, they try and bring in some of the trust signals we've seen evolve. I think a lot of credit to tools like deps.dev from Google that try and give you this, like package health, package security, package trust. Does this have a wide committer base or a wide download base? Is this updated frequently? Does it have CVEs? 
 
I think it's really good to have in the registry this context for reputation signals. However, I also call out that this registry, in attempting to sort of, I would say very rapidly develop this idea of trust signals, produce some footgun. The example I give here is that I could not find any documentation of how this registry determines that packages are official or not, that servers are official. 
 
And so there's an Azure MCP server that is very brightly Twitter-style verified official that is definitely not from Microsoft. And that is a really bad trap. And that's why I'm really excited that everyone seems aligned. We should have an official registry where hopefully we will more clearly take some of the lessons from, you said, NPM. I would say also Chrome extensions are a really interesting model. Just as a slight tangent, Chrome extensions are an interesting model for MCP servers because of the permissioning that's present. 
 
Specifically, with Chrome extensions, there's this long history of figuring out that you probably don't want every Chrome extension to have access to every possible function or permission in your browser context. And that's how we ended up with this idea that adding a Chrome extension is relatively safe as long as it doesn't have access to all of your browsing data. You can get a little more granular. And with MCP, there are some open source attempts, some industry attempts to explore what that sort of first sandboxing and then defining permissions would be. Because you can imagine that an MCP server that runs locally, that is sandboxed, is in a container. If it runs malicious code, good for it. Go hang out in your little container. And then can't make network calls is, to some extent, dramatically defanged. 
 
If I wanted an MCP tool that provides an interface to some local data source, say, I could limit it just to that data source, I could limit it to no network access, and you could picture a much more sensible model. And so I like Chrome extensions because it's both this registry concept, right? Chrome, Google are somewhat taking responsibility for reviewing things, but we've seen a ton of risks there. 
 
And also, there's a sense that even with that, you still need to look at permissioning so that we're not grouping every MCP server in the same category of extreme risk because the risk present with an arbitrary binary on an employee machine is like, for many, many, many companies, a game over risk. And that's why I think actually security is - one of the hard things about security today is many companies are doing security well in a lot of areas. And then as soon as someone gets malware on their laptop, it's game over. And this MCP would be a great watering hole distribution mechanism for some of the malware we've seen targeting developers in recent years. 
 
[0:28:42] GV: Yeah. I think we're going to move on from MCP security shortly, but I think kind of leaving the audience with, I guess, just some - maybe in security, it's kind of fairly, we might say simple things to look for. But we do have quite a wide-ranging audience from a technical perspective, and I think it never hurts to kind of remind the kind of basic things you might want to still be looking for. As you call out, there is no official registry. And even if there was, it doesn't mean you're sort of absolved of all your responsibility now. 
 
There's kind of four concepts you touch on that crossover back into just general package manager registries as well. We've got typosquatting, impersonation, rug pools, and account takeovers. And maybe we could just sort of go through those very briefly, just to kind of remind people what they should be looking for. 
 
[0:29:31] RM: Yeah, I love those because they're generic. We'll talk about them for MCP, but it would apply for your NPM packages. Typosquatting is the idea that names are not meaningful. Just because an MCP server is called Azure official does not mean it's from Azure. And similarly, there might be an Azure official and an Azure official that is either has swapped two letters around. Or, potentially, I'm not sure that MCP has gotten to the stage of dealing with Cyrillic characters, right? Some sort of alternative character set. And so that's one. 
 
Two is impersonation. Impersonation is this idea that someone is going out of their way to try and represent themselves as a known project or organization, right? This is not just, "I'm a developer. I made my personal Azure MCP, but I'm going to make a landing page that says 'GitHub launches official GitHub MCP," and really try and mislead you as to the affiliation. 
 
Rug pulls are more of a temporal aspect. They have to do with time. This is where, at the time you review a package, maybe it is safe. But again, if you're consuming updates to these servers, someone could later introduce malicious code, and that could be either the original owner of the package sort of creates a useful package and you wait a while for a bunch of people to adopt it and then you pull a rug out by introducing malware. Or it ties into our fourth point, account takeovers, where you either compromise, or as we've seen in Chrome extensions, purchase a legitimate MCP server that's popular and then turn it to malicious gain. 
 
I would say that all of these just go to show that you should not think of a search term and Google around for MCP servers. You should work backwards from tools you knew, use, organizations you trust. And especially, I think that this is where social media can be dangerous for security. There is a lot of virality to MCP tools. Someone will post a really cool demo, and you should be really thoughtful about sort of going directly to a GitHub, or download, or installation link without transiting through some sort of process where you build a little more trust in who's providing you that MCP server. And Ideally, I'd love to say read and review the code, but no one does that. I'm not going to lie to myself. 
 
[0:31:47] GV: Yeah, and I think it's a good callout here where the same tactics always still apply. And social engineering is kind of effectively - a lot of that, as you call out, it could whether deliberate or not exactly, but something that appears in posts, and then maybe there was some reason why that one appeared or maybe not. Unfortunately, it was just sort of innocent. But anyway. Rug pulls is interesting as well because, yeah, we've obviously seen that pre-MCP in the sense of - and again, social engineering coming in there where someone kind of befriends the repo maintainer. And then the maintainer sort of says, "Oh, I'm getting a bit stressed. I'm going to leave now." And then the person who befriended them has actually got much - 
 
[0:32:30] RM: Yeah, that was xc_utils, right? I think it's actually the specific case you're talking about. Yeah.
 
[0:32:34] GV: Yeah. But I wouldn't be surprised to see something similar happen again, right? 
 
[0:32:40] RM: Chrome extensions, enormously common if you own a Chrome extension or run one and it gets popular. Someone will be in your DM's offering to buy it, so they can use it for Adware or to turn into part of a proxy network. Really common. And so I think to address rug pulls, we have to think about what versioning and updates look like, right? And so the takeaway there would be just think about, like, "Go look for your stack of MCP tooling. Do you consume updates by default? Do you version bump? And how do you make sure that when you do the review, you put in place at the outset holds true?" Same with any package. 
 
[0:33:13] GV: Yeah, and I think it's also - yeah, it's a good call out at this stage in the game, if you want to call it that, where a lot of MCP server have flooded the market effectively. And what does that look like retroactively in terms of retroactive assessment? As you say before, an official registry comes online where, in theory, I sort of like the Chrome extension example, there would at least be some potential party doing some kind of verification at least at that point in time. Again, what does the retroactive version of that look like remains to be seen, I imagine. Yeah. 
 
Maybe you just switch gears a little bit. Obviously, I think the audience probably has a good grasp now of just what to look for in MCP security. Another piece of research that you did, I believe it was whilst, or it must have been very early, when you join Wiz, tj-actions. And I wondered if we could just - this is a slightly more complex case, but I just wondered if you could maybe just talk through that. And especially from your perspective as a researcher, just walk us through the life of a researcher. And sort of from start to finish, what was this? Yeah. 
 
[0:34:22] RM: Yeah, I was six weeks into Wiz when Tj-actions kicked off, which actually worked out perfectly. So, TJ-actions, in case anyone in the audience isn't familiar, was a GitHub Action-based supply chain attack. GitHub Actions are CI/CD runners that allow you to use sort of composable little bits of code from across the ecosystem. Again, everything is supply chain, it seems like. And in this case, there was a multi-step attack where an attacker was able to be very abstract, compromise a GitHub action that gave them access to a more popular action, that gave them access to an action called tj-actions changed files that was both very popular and more importantly to the attacker used by Coinbase. 
 
And so this was an attempt to do a multi-step supply chain attack targeting Coinbase. The attack on Coinbase failed, at which point the attacker sort of broadly changed the action to be malicious, like we talked about, a rug pull, on TJ actions, although coming from a malicious takeover. And at that point, the outcome was that folks who were using secrets in their CI/CD pipelines would change files, ended up having those secrets leaked in logs, which is, if you were using public repositories on GitHub, the logs were public. That's the super high-level on the attack. 
 
I was six weeks in at Wiz when this sort of floated by on my feed. And that's really good thing, I will say, because I was onboarded enough to be empowered and situated to know what to do, but not so onboarded that I was already deep under water and this threw my life too much for a loop. The good or bad of being a researcher at Wiz is that anything in security probably applies to us and our customers. Wiz is a cloud security platform, but we also have code security tooling at this point. Our customers are half of the Fortune 100, right? 
 
If you can think of a technology, it probably impacts our customers, and they're going to come asking. And so this sort of attack that clearly was having broad untargeted impact meant, "Okay, we should serve our customers by making sure we can tell them exactly who is impacted. Is the attack over? What do you need to do about it? How did this happen? How can we protect you from future attacks?" 
 
And what's fun about research to me is that it's sort of technical and social. And so tj-actions, there's a very fun Slack artifact inside Wiz where I - it was 11:30 pm in Sweden and I was scrolling on my like news feeds before bed and saw someone share out. I think it was the step security folks who initially identified this credit to them, share out that like, "Hey, we think tj-actions changed files went malicious. 
 
I've used this action before in past roles. And so I was familiar enough with it to be like, "Oh, this is a thing that people like myself who are thoughtful about what they consume still are using it." And so there's a message inside with Slack that's like, "I think this one's a bad one." And then I woke up the next morning, and it was truly a bad one, and folks had started to rally. 
 
And so what did this look like? Well, there has been a lot of research in the past two years about GitHub Actions' security and about specifically sort of a set of common misconfigurations that can be made, coding issues that can be done that allow for actions to be the secrets associated with actions to be compromised. I was lucky at Figma. One of the things I was responsible for was hardening our GitHub Actions' security posture by happenstance. And so I had this leg up where, as a researcher, this was an area where I was already pretty literate. 
 
And frankly, with Wiz, the joy of the research team is it's big enough and strong enough that, with any topic, someone has that level of literacy. And so the job to be done was sort of fragmented. It was let's investigate this from a research perspective. What happened? What was the root cause? Because when this was announced, no one understood how the compromise had happened. Let's talk to product and product analysts and talk about what sort of detections can we validate fired, right? Did we detect this in our customers proactively? In our case, we did. But also, are there additional new tactics, techniques, detections, but also configuration management controls we should be building? And then third, how do we speak publicly about this? What is the information we should be giving to customers through their account teams, to customers who were impacted, who we now have to give the heads up that, "Hey, you have some cleanup to do." 
 
What's fun about TJ-actions is there were, I don't know, a dozen security research teams that wrote blog posts sort of regurgitating the same information on that first compromise. And I felt very satisfied that we got to write the blog post where we actually found the next step in the chain. We were able to find that how did this TJ-actions action get compromised? Well, we're able to find that they were compromised upstream by an action called reviewdog/ action-setup. And actually spent a lot of time over the next week or so in touch with that maintainer to sort of talk to him as he had, or as they had the hard job of sort of cleaning up now, right? You're an open-source maintainer. You make a little pet GitHub action out of the goodness of your heart. 
 
TJ, who is the TJ behind Tj-actions as well, had really rough weeks. And so a lot of credit to them all for the time they spent cleaning this up. And just worth pointing at supply chain and how we treat our open source maintainers. I think I was very focused on making sure we, through the way we talked about this instant, through the sort of support we offered, didn't make their lives harder, while still focusing really hard on eradicating the root of this attack. 
 
And similarly, we stretched up through Reviewdog to find the source of compromise in that direction. And we also identified and disclosed to Coinbase once we discovered that Coinbase was the eventual target and tied that attack chain together. And that Coinbase discovery was simultaneous with a couple other teams also sort of tied those dots together. 
 
[0:40:19] GV: Yeah. I think just want to kind of go back on a couple of bits there. Again, it kind of comes back to people. The first bit I just want to touch on is it was interesting at the start you say it was Twitter effectively, I think, or X, whichever, that you were - do you find that's kind of still for you where the first flash of something being announced kind of happens is in your network effectively? Because I think there's maybe a misconception that sort of somewhere you go to find out about latest attacks, etc. And usually that's a huge lag for various reasons on the reporting effectively. What's kind of your approach on that one? 
 
[0:40:59] RM: My stump speech is where you go is Wiz. If you're a whiz customer, we do all this work because we do - this is the challenge. And so Wiz has a threat center, and we put out public thread information as well as to our customers, so that when something like this happens, you don't have to scrounge the GitHub issues, the X threads. Hacker News had some really helpful threads in this case. The research blogs, the private Slack groups, and Signal groups were all in, right? The information is distributed. And we and other research teams, I think, try and consolidate it. 
 
I will say the changes to X in the past couple of years have made it much less so. The firehose it used to be for this sort of research; there were a couple of years there where I could feel pretty confident if there was breaking security research, I would find it in short notice on my X feed. And given the diaspora of security researchers to platforms like Bluesky, and Mastodon, and folks who've just disconnected from social media in past years, it's less true. 
 
I will say that security, especially, currently benefits from private Slack and Discord groups that are communities of practice. One I'm heavily involved in, it's open to practitioners. It's called Cloud Security Forum. It's associated with the fwd: Cloudsec Conference as well. Folks can Google it. There's an invite form. Please feel free to join. And those are the communities I expect to either see this discussed. Or if not, when I'm the person who stumbles on it, those are the places I sort of start a community response, right? How do you trigger the security communities' immunoresponse while you have some of these more public Slacks and Discords, and then you have some more private Slacks, and Signal groups, and whatever else where you can get the word out just by nature of being in the industry for a while? I think I'm second or third degree connected to every security team in the US, right? It just happens if you're in security. And so the word does seem to still get out organically. But nonetheless, I am disappointed that X is no longer quite the useful source of information. But you do still occasionally catch things. 
 
And I will say the joy of being a researcher, one of the many positives, is that I'm paid to pay attention. And I think if you are the average security engineer and you're someone who's spending four hours a day monitoring social, you're probably not being very effective unless that's your role. But for the research team, especially for our threat research team, they spend a lot of time pre-sophisticated infrastructure from monitoring everything from social to news feeds, to research, to disclosures, et cetera, to private feeds. 
 
[0:43:32] GV: And on the other side, in terms of making contact with - I mean, in this case, just individual, it sounds like, in the open source world. I'm aware sort of this may be more in the hardware world, these edge devices. And unfortunately, the problem there is when, say, a company, one that comes to mind is Watchtower, I'm based in Singapore, they're a Singapore-based company. I often just like to throw them in there. But I think one of the challenges they face is once they've discovered vulnerabilities, say, with a router or such thing, promise is vendor-to-vendor. And the vendor of that router is, unfortunately, I think often not willing to admit within certain parameters if there's a problem and over what time frame they're going to patch that thing. This feels quite different because you're talking to an individual who's I imagine quite - I'm not going to say overly receptive to hearing that this is a problem, but maybe more receptive given that you're kind of coming in as Wiz to help them with this. And they don't have such a commercial interest in sort of keeping quiet about it. - 
 
[0:44:38] RM: Yeah. I will say disclosure is challenging. And I think there are a lot of challenges for organizations as well. There are companies that take disclosure poorly. And then there are companies that have 90-day release cycles. And so as a researcher, you hope to write something next week and the timelines are just really hard to reconcile. Wiz has a really active vulnerability research team. They did Ingress Nightmare in Kubernetes. And when you find vulnerability in something core to Kubernetes, you are not publishing in under 90 days or whatever it is, 180 days, just because the level of coordination and community stewardship with these open source maintainers and, frankly, with people in the community, because there was collaboration here, not just with open source maintainers, but across research teams, right? 
 
There was certainly back-channel conversations between myself and other people working on this problem, whether they were independent researchers or even competitors, right? We're all trying to get to the same answers. That being said, I think with open source maintainers, people handle it poorly in a lot of companies. Not to like vigorously pat myself on the back, but I was really aware that I was ruining someone's day over their pet project. And so we want to be very thoughtful about presenting this not as Wiz has got shed, right? Like, "You've been hacked, here's our blog post. We'll let you read it in the news." But rather, we actually lagged our reporting by as much as we could to respect the maintainer's ability to understand what was about to come down on them versus launching first, apologizing later. 
 
And frankly, we did the same thing with Coinbase. With Coinbase, we went through their disclosure pipeline when we found out they'd been targeted. Even though this information was all public, GitHub has a public audit log, so you can actually investigate these attacks in real time with public data. Despite that, we weren't going to just put out a big press release that Coinbase was the target and try and get attention for it. We wanted to make sure Coinbase had a chance to detect, respond, recover. 
 
And so I think it's the same with these open source maintainers. If you go in with good intentions, treat people respectfully and are empathetic to the challenges, they're going to be much more receptive to working with you. If you send in a big bounty email, then they're probably going to be less helpful, right? I think we have an obligation for how we approach this. But also, yeah, I can expect better things from the average open source maintainer, just 'cause I expect that they are someone who's in it for the love of the game as it were versus vendors who have really complicated incentive structures. 
 
[0:47:15] GV: We are coming to time a little bit, but I'd love to just touch on one other thing before we try and wrap up, which is I guess just you've done - this is, I guess, pre-Wiz, especially things like jus sort of critiquing state of cloud security reports and sort of what things should be measured. Do you have a kind of TLDR on your analytics within Wiz's opinion, kind of what should be getting measured here? 
 
[0:47:40] RM: Yeah, to reflect my personal opinion on that piece I wrote before I joined Wiz, the summation of that piece on state of cloud security reports was that a lot of vendors take their customer data and they write a report saying that people are not fixing problems. And I always found that really funny if you are a cloud security vendor to write that year over year, 90% of organizations have publicly exposed sensitive data, right? If you are a security vendor and your customers who are the data you have access to have publicly exposed sensitive data and they're not fixing it, what value are you providing? 
 
And so I think actually the things we should focus on as vendors when measuring progress is what are we helping customers fix. And also, what are we helping the industry move towards uncommon? Examples of what we're helping customer fix, Wiz does this huge thing around zero criticals. We really try and push customers by emphasizing what the important findings are, and then having them have a clear goal of killing those findings. 
 
On industry-wide, IMDSv2, right? In AWS, there's this instance metadata service tied to a bunch of instance, including Capital One. We as an industry pushed the platforms to make it better and more supported in migration path, open source tools, reporting, adoption, advocacy. And I think as an industry, we can congratulate ourselves when no one is on the vulnerable IMDSv1. 
 
And so I think when I look at state of cloud security reports, I don't care about what percent of your customers are not fixing misconfigurations or hygiene issues. I think that reflects more on you as a platform. I want to see sort of what are the real world attacks that are happening and how are customers building resilience, not just building that first layer of defense. 
 
I think Wiz was founded on this idea of toxic combinations, right? We don't just report that you have an open port. We tell you that you have a service exposed on the edge with a known vulnerability that is pre-auth, that is tied to sensitive data inside your infrastructure, right? We want to show you the whole attack chain. And I think similarly for these state of cloud reports, going a little further than publicly exposed S3 bucket metrics, are the common bad cases important. And I've worked on this. I've peer reviewed a lot of reports for companies pre-emptively, even before I wrote this. And I think I'm constantly pushing it back against the easy metrics. Because the problem with metrics are that easy metrics or even achievable metrics aren't always the useful ones. 
 
And so it's really easy for vendors to give you a number on public S3 buckets. But many vendors don't have the sophistication to tell you whether those buckets are sensitive or not. And that's like the difference I'm pushing for and Wiz has always pushed for is it is not important if a bucket is public. Many buckets are public. Many buckets should be public. It is important if there is some combination of intention, or sensitive data, or tied to systems that matter. And so I think there's no there's no one golden metric, but caring about moving things towards invariance, caring about the critical issues, which are often toxic combinations are what I'd leave folks with. 
 
[0:50:47] GV: Yeah, I think that's a great place to leave it. And I think that's some very sage advice just in terms of whether you're a CISO, or a security engineer, or just a developer. If you're a developer who should always just be thinking about security, it's not just about numbers and red warnings, it's about combinations effectively and about people. 
 
Just before we go, I do like to ask this question, and I think it kind of also helps anyone listening today that's maybe towards the start of their career, whether it's security yet or not, but what's one thing that maybe you could tell, Rami, of coming out of college that you know now that you'd love to be able to sort of tell that person and maybe that helps someone else today? 
 
[0:51:30] RM: It's hard to give advice to people coming out of school today. The industry is in a weird moment. One practical thing I would advise people is do not underestimate the compounding value of working in public. And this is something I learned in my journey and have applied and seen a lot of return on. And what I mean by this is it's never too early to start blogging, talking publicly, building your network, being in the room where things happen and putting yourself out there. And this doesn't mean you have to self-promote. I mean, it's embarrassing. But you can scroll back on my blog and find the first things I wrote, and they are very small, and they are not particularly novel or interesting. But getting in that practice means that in the year of my sabbatical, I wrote 35 posts. And the kind of funny thing is that so many of those self-reference. 
 
I have started to accumulate this momentum that now I can self-reference a lot. When I have new challenges, I can often sort of have written down and published my perspective. And it is, I think, advice I give because it doesn't require you to do anything special. You don't need to have an audience. You don't need to have a network. You don't need to have any expert level of knowledge. If you have learned something, write about it, share it, find places to learn from, and just know that the first three or four years are mostly practice. 
 
There's some advice on fiction writing that says you throw out the first five books you write. It's that. But I think that while you're trying to make the rest of your career come together, you will never regret having spent some time getting better at documenting what you work on, how you think about security. That's also a way to pressure test ideas, which has outsized value early on. If you are early in your career, and you're brave enough to write something down and get some feedback on it on some of your views on security, you can learn a ton. I know I have. 
 
[0:53:20] GV: That's amazing. Yeah. And I think it's a great callout. Just the word obviously that kept popping out there is the verb writing, which is maybe anyone coming out of high school, college wondering, "But an LLM can do that for me." And of course, that's the whole point. Writing is thinking. And I think that's a great piece of advice for anyone, myself included. I'm trying to do more writing. 
 
Thank you so much, Rami. What a pleasure. And Wiz is obviously very lucky to have you. And it's been great just to hear about what Wiz is up to with your help as well. Thank you so much for coming on. 
 
[0:53:51] RM: Awesome. Thanks for having me and for the great questions. 
 
[END]