EPISODE 1969

[INTRODUCTION]

[0:00:00] ANNOUNCER: The rapid rise of AI has set off a flurry of legal and regulatory activity. There is no single AI law to follow. Instead, there are overlapping circles of regulation, where privacy, cybersecurity, consumer protection, employment, and IP laws all continue to apply alongside newer AI-specific rules. 

Europe has its EU AI Act, but the US has no federal equivalent yet. In its place is an emerging patchwork of more than 150 state AI laws, and the picture shifts every few months as new legislation and guidelines arrive. For engineers and tech companies, it can feel impossible to know what applies and where the real risks lie. 

Shannon Yavorsky is a data lawyer who co-leads the Cyber, Privacy, and Data Innovation Group and the AI practice at the international law firm Orrick. She advises companies across sectors from health technology to financial services on how to develop and deploy AI responsibly amid this evolving regulatory environment. 

In this episode, Shannon joins Kevin Ball to make sense of that landscape. They discuss how to map concentric circles of regulation that actually apply to you. The risk spectrum separating a low-stakes manufacturing tool from a high-stakes system making loan or healthcare decisions, the problem of shadow AI, and why recorded meetings can become a litigation liability, how to build an AI governance program, and more. 

Kevin Ball, or KBall, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action Discussion Group through Latent Space. Check out the show notes to follow KBall on Twitter or LinkedIn, or visit his website, kball.llc. 

[INTERVIEW]

[0:02:07] KB: Shannon, welcome to the show. 

[0:02:09] SY: Thank you so much for having me. 

[0:02:10] KB: Yeah. I must admit, I'm personally very interested in this topic because it's an area of uncertainty for me. Getting to dig in with an expert about the state of regulation will be good. But let's start with you a little bit. So can you give us a brief bit about your background and what brought us to be speaking together today? 

[0:02:27] SY: Yeah, sure thing. So I describe myself as a data lawyer. I spend most of my time at the intersection of technology, regulation, and business. I co-head Orrick's Cyber, Privacy and Data Innovation Practice, and I also co-head our AI practice, which is a lot of people think about, "Oh, are you helping people implement legal tools like Harvey?" And I definitely use those tools, but that is not my job. I advise companies on their development and deployment of AI technology. And just like a deep background, because I think this is helpful, I went to law school in England, and I started off life as an IP lawyer, as an intellectual property lawyer, and sort of moved from IP to tech transactions to privacy. So the lens through which I look at AI is really intellectual property, commercial agreements, and data, like private personal information and all of those things like really converge in AI. So I practiced in London for about 12 years and then moved to the US about 13 years ago. So I'm evenly split in my US and European legal experience. 

[0:03:47] KB: Well, and I feel like that's really important right now because they're pushing regulations in a pretty different way at this point. Let's maybe start with kind of just getting the lay of the land. I don't know about you, but it's not just regulatory, but anything AI, I feel like I'm constantly behind what's going on, what's not. So what does the regulatory landscape look for in this space right now? 

[0:04:10] SY: Yeah, it's a really good question because it's a really complex landscape. And I would describe it as overlapping circles rather than one giant AI law. Even in Europe, where you have the EU AI Act, there's still a whole raft of legislation that also applies to companies who are navigating this space. 

So if you think about consumer protection legislation, privacy law, cybersecurity law, all those laws continue to apply alongside AI laws. That's Europe, you've got the EU AI Act, and that was supposed to be one law to govern AI, but it is supplemented by all these other laws. And it's very similar in the US in that you have consumer protection legislation and privacy legislation, employment law. We'll maybe talk about this a little bit like discrimination and civil rights laws. But the differentiating factor in the U.S. is that you don't have an EU AI Act. You don't have a single federal AI law, at least yet. And you have this emerging patchwork of state AI laws, of which there are now over 150 state AI laws that companies are trying to navigate, again, alongside IP laws, cybersecurity laws, privacy laws, employment laws and contractual obligations. So it's a really complex and quickly evolving patchwork. 

[0:05:51] KB: That feels overwhelming, honestly. We're talking about over 100 different laws and different things. And I have deliberately stayed out of healthcare tech, but I know healthcare tech navigate this all the time with 50 different states having their own health-related stuff. But are there ways to think about this that are going to help you navigate that complexity or reduce it down to some sort of underlying principles or anything like that? 

[0:06:14] SY: Yeah, I can really understand that it would be complex for a founder in the health tech space to hear me talk about the thousands of laws and just be like, "How am I going to manage this?" But I think the important point for, well, let's say a founder is that you don't necessarily need to become an expert in every one of those regimes, you need to understand which of the concentric circles your product falls into. So is it - maybe start with geography. 

[0:06:47] KB: Yeah, let's map out what are those concentric circles. Yeah. 

[0:06:50] SY: exactly. Map out the concentric circles. If you're a health tech product, are you subject to HIPAA, which is the federal health data legislation in the US? So what's your privacy regime? And then look at whether you're regulated by any other federal agencies or state legislatures that may apply to your product. And then are you subject to what other geographies are relevant for your product? Is it just the EU and the U.S. or is it the U.S. alone? And then you can sort of narrow it down. So I would really start with the product in the use case rather than this giant list of laws. I think that's probably a better starting point. 

[0:07:36] KB: We're in the AI world. You said there's a hundred different laws, but start with what's applicable. So let's imagine I'm starting in the US, but I probably want to sell into anywhere in the US. I mean, I think most of us don't think about within the US geography, where is somebody buying an online product from? Maybe we have to, maybe we don't. How would I even start with that? Is it I have to read through these a hundred different laws or there's like some sort of of underlying structure that can be derived out of them? 

[0:08:04] SY: Yeah, there is because there are triggers for applicability for these state laws. As an example, if you're thinking about whether the privacy legislation applies and there are - in the privacy law world, there are about 24 different comprehensive state privacy laws and they're triggered on whether you have a certain level of revenue or whether you're collecting a certain amount of information. So you have to determine whether you hit these thresholds and whether you're going to trigger, even trigger the application of the laws. You're looking at the regimes that are overlapping the concentric circles, the ones where you fall into. And then you're looking at, okay, do I even trigger the laws in these states? And work from there. 

[0:08:58] KB: Got it. Okay. Now, I personally have started walking down the road of understanding more like some of the traditional data related regulations and how to become like SOC 2 compliant. And then what does it mean to add GDPR and these different things? And one of the things that's been very helpful for me in that is kind of shifting from, "Oh, there's just all these requirements." There are, but there are tools that help with that. But like, "Oh, fundamentally, this is about legibility and process and making sure that you are very clear about what's going where." Is there a similar kind of mental model that we can apply to these new layers of AI that are being onioned around this? 

[0:09:39] SY: So how to think about the applicability of the AI laws? I think where I come out on this is if you look at how companies dealt with a similar emerging patchwork of privacy laws around the world, where you started - and there were other laws before, but you kind of started with the GDPR, the General Data Protection Regulation in Europe. And then from there, you just had this wave of privacy law emerging around the world. And companies faced a similar problem that they do now with AI laws. How do I manage compliance against the backdrop of this really complex world of laws? 

And I don't want to go to local lawyers that are expensive. I don't want to go to local council in 30 countries and get guidance on whether these laws, how exactly they apply to me. So what we did with privacy law was help people understand how to build a foundation and build compliance around core principles. And it's very similar to what we're doing now with building AI governance programs. 

So instead of going to each country around the world, local council in every country and saying, "Okay, what's happening in your geography?" It's a little bit of a risk-based approach and you determine, "Okay, we're going to build this foundation around these core principles of transparency and safety." That's our foundation. And from there, we'll build layers of more nuanced compliance. 

[0:11:17] KB: I like that a lot. You mentioned two principles there, transparency and safety. Are there more? What are the core principles behind a good AI compliance program? 

[0:11:26] SY: There are a number of self-regulatory frameworks that have emerged around the world in relation to AI governance, and they are really similar. Transparency is one of them. The OECD actually published very general principles around AI. And they really aligned around transparency, fairness. So think about bias and discrimination in data sets. We hear a lot about that. Safety, accountability. 

And accountability was a core principle in privacy legislation. It's actually enshrined in the GDPR, but it spans all of those things. But accountability is a really important one. And it's really about... documenting how you were thinking about particular decisions. So we thought about bias and discrimination, for example, within an organization and looking at these data sets. Here's the process that we decided to implement to mitigate risk there, so that if a regulator or a litigant in any event are looking back on things, you have a record of the organization's thinking on a particular topic. 

So I think that accountability point is really important and maybe even more important in the context of AI because there's a lot of talk about the black box of how AI makes decisions. So I think it's important to document how the organization was thinking about implementing AI. 

[0:13:03] KB: Yeah. Honestly, you connect there something that was a major unlock for me when I started having to deal with regulatory frameworks. Because as a software engineer coming in, I was like, "Oh, there's a right answer I have to get to." And I don't know what it is. And that's terrifying, right? Because now there's this external body that could just come in and say, "Hey, you're wrong, you lose this certification or whatever." And really shifting to it not being something where there's one right answer, but like you need to have thought about and documented and made transparent how you're approaching a bunch of different problems. And your approach might differ across different companies. Are there any kind of non-negotiable pieces in here? Or is it all in those like shades of gray? And remember, for software folks, we're used to like hard cut and dried. Is this true or not true? Does that exist in law here? 

[0:13:51] SY: Yeah. I live so much of my time in gray areas. We talk about there aren't a lot of bright lines here, but we talk a lot about with clients, the spectrum of risk, right? And the sliding scale and what puts you in a better position with respect to compliance. Here's the gold standard, which we think is like the best possible answer, given that maybe there is a law, but there's no guidance on how this particular law might be interpreted because it went into effect last month, right? So we help people understand the spectrum, and then it's down to the company to determine where they want to fall on that spectrum of risk. 

There are things that are just a hard no, or at least we tell companies that it's a hard no. It happens a lot where it's very clearly a violation of an important law, what they're trying to do. Sometimes this comes up in biometric data collection. There are a lot of novel use cases with respect to recording equipment, people wanting to record ambient sound, ambient conversations, to use that data for training models or to inform digital twins. There are lots of different use cases there. 

Recording is one of those really sticky topics in the U.S. anyway because of really old laws that were designed to - the state wiretap laws. So there's this basically ancient privacy laws being applied in this very new context. So sometimes you really run up against a pretty hard line, and that's a difficult conversation with the client that what they're doing is actually pretty illegal. 

[0:15:41] KB: That's an interesting just like a micro diversion here. I feel like one of the things I've run into a lot in terms of video calls is different states have different requirements about informed consent. If you're in a two-person call or many person call, do all parties have to consent to this or does only one party have to consent to this? 

[0:16:00] SY: Yeah. And there are different laws. 

[0:16:03] KB: They're different and different tools take different stances, right? Google's Workspace does a nice job of saying, "Hey, everybody must consent." Nice big warning. We've got all these companies shipping AI note takers that plug in on one person's computer, and that's legally gray. 

[0:16:20] SY: Yeah. Yeah. That was a big issue whenever all of those different note takers came out, because lots of people were just clicking through. They make it so frictionless to just download a tool. As an employee, you're like, "This is really going to help my day-to-day. I can't be in this meeting. It's going to record everything and send me this neat summary and the follow-up items. In fact, I might just take a half day off." People didn't make that decision. But there was a huge issue when those first came out because it's part of a different problem of shadow AI of people onboarding AI tools without a company's knowledge. 

If a company has an enterprise version of an AI note taker tool, they can implement all of the controls that are needed to dial down risk. And it makes it a lot less risky for the organization to allow people to use those tools because they are really helpful. They're super useful. You don't have to write down every single thing as you're listening to a call, but there are lots of potential issues there, even outside of the state wiretap laws, right? Where it's going to be stored? Who's going to have access to it once it's been saved? Whether those summary points are actually accurate and you can draw a line to what was said in the conversation. 

And then we sometimes talk about the ineffable changes, like whether it has an impact on what people actually want to say during that meeting because they know it's being recorded. And then as a lawyer, anything that is recorded is automatically a record that is potentially discoverable in the context of litigation. So you can imagine the uproar at firms when these things rolled out to all clients making all these records at once. 

[0:18:16] KB: Let's actually double tap on that a little bit because it's not something that folks in tech are used to thinking about here. What does that mean that, "Oh, suddenly, all these things that were previously not discoverable are now discoverable?" 

[0:18:29] SY: Yeah. Anytime you have a record of something, books and records that is not protected by some kind of what we call privilege, it can be potentially discoverable or have to be disclosed in the context of litigation or an investigation by a governmental authority, regulator, as another example. If they're looking into an issue, they can say, "Give us every record that you have related to X topic." And that means the company has to search through any records. And now if you've recorded every single conversation that takes place at your organization because it was convenient, those things are now going to be picked through in the context of this litigation or investigation. Again, if it's not protected by privilege. 

[0:19:22] KB: Yeah, I think this is an unintended side effect of our now - certainly my company, most meetings are recorded by default. And I hope we're not doing anything that's legally shady. I don't think we are. But yeah, it is interesting to think about. That just kind of changes the landscape. 

Coming to a slightly different question. So you mentioned shadow AI here. And one of the distinctions I'm often thinking about is application of AI in operations and company operations versus in a product that is being used by people outside of the company. Are those legally distinct in some way? Is that an artificial separation when it comes to the law? How do you think about the different types of AI systems and their use in business? 

[0:20:03] SY: Yeah, so we're talking about something that's being onboarded versus something being developed and deployed at an organization. I think there are a lot of really similar considerations. If you're onboarding an AI tool - and let's stick with the note taker tool because I think it's one that everybody sees all the time. If you're the developer of the AI note taker tool and shipping that out, you're putting all the onus on the company, on your customer, to make sure that they're complying with the law. So you'll see all of those agreements say, "You will comply. You company, customer, will comply with all applicable laws related to this tool. We're just providing you with this tool." 

One thing they do have to do is ensure that it can be configured accordingly. And by that, I mean, there can be options surfaced to the user to say, "Do you consent to this call? Do acknowledge and consent to this call being recorded?" As the developer, you could get into trouble for not including those features. And then if someone is suing the company that is using the AI note taker tool, they might try to add the AI note taker company, developer company to the litigation in any event, because they just want to sue everyone in the chain. So whether or not there will be valid claims there, has to be played out. And then that company is using resources and time and money to defend itself. So it's in that company's best interest to make sure that they're doing everything that they can to make sure that their customer is using the AI notetaker tools complies. 

[0:21:48] KB: Quick question on that, because this is a pattern I've seen. I've seen tools that they don't have an opt-in so much as they allow you to configure. So it'll send a message saying, "Just so you know, I'm recording this." Does that count as consent if somebody sees that? 

[0:22:03] SY: It's a really good question. And it's not one that's been heavily tested, I would say. You'll notice as an analog, when you call your bank, they'll announce on the call, "This call may be recorded for quality and assurance purposes." Right? That's their similar notice that the call is being recorded. That's their nod to compliance with the state wiretap laws, but it's not necessarily the level of consent that's required by the legislation. But because there's really no other way to do it, that's what people acknowledge as being the safest thing to do. 

Similarly, you'll probably have seen in different places, sometimes on cars that are going around recording for LIDAR or street cameras for autonomous vehicles, they'll have a QR code on the vehicle that you can look at to see their privacy notice that will say things like, "We're not identifying individuals, we're just recording the street, the public street." Or sometimes on a receipt at a store, you'll have a similar QR code or a link to a privacy notice, again, because it's difficult to present notices to consumers. So we find ways around. 

[0:23:28] KB: And that's fascinating because there's a difference between notice and consent, right? And in the case of a phone call or a video call, I could hang up in theory. Am I going to? Probably not. But I could say, "Oh, no, I don't consent. Goodbye." I don't think I can do that to a Fleet Street cam or whatever. Is there law applying to that? Is that a different section of the law? This feels very hairy to me. 

[0:23:53] SY: Yeah, this is an ongoing discussion. And it goes to constitutional privacy laws. And the fact that you have places where people have a reasonable expectation of privacy do not include public streets. So if you're just on a street, people can sort of freely record in a car, provided you're not trying to identify individuals from that recording, which was a big problem with - I think the company still exists. 

[0:24:26] KB: The flock cameras. Yeah, we did an episode on that. 

[0:24:29] SY: Not the flock cameras. It's the one that does the reverse image lookup. Clearview. I think it's called Clearview. Okay. Clearview AI. And that had a lot of different privacy issues. 

[0:24:41] KB: Yeah. So momentarily bringing this back to the ambient recorders, I've seen a lot of folks that say, "Oh, we should record in-person meetings as well. Oh, we should record this type of thing." What is the level at which that is covered by this like public spaces? Sure, I don't need necessarily everybody to consent, versus I have a reasonable expectation of privacy. Now you have to get my consent. 

[0:25:03] SY: CCTV is a good example. So you can have CCTV in certain spaces. For example, you cannot have CCTV within an organization, like at a company, in the locker room as another example, because people have a - there's an expectation of privacy in certain spaces. 

[0:25:24] KB: Okay. Well, let's maybe dive back to - let's imagine we're a company, a founder or a software engineer at a small company. We don't have an existing AI compliance program or regulatory standpoint. How would you think about getting started saying, "Oh, I'm an implementer. We need something?" Where do we even begin? 

[0:25:46] SY: And it seems like such a really monumental task of tackling AI governance. So where we tell companies to start is just getting their arms around what tools are actually in use. And that in itself can be a huge challenge because of shadow AI, people using lots of unsanctioned tools, which is a problem. But even within the last couple of years, the procurement teams have onboarded lawfully and enterprise versions of AI tools, but there hasn't been like a complete record made of all the tools, the list of tools that have been onboarded at an organization. And sometimes they're older tools that have just deployed AI features. And people will have seen that with lots of tools that they use every day. It'll have a pop-up that'll say new AI feature or new AI companion. And it's so easy to just slide the toggle. 

So there are lots of AI tools that are in use at companies that there's not a full awareness of what those are. So that's sort of step one is getting your arms around what all of those tools are. And then potentially in the context of that exercise, shutting down really egregious uses if there's like unsanctioned use of AI tools that are really posing risk to the organization. Maybe blocking that so people can't access those tools at all. So starting with that, getting that together and then putting together AI use policy. Internal AI use policy. Everybody knows what the rules are. And that was an issue early on is that there were tools that were being deployed, but there weren't any guardrails put around the use of these amazing new tools. So helping people understand where the lines are. 

And what we saw as lawyers, we're used to writing 10-page legally compliant memos, but having a very stark realization that not a lot of people were reading those cover to cover. And so trying to tackle that. 

[0:28:08] KB: I mean, I'm going to start with, yeah, you're mentioning great things about where's the risk and whatever. Imagine someone starting from scratch. What's a risky behavior in this context? 

[0:28:17] SY: Downloading and clicking through the terms on new AI tools without it going through procurement as one example. Using something suggested by AI without a human in the loop. So let's say in the HR context, using a tool for hiring without having someone review what the AI had looked at as another example. Really, do's and don'ts. Where we found it was really helpful on top of that AI policy that helps tick the box for compliance and accountability, but rolling out something that was a little more black and white, to your point. Like, "Here is what you can do. Here is what you cannot do. As a regular employee at a company, these are the bright lines for you." And that was a lot more helpful, we found, than just kicking out a 10-page policy that was then available on the intranet. 

[0:29:18] KB: I'm imagining like a for dummies guide to here's your company's first AI security guidelines or something like that. Where would someone who's new to this space, maybe they're grappling with the changes, they're doing that, or they're getting pulled into a company that said, "Guess what? You're in charge of our AI systems now." Where would they go for this research? Do you have this on Orrick's site? Where does this start from? 

[0:29:44] SY: We have templates that we start from, but I feel like it also has to be calibrated to the organization's risk tolerance. So going back to how risky at the sliding scale of risk, like where the company wants to fall on that scale will inform what that policy looks like and where those do's and don'ts are. 

And we're in Silicon Valley, right? There's a long history of moving fast and breaking things. So we have found that people want to sit on many different places along that sliding scale. So it has to be tied to where a company wants to fall on that sliding scale. 

[0:30:29] KB: That makes sense. Related to this, you mentioned early in our conversation different tiers of applicability, right? You're maybe not exposed until you're some certain size or you have certain number of customers or certain types of data. Do you have in mind what the rough thresholds are there? Like, a company doesn't need to worry until they have a thousand clients. Or what does that look like? 

[0:30:55] SY: It's a really good question because sometimes we have early stage companies come to us and say - going back to our health tech example. We have this NovelAI use case. We're using PHI, like Protected Health Information. That, you have to worry about very early on. That, you have to build into the product from the very start, right? Versus a company that is deploying something that is maybe it's AI for manufacturing and just the risk to people is a lot less. 

And maybe that's a good place to start as well as like a really high-level trigger for figuring out what the legislative landscape looks like is what harm could this cause? Where could this go wrong? Where could it harm people? And we see that that's how AI legislation is emerging. It's really emerging in response to what could potentially cause harm to individuals, to companies. 

[0:31:59] KB: So do you have a a way that you would guide somebody through thinking about that? It sounds like health PHI is way on one side of the risk spectrum. And I imagine way on the other side is something social media, like open public information or something like that. What are the layers of this risk spectrum for someone who's thinking, "Oh, I want to build this NovelAI tool." Or maybe my company is utilizing novel AI tools and we want to do things with them. Where are our buckets here? 

[0:32:30] SY: So I think you think about what are the use cases, potential use cases. What geographies am I in? What kind of data is being used in the context of this model? Are we talking about manufacturing data that's going to be used to further train the model that is potentially lower risk? You may have IP conversations with the counterparty around what data you're allowed to use to train your model. But if there's personal data that's being used, then you have to think about what kind is this financial data about individuals? Is it health data? Or is it less sensitive personal information? And we're just talking about name and address. So keying that to the sensitivity of the data I think is really important and informs where you start. 

[0:33:25] KB: And so as you're going down this road, then let's maybe map out for each of those layers of sensitivity, where would you start? 

[0:33:33] SY: Let's talk about an AI model that is making loan decisions. So you know that this is risky. This is deciding who gets a loan or not. And the consequences are potentially very significant. If someone gets something wrong in manufacturing, maybe a product gets made incorrectly. If AI helps determine if someone gets a loan and makes a mistake, it can materially affect someone's life. The higher the potential impact here, the greater the governance burden that you have. 

And for that particular product, you would look at the existing federal legislation, like the Gramm-Leach-Bliley Act, we're using consumer financial data. Then you would look at the state laws that have emerged around AI and loans as another example. And from there, you would start developing your compliance framework. 

[0:34:37] KB: It definitely sounds like one of the pieces of this is doing your research, right? What laws are applicable to me here? Now, it feels like this stuff is changing all the time. You mentioned you all have like a tracker for like - if I'm trying to find, here's the set of laws I need to worry about. How do I do that? 

[0:34:55] SY: Great question, because this is something that we were having a challenge with at Orrick. Specifically, I was having a challenge with because there were so many bills being proposed at the state level. I couldn't keep on top of it. I was spending the weekends reading all of the new bills and it quickly became apparent that that was not a tenable position. There were, in the last year, about 1,500 AI laws proposed at the state level. And some don't make it, right? Like a lot of them don't make it. So we developed an AI law tracker that just shows the laws that are in effect. And that was really to help guide our attorneys on what is the legislative landscape that applies to our client and their product or service. And then we made it available on our website. 

So we have the Orrick AI Law Tracker, the AI Law Center, and it has a tracker that shows a map of the U.S. and all of the different laws that have gone into effect. So you kind of cut through the noise of all of the AI bills and you just have the laws that are in effect, which frankly, it's still a lot, but it's a lot easier to parse than 1500. 

[0:36:14] KB: Yeah. When it looks like it's filterable by type of scope and searchable and things like that. Now, did you use AI to develop this? 

[0:36:23] SY: It's a really good question. Sadly, no, because AI has a hard time going into the different - and maybe this will get better, but a hard time going into the different layers of the state websites, we found that it was not terribly helpful. So AI was actually initially a guy called Dominic sitting in a room figuring out what laws had been enacted. Now it's a little bit more informed by AI, but it's not entirely populated by AI, sadly. 

[0:36:55] KB: AI in this place being actual intelligence rather than artificial intelligence. 

[0:36:59] SY: Yes. 

[0:37:01] KB: Yeah. These models keep getting better. They're breaking unbreakable math problems, but they still cannot deal with government websites. So if anyone's feeling bad about their ability to navigate government websites, you're not alone. 

Okay. We've covered a lot of different things here. Now, one interesting thing talking now, we're speaking mid-August, it feels like a lot of stuff is kind of - I mean, I guess this is always true in AI, but a lot of stuff feels like it's like taking form now. It's happening now. We've just seen these announcements about, "Oh, we're going to watermark our texts to comply with the EU AI Act." What's the current events going on? If I'm coming in, I'm like, what is all this going on with the EU AI Act is starting to apply? What does that actually cause folks to do? What do they have to do? And when does it become applicable? If I have one EU customer, do I need to worry about the EU AI Act? 

[0:37:51] SY: Yeah. So I think where we are now, it's a really interesting moment because the larger political discussion around AI legislation is really changing. So the Biden administration was very focused on developing an AI compliance framework and ensuring that there was AI governance. And day one of the Trump administration - day one, there was a lot to do. And as we all know, a lot happened on day one. But on day one, that Biden executive order on AI was struck down or invalidated. 

So the administration signaled that deregulation was important, consistent with the idea that the idea was to stimulate innovation and ensure that AI could very freely develop. That has been the case for the better part of the last two years. And in fact, as part of the One Big Beautiful Bill Act, there was a proposal to impose a state AI law moratorium, meaning that the state AI laws would fall away because the administration saw that there's this emerging state AI law patchwork that was hard for companies to comply with. It's tricky. It's regulation. You have to unpack it and figure out what applies and hire lawyers, and it's complicated. And the idea there was that it's stifling innovation, and that's not something that the administration wanted to do. That did not pass, that part of the One Big Beautiful Bill Act. 

But then in December of last year, there was an executive order that was designed to review the state AI laws and do away with those that were overly onerous. And that is still kind of a little bit in process. I'm not aware that any law has been struck down as a result of that executive order. However, in the last couple of months, in fact, since the mythos model was released. 

[0:40:11] KB: Yeah, Mythos, and Sol and all these things. There's been a change in tune. 

[0:40:16] SY: There's been a change in tune. The administration is like, "Oh, actually, I think we might need a law here." There's been a real reckoning in a way of like we don't want to stifle innovation, but we think we need some guardrails here because this is a national security issue, which is not something that I think was really considered before. And now it's very apparent that these models are incredibly powerful. And so there is a sense that there's a responsibility and a need to develop legislative landscape or AI regulation that meets the technology where it is. And right now, the thinking is that we need some better guardrails, especially around incredibly powerful models. And that has been playing out just in the last couple of months. And I don't know where it's going to land, but it's been just fascinating to watch. 

[0:41:18] KB: Yeah. It's interesting. I think if I recall correctly - and obviously, you're going to know this better than I do, but the Biden related stuff was more focused on these questions of fairness and privacy and things like that. Whereas it feels like, at least in the US, the conversation now is very much around security and hackability and all of these different dimensions. 

[0:41:38] SY: Exactly. It's around national security. It's also around competition. So alongside this is the arms race with China to some degree. We want to, in the U.S., have the best, most powerful AI. And in order to do that, we can't restrain development. So there are two things that I think are at play. There's this idea that whoever wins AI will win the global world order. It sounds really like we're really zooming out here. But that, I think, is in the backdrop to all of what's been playing out in the past couple of months. 

[0:42:22] KB: We definitely have that odd stance here in the U.S. where there is this apply the gas and the brakes at the same time, put it in the right spaces. What about that EU AI Act? It feels like that is, as I said, just kind of starting to bite in that we're seeing U.S. companies start to roll out functionality in order to at least nominally follow the requirements of this act. What is their stance? And at what level of usage or involvement do companies need to start worrying about it? 

[0:42:52] SY: The EU AI Act doesn't apply to all companies around the world, but there is extraterritorial applicability. So the extent to which you're deploying AI within the EU, the legislation will potentially be triggered. And it's not fully in effect yet. It has different phases of applying. The transparency guidelines were actually just finalized within the last couple of weeks. And so companies are getting up to speed on the guidelines that are meant to help them understand how to comply. These things are happening in like every couple of months. There's something new that is - deadline that's coming up and guidelines that are released to help folks comply. 

The EU AI Act is a comprehensive AI legislation that's designed to - it's really like a risk-based framework for AI with obligations that depend on what the AI system does, who's using it and the role the company plays in the AI value chain. So there's some prohibited AI, so certain practices that are banned outright, and those went into effect pretty quickly. High-risk AI, which is the most heavily regulated part or bucket of AI with extensive governance and technical requirements. And then there are transparency requirements and then minimal or low-risk AI. So those are the core buckets. 

And people love to hear about the prohibited AI. They're like, "Oh, so what's actually prohibited?" And social scoring is prohibited. I know that exists in other geographies. Manipulative or deceptive AI that causes significant harm. Emotion recognition in workplaces, in schools, which is subject to certain exceptions. And those things, it makes a lot of sense that they would be prohibited. 

And high-risk AI is some of the things that we've already talked about, which are if there's health data at issue, as one example, anything that we talked about a loan AI used in deciding who gets a loan, like that kind of - or who gets a job, those applications are generally considered high risk. For organizations thinking through, they have to go through and figure out what level of AI their technology falls within, whether that piece of the law is in effect already as a sort of starting place for thinking through the EU AI Act. 

[0:45:48] KB: So we're getting close-ish to the end of our time, and we've covered a lot of different things here. I have one more question I want to ask. And then I'd also want to open the door if there's anything we haven't talked about yet that you think would be important. But the last question I have is kind of around, once again, this sort of climbing the learning curve here, both in terms of if you're an individual, where do you start? How do you start going down this? And then if you're a company, let's say you're starting from quite small, what are the stages at which you should really start to worry about up leveling yourself in terms of understanding and applying these different principles for regulation? 

[0:46:25] SY: I would say any company using AI tools should really start thinking about putting in place some really basic protections. So an inventory of the AI tools, as an example, a basic policy around AI use within the organization to hit on the biggest risk areas. Procurement is another one where we see a lot of potential risk there. So thinking about which tools to onboard. And this is a surprisingly complex question, especially for larger organizations, do we already have a tool that does this thing? Is it easy enough for us to use Claude or OpenAI to build this thing? Do we actually need this? Is this application useful at all? Procurement has been - really, really interesting issues. 

And another one that's come up in the context of procurement is a lot of these, the application layer, are brand new companies. They've been around like a hot second. And so in the contracting, they have sometimes immature contracting practices. And they'll say in their contract they'll shift all of the liability over to the customer, to the user. And you have to really think about, "Okay, if I'm using this tool, they've shifted all potential liability to me." Or actually on the other side, maybe they're offering unlimited indemnification. We, new company, will be your insurance policy, but then they've been around six months. If something goes terribly wrong, you're not going to have deep pockets to reach into to make it right, to make yourself whole. So there's that thinking as well, that we have to help clients sort of calibrate. Like, what's the risk here? And how are you actually going to be on the hook if something goes terribly wrong, which is a really interesting discussion. And do we need extra insurance? Because we can't rely on the deep pockets of this company if something goes wrong. So lots of really interesting questions around procurement and lots of risk there. 

So those are key points, I think, are what is the AI that we're using? Do people know how they're allowed to use it? And then in the really early stages of onboarding tools, are we onboarding the right tools? And are we taking on a reasonable amount of risk for the company? 

And then the last thing I would say there, and this has just been incredibly important, is education. So lots of these tools are brand new. The use cases are brand new. And helping people understand, getting them sort of up to speed on what the risks are, what could go wrong, has been a really important part of our work, as an example. And it's kind of like cybersecurity in some ways. It's little and often. It's almost like you get an email now once a week from about, "Just to let everyone know, there's this new kind of phishing attack." It almost needs to be like that for AI education as well, little and often. Just as a reminder, we have these great tools, but here's a rule that we want you to be aware of. So that education piece is really important. 

And then the very last thing is I would just say accountability. Having someone who is accountable. Who's in charge? So that it's not everyone pointing at someone else if something goes wrong. And sometimes you don't necessarily need a chief AI officer as an early stage company, but maybe it's a product leader or maybe it's a group of people, a committee. But somebody needs to own the question or questions that arise and help figure out solutions and make sure that it's workable for the company and help the company move faster because people know what the boundaries are. 

[0:50:38] KB: That feels like a good wrap. Is there anything else that you wanted to cover before we do? 

[0:50:42] SY: No, I think we're good. 

[END]