EPISODE 1816

[INTRO]

[0:00:00] ANNOUNCER: Agreements and contracts are a fundamental innovation and govern everything from personal commitments to major financial decisions. They function as trusted artifacts to capture the nature of a commitment and provide clarity and accountability. Software has revolutionized many business functions, including the basic mechanics of digitally signing an agreement. However, the process of managing agreements systematically and at scale with type definitions, programmatic document creation, and storage schemas remains a complex and largely unsolved challenge.

Dan Selman is the product architect at DocuSign and was previously co-founder and CTO of the Smart Agreements Platform, Clause. Larry Jin is the VP of Product Management at DocuSign and previously worked at Amazon, Microsoft, and Google. DocuSign recently released a developer API focused on fully modernizing and scaling the agreements process. In this episode, Dan and Larry joined Sean Falconer to talk about the frontier of digital agreements.

This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him.

[EPISODE]

[0:01:20] SF: Dan, Larry, welcome to the show.

[0:01:23] DS: Hey, there.

[0:01:22] LJ: Hey, Sean. Thanks for having us.

[0:01:24] SF: Yes, absolutely. Thanks for being here. So, we're talking about digital contracts and agreements today, but before we get into all that, since there's two of you, let's have you introduce yourself. So, Dan, let's start with you. Who are you and what do you do?

[0:01:36] DS: Yes, thanks, Sean. Thanks for having me. My name's Dan Selman. I'm a product architect and distinguished engineer at DocuSign. So, I've kind of worked my way up through the tech stack. I've done most of the engineering jobs as an individual contributor, and now I do a lot more architecture and strategic work.

[0:01:54] SF: Awesome. And Larry, same question to you. Who are you and what do you do?

[0:01:58] LJ: Larry Jin, I'm VP of product management here at DocuSign. I lead a number of areas, but really focused on building a platform for our developers and partners to build and extend our agreement capabilities. Before this, I've built platform products as well at Alexa, at Amazon, and Microsoft as well. 

[0:02:17] SF: Awesome. Well, what ended up, I guess like interesting you in DocuSign and what was particularly interesting about the space that led you here?

[0:02:26] LJ: Yes. First off, what personally led me to DocuSign was the brand, the familiarity with the brand. I had good friends who worked at the company and said nothing but great things about the people in the culture. But what's really kept me at the company, I would say, is I think it's a pretty interesting space agreements and contracts. It's something that everyone deals with at all times. I'm sure everyone signed an agreement or a contract for whether you bought a house or rented an apartment, bought a car. Even things that you don't necessarily think about kind of day to day, like giving consent to share your healthcare record information with one of your providers if you're going to the dentist, if you're going to the hospital.

But it's kind of interesting how little sort of innovation there's been historically on this front and how much opportunity there is just given people deal with it every day, businesses deal with agreements when working with each other, and then contracting with each other. Individuals, like some of the examples that I just mentioned. You're dealing with agreements all the time, but a lot of the processes, even though DocuSign has spent a lot of time digitizing that signing experience and making it easy to sign that document, there's still a lot of parts of the workflow around it that are pretty broken, disconnected. We could talk a lot more about it, but that's really the heart of what DocuSign is doing right now, and I think what's pretty in this.

[0:03:45] SF: Yes. I mean, it is kind of, when you take a step back and you think about like your interactions with business or business to business, things like you said, renting an apartment or buying a house or whatever it is, there's so much contract, agreements, like paperwork involved in that process. It stayed essentially relatively the same throughout like decades and decades of essentially the same paper processes.

[0:04:07] LJ: Yes, absolutely. There's an analogy that I think works really well here. You think about maps, you think about the journey that kind of maps have gone through 20 years ago, if you wanted to go off on a road trip somewhere, right? You'd probably have a physical version of the maps, that giant thing that you would unfold eight times and it'd be the size of - take up your entire dashboard. That's how you kind of navigated and got around. Then we transitioned to digital versions of maps where you could kind of have a digital representation with MapQuest and things like that.

But they would still be kind of clunky because you would still end up printing it and then having a version of it printed offline with all the directions in turn by turn. Then what happened? Then you got mobile phones and you had the ability to use GPS and pull up the map in real-time in directions. Now, you think about the experience. It's so integrated. It's so seamless. It integrates with your apps. You look at the restaurant that you want to get to and it automatically kind of takes the directions. Now, with cars, it'll sync it to your heads-up display. We think about kind of agreements as we've gotten one big step forward in the last decade or so with taking what historically was this piece of giant stack of papers that you would have to read through. You'd have to initial 17 different places you'd have to sign.

Now, we've gone to a point where that representation is digital and you could sign a PDF version of it, which is obviously a lot better, but there's still so much more left to do. I mean, all the steps leading up to creating that agreement, making sure that you have the right templates, the right language, the right clauses. I mean, maybe some of our listeners probably don't have too much experience with having to go create this stuff. It's actually a lot of work and it's super complicated, the exact language and specifics of what clauses and terms you can use or not use.

So, there's a lot that actually goes into that creation part of it. Then there's the actual signing of it, which seems simple again, but it's actually pretty complicated depending on where you are. If you're in Europe, there's some pretty heavy requirements around proving who you are. You have to make sure that you upload a copy of your passport so that you have your identity proven. You might have to notarize it in some parts of the US, depending on the transaction. There's a lot going into signing it. And then the really interesting part, Sean, is what happens after that gets completed, right? What happens after you sign that agreement?

So, most of the time you get a PDF, you have the signature stamped onto it, and then it kind of goes into a box somewhere, right? But in reality, there's a lot more that you have to do with it. You might have to take all that information and you have to throw it into your database, think about like a banking transaction, if you're going to create an account, if you're going to update your personal information, your address. All that information that comes off of that agreement has to go back into that system, into that banking system. Same thing with those business-to-business transactions. If one company is buying something, you're buying widgets from another company. The information about how many widgets or the price per widget, when is it going to get delivered, for how long, all that information has to go back somewhere. And then companies have to go retrieve that information. They have to go find out what got signed, how many units did we buy, what kind of discounts did we get. Depending on what industry you're in, that information also becomes really specific to your industry.

If you're manufacturing, you care about the quality, the tolerance of the manufacturing, things like that. So, that is kind of the entirety of the space. I think what's kind of interesting about DocuSign right now is, we were really good at that first part, which is helping you sign that PDF, getting it completed, making that experience a lot simpler so you don't have to literally download a stack of papers, print it out, and then sign it with a pen. But again, I think there's so much more to do. What we've launched is this new product, this new offering called Intelligent Agreement Management. We won't launch it last year. And it's really to help in all for kind of all different kinds of customers. We have small customers like mom-and-pop shops all the way up to those really, really big ones.

That's why, I'm interested in this space and it continues to be a pretty interesting one that I think, a lot of companies don't realize that there's such a big opportunity here. I won't bore you guys with all the big numbers and like the market stuff, because then I'll catch flak for being the product guy. But it's a pretty big opportunity. I think we're both excited about it.

[0:08:20] SF: Yes, it's interesting, there's, I think this like well-worn path with these transformations to happen when you're going from like sort of paper to digital. The first sort of step of that transformation is like, let's literally take the paper process and make like a digital version of that. That is like a step forward, but it's not like an innovation step. It's like we've gone from physical maps to kind of like a map manifestation on the internet or something like. And then now, we have a like experience with maps that's hard to even imagine interacting with maps in a different way. It sounds like sort of the e-signature part of that was maybe the first step of making this paper process a digital process. But then there's so much more that you can actually start to innovate with.

Dan, there's a lot of essentially money in my understanding is lost essentially like poor agreement management. You mentioned Larry, the Intelligent Agreement Management platform. How does that fit into trying to help businesses essentially reclaim some of that value that are lost due to poor management?

[0:09:20] DS: Yes, that's a huge motivator for most of our customers. As Larry said, agreement management is that sort of tricky last mile for most business processes. Over the last few decades, we've digitized most business processes in the back office. But that last mile, like getting the signature, getting the agreement done has been surprisingly resistant to digitization. That leads to these inefficiencies. So, I think there's something like $2 trillion or something is lost due to inefficient contract management. That comes in a couple of forms.

So, on the one hand, you may be signing things that contain risks that you're not aware of. You can think about things like service level agreements. Perhaps you're signing up to very aggressive service level commitments with your customers, and perhaps you're not aware exactly of what those levels are. Because as Larry said, they're kind of just baked into PDFs and stored on a file share somewhere. It gets very difficult for the human beings to manage all those commitments.

On the other hand, you may be entitled to something that you're not aware of. Maybe your supplier isn't meeting their service levels and you could be getting discounts or better business terms that you're not aware of. That's in aggregate. Every day, millions, if not billions of agreements are signed. They encapsulate these rights and obligations, and when we don't manage those rights and obligations efficiently, then we get this contract leakage, this agreement trap problem, as we call it, a DocuSign. 

[0:11:03] LJ: Yes, I'd love to maybe add another couple of examples. One that comes up a lot too, like the one that Dan mentioned, is renewals. We call it sneaky renewals. You get these contracts that get signed for, I don't know, could be a vendor, could be some software license in your organization for 12 months, right? And then it says, "Oh, this contract is going to auto-renew on January 1st the following year." Then everyone says, "Okay, well, we'll think about it one year from now. We'll to make sure somebody has a reminder in their calendar to go, renegotiate this or maybe revisit, like do we actually need this particular tool or you know software?" Then, guess what happens, no one does that. Then gets auto-renewed and then you're on the hook for another year of spend, whether you like it or not. 

The other one that comes up a lot too is actually having to take the information out of those agreements and then put it back into your main database. I talk to a lot of our customers in like banking, like credit unions, these are smaller organizations that still move a lot of money around, but they're not necessarily the most technology savvy. They're not leading IT firms. So, they end up having people, back office folks, who spend a lot of their time kind of just taking that information. We call the swivel chair. You get information from one place, you swivel to the other part of your screen, another tool, and then you just have to manually re-enter that information. It's like, well, we got you a digital version of the
agreement, but it's really not that much better than taking a physical version and retyping everything into your computer. I mean, it's only kind of a half-step better than that.

So, there's a lot of opportunities to kind of just make those everyday life simpler for folks like that.

[0:12:37] SF: Yes. I mean, you're taking a lot of essentially all these agreements and contracts are unstructured data. It's just a written contract, but they encapsulate a lot of these rules, SLAs, when renewals happen, all these types of things that it's very easy for that to pile up and slip through the cracks. I think historically, like businesses essentially just haven't had a way to like mine that information other than like assigning a person to go and essentially like transcribe that into a software. 

[0:13:03] DS: Exactly.

[0:13:04] SF: So, how are you solving that problem? Are you essentially bringing in some of the latest developments? Generative AI, to help you mine and automatically extract some of that information? 

[0:13:13] DS: Yes, for sure. As Larry said, we kind of think about this as an agreement process, right? You go from wanting to enter into an agreement, so you probably produce some sort of draft or template, negotiating that agreement, signing the agreement, and then post-signature you have to manage the agreement. AI applies to all of those different phases. The one you touched on was probably like, "Okay, we've signed this unstructured thing. How do we get some structure out of it?" There, obviously, all the latest cool stuff in terms of NLP and LLMs is super applicable there.

We have a range of data extraction models that we can apply to unstructured documents and they pull out those semantic elements of agreements. Not just like the names of the parties and the expiration date or something, or the renewal date, but actually the fact that, okay, this agreement contains a renewal clause. It's an auto-renewing agreement. It will auto-renew on this date and we build this very rich semantic model of an agreement based on dozens and dozens of data extractions that we then index and make searchable and expose via our navigator user interface.

[0:14:26] SF: Can you walk me through a little bit about essentially the life of a contractor and what's happening on the DocuSign side? What parts are coming into play in terms of being able to improve the process, but also automate some of this?

[0:14:38] LS: So, there's sort of these three big parts of that journey, the life of an agreement, we call it. The first is the creation of it. Like I said, there's actually a lot of kind of different steps that go into it. Then there's the actual execution or commitment where both parties come together and actually sign the thing. Then there's all the stuff that happens after what you call management, agreement management. So, the creation party really starts with, what is this agreement that we're trying to get signed by counterparties?

That usually starts with a template. So, a template could have been prepared by the legal team or a bunch of people coming together. You author something in Word or Google Docs, and that forms the template in the basis of the agreement. It could be like MSA or sales agreement. It could also be like an apartment lease, right? Usually it's a template. People don't create those every single time you want to get something signed.

Once you have the template, then you need to fill it with kind of the information specific to the person that's signing it, right? You think about it, anyone that's ever signed an offer letter, you can kind of tell which parts are boilerplate and which parts are specific to them in terms of their name, their title, their start date, maybe their offer package, things like that. That's sort of the dynamic information that gets bound in from whatever is generating that contract, could be like Salesforce or it could be like a workday, things like that. Sometimes there's also, you might have to dynamically pick and choose different language and parts. Oftentimes, what state you're in, what country you're in, there's additional regulations. People who are signing employment contracts for a company that spans lots of different countries. If you're based in the UK or based in France, you might have very specific language and additional like documents, additional writers that kind of say additional terms, right?

All of that kind of gets thrown together into this thing we call document generation. Then, depending on whether or not there's additional approvals, it might go to your legal department. So, there might be reviewing and what we call a redlining, which is kind of looking at specific terms. Then literally, lawyers used to take a red pen and used to underline kind of the terminology that needs to get changed because they say, "Well, we don't like this, we don't like that."

So, that draft document then, we'll get sent over to the counterparty and that goes to the second part, which is the actual execution of that agreement, the signing of it, right? When two businesses get together to sign something, agree to buy something, they'll send the paperwork over, and then the legal team on the other side will review it, maybe make some edits, make some revisions, and then eventually get signed. If you're buying a house or buying a lease, you probably won't have your lawyers review it although sometimes you do. But in most cases, you just sign it. You might have to notarize it depending on you know, if you're like buying a house, you might need an online notary.

Then eventually, once that gets signed again, that saved version is kind of set in stone, memorialization of always agreed to those terms that Dan was talking about. And then eventually, depending on what was in there, down the line a year later or two years later, you might need to go back to that agreement. No, this happens a lot where all the time you get some new regulation comes out, like GDPR, VNEX comes out. And then companies are like, "Oh man, is there anything in our contracts that this might actually be problematic? We might have to go back and change things or we have to reach out to our clients and customers that signed a version of our agreement years ago, and we might have to change something." That happens a lot now where the regulation is moving so quickly, new things get introduced, things get rolled back. So, legal teams are always scrambling to figure out what was agreed to before, and then what do we have to go change, or who do we have to reach out to go make sure that we're in compliance.

It's pretty tricky stuff, and you look at every single part of it, and there's just opportunity because we've done a really good job with that one step in the middle, but there's a lot of others that are still pretty manual today.

[0:18:26] SF: Where are you from engineering perspective like applying AI, other forms of automation to help essentially at each part of that? So, you mentioned sort of being able to extract some of those rules automatically through understanding what's going on in the document once you've sort of in the last mile of the contract, but what's happening ahead of that? Are there other places where you're applying some of that type of technology?

[0:18:47] DS: Yes, definitely. First of all, my background is in good old-fashioned AI, which is it's all about rules and process and expert systems. That's kind of how I got into this space. So, if we zoom out, what our customers are trying to do is to digitize these agreement processes. Most of those agreement processes are deterministic and precisely described. We need to do this, generate the agreement, and then we need to send it to the counterparty for negotiation, and then we need to sign it, et cetera.

The first thing that we have is a process designer, essentially, which we call Maestro, that allows the process creators wiring together boxes and arrows in a fairly classical way to describe those agreement processes. Then what we see is, within those agreement processes, there are specific steps where it's useful to transition from unstructured data into structured data. In the front of that process, it could be during negotiation.

Larry and I are negotiating an agreement. He wants this term. I want that term. First of all, what's the difference between the terms? Can we have AI summarize the red line, essentially? That will be a great accelerator for the people that are involved in negotiation. They're perhaps operating at slightly higher-level abstraction. I will see a summary of Larry's changes. Larry's requesting a discount of 70% instead of 65%. Then we get into approval. It's like, well, do I have permission to approve that or does it have to go to a VP or GM or something to approve that level of discount? That's sort of front-end application of our AI summarization differences.

Then there's also the description of some rules. Usually these negotiations, they're not kind of free form, right? When we go into negotiation, we have a set of rules that often is called a playbook. So, when Larry does X, typically, what do we do? Okay, if Larry asks for a discount above 50%, well, typically, we will offer will offer up to 70 and perhaps change some other terms, et cetera. So, understanding those rules, interpreting them as natural language and then applying them during negotiation is, again, is like a key application of AI.

Once with post-signature, we've signed this unstructured document in most cases. It may have been negotiated, it might have deviated away from our templates, and we need to get back to some structured information that we can reliably index and search. There we do fairly classical NLP data extraction. We use a variety of models, some built-in house, some open source, some commercial. As I say, we have dozens and dozens of these models that we apply to these incoming documents. 

Probably the hard part of that from an engineering perspective is once we've extracted the data we need to understand the semantics of the data and we need to land it into a semantic model of an agreement. So, we've built out a very rich definition of an agreement. An agreement is composed of clauses. A clause could have a type. One of the clauses could be a renewal. Renewals typically renew on a certain date to auto-renew you know and so on and so forth. I call it like peeling this semantic onion of the agreement will drill down deeper and deeper into the semantics of different types of agreements. With the ultimate goal being that you can get them into Navigator where you can slice and dice them and run reports on them and say, "All right, find me all employee agreements signed in the last three months where we offered, I don't know, more than half a million RSUs, and the employee lives in California and they have a non-compete agreement more than 24 months." That's the kind of information that we want to give. But at a macro scale, right?

So, if you're a Global 2,000 company signing thousands of agreements a day, you get this cockpit, overall dashboard that gives you that visibility.

[0:22:53] SF: In terms of building semantic model for these agreements are you starting with some sort of base model or you essentially bottoms up sort of reverse engineering that semantic model based on the things that you're able to pull out the entities from NLP, and then essentially figuring out what those relationships are and then continuing to build sort of this ontological structure that describes the agreement?

[0:23:14] DS: As part of the product, we give you a base semantic model that we've developed over the years. That's pretty rich. Then that model has some extensibility mechanisms built into it. I think we ship around 30, 40 standard agreement types. So, things like employee agreements, offer letters, MSAs, sales agreements. But we kind of understand that customers do all kinds of unusual things and they might need to extend that set. So, the ontology is designed to designed to be extended for a given either vertical, which our partners will be able to do, or into a customer account. Customers can directly customize the ontology.

[0:23:56] SF: How do you test all this? You're using a lot of different models. You mentioned summarizations. You're probably using some generative models, but you're also using classical predictive ML along the way as well. How do you combine all those things and make sure that the output essentially is matching the customers in your own expectations and that you're continuing to improve those things.

[0:24:16] DS: Yes, I think that's a great kind of segue or point. It's not enough to just have an OpenAI account and upload a PDF and get some output, right? That's sort of 5% of the problem in a way. Yes, we have a very mature kind of data pipeline, AI/ML practice you know that is assessing is assessing model drift and has a pretty extensive kind of test set of documents that we use. Then we built in some direct customer feedback about extractions into the product itself. So, if customers disagree or agree with the data that we've extracted, they can send us a signal that we will monitor to assess our models and improve them in the future.

[0:24:57] LJ: I'll give one further example of how it's actually, the kind of human feedback and reinforcement is pretty naturally woven into the agreement kind of process. What I mean by that is if you think about that legal kind of negotiation scenario that Dan was mentioning earlier, oftentimes your legal department will get a contract that some external party is trying to get you to sign. What we have is sort of this AI assist technology that will suggest revisions, even changes in language and terminology based on that playbook concept, based on kind of your internal rules about what's acceptable, what's potentially risky. That's an opportunity for us to iterate and present new options, new language. Because it's a sort of legal persona who's looking at it, they're the ones going in accepting those changes or rejecting it. So, that's a pretty natural kind of signal the accuracy of it, it's almost similar to classic search results. Which ones did they select? Which ones performed the best? Was it the top result that we gave back, or was it the fifth or sixth one?

That's just another example where Dan's point is some of it is obviously using our own golden set and being able to test for improvements or regressions, F1 scores, or it's using our users who are kind of naturally giving a signal on how well our models are performing.

[0:26:16] SF: In terms of that feedback loop that you're getting from the people who are using this, are you able to essentially customize that at a customer level, or is this an aggregate customization that essentially my feedback might influence the model wholesale across all customers?

[0:26:30] DS: I would say we're closer to the beginning of that journey than the end. We do support custom extractions. That's where customers are defining their extractions. I don't know, maybe I'm a company in aerospace, right? Most of my contracts talk about aerospace engines or something. I have an internal taxonomy of different types of aerospace engines that I sell. So, our customers can bring those extractions into the product, and we will use them. That's incredibly powerful. I think we want to do more in the future. But that's where we are right now.

[0:27:03] SF: In terms of the testing, you mentioned having your own documents that you can test with. Are using any sort of like eval frameworks that exist for AI today or is this all stuff that you've built custom?

[0:27:16] DS: It's fairly custom. These ML pipelines are very custom and I would say sort of a sidebar, our biggest asset as DocuSign is our brand and the trust that companies place in our brand. So, we take this issue of sort of data governance and the data that we use for evaluation and testing incredibly seriously. We have a very straightforward opt-in AI training policy, so we will only use customer's data if they explicitly opt-in. We will go to great pains to anonymize and aggregate all the data.

We've had to build a lot of our own custom pipelines to support that sort of functionality at our scale.

[0:27:57] SF: I think that makes sense. I think that's consistent with what I've seen from most people who are doing this in production for real customers. I think a lot of the tooling infrastructure is just not there yet, especially for probably doing something at the scale that DocuSign is doing this at.

[0:28:11] LJ: Even though I would add, some of the industry benchmarks around what constitutes good anonymization, being able to recognize what's sensitive information and being able to ensure that that doesn't feed into your training system. Some of these benchmarks are relatively nascent and honestly, the industry ones maybe aren't as high as they need to be when you kind of think about the level of trust that your biggest enterprise customers and very sensitive industries like healthcare, financial services, manufacturing, there's a very high bar. I think as a general, the industry, and whether it's heuristic like an F1 score or the definition, what's your accuracy in anonymization, some of this is I think is still getting established. I think internally, we try to have a very, very high bar because as Dan mentioned, the trust in the brand, that kind of that DocuSign brand is just so important to what's made us successful.

[0:29:09] SF: I think also when it comes of like anonymizing sensitive data or detection sensitive data is very context-dependent. There's obviously the sort of Paris Hilton problem of like Paris, the person versus Paris, the city. But I think it goes beyond that of like if I have a contract, I can have things that are particularly sensitive to my business and the people who are involved in this agreement that is not necessarily sensitive to other people. I don't know, the formula for Coca-Cola or something is clearly something that they would want to protect. That's very, very context-dependent in that company. It's a very hard problem.

[0:29:41] DS: A little side note, OpenAI open-sourced a set of chat logs for researchers a couple of months ago. I think it was two, three months ago. It feels like years ago in this crazy AI world we're living. But if you actually trawl through those logs or search them, you will very quickly find fairly sensitive data in the chat logs, even though they've gone to some effort to anonymize them. So, as you say, it is a challenging problem.

[0:30:06] SF: Well, I know you worked on Alexa, I worked on Google Assistant, like I used to say, 
when it comes to like chatbots, conversational AI, like chatbots essentially see everything. They're like a keyboard. You have no idea what someone's going to put into those things. So, it's very, very difficult to keep sensitive information out of it.

In terms of thinking about the extensibility of everything that you're building. A lot of this is built on APIs, developer platforms, developer ecosystem. Why was it important to focus on making this something that people can essentially take these building blocks and extend and build their own experiences around?

[0:30:40] LJ: Yes, it's a great question. Obviously, important to this audience, kind of make it a little bit more relevant. So, it's interesting, e-signature in general, but definitely, DocuSign specifically, a lot of the use of it historically came from our developers and using our APIs, right? You think about e-signature, it's not a product like a Slack or a place where everyone goes to. A lot of the times, signing a document happens as part of a workflow that originates somewhere else, right? If it's like an employee offer letter, it gets generated out of something like workday. If it's a sales contract that you're trying to get your customer to sign, it probably came out of a CRM system like Salesforce or HubSpot. 

So, developers played a really important role because they would effectively integrate our APIs, or we also had language-specific SDKs. We also had iframe embeddable versions of that signing experience that developers will integrate into their custom product. It could be part of software they sell to others, or it could be some internal solution they're building for their own organization. So, we have a long history of that. Like I said, about 50% or more than 50% of our documents that get signed those transactions were actually API generated in some shape or form. 

So, kind of thinking about the future and a lot of the problems that we wanted to solve with the technology that Dan and I were talking about, whether it's AI extraction, understanding of agreements, whether it's helping you create and negotiate agreements. It's really important that we recognize every customer has a slightly different need. Depending on what industry you're in, depending on what country your business is in, whether you're a small customer or a big one, there's just so many different variations of agreements, the data structure, the workflow and process around it, what degree of trust and authenticity of the data you need or the identity of the signer.

There's all these kinds of points of extensibility. There's the need to extend our UI and product surface area. There's a need to extend the workflow with custom steps and data input-output processing or file processing. And there's also extending the actual semantic model of the agreement itself with additional rich attributes. Because if you're in manufacturing, you want to be able to express key terms around quality and tolerance and units shipped and all of that stuff. You want to actually have those attributes in your data model for your specific agreement definition so that our AI is able to extract and recognize those.

So, as we kind of built out this new Intelligent Agreement Management platform, we thought about all these different scenarios and use cases where developers would want to come in and be able to build extensions to our core products to be able to support those kinds of use cases. We kind of modeled our APIs after that. For example, we have the ability for you to integrate basically an external service into our workflow, kind of similar to like an Alexa or Google skill. Whenever something happens in our system and we need to go call out to a third-party system to pull in some additional data or verification of an identity as an example. We have an extensibility model where developers can kind of build a service in a programming language of their choice, host it in a cloud provider of their choice, and then they would basically give us some metadata so that we can call into that external service. Almost, again, I'll kind of use the smart assistant analogy. It's like building a skill that can plug into our framework.

[0:34:05] SF: In terms of, from a design perspective, figuring out what the right abstraction layers are. How did you think about that? You want to essentially take enough work off of somebody's plate to make this worthwhile of getting value from these systems, but you also need to have essentially enough flexibility that you can support all this context-dependent either like regulations that they might have to deal with or workflows. How do you work through that design challenge?

[0:34:30] DS: Yes, it is a challenge, right? And DocuSign been around for a couple of decades. So, as Larry said, we have a lot of experience of solving our customers' problems. I think it is a stack approach, essentially. We have lower-level primitives, SDKs, and APIs that are extremely flexible and extremely embeddable, but they come with a pretty significant learning curve. For example, if I wanted to build a completely custom app and embed sending a document and signing on a web page in that custom app. We have all the tools and SDKs for you to build that custom app. Maybe you're building your new Neobank and you want to do KYC as you're onboarding new customers, and there's probably some regulatory documents that you need to sign, and you will have a completely embedded experience.

That's sort of the lowest level primitives. Then above that, more recently we've added features like Maestro that allow you to do a lot of those common tasks but in a much more declarative way. So, instead of writing like node.js or C# to orchestrate those tasks, I'm using boxes and arrows and I'm building a process definition. And then our process engine will run that process definition for you. One of the steps in that process definition could be a call to our document generation API, or you could call the document generation API yourself by hand. We kind of have that sort of Russian doll approach, which is really grounded in the problems that our customers are solving and their use cases.

We want to make the 80% super simple as declarative as we can, five minutes to wow, whereas we do acknowledge that there's always that hard 20% where you need to drop down into the code and you probably need to be a more technical user to build those super customized experiences.

[0:36:19] SF: We started this conversation talking a little bit about making this analogy to maps and evolution that's happened there. If essentially, e-signatures is maybe the MapQuest version of this, then now with this investment, you're perhaps in the early days of Google Maps, what's this look like 5 to 10 years from now in terms of the future of agreement and their management and making this, essentially cutting into this like $2 trillion that businesses are losing because of poor management of these things?

[0:36:47] LJ: Yes, well, first off, if we can get a small percentage of the two trillion, I think we'd all be happy with that.

[0:36:52] SF: Yes. I mean, I'm not asking for 10% here, just a small fraction of percentage.

[0:36:57] LJ: Yes. We'll take a small cut of that. Yes, I mean, I think a lot of it is around automation. And you think about kind of this convergence that we're seeing between Generative AI and also automation of business processes and workflows, and obviously agents is kind of the hot topic these days. You look at every major enterprise company, super, super gunk hoe about it, right? Whether it's Salesforce or ServiceNow, Workday, Microsoft.

I do think, putting aside kind of the marketing hype machine aside, I think there is something real to be said about the quantum leap with Gen AI and the application to business process automation. I think it's pretty substantial because in the past, the big challenge was, it wasn't necessarily that you couldn't automate stuff. I mean, lots of vendors out there have built workflow automation, BPM, iPass, pick your acronym. But I think the challenge was always, you need to have someone sit down and really think and map out, well, here's what my business process looks like on paper and here's the boxes and arrows and let me basically map that like into this tool and use the tool to go create the digital 
representation of it.

But it's a big effort. There's a big learning curve. As much as you want to like democratize that and have Sally who's an operations person in HR be able to do that, it's pretty tough. People have their day jobs and for them to want to be effectively like almost like developers to go and do that is pretty tough. But I think what's really interesting with Gen AI is really being able to take a very, very natural language sort of prompt and here's the problem that I want to go solve and using kind of agentic reasoning and be able to figure out what steps to go execute, potentially even what external sources of truth and knowledge, or what external systems and APIs to call into, to go and carry that out in kind of a multi-step fashion.

I think it's still early, but I think that's what's really, really exciting. And if you think about the agreement process and all the things that we talked about, these are pretty complex business workflows. Some of these are the most complex that exist in big corporations, even small businesses. So, making it easy for people to be able to automate those, all these complex processes using agents in Gen AI, I think is pretty interesting. So, I think maybe 10 years from now, a lot of the steps that we mentioned could be pretty automated. That doesn't mean that there isn't work for lawyers and for other people as part of the process, but I think we could take a lot of those manual kind of wasteful, inefficient steps out of the equation.

[0:39:30] SF: Yes. One of the examples you mentioned earlier was the idea that there's some sort of regulatory change. For example, India just passed DPDP is now being enforced there. That's probably going to potentially have some sort of shrapnel damage to existing contracts. Now, the state of the art today is I might have to go pay somebody to essentially review all those existing contracts and see, does DPDP, is there anything we need to change here or take into account? Like have an agent go do that work for me and then have essentially an expert in the loop to evaluate whether the what is correct or not. That's hugely valuable.

[0:40:03] DS: I plant a seed maybe, as we're thinking about the future, right? Lawyers, they talk about this concept of a company as a nexus of contracts. A company is just a set of contracts fundamentally, contracts with employees, with suppliers, with the government. I think traditionally, we've thought about those contracts as very static things, passive things. They're on pieces of paper in a filing cabinet traditionally. We're transitioning now, I think, in the coming decades to a mode where those are active running entities, agents, programs, whatever you want to call them.

So, I don't know, if I have a supply agreement with a company and that company declares bankruptcy, that information should enter my enterprise and ripple through my whole enterprise. That's a risk event, right? I think we're now able to build systems that could be much more reactive like that with, as Larry said, human oversight, kind of human in the loop, but we'll get those signals in almost real-time. Or I don't know, there's a typhoon in Southeast Asia and it's going to impact our delivery schedule from our factory there. How does that impact the SLAs that we have with some of our customers? Can we get ahead of that situation? Reach out to them proactively as opposed to them getting angry that we didn't meet our commitments and then suing us. It completely changes, I think, the way that we think about the role of contracts in the enterprise.

[0:41:31] SF: Yes. In a lot of ways, these become less static assets that are shoved in a file cabinet somewhere and these become essentially more of a living changing artifact over time as these changes happen, updates can happen. Or they essentially impact business processes as there's new information available.

[0:41:50] DS: To your point, you could have an agent that's just sitting there, monitoring Indian government privacy regulations. It's just sitting there doing that all day every day. Then when it sees something interesting, it could kick off some work to replay personal contracts.

[0:42:04] SF: So, what's next, I guess, for the Intelligent Agreement Management in a more foreseeable future the next like 6 to 12 months?

[0:42:13] LJ: Yes. Got to ask the product roadmap question to the product guy, I suppose. I think it's still early days and realizing a lot of the stuff that we talked about, where I think we've spent a lot of time building some of the foundations over the past year, some of the no-code workflow automation, AI-powered extractions, and being able to do that at high scale for all kinds of different types of agreements, building out also integrations to a lot of the major tools and vendor software that customers typically use like CRM systems and things like that.

So, I think what you're going to see this year is really kind of continuing to solve different parts of that agreement lifecycle. We're investing in agreement, creation, preparation, and simplifying that. We're also going to obviously invest more in our AI understanding and extractions. You could always try to add more extractions and more out-of-the-box understanding of different common attributes. But of course, investing in the extensibility of it, letting customers be able to define those custom attributes and extractions, package them up.

Then, I think a big part of this continuing to work with our developers and get them excited about what's coming. We had an event last year, late last year, where we kind of relaunched a lot of what we're doing for developers and new APIs, new tools. This year, we're going to continue to do that with participating in developer events, having hackathons, and then kind of showing our developer ecosystem that there's a lot more here than just calling an e-sign API to send out a document for signature that there's actually a lot of cool stuff that we're doing and expose it via APIs, whether it's some of the extractions and that semantic data model, actually providing APIs for developers to be able to integrate that and maybe hook it up to their BI or visualization system.

So, I think there's some cool stuff coming on the developer front and then just a lot more on the core kind of product front, and solving some of those problems around the agreement process that we talked about. 

[0:44:09] DS: Yes, I just jump in as well. The other thing that kind of keeps me fascinated with this space is that agreements are contextual, right? The way that you sign an agreement in Japan is not the same as the way you sign it in North America. Obviously, AI extractions are heavily language-dependent. We want to cater to all of our customers in all markets. So, you'll see a lot more focus on non-North American data extraction and semantic modeling of agreements, which is very important. 

[0:44:38] SF: Yes, absolutely. There's, today, a heavy bias to Americanized English for a lot of these systems just because that's where the original drive of innovation is coming from. But that's certainly a fascinating area and a lot of work to be done there.

Dan and Larry, I want to thank you so much for being here. This was great.

[0:44:54] DS: Thank you, Sean.

[0:44:55] LJ: Thanks for having us, Sean.

[0:44:56] SF: Thank you. Cheers.

[END]