EPISODE 1777

[INTRODUCTION]

[0:00:00] ANNOUNCER: Palantir Technologies is a data analytics and software company specializing in building platforms for integrating, analyzing, and visualizing large data sets. The company's tools are designed to help analysts and decision makers collaborate on data-driven solutions to complex problems, and they have worked extensively across the intelligence, defense, and commercial sectors. Akshay Krishnaswamy is the Chief Architect at Palantir, and Chris Jeganathan is a Group Lead at Palantir. They join the podcast to talk about the evolution of Palantir, its technology, the AIP platform, and more.

This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him.

[INTERVIEW]

[0:00:52] SF: Akshay, Jeg, welcome to the show.

[0:00:54] AK: Thanks, Sean. Thanks for having us.

[0:00:56] SF: Yeah. Awesome. Akshay, you've been at Palantir a long time, I think, around 12 years. I wanted to get a little bit of background on the company, its area of focus, how it's changed during your time for anybody who maybe is less familiar with Palantir.

[0:01:12] AK: Yeah. I think it's always a tall order to try to summarize things, but I guess being here for 12 years makes me a bit of a Palantir boomer, I think, in at least relative terms. Maybe just to give the very abridged version. The company was founded in the aftermath of September 11th, very much focused on a set of counter-terrorism missions across the US and allied governments. The principle problem at first was, how can you solve the fragmented data problem to be able to make better decisions in the context of certain national security missions?

It turned out, there was a lot of technology that had to be built around data integration and being able to enable frontline analysts and soldiers to be able to make better decisions. The core technology there around being able to integrate data and then link it to operations ended up being this common theme, which took us throughout other parts of the government sector. Now, we're all across not just the defense side, but also, veterans affairs, CDC, working with the NHS in England, which Jeg probably knows more about than I do. But also, that kernel of technology and the ontology concept and system that we built from there took different renditions as one of the commercial market, first in the financial sector, other areas where it was a lot of regulation, a lot of fragmented data, and then into domains in manufacturing with Airbus, BP, etc., and all across the commercial sector since then.

It's been this process of being proximate to problems, seeing the complexity in them from a data and an operations perspective, and then building these platforms. Kind of, you could argue backwards from that.

[0:02:36] SF: The company itself has been around, as you mentioned, founded after 9/11, so I think it was 2003 is around the official founding day. Now, over 20 years, this probably started as I would suspect, an actual desktop application and then moved over time to the cloud. There's been a lot of transformations that had happened from the technology standpoint in order to essentially modernize, as well as the business standpoint where you started to move more commercial outside of government.

[0:03:01] AK: For sure. Maybe I can give a little bit of that boomer context, and Jeg can give a little bit of the modern version of how we've evolved to the current stack of technologies. Yeah, you're right. It started off as a set of Java services and a Java swing client. I still remember one of the first things we had to do early on was like, how do you debug the runtime errors on people's particular machines to get them to actually use the thick client correctly? There's a whole different set of issues when you have people on rugged laptops running all the services local in a disconnected environment, that's like, what's the boot sequence? How do we get the logs off this thing? If it's a security gap, how do you do that?

It's like, a lot of the infrastructure we built to enable more flexible deployment options and also the current experience of using the platform largely through a browser-based set of interfaces came from a lot of the pain of saying, okay, how do you build these things early on using the technologies at the time? There's a whole journey there, and maybe Jeg can talk about this too, of transitioning from those technologies where we built a lot of sophistication for those tool sets into more modern tool sets over the past several years, while not leaving people in a lurch who had all those requirements build up over the years. Maybe, Jeg, you can add some color to the current stack, too.

[0:04:07] CJ: Yeah, I find that, because I joined about five years ago. I remember even in my onboarding, we were still using the Java Swing client to use the product. That swing client has only very recently been completely removed from our product. We've moved to a more standard web stack, but it's been quite interesting from this idea of having all these services running on a user's end laptop to us now thinking like, one of our customers somewhere probably has a 2013 ThinkPad that has no memory to it. How do we get these products to run on any device that our users are using?

Then I often joke that we've gone full circle, that we were talking about how we were moving to the cloud. Now, you can see our software running on trucks on the edge and in any environment that we can get them to. Ultimately, yeah, we've definitely moved a long way. I never got to experience the golden swing days myself. Everything we do right now is web-based. But still, we are now coming back to that story of, can we run in any environment, whether it's on the edge? I think that's been consistently in the DNA of what we've produced.

[0:05:15] SF: Yeah. I wrote more Swing code that I'm proud of and glad that those days are now behind us for the most part. I would think that, one of the update cycles, like when you're deploying some of these things to government and military and these scenarios, I think when I think about government, a lot of times, I think about they're maybe cautious, they're not necessarily going to do the latest update, because if something's working, there's always risk inherent with updating. Are you always in a situation where you're dealing with an older hardware, older operating system, these types of constraints?

[0:05:46] AK: Yeah. Maybe I'll again turn over to Jeg to give the more detailed version here. A lot of, I think, the complexity there that you talk about, Sean, is why we built our Apollo platform. Even about the user facing platforms, historically, first it was Gotham, which has evolved now into a set of defense offerings that we broadly call The Defense Suite. It's the core Gotham workspace, which is now web-based. It's meta constellation, which is more satellite imagery oriented and a few others as well. Then there's Foundry, which is the broad-based platform, which now even those defense offerings sit on top of and there's AIP.

All of these are microservices architectures, which are now backed by the Apollo platform, the continuous delivery platform. I think continuous delivery is one thing, right? How do you shift code incrementally? How do you deal with somewhat different constraints in environments? To your point, you have regulatory reasons why you can only upgrade in certain cadences. If there's a complete tool chain of additional requirements that have to go and validate the binaries you're shipping beyond that, how do you still get the efficiencies of scale without having to build one-off forks of your delivery platform for every single customer?

Then, how do you deal with the fact that you might have to do all of this continuous delivery in a disconnected hub that exists in the classified network, or somewhere else? What we try to do is get as close as we can to what we can do for commercial customers typically, which is just transparent, SaaS-based commercial delivery, right? Or continuous delivery of software updates for every part of the service mesh.

Apollo allows us to then say, well, what are the risk tolerances you want to impose and how do you want to make that happen in, let's say, a more sensitive government environment, or even a manufacturing environment in commercial, where you might say, okay, there's blue-green deployment channels, of course, and there's ways in which we can have soak times for new versions of services. Then there's additional checks that the agency, or whoever else has to do on top of those new service versions. Then they can get automatically promoted to a new branch, or a new version.

Essentially, this was all us doing this in the field as forward to point engineers, right? We had to build, as our CTO calls it, robots in the software instead to do this stuff. Actually, what's cool is Apollo as a platform that does all of these different requirements, gating, promotion, rollback, security patching, etc., is now being used by some of our commercial partners, like Cisco and others to deploy their own software in these complex heterogeneous environments. Maybe, Jeg, as one of our development leads, you can give a sense of what the life is like now at the Apollo working at scale.

[0:08:03] CJ: Yeah, I have to admit, Apollo makes my life ridiculously easy. I don't really have to think about what I'm doing as a developer. For example, on our front ends where I normally spend most of my time, we make a cut of our code every Monday and Thursday, and then we have various soak times in different environments. With Apollo, I can very easily go speak to one of our customers. They'll say like, "Oh, when am I going to get this feature?" I can predictably say, "Oh, have a look on Tuesday. I'm pretty sure it'll be there." We can fast forward delivery whenever there's a requirement from the customer. But we can also freeze their stacks if they need it.

All of our customers have different requirements. As we were saying, that tolerance to risk where, I don't know, if you're just developing on the platform and you just want the latest thing, we can get you new versions of code within two days of it being written. But in the environments in which you have changed freezers and all of these SDLC processes, Apollo really lets us capture exactly what that customer needs. The developers on our side are just coding, getting their stuff shipped and then worrying about the code getting to the customer very little, just having them their timelines. It's a very easy life.

[0:09:17] SF: Then, in terms of given these constraints of some of these environments that you have to deploy to in the world of government, or military, have working with those types of agencies under those constraints, helped inform parts of, essentially, the commercial product and how you think about building products there, or even had been able to solve things for a commercial company that you wouldn't have been able to solve if you hadn't had that experience?

[0:09:41] AK: Yeah, absolutely. There are a couple of things stand out. I think early on in the government space, it was there were two pillars to the technology early on. There was the data integration piece, which was like, how do you deal with all these different forms of data coming in, structured data, unstructured data, even at the time, like it was documents and imagery and things like that. How do you reconcile them together inside of a common, essentially world model, like an ontology that you're building? You have a common substrate that these technical teams are working on to enhance and then is immediately accessible to different types of business users for operational teams.

That then necessitates the second thing, which is having a security model that can be super granular, right? It's not just the fact that I have these objects, or these links, but it's which version of the object, like maybe Jeg added this person's eye color is blue and I said that it's brown and you're our manager, Sean, so you can see both things, where we can't see each other's and it's the same object, ultimately.

All those things, I think, through architectural pain, I would say, we got to a point where we understood like, how do you build some of those core frameworks in place that can be generically applicable? For modeling, like mortgage fraud, or oil and gas work, or the A350 assembly, it's like, you're continuously learning, okay, there are some of those lessons from the integration space, from the security space that make it possible now for us to function at a place like Airbus, which has operations put across Spain, Hamburg, Toulouse, and has all, probably the most regulated manufacturing environment in some ways in the world.

Then I think from those environments too, I'd say vice versa is true. When we say, okay, at a place like Airbus, or at a place like the Cleveland Clinic, you see new requirements around how the applications have to be able to capture user context and then feedback to the ontology and allow technical users to reconcile those changes, that then gets shipped back. Because it's all part of the same platform, the government users benefit from that work, right? I think it started off in the direction you're talking about. What's cool to see now as it's the flow lines come in from both sides of the business. Maybe Jeg can add to that, too.

[0:11:34] CJ: Yeah, I definitely saw this and I was talking to a customer two weeks ago, in fact, and maybe I don't have the right picture with this customer. Was there like, do you know what the thing that you clearly learned from government is that you built everything with permission as a primitive? I was like, "Oh. Well, I've not really thought about it, but it makes a lot of sense." Because it's just how we think day-to-day that permissions were the first thing that entered our platform, because we were looking at government users. Then we added the feature set on top, where a lot of other companies are doing features and then working out how to apply security around them, and even the way that we talk about how we build the primitives and the concepts within the platform, like the language that's used internally is always around the security model and our provenance systems.

I think the fact that our whole system is grounded in this concept of provenance, like for every piece of data within the system, wherever you're using it, whether you're using it in AI, or in data integration, or in our ontology, the first thing we want users to know is exactly where that data has gone, where it's being used from, and what the lineage is. We absolutely love our graphs here. It makes a huge difference to one, the way we make our product, and two, the way our users can consume it and apply their own software development, life cycle processes, all of the things that they care about once they're starting to scale their systems up.

[0:12:57] SF: Yeah. I mean, I think that it sounds like, you took a very first principles approach to making this secure by default, because you had to, essentially, based on who you're selling into, but it's very difficult, I think, for companies that are maybe coming from the commercial world, and then hadn't really thought about that, and then they try to layer these things on, and I think that's where they end up getting in the trouble from a security perspective, because it's just very difficult to do that after the fact and do it correctly, unless you really put the investment in.

[0:13:24] AK: That's right. I think even being candid about our own journey, it's like, we built an initial security model in the first versions of Gotham that scaled so far. It had this idea of role-based controls, more granular controls, within the expressiveness of that current ontology, which is mostly objects, links, properties, and things I would say are more semantic elements in the traditional sense. You expand that now to the kinetic behaviors you're modeling to. It's actions that are backed by arbitrary logic. Like Jeg was saying, it's like, we need to be able to model holistically an AI technique, or function as interacting with any of that stuff.

It required us to actually upgrade and build new architectures under the security model over time. I think we all remember different points in the scar tissue of oh, that was when this particular iteration didn't properly scale. Or, it was scaling up to the point, but now when you really turn the spigot on, when it comes to users doing these sorts of operations with much more granular statistics or something, that suddenly blows up the count of this type of security tabulation, like we have to think about how to address that again.

I think, because we have context, I think, frankly, that have always kept us honest about you can't release a new feature here, unless it abides for the security model, that forces us to keep working on the underlying model and the provenance service and all these things, like Jeg said, because we literally couldn't ship these things otherwise.

[0:14:39] SF: You mentioned ontology a couple of times, and that's really how Palantir originally got on my radar was I was working in the world of ontology during my PhD and postdoc work, but outside of Palantir, I think no one outside the academic world, I think, relies on ontologies at the same level as maybe Palantir does. Maybe, can you give a little bit of background on how you're using ontologies and what's that help inform within the Palantir platform?

[0:15:04] AK: Yeah. Maybe I can give, again, the history lesson here, and then Jeg, I'd love your take, too. It's like, I think only a company that had no marketing team for 20 years could have probably built the entire technical story around a term like ontology. I think there are people like yourself who appreciate it, but it's always an - I'd have a philosopher as our CEO and founder, but it's like, I think it came from this recognition that if we're going to integrate data and we're going to make it accessible to operational teams, what is the commonality between, essentially, the technical work, or the analytical work and the work that's actually mission driven?

I think the idea was like, if you represent the world, call that an ontology. It started off by saying, okay, they're the people, the places, the things, the objects, the links, the world that I think was honestly first there to help people with investigations, what you might think of as read heavy operations with some amount of right associated with it. I think what was interesting, and I think what we probably couldn't have predicted, but the additional mission context, and really this was the transition from intelligence workflows to more forward deployed, warfighting workflows, much more operational, much more kinetic, and then the work in the commercial sector was much more industrial and manufacturing oriented, forced us to evolve what that ontology system did.

Actually, the Foundry platform took the original Gotham ontology idea and actually re-implemented it, and now that is the common ontology system for all the platforms, but it expanded on the expressiveness of those core objects, links, and traditional read oriented elements with these ideas of co-modelling behavior as well. I think the way that we think about it is like, it's a decision-centric system now, and that's buzzwordy, but what we mean by that is like, it's modeling the data, the logic, and the action together. It's like, what is the information at my disposal from looking at a supply chain reallocation strategy? It's like, there are semantic elements there that make up the world of the packages, the customers who might be shorting me on the supplier side, the customers, etc., like on a graph. We love our graphs, as Jeg said, or something else.

Then, it's like, what can I now do about this? It's like, there are sets of behaviors here that maybe come from business logic in my ERP system, or maybe I've defined the rules for inside the Foundry system that I want to be able to then evaluate, and maybe those are contextual. There's the graph, it's the data graph, and now there's the state machine of what I can do. This is, I think, part of the overwrought description, you'll sometimes hear from Palantir, of what is the ontology? It's like, well, there's a data element, but also there's this workflow and state machine element and what you're defining in the behaviors. Then those two things can be represented to AI systems as well, if you want them to help your orchestrate or do any part of that process.

It ends up being, I think, this combination. I think the way I would describe it to most folks is it's like the object layer that, I think, people talked about in the era of Xerox, Park, and Small Talk, or if you think about Sun and Next in the 90s, it was like, what does an enterprise object system look like in the grandest sense of the word? I think that's what the ontology feels like to me now.

[0:17:50] SF: I'm assuming you have probably some baseline ontology, and then on a per customer basis of building, essentially a instantiation, it's like an abstract class and then someone's building their own defined class for their specific use case and their understanding of the world and their business requirements.

[0:18:07] AK: Yeah. Again, just being candid, it's like, we've gone through many iterations of what those archetypes, or those abstract classes could look like. I think, we're always skeptical of them, to be honest. Because you show up with a starter pack. It's like, here's the ontology and of course, if you're working, I think you come from a bioinformatics background, there are disease ontologies, or things that you just need to have if you're working in certain spaces that are table stakes. I think we always think about those as a starting point, because what you're really layering in because of the combination of data and behavior is the specific workflow dynamics of your enterprise, which are always hard to think about in the generic.

I think, we always see like, here's a starter pack and you're doing quality work in automotive, you're doing flight work in aerospace, but no, it's just a starting point for you then to be able to evolve and make it your own through a use case or two. I think we're probably, we come in and say, "Cool. Let's start with this default starter pack." The first ontology will emerge once we've implemented our first use case is how we think about it, typically. Jeg, I'm not sure if you have other thoughts there.

[0:19:05] CJ: No. I think you hit the nail on the head, that if we often just come in with what we think a business will look like, even if it's been successful with another business, I've always seen that ontology be mutated in various ways, because the businesses we arrive at and the customers, they already have a way of thinking a way of working and their own articulation of their world. For one automotive industry, they might have a different ontology explicitly to another. We can try our best, but we're not prescribing exactly how to run your own organization.

[0:19:40] SF: Got it. We've been talking about some of the history of Palantir, or some of the transformations that have gone through from originally very defense and government focus to being commercial, the dairy or some of the platforms that you've built. I think, feels like to me that you're going through another transformation now to becoming an actual developer platform with things like, AIP, which is your AI platform. I mean, how would you describe AIP, just to start with?

[0:20:06] AK: Yeah, maybe I can take that one. Then Jeg, you can talk about the developer platform piece. AIP was very much the recognition of like, if we think about the existing set of services we had built in the Foundry platform around data integration, around the core ontology, around the application building layer, it's like, hey, a lot of this is it revolves around this idea of modeling the enterprise, this ontology system, which is modeling, like we said, the objects and the data, but also the behaviors.

I think AIP is the idea that you can connect all sorts of generative AI models, or that was the conviction. You can take the power of these emerging models and we had seen some antecedents in the government space that were not quite what we saw in the commercial sector, but gave us a hint of where things were going. It's like, well, if these models have these reasoning capabilities, they're going to need to interact with the data, the logic and the action in the enterprise to be useful. It might be helping us just structure data, it might be helping us, essentially, in a co-pilot function look for different types of pathways we might evaluate in the workflow. Then maybe at the limit, it's actually an agentic workflow, where it's working across the entire state machine, or finding new avenues within the security boundaries and limitations we would affix to users, right?

I think it was this moment of somewhat semi-serendipity of saying, well, a lot of what we had built because of this very heterogeneous collaborative model of different types of users interacting with this ontology across enterprises we had been working with, that can be just model AI as another user at the table. AIP was the entire set of infrastructure needed to enable that. It was the language model service we built to be able to interact with all sorts of different models securely. It's like, we love our partners at Azure with the open AI services and anthropic through AWS and the Google Gemini models. We have our own hosted model hubs for the open-source models. You can bring your own and register them. The idea is like, can you provide the platform? I mean, this gets the dev platform side for allowing people to build AI enabled applications that are fundamentally going through that ontology system, not just as a semantic source of grounding, but also as a security and governance system, to Jeg's prior points around provenance and things like that. I think that did segue as we caught wind with that to the dev platform side of things, too.

[0:22:03] CJ: Yeah. I guess, I can jump on the dev platform side of this, which is really describing the full journey that we've gone through where we started building this product at customers with governments, where we were literally just trying to make tooling for ourselves. We wanted to enable our most technical employees to be able to build things, until we found that this definition of an ontology actually made it very easy for our operational users to interact with the system to understand the semantics and the kinetics of their business and build things using our no code tooling.

We came full circle, where all of those things we're describing, like ontology get you very close to the HCI principles that we've been thinking about for a long time and object-oriented programming, where you're describing the model and then it's the thing you actually want to interact with when you're building software on top. The idea of collecting the exact objects that you want to interact with, which are a subsection of your business and then getting the actions that you want to perform on those and calling them as if they were methods is actually very attractive, especially as your business scales, because you're not thinking about that same modeling problem over and over again when you're doing software engineering.

We literally have a shopping cart experience of like, I'd like these objects today and these actions and maybe these AI enabled functions and this is exactly how I'm going to power this particular interface decision product, maybe application. I like the term semi-serendipity, because it carefully highlights the point that if we called it serendipity, a load of engineers here would be annoyed about the amount of effort they put in. But the semi-serendipity of us actually getting to a developer platform seemed very obvious, once we got to the point where we were like, yeah, now we want to code against it.

[0:23:50] SF: How long has Palantir been investing in large language models?

[0:23:55] AK: I think we all collectively caught wind of things, I think, after the release of ChatGPT. A former Palantir director of engineering, Bob McGrew, actually was the head of research at OpenAI and Bob was doing amazing, awesome work there until quite recently, and he's moving on to his next thing, I think, publicly he said.

I think we caught on around the same time, I think really in earnest that most people did with the launch of ChatGPT. I'd say, there were a couple of things that were antecedent to that. One was, I think we saw some - we were following some of the literature, and some of, I'd say, the precursors in the transformer space because of our work in DoD and other fields and just staying attuned to things. We also had some sense of what was going on at a high level at OpenAI in other places, just because they were all publishing most of their work. It's even open source up until through GPT 2.

I'd say, we got serious about it probably with the sense of like, we have the conviction to try to build something here shortly after I'd say, that fall, that winter when the ChatGPT boom happened. We said, well, now that there's really a there, there, it got to a point of congealing internally where it said, okay, there's all these things we're also building in the foundry system that will, I think, coalesce and come together to enable something really interesting that would merit something like AIP.

[0:25:06] SF: Did it feel like an natural fit based on some of the things that you had already previously been doing, combining maybe traditional machine learning and AI approaches with everything that else that you were doing in terms of these various platforms?

[0:25:19] CJ: I can definitely talk to this, just because I had the opportunity to work on some of our earliest AIP pieces, our initial interactions. I think it was very much, we were seeing this technology come through and it just aligned so well with the ontology concepts that those were the things we wanted to try. We wanted to see like, does retrieval augmented generation work with objects when we put them together? I think many people, like engineers and for deployed engineers are like, entered this research mode where it was very much out of sheer curiosity, do these things connect well together? What happens when we start defining systems where we, like Akshay said, create AIS the next player in the multiplayer environment.

Then, as soon as we saw that, actually when you're using these products initially, we're trying to get people to a point of decision making where you don't normally get these with a large language model, where you like, where did my citation come from? What is the actual meaning behind this decision? Can I actually go through the working out? We found it to be a lot more clear when we could say like, cool, this is the corpus. This is the truth of the world. If we give you this truth, you have to refer back to the truth and tell us exactly how you came to a decision, or what comes next.

By giving our users and the AI the exact same image of the world and saying, "Cool, this is how I came to my decision, and these are the constraints," we found very consistently in multiple workflows that this came together very well, and then we just started iterating very explicitly on how do we build trust in a system that's using something that is non-deterministic by nature, but enables people to do their workflows so much faster.

[0:27:05] SF: What's it like building on AIP? Maybe, Jef, you can walk me through what is the equivalent of building the hello world application on this platform? What's it take to get started as someone fresh to it?

[0:27:18] CJ: Yeah. It's interesting, because I recently thought I'd make a notes app. I was there like, what is the hello world here?

[0:27:24] SF: Yeah, like a tic-tac-toe notes app, a lot like get starter projects.

[0:27:29] CJ: Exactly. I was trying to work out like, what's the hello world of AI? Now, we have the idea of retrieval augmented generation. Can I make a notes app that when I type in a question, will tell me an answer, but go like, "Hey, look at this note where I found your answer." What am I having for dinner? Well, your groceries has a chicken on it, so I guess, you're having chicken. Getting started then in this narrative is like, cool, you make the notes app just like you did before where we're not taking that away from the engineering experiences how you augment it and you use our internal tools and AIP to define the logic as you walk through what is a note? How do I bring a note into the application? What is the question I want to ask? How do I embed into a semantic database?

In our platform, it's very much click interactions, where I don't know how many times you've tried spinning up your vector database at home just because you wanted to try semantic embeddings and then connecting to various APIs. But internally, these things that are the basic primitives of AI are just baked in, so then when you're writing your code it's, "Cool. I'm going to call this endpoint, tell it that I'm working with note objects which are already defined in my object model and I want it to tell me exactly based on the content. Can you find a note that matches and give me information?"

We very much expose the APIs that are the primitives within the AI platform and within the gen AI space that people are using day-to-day, whether it's vision, or whether it's semantic searches, whether it's rag, and then allow you to just define them as an SDK that gets generated for you. Instead of having to think through like, "Oh, what is the infrastructure I'm going to spin up today?" Because there's a new white paper on some new technology, or there's a new language model with a different multi-modal interface. It's very much, cool, I have an SDK. I can call it in TypeScript. I'm going to just use it in my notes app and not think about it as a difficult problem in the same way that making a to-do app isn't a difficult problem.

[0:29:34] SF: Does the platform have an opinion about how some of these different components, these base units of AI work so that as someone who just wants to essentially produce this notes app, I don't have to think about what is my text chunking strategy? Those things are just an abstraction. I have an API endpoint that I'm calling.

[0:29:55] CJ: Yeah, exactly. We have very easy mode versions of these things. I describe it as for every interaction you want to do normally, then there's usually a one-click, "Hey, I want semantic embedding," or, "Hey, I want to do text extraction." Then, there's usually a drop down somewhere, which is, cool. Now, every time I talk to a set of customers, someone always asks like, "Can I adjust the seed?" I personally have never changed the seed in my life, but I know you can change the seed. It's these kinds of interactions where the more competent you are and the more of the work you want to expose to yourself, especially the interfacing, you can do that, all the way down to like, if you want to hand code the interactions that you have with a new language model and then register that as a contract within our product, and be the other side of the wall, you can bring that as an interface.

The way I've experienced it myself, because I got myself one of our free accounts just to see whether I'd enjoy it or not is you can very much build exactly what's worth building very quickly and go like, "Cool, this is the USP of the thing I want to make." Then for every individual component that you are adding to your system, you can decide like, is this worth as a microservice pulling out and writing it myself, or should I just use the Palantir default for every single step? So, if I just want to use the default embedding model and not think about it, because that isn't the most valuable thing for me to do, then I can do that. Then the next day, I can go and I can spend a day researching what the latest in embeddings is.

[0:31:34] SF: How much of it is your platform-driven, building an application? Am I in control of what I'm using on the front end? Can I go and use React, or Angular, or whatever framework I want to use there? Then I'm slowly using AIP for the API ware in the backend? Can you explain a little bit about what are all the different components and how much of it is the platform, versus things that I have to build myself?

[0:32:01] CJ: Yeah. I mean, you're fully in control, basically. We provide a Vite template that will build your classic React template, or Next.js. But at the end of the day, what we're really exposing are restful APIs. We're doing the most raw integration that we can do with the platform, so that if you turn up and you wanted to write all of your code in Rust, that's your prerogative. I wouldn't recommend it for a front-end application, but it might be something that you want to do. At the end of the day, we provide SDKs on standard languages like Python, TypeScript, Java, but we are fully giving you the opinionation onto how you want to build your product, whether you have an existing application that you just want to add AIP to, or whether you want to use Angular, React, Vue.

Ultimately, we will provide opinions, because as Palantir, we have opinions as to how you could get started, how we can make your life a lot easier, what we do in any environment. But we've had customers just building things that we hadn't thought of, or building the first version or something, because they were insistent that the first time they were going to use AIP was going to be in an expo app. I had someone going like, "Hey, can you set up an expo app?" I was like, "I've never made one, but let's work through it together."

[0:33:20] AK: Part of what, I think, Jeg said earlier as well around the things that were coming together around the time of AIP, I think, it reflected a lot of learnings and requirements from enterprises, as well as said like, if I'm going to rely on this ontology layer, or its ontology system really, it's like, you need to provide me the leverage around being able to use different data compute engines in the pipelining layer. I want the provident service. I want the security. I want all these things. But I want to not only be able to use Spark and Flink, but also be able to use lightweight transforms, like Pollers and I want maybe Clickhouse, or an open source engine or something as well and plug that in in a way that actually is seamlessly integrated with all those things I like about the integrated system.

Same thing, like you said, I think earlier, you said, Sean, were there other types of models that forced us in a way to have some of the infrastructure ready for the AI, the gen AI models? That was very much true. It was the not just traditional machine learning models that people were building and dockerizing and they wanted to bring into the platform, or hit externally, but it was also optimization models, the things that are solvers that are being used to run gas networks, or airplane networks, or things like that. How do we have you able to have an ontology action that is backed by one of those models, or any type of model generically with some contract you're defining? All of that complexity. Being able to mix and match and evolve these things over time, I think, now comes to the benefit of the developer platform, where you can come in and say, there are lots of sensible defaults, like Jeg said. But if I want to slot in and out all these different types of data and compute tech within a common perimeter and a common interface through the ontology SDK, I think the experience has been it provides a lot more leverage than like Jeg said, having to do all that stuff yourself.

[0:34:53] SF: How much is plug and play? Can I run my own private open source model, or plug in my own vector database if I'm doing something with reg?

[0:35:01] AK: Yeah, absolutely. Maybe Jeg can give some of the patterns here. But it's like, I think one of the things that - this has gone, I think, very deep, again, because of the requirements from 55-plus sectors now at scale. It's like, people have their existing data platforms even. It's like, can I virtualize data from the existing Snowflake, Databricks, BigQuery instance I have? Can I bring in the metadata from my Informatica system, or my Elation system, or something else? My semantic definitions may be sit inside of some Owl file somewhere. Can I bring those in and map those into the API as Jeg mentioned?

The model building, yeah. It's like, I want to connect and pull open-source models from HuggingFace. I want to bring my own models from my existing Dataiku cluster, or my data robot cluster, whatever it is, or just my EC2 environment. All those things are pads that we've had to make smoother and smoother, even though they were technically possible, because those are the things that people - every enterprise architecture team is asking about. I think what's cool is when the developer platform turned on, by no means perfect, and still working on it, it's like, people got a lot of that leverage from the years of having to build all those on ramps for different types of integration patterns

[0:36:01] CJ: Yeah. I think it is very interesting to describe the journey of when we were asking people to do data integration originally, we were like, very explicitly you must use this language and you must use these technologies. For a long time, we were like, you're writing Python and you're writing Spark, and that's the journey of data integration. Then slowly over time, we've exposed more and more of the infrastructure layers, so that people could have that level of customization, so you're not just picking your language, or picking the environment in which your compute is run, but we've gone as far as going like, cool, you can bring your own containers, and then our job now becomes, how do we show you very natural interface points, so when you bring a container in, then it's treated correctly in our provenance service, you can see it being used in your ontology, you can define an action that runs through some logic that maybe your business wrote 40 years ago and refuses to change, because they don't have any engineers who can write Cobalt, for example, right? These are real problems that we've had to solve and face.

[0:37:03] SF: Now you have your own unwrite the Cobalt, right?

[0:37:05] CJ: Yeah. It's true.

[0:37:06] AK: I was just talking to somebody who's like, all you people now who have these LLMs, I had to do this by hand two years ago. Get off my lawn, right?

[0:37:17] CJ: You do have to admit, the engineering problems have have become a lot easier now. I feel like I write less code myself. But ultimately, yeah, we can bring your containers, we can bring your front-end applications. Egress to other services is completely designed around just making it native to the things that are in your ontology. Because in reality, if everything was inside AIP, then your business isn't actually running. It's like, the connective tissue between our products and exactly the things that are your system of record, things that actually mean that something changes in your organization means that we've had to force ourselves to integrate with every product under the sun, whether we'd like to or not.

[0:38:00] SF: How does having the ontology at the core of a lot of this help with things that are challenges with LLMs today, like hallucinations, lack of domain knowledge, some of the security and governance over the data?

[0:38:12] AK: Jeg mentioned a couple of things that I think are specific elements there and how they think the ontology serves the grounding system for the LLMs. I think the core thing, the core central assumption is it's the same image of the world, like Jeg said. It's the same rendition of the world between humans and AI. When you have to go and look up objects and run object queries using the object set service, or you have to invoke an action through the ontology system as a human user, they're all these junction points of like, here's how the provenance service tracks that, here's how the action gets run, here are how the results get staged, or in a scenario, or get executed and then written back to an SAP system, or somewhere else, and all of those things have defined behaviors protocols frameworks around them.

When we introduce the AI capability here and you're defining an AI function that is looking through a support ticket, figuring out which customer orders might relate to it, you're literally saying, here the customer order object tools that can interact with that are whitelisted for this particular function, here's the actions it might consider, these are all actions defined in the ontology. I can literally run these dry runs even as, and this is the integrated development environment in the AIP logic application, but you can also do it programmatically. It just shows you, here is how it's interacting with all those different object-based tools, or I'm sorry, ontology-based tools, objects, actions, etc., in the same way a human - I could literally go and do those same exact things.

I think like, being able to have every single run, every permutation of the interaction, whether it's just feeding information, or doing something agentic, always be in that same system, that same set of guardrails of the human users are all working within actually serves as the most natural on-ramp for introducing these capabilities in different customer settings. I think about some other, just very quick customer examples like, eating the big manufacturer does a material reallocation and supply shortage disruption handling all the time with human users, and they built this ontology for doing that. They could say, "Well, there's actually this part of the workflow where the LLM could help us do better retrieval, or turn our alert inbox into a solution inbox as proposing things we might do."

That was just something we were already doing. It's just coming into the workflow is like, one Japanese customer called it the intern on the team. The intern knows nothing, but if we give it some context, it may be useful. You wouldn't trust an army of interns to do everything, but it can do some things. Then nurse scheduling is another example at the Cleveland Clinic, where it's like, you're going to have somebody who is qualified to sign off on the final schedule, but maybe it's able to find disjunctions, or look for other patterns that you're not considering and give you some additional options there underneath the three that you selected. It's coming right into the workflow, same model of the world, I think, is the fundamental conceit.

[0:40:46] SF: Who's the target user of this and who do you see as competitors, essentially?

[0:40:51] AK: Yeah. This has always been a, I think, a tricky question to answer, I think on the user side, because it is cross-persona, right? It's like, we always think about the operational user backwards. I think even with our smaller customers, we think that way too. Who is the engineer? Who is the analyst? Who is the person that - the metaphor we use is they were building the ironman, or ironwoman suit for? It's like, their life gets better, because they can make better decisions through some application, or front end.

Now, behind that, there usually sits a set of personas. There is the data engineer. There is the maybe traditional application engineer now. There is the data scientist. It's a tiger team of folks that are typically working in a circuit with the operational user. In their teams that, by the way, just did dev work and don't have as much operational work, but we always think about like, if you ask any classic Palantir deployment, who were those champions at VP at United Airlines, at PG&E, and others? It's always the electrical engineers, the network operations folks, somebody, right?

I think those are the users that we orient around, and then we have technical personas in service of. I think that's probably a different orientation than most software systems that are out there. That's one thing. I think the result of that is because it has this phylogenetic tree structure to the personas, it means that the competition thing gets a little confusing, too. There's a cheeky answer and a non-cheeky answer that I typically give to this. The cheeky answer is like, it's the internal build. It's like, when you're done with the data lake and the analytics thing, you're like, now I need to go into application space. It's like, that's the wild west. I got to go hire some consultancy, or staff up a dev team and try to build something custom. Maybe we have talented people and we do that, but then it's like, I got to build the next app, the next app, the next app, and it's like, this is getting crazy.

Some people can do it, but I think what we typically get - what we typically hear is people told us we would be the next Netflix when it came to building internal apps. It turns out, it's hard to do that without a system that handles all the things that they have built internally. It's like the internal shop that's trying to do these things, the more and more when they see this as a - it's not a buy versus build as we say. It's like, I buy this so I can build. Especially now at the AI stuff adding more requirements, that's the first thing.

The second thing and I think especially with offerings like our new warp speed offering, which we think about as an operational manufacturing-heavy version of Foundry and AIP is like, it's the traditional operation software providers. Think about the world that's locked into ERPs and PLM systems and all of these things that run the core functions of the business. Right now, we treat these things as inputs. But more and more we see with large manufacturers, energy companies, utilities, hospital systems, etc., is like, it's great that those systems have the ledgers in them, but it's like, I need to model them into my world, my ontology and then build, and be able to permute that, as Jeg said, as my world changes.

I think there's legacy operation software, and a lot of that's domain specific that I think we end up being much more competitive with, than data and analytics software, which are usually plugging in.

[0:43:42] SF: Jeg, you talked a little bit about this earlier, where when you start to plug in these units of LLMs, you're dealing with a non-deterministic world. I think in some fashion, building with LLMs requires a bit of a different mindset than traditional engineering, where you're used to algorithmic inputs and outputs and what you get out is presumably what you expect if you program the algorithm correctly. As Palantir were starting to build in some of these technologies, do you take effort to get the staff, essentially, comfortable with the idea of building on LLMs and be ready for dealing with these non-deterministic boxes?

[0:44:19] CJ: I think, it's an interesting take of exactly whether we were comfortable with non-determinism. I think no one is comfortable and ready to accept it. I think the example you often see is if you give an LLM a calculator, a tool, then suddenly, it can do math. I think this is the journey that we've seen, which was the more of that determinism that we can inherently bring to the table, such that the language model is just solving the exact thing that we need solve, that we can trust it to do, whether it's summarization, or dealing with specific semantic layers, then the less this ends up being a natural conversation of like, hey, are we okay with the risks that we're taking?

We explicitly minimize what is a risky interface, what is a thing that is an explicit decision by, like Akshay said, taking the actions that we've already defined as safe paths for a human to take that are already validated, maybe dry run. Then on top, we add concepts, like evals, evaluations, test suites, where the things where if we're using a language model and we expect the decision to be deterministic, then we should have deterministic tests in the environment. We basically just ended up building the suite of things that would make us confident that we weren't taking risks, that we knew exactly what the language model is doing is the thing that is adding on top.

I almost would argue, it's like, the first time you're introduced to a reverse proxy, it's just another piece of software engineering tooling that's in your back pocket. It's not something you'd slap everywhere in your system, because it makes no sense. You go like, cool, where are the LLM-shaped holes in my organization where a workflow could be better? But I'm not going to turn up to the table and be like, cool, well this whole workflow, we'll just put an LLM in front of the APIs and hope for the best.

[0:46:05] SF: From engineering and technology perspective, what do you feel the most excited about with, essentially, this transformation that Palantir is making now around helping people build these AI-powered applications through this platform?

[0:46:17] AK: I can almost think about, it's almost like the venom symbiote from Spider-Man in my mind is maybe it's like, it affects every part of the system. If you think about more and more detail being built into the ontology system that represents the reality of the operations of the enterprise, I think there's more and more space for the LLMs to be able to interact and add value in the ways that Jeg mentioned, where it's like, what we're seeing even right now, it's like, the initial versions of how we incorporated LLMs into data pipelining I'd say was like, okay. Then we built some additional capabilities that were much more targeted and much better received.

I think, that's been the case for every part of the elephant. It's like, well, how do you then think about, to your point, Sean, structuring default ontologies? Turns out, you can actually do some more interesting targeted stuff with LLMs if you have that as part of the bootstrapping process. With how you define compute and the action framework as well, the application building, how can I start there? We've built a lot of the application front-ends. In the platform, you can also build using your own React frameworks and environments and other things using the SDK. But even the way we build applications in the system is still very free LLM. It's like, why is this not more interactive helping me contour my way through that's the name of a product, pun intended, sorry. Of just being able to get to an application state that makes sense as a non-technical user and then work with it.

We're right now injecting the LLMs into the existing frameworks, I'd say, still and large. But there's going to be entirely new experiences that can be imagined in the application space. I'm particularly excited by just seeing people push the frontier and how much productive, like the reverse proxy analogy I love. How many critical junction points can you actually bring this to that are value additive? What can we learn about how to then build more and more capability around those points? I think, this is where being forward deployed, having all of these deep touch points with all of our customers is really, I think, showing us where the wheat is from the chaff, and helping us continuously orient and call BS on even just internal assumptions we have.

[0:48:10] CJ: My take is almost very low level in comparison, which is for the individual developer. I think we're in a very exciting period of new technology comes up on a Monday, you're like, "I want to try it." That's an individual who spends their weekends tinkering on software projects. I'll spend more time trying to set up whatever this new thing is, working out if there's a dockerized version of it, or whether the platform or my laptop will actually even run this model. Then, it's a whole different story about whether I can actually use it in a meaningful workflow. The ability to come up with these new technologies that are being delivered to us daily at the moment, and then bring them into a workflow that we would consider productionized and say like, "Cool. I'm actually going to bring this into an AI infrastructure system, such that I can use it against my ontology, against actions that I can actually define a workflow that will use this video model," for example, makes a huge difference in terms of me going from zero to one in terms of actually building something that's usable.

Because I think at the start when we didn't have this tooling, I actually now use Palantir for my own home projects. I would spend so much time just spinning up some new service and just trying to get the infrastructure right, just so I can start making the application on top and being able to go like, "Cool. This is the thing that's different, and then this is actually everything that I already had in my system that I can now wire into this container, whatever," makes a huge difference to the velocity at which you can start adopting these new technologies.

[0:49:42] SF: Yeah, absolutely. We're coming up on time, so Akshay, Jeg, I want to thank you so much for being here. This was a lot of fun.

[0:49:49] AK: Thanks so much, Sean.

[0:49:50] CJ: Yeah, thanks for having us.

[0:49:51] AK: This was awesome.

[0:49:51] SF: Cheers.

[END]