[0:00:01] J: James Cowling, welcome to Software Engineering Daily.

[0:00:03] JC: Thank you, Jeff. It's really nice to be here again.

[0:00:07] J: You are the Co-Founder and CTO of Convex. Convex, can you talk a little about what it is? I wrote down in my notes, it's primarily, according to my own assessment, it’s primarily in the business of developer experiences is how I read it. Is that fair to say?

[0:00:26] JC: Yeah, that is fair. Actually, I like that you said it’s a developer experience platform and not a database. Because it is a database We built a database. But convex is designed as a labor-saving initiative ultimately. Our long-term vision, which you won't see written on our website right now was to make mostly back-end infrastructure a thing of the past and get us to a future where the vast, vast, vast majority of engineers are building applications, or building products that people use.

What is Convex? Convex is a platform as a service. You might think of a little bit similar to say, something like Firebase. It is a platform that has a database that handles execution and you can use it to build applications, primarily right now web applications and mobile applications. We can talk in a lot more detail about Convex as needed. But really is designed as a platform where you can get started very, very quickly and build productive applications very fast in a product full-stack applications, but then it's something that's designed so that it will scale with the developer and with the team.

It's not something that you start using and product type and app and then grow out of and have to have to hire a back-end team. It's designed really so that you can use Convex for the full life cycle of your company.

[0:01:46] J: Okay. I'm going to say, how does it manifest? When I get started with Convex, I suppose there is a big Convex idea I download. I know it runs off TypeScript and/or JavaScript. Is it basically a bunch of libraries and you compile it and it does the – What does it look like when I get started? Yeah.

[0:02:10] JC: Yes, specifically what’s it look like, right? We have a website, convex.dev, and we have a dashboard, dashboard.convex.dev, where you can go and see your data, etc. The main interaction you'll have with Convex is throughout development libraries, right? There's a CLI tool to create a project and then Convex currently is a really great fit for React developers, so people who’ve been in React applications and you pull in the Convex library and you code against those.

Specifically, how you interact with Convex is we like to think of a database source as a back-end not as something where you send queries to or exchange rows with, but a platform where you actually run programs. If you want to say, write a query, let's pick the classic chat application, or let's say, a to-do application and you want to write a query that says, give me all my undone, unread to-dos. You write that program in JavaScript, or TypeScript very naturally like you would write any code. That runs on Convex servers on the database, it fetches the results for you, but very importantly, we actually run that code in a V8 isolate so we actually know what that code is doing. As a result, you can subscribe to the results of that query.

If anything changes, for example, if someone marks one of those tasks as done, we'll automatically send those results to you over a web socket. Because we know if those inputs to that query function have changed, we can automatically cache this data for you at the edge. The idea with Convex is there's a set of design principles, that a design such that you don't have to think, or peer through the abstraction boundary of the backend.

Your interface with Convex is writing functions that run on data and then execute them, subscribe into them. Now, the platform has storage, it has search, it has all the things you would expect out of infrastructure. There's really designs so that you don't think of this as a bag of tools, but it's one problem that one platform that makes your problems go away ideally.

[0:04:17] J: the way I visualized it when I had my cache look was that basically, you almost write your code as a developer. Based off of that code, it infers the data model and does all the backend stuff from infrastructure to schemas, to everything so that you just basically write your query and it figures out what the data would have to look like. Is that vaguely on the money?

[0:04:51] JC: Absolutely That is certainly part of the platform, right? We have this catchphrase inside the company where we say, day one is year two power. This is the experience you want to developer to have. Most of the team here has run very, very large infrastructure before knows how to build databases and knows how to build storage systems, has run backend teams. But doesn't want to go to product developers and say, “Use Convex, you'll be happy about this in a couple of years.”

When I go to developers and say use Convex, you'll get your product shipped to 10 times faster than you would otherwise. It just so happens, we've chosen a set of abstractions such that if you do this, you build your product out of your deterministic composable functions. We handle the caching. We handle the subscriptions for you. We have our own web socket library for you that the product and the platform will scale with your company.

Convex is designed not to be a toy. It's a fully relational data model. It has caching, it has schemas. One thing you made reference, too, is schema inference. This is part of this day one is year two power. People love MongoDB. Old meaning. Five years ago, 10 years ago, people really love MongoDB. Partly why they really loved it is you could just take your JSON and then dump it in this thing and it would go away. There'd be database people. Mongo actually has improved tremendously in terms of their correctness story. There was database people at the time who would were very critical of Mongo and like, it's not a proper database. It doesn't do this and that, but it solved the needs of the developers. They’re like, just give me something to throw some JSON in and then let me have it back.

Convex allows you to do this. You can write your application. You don't have to define a schema. You can just dump your data into Convex. We actually have in the background a schema inference algorithm that figures out the minimum schema that represents the type of your data in the system. That is done very efficiently. It has very cool algorithm author and source sometime. When it comes time to define a schema, when you say, “Hey, I actually want type safety right now. I want to validate the inputs to my function. I want to make sure that only a certain type of data is in my system,” you can just – we’re not exactly one click, but you can just go copy the schema of the dashboard and use it. You can obviously define your own schema as well.

This is this tension we’re threading, but it doesn't feel like attention is working quite well, right? You can get started incrementally. You can start building and you can have a very good developer experience. I think, if you're a React developer, you should go try it out. I think you'll really like it on day one, but I don't think this is a platform you're going to outgrow, or find that it's encourage you to make bad choices.

This is this tension between a front-end and backend. If you just try to solve problems purely on the front-end side, you can make life very easy for developers, but maybe you're encouraging people to make some bad choices, maybe to take some shortcuts, maybe write some things that don't scale. If you solve things purely on the back-end side You're going to build this grandiose, wonderful infrastructure that no one can use. Convex is desired as a company is to unify these both, right? It's basically, infrastructure built for front-end people.

[0:08:12] J: Yeah, absolutely. Two last points, what this reminds me of is a little one-panel cartoon that I've seen where employees walk out of a fail startup, carrying the leftover laptops and stuff and they said, yeah, it's like, one engineer saying to the other, “We added absolutely no business value, but our schema was pristine and our infrastructure is so robust.”

[0:08:35] JC: Yeah. In many respects, Convex is almost my apology to the world for a lifetime of being an infrastructure person and leading infrastructure teams. I say this would love to all my to my infra homies out there, right? We only exist in service of the product. Just don't get this the wrong way, or other way around. Nothing matters, unless someone's building a product on top of it that affects a human being. In companies, you talk about top line and bottom line. Top line is how do you increase profitability. How do you get new customers? Bottom line is how do you minimize your costs?

Infra usually is a is a bottom-line optimization strategy. How do you keep the company running? How do you save money? Product application normally is a top line maximization strategy. How do you get more customers? Infra teams, back-end teams always have to make sure the company will continue running, will scale, all that stuff. But they live in service of the product. I think sometimes, we get this the wrong way around, right? We don't realize that companies exist to solve a problem.

My dream as I'd mentioned is with platforms like Convex to get us to the point with a vast, vast majority of people, I just go in and building products for humans to use. They're having a good time doing it and it works and it scales, which I don't think Convex is amazing, wonderful and the answer to the world's problems, but I don't think outside of this we're really at that future yet, because even with infrastructure as a service, AWS is amazing, it's still a bunch of tools you need your infrastructure team to put together to actually solve a problem. The interaction surface with the application needs to be all the way at the application surface, at the libraries people are using to build their application.

[0:10:33] J: How do you know how to tackle that problem? Because like you said, your background is infrastructure, as is mine mostly. I'm a dev-ops platform engineer, lots of infrastructure, lots of server maintenance, lots of cloud. I personally recently tried to actually do a little web application. I realized, I have absolutely no idea how to approach it, because I've never been a developer. I learned HTML, I think, four or three, 15 years ago. Everything that happens in the browser nowadays is a mystery to me. I've never acquired those skills.

Talk to me about your background and the background of the other people at Convex. How did you manage to get all the kind of this breed, I suppose, pieces of the whole developer and operations experience together, not just as stale bits of knowledge. I know this, I know this, but also, the problems that need solving in these domains and how we can solve them all at once, which is what you're doing?

[0:11:39] JC: Absolutely. The first thing I'll do is just say that your experience absolutely resonates with me. The genesis of this company, the co-founders, Jamie and Sujay, we all worked together at Dropbox for around eight years. We built a lot of the very large infrastructure systems there, so the big distributed storage system that was multiple exabytes, close to a million nodes. The big distributed database there that round four plus million queries per second. There's a lot of very large infrastructure. We've built databases many times before and we came out and we promised never to build a database again.

Partly, because we felt there's a lot of startups and companies out there that build infra first and then go around trying to find a use case for a second. Again, I say this with love, but I will throw a shade in all the companies has come out of Google where they just take some big piece of Google infrastructure and try to convince the world that a two-person startup needs Kubernetes, which they don't.

We came out and said, we'll never going to write database again. We promised ourselves this. Then we'd have exactly your experience. We try to build other applications, front application and we were like, wow, this is just needlessly complicated. It got worse over time. There was a time where there was the lamp stack, some of the maybe the older listeners will remember where you had – go around this correctly, Linux, Apache, MySQL, PHP. You could get a company going in a day. Many people did. Nowadays, the amount of complexity you have to deal with the builder, a product these days is out of control.

We realized that the reason for this is the infrastructure that exists on the market reflects the shape of the underlying technical solutions, not the shape of the problems people are facing. This is the genesis of the company. Now, how do we build a team? How do how do we think about building composable abstractions? When we built the storage system at Dropbox, which was a very, very large system, we were the first team in the world to deploy something called shingled magnetic recording at scale, so new hardware technology, at least at the time. Perhaps now, the most efficient storage system in the world.

We had a very, very small software team. Some of this time, less than half a dozen people. The only way you can do something like this is to think very deeply about simplicity, composability. How to not paint yourself into a corner. Thinking very clearly about that abstractions and I can talk more about abstractions in a second. I think we came into this with a very keen sense of how to design things simply and how to invest effort in building elegant, composable components.

But we have to build as a team. I think, the DNA of Convex right now is there was a lot of Dropbox folks. We have a lot of folks who've come through Google Photos and have a very good sense of developer – a very good product sense. There's folks from the old Bump team, if folks remember bump. Bump was, I think, one point the most popular app on there on the iPhone store, who – it’s a real killer, killer software team? Asana, etc. You asked about abstraction composability now. I think that the core influence in central to Convex is programming language design, of all things.

Jamie and Sujay, my co-founders are big programming language nerds. I'm a slightly less of a programming language nerd, but I did my PhD with a woman called Barbara Liskov. Barbara Liskov, who’s a professor at MIT, very big in the program language world, the pioneer of abstraction in computer science. She thinks very, very deeply about, how can you make things simple? Which we take for granted, but it's actually very hard. I think the programming language community has a lot to teach the systems world about developer experience, composability, understandability, expressivity. Not just thinking of the world in terms of cues and databases, but thinking of the world in how do you express things in simple ways?

You can look at languages like Haskell with software transactional memory. A design of Rust. People thinking about cues, instead of locks. There's a lot to be learned from these communities. There's a lot of influence from languages like Haskell in Convex. If I think about what does abstraction mean? This is a vague word. To me, abstraction means taking a complex, large set of information, putting a box around it, such that you all you have to understand is the box and nothing inside of it.

A successful abstraction is one that has a simple shape of the box, so it's composable. It fits together with other components. It's powerful and it's useful, but it's also working. You don't have to open the box. If I think of like, if I have a company and I hire a head of infrastructure, or a backend team, their job is to make backend problems go away. Their job is not to come to me and ask what version of MySQL I want to use, or do I want semi-sync replication, or sync replication, or read committed, or serializability, or even what regions I should deploy caching in.

Their job is to make problems go away. They abstract that problem for me, so that I can then focus on other things and drive the company forward. If we want to build a good development platform, it's very, very important that platform has a good abstraction. We don't ask our developers what region do you want to cache in. We just figure it out. We don't ask them what what consistency model they want. We just provide them – I have a strong consistency. I don’t have to think about it.

A lot of Convex is designed with this influencer programming languages, this influencer composability, this influencer abstractions, such that you can just take problems away from people, not add problems to them.

[0:18:09] J: That makes a lot of sense. The way I always think of abstraction is making problems, specific problems look like other problems. Because at the end of the day, if we abstract them, to use the very words in the definition, they are all problems we've seen before. We can't reinvent the wheel every time there is some problem, so we just use abstraction, be it through common interfaces, be it through – usually, just common interfaces, isn't it? Because then, our brain can look at a complex problem, divide it in huge Legos, say, “I know this, I know this, I know this.” Maybe there's a bit of interesting stuff we can re-engineer then, do something novel, but then we can actually solve the complex problem and bring value to the customer, like you said.

[0:19:07] JC: Yeah. I love that you've used the example of human psychology, because I get asked a lot, if someone wants to be a good engineer, what should they read? What log should they read? What books should they read? What topics should they learn more about? Generally, I would say, you should learn more about human psychology. It's probably not something you hear a lot, and for two reasons. One, is engineers, we generally over bias towards technical stuff and we under bias towards the human side. If you're a software engineer, huge generalization, there's a good chance of something you can learn more about human psychology.

The second is it's just so, so informative in terms of how you can build good products and how you can communicate effectively as an engineer. One thing we discuss sometimes, it's very similar to what you said about the working set in your mind. You approach a new problem, you come and use Convex. When you first start, you are confused. You're a new developer, you come to this platform, I don't know what this is. I don't know how this works. It's using up all this space in your stack, let's say, in your mind. There's so many new things and new ideas and new words and new topics in your head, you don't know how they fit together. It's in your stack and that slows your brain down.

The way you get better at any discipline, any domain is not to keep adding things to your stack. It's eventually to take things on your stack, compressing them down to an abstraction and then putting them away on your heap. This is not exactly how the brain – I don’t know how the brain works, but this is my model for this and I think it does – it resonates with me. When we're introducing topics to people in Convex, We have to make sure that they introduce in such a way that they can be compacted down to an abstraction and stored away for later.

There were times early in the company where I think we made some mistakes, where the shape of Convex – this is going to sound – It wasn't Convex. It wasn't around. We had some gaps. It was like, Convex is a database and it was a sync protocol, but we didn't handle storage, we didn't handle search, we didn't handle execution of arbitrary functions. People would struggle, what is Convex? It wasn't able to be simply modeled in their brain and stashed away on their heap, so they can start absorbing new information.

We added more features to make Convex – because we have some differentiator features and some undifferentiated features. How we run transactions, how we have our sync protocol, how we do subscriptions, how we do automatic schema inference, highly, highly differentiated. How we handle storage, not particularly differentiated. We handle storage like everybody else. Now, one could argue, hey, focus on the differentiator stuff, not the undifferentiated stuff.

Our experience was it was much harder to understand Convex without the full picture. We added some more features. We rounded out the shape of Convex and people find it much easier to understand now. It's more approachable than they can continue forward. I think, thinking about that human psychology side all the time, it's actually an engineering communication, don't hit someone with a million details at once. Even the big picture allowed them to simplify things, stash it away in the heap and then they can continue.

[0:22:30] J: Yeah. The word developer experience, I think, is chosen quite aptly in it. It's so similar in that sense to user experience. In that, you can't, yeah, you can't just have warned the user with a million things at once. What I'm thinking of is, I've played Portal. Absolutely fantastic video game. At some point, I've played it, I think, mistakenly with the developer commentary on and it actually walks you through how the levels are designed. At first, a user learns a novel concept and then does it two, three times, until it's not just a bunch of new functionality, but it becomes like you say, it's compressed. It becomes one concept. The brain has one, I like to call it concept, one function that it can then just use without having to process it all again, then the user learns a new thing, and then they learn a new thing. Then you towards the end, you're able to solve these great puzzles in, I don't know, level 20, or whatever. That you would actually, but not know where to start if you enter that as the first one.

[0:23:44] JC: Game design is such a great example, because I never had a personal experience of things like Roblox and Minecraft, but I've seen friends’ kids just build amazing things in these platforms. I’m like, how do you build a rather complex set of emergent properties in a way that a child can understand? It's very impressive game design. It'd be a huge mistake to underestimate how much talent goes in to thinking about introducing concepts in a way that can sink in to even a six-year-old. They can understand. They can be familiar with and then they can take that. Once they feel comfortable, once they understand, then they're off for the races and they can achieve so many things.

[0:24:27] J: Just for discussion sake, I'm wondering what would the opposite case here is. It must be language. We have children, babies learning language. We don't, as parents where we talk, I'm not a parent, but as humans who are grown-ups, we talk to children. We use some baby terms, but mostly we just talk to them normally. We explain what we're reading in a newspaper, what we're seeing out the window. We're not using a simplified grammar, until they know how to use that and then we speak to them all.

It works, but it only works after they've been exposed to it for a year or two straight, and then they just take in all that information. Then at some point, they know how to speak. I think that's quite an exception.

[0:25:13] JC: It's exceptional, because you have to learn language to function mostly in society. I mean, I think the quality of an abstraction relates even to how we interact with a domain. I'm thinking right now about avoiding politics, but tax code and regulations and a lot of things in the legal domain that have so many exceptions, so many corner cases. It's quite hard to understand the clear guiding principle behind why various tax exemptions exist. Most people throw up their hands and they have no idea how it works they say, “Well, I will never understand it, and I'll just go through the motions and I'll click some buttons in TurboTax, or whatever.”

There's people who look, for example, at an engine and to see a million pipes and boxes and they just – their brains, which is often they have no relationship with the object. I personally love mechanics, and because over time I have developed a sense for the abstractions in an engine and I can look at it at a car engine and see what the components do, what they mean and I find it unintimidating. How someone interacts with an object, with a domain comes down a lot to whether they understand it, not just whether they can recognize a bunch of disparate components. I have an example of that abstraction at Convex and how it simplified things, as opposed to adding, if this is a good time to bring it in.

[0:26:39] J: I think a podcast about Convex is a very good time to bring it in.

[0:26:43] JC: All right, here we go. I have an example of, at least a positive example of where abstraction has helped in the design of Convex. This is an example that makes me look good as a software engineer, where there's not that many of them these days. So human and be with it. Convex is a database and we run functions on behalf of the user. The source code for those functions are stored in the database and we have automatic caching, etc. Now, when we first put the product, there was no automatic caching. My job was to add caching to Convex.

Consistent caching is a very hard job. Some people say, consistent caching is one of the hardest problems in computer science. Getting this correct. It's hard for a variety of reasons. I'll give you an example of why it could be hard. You run a function on Convex and then you want to cache the result of that function. Now, there's a whole lot of conditionals that affect whether that function is – whether that cache result is correct at a given point in time. Convex is the optimistic concurrency control database, where everything's serialized, so we can know as of a point in time whether that cache value is relevant, but there's a lot of things you have to check.

For example, if you run a function, we cache the results and then you change the source code for that function. That should invalidate the results of that cache, because now it’s different source code. Now, that adds another conditional. If the inputs to that function change, that also is a conditional that should invalidate the cache. If you take a relatively naive approach to systems design, you're going to start this writing down every situation where we should invalidate that cache. Over time, you'll build a very complex system that will break one day.

Now, how caching works in Convex, we have a subscription mechanism. When we run any transaction in Convex, we track all the data that that transaction has read. We call it a read set. We know all the ranges of data that have been read.

[0:28:53] J: Could you define what you mean by transaction in this context?

[0:28:56] JC: A transaction is a function that runs on a database and it runs transactionally. It will commit all or nothing, and it will be serialized. When that transaction runs, it reads some data from the database and we record in a efficient, compact structure all the data that has been read, all the ranges that have been read. Now, we know that that result will stay exactly the same, unless anything that was read has been changed. Every time you do a write in Convex, we intersect that write with any active read sets, and we know if any of those subscriptions to those queries have changed.

We can vary efficiently in linear time, fend a subscription refresh message to the client. Let's think about how caching would be implemented. Well, caching is very similar to a subscription. I have some data that’s stored server side, or on the edge that subscribes to the result of that query. You might say, “Well, wait a second. What if the source code changes? I'll have to go manually invalidate that query.” Well, that's not true, because the query has to read the source code first before it executes the code.

The source code is in the read set to the transaction. If you upload a new version of the function, it just changes the – it conflicts with the read set, changes the result and invalidates the cache. I was able to implement caching in Convex in literally one day. I am not a fast programmer these days. I've been a senior guy for far too long. I'm quite a slow programmer, but I was able to implement caching correctly and deployed in one day, because of composable abstractions. These take serious work to get right, but you just know when you're on a set of composable abstractions, because everything just works.

When you're not, everything's a disaster of if statements and retries and checks. You can see code that's been written. There's locks everywhere There's all these conditional statements, there’s big notes and warnings and you can see code that's been written with clean abstractions, it just fits together.

[0:31:14] J: Does this relate to gradual rigor? If so, how? You mentioned on your website the concept of gradual rigor, which I think is mostly what we talked about at the beginning of the podcast, that you can start by just as a developer, just writing – basically, writing the business value and it won't come back to bite you later, because it'll scale with you and you can, for example, become more rigorous about your schema, which initially was completely inferred. Is that also a sort of abstraction?

[0:32:03] JC: Absolutely. I mean, now the standards are different for the application and the platform. The platform itself has to be rigorous from day one. If you think about a typical application where you want to do something dynamic, let's say you have a shopping cart, or shopping website and you want the prices to update live, you could write code that manually goes and polls the server and checks this every now and then, but you have to decide how often to poll the server and you start worrying about what happens if one component on the screen updates out of sync with the other one, are they going to be correct? You can add some conditionals and you can check that.

Or, you can just use a subscription in Convex. We guarantee that everything will show up consistently on the browser. Do you have, say, two separate components on screen and they're both affected by a number. Let's say, you have a shopping cart on the right and a list of products on the left. Someone changes the price of an item, you want both the cart to update the price and you want the shop, the items to update their price. In Convex, they're all – you have two separate subscriptions. We understand both those subscriptions are watching the same data. There's a single web socket that all your components subscribe to and we guarantee, we send you the results to the client in an order that will show a consistent view on the client.

That's, I guess, an example where we are required to be rigorous. There are parts of the Convex stack that we don't bother explaining to a day one user, because we don't have to talk about sync protocol and consistent client views and how we designed the sync protocol at Dropbox, and at least that the sync protocols have been used for the last five years and how that influenced the design of Convex. That's a detail.

Well, all the customer needs to know, or the client needs to know is it just works. Don't write your own sync logic. Don't write your own polling logic That's the stuff that compounds and becomes worse and worse and worse. Use composable abstractions. You don't have to think about those problems again. Don't add your own ad hoc caching, because one day, you're going to accidentally read some incorrect data, because you forgot to invalidate a cache. That's the promise of Convex is you can, I guess, ignore whole domains of rigor. You don't have to worry about the composability of some of these infrastructure components, because it's handled for you.

[0:34:35] J: If, last when, it will become a concern. Obviously, as you've stated before, Convex is the solution to all of the world's problems, but since we are serious engineers, we must acknowledge that there will always be cases where it's not the right solution. Something extremely specific, fringe. I assume, you can correct my assumption, since it's still effectively programming that you're doing, you can override anything you want ,you can make anything modular that you want and if you really need for part of your data to have a different way to store data, or a different consistency model, you could program it yourself if there is really no other way.

[0:35:29] JC: Absolutely. I mean, it's software. You can use it where you want to use it. This is a podcast where we're discussing Convex, but I love to be generally interesting to the audience, right? We can think about philosophies for designing systems. How do you design the systems that may become complex over time? Let’s assume that Convex doesn't exist, because everything gets boring when you have the greatest platform that solves all the world's problems, obviously.

I've been part of building some very large systems before, and systems that have budgets, I don't know, in the half billion dollars in terms of the money have gone into deploying their systems. There's a great fallacy that in hindsight, people think that the authors of these systems sat down and had this genius, precent understanding of every step that was required to get to the destination. They just did it step-by-step and everything worked out, which is completely false, right?

Then there's a second fallacy, and you may be able to get me to rant about – rant against data-driven decision-making here. The second fallacy is you can sit down and run a bunch of experiments and switch off your brain and let the data make a decision for you. The result is neither of these things The result is that when you're approaching your domain, you do not know how to solve that problem. You have to have a rough idea where you're heading. The real key is to break down incremental deliverables that will help get you towards your destination. I call these stepping stones at certain. This is for management. Forgive me, buzz –

[0:37:10] J: Wouldn’t those be data-driven though? Isn't the current key set to design the standard way to, let's say, build a startup for simplicity sake, you start with an MVP, which has to be partly, obviously, hopefully data validated, hopefully you did some business research to find niche, but partly also, it's just your vision, so you have something that brings a minimum amount of value and then you build on top of that, hopefully, also data-drivenly really by figuring out what do customers want, by looking what features are already using, doing surveys, looking at competitors?

[0:37:49] JC: Yes and no. I mean, data is great in hindsight and data is very ineffective in foresight. Data will allow you to evaluate whether the thing you've done achieves the goals that you have stated. If you say, “We're going to change this thing and it's going to save mind to the company,” and it did or it didn't, well no. You said, we're going to build this thing, we're going to get new users. We use data to figure that out. But data won't tell you what to do in the next two three years. That's going to require intuition.

One phenomenon I do see it happens a lot at bigger companies is people start getting afraid of intuition, because intuition can get you in trouble, all right. It actually requires a lot of talent to have intuition. You shouldn't let everyone go off and make whatever decision they want. You can use incremental deliverables to de-risk things. But the reality is when you start a company, there is no data. You start a company, there's gut feel. Maybe you did a little survey that's not statistically valid. You have an observation around about the world and you have an idea for how you can make the world better via some technical solution, right? You have to keep that strategy in your mind at all times.

The role of leadership and technical literature in particular is to is to build a strategy and build a series of deliverables you can produce along the way, that help get you to the destination and learn things. The best thing that could ever happen out of the next product you do is that you learn more about your domain. You understand your domain better. You understand your custom better. You understand how to solve it technically. Then you move forward. You can use data to validate that. Sometimes, it’s part of a thought. You just have to be in the dark for a little while. You got to build things for a few steps and survive on the strength of your conviction.

Sometimes you have to intentionally go in the wrong direction and see negative results, because you know that that's going to turn things around and drive you forward. There's always a role for data when you decide it's time to evaluate and you should try as much as you can to shorten these timescales, build things very fast, build things you can test against the market, against the customer, against the problem as quick as you can. It'd be a huge mistake to sit there.

If you say you want to build a database, or a company, or a storage system, or any product actually, it's a huge thing to say, “Hey, everyone. Go away for two, three years. Just let me do my thing. Trust me, it's going to come out perfect.” You have to figure out a way in three months, or less to do something useful that you can validate. The value could be that you get it out in the market and people love it. Well, the value can be that you figured out way better how to proceed from there. 

I just have this a little bit of like a pet peeve against blind adherence to data-driven decision making, because I think it just – it really devalues all the immeasurable things. I've had scenarios before where we've talked about, hey, let's build a database, or let's not – actually, that's not the right question. How do we solve the data needs for this company? There's arguments. Well, what we should do is just try all six databases and then pick the one with the highest queries per second, or lowest latency. That's measurable. It's not a particularly nuanced framing.

Do we trust the team? Is it written in the language that can we edit the source code? Is it have good consistency properties? Is the way that the team thinks about the future aligned with us? Does it have adoption? All these things matter in making a decision. Fixating on just measurable stuff means you're going to devalue the immeasurable stuff. I’m a big believer in data-driven reflection. Looking back and saying, did we achieve the goals we stated?

In a certain point, the rubber meets the road. You have to be held to task for the impact of what you do. All engineers, especially senior engineers evaluated on the basis of the impact. But data will not write a strategy for you. That's on you. You have to use your brain, you have to use your life experience, you have to use your bravery sometimes and you have to use your organizational sense to figure out how to get the team to the next goal.

[0:42:29] J: I absolutely know what you mean, annoyingly so even mine. I made a few notes to answer to this, but then you talked more and you basically said what I was going to say. The thing about bravery. I think it's really interesting. Again, it brings us back to the human psychology aspect of it. You mentioned it before that in bigger companies, people are often afraid of exercising their intuition. I find it also really fascinating how to foster the culture that encourages people to the whole spectrum of, to have a balanced life, to have a free and non-preoccupied minds that is then freed up to be able to have intuition and vision and feel safe in their environment to dare to bring, or new ideas even if they're not a 100% rigorously tested by data.

[0:43:24] JC: Yeah. This is not an excuse for arrogance, right? It's not an excuse for just going and making unbridled decisions. It does mean you have to really just think about, how you're going through the motions, or are you doing the thing that rationally makes sense? Now, there's a saying, the customer’s always right. The customer’s always right about the validity of the problems they experience. The customer’s experiencing some pain. They're having an issue, that is true.

The customer probably doesn't know what solution they need. That's your job as the expert to figure out how to solve that problem. I guess, that's called the Henry Ford thing. I'm not sure if you really said this, but if I asked the customer what they wanted, they would have said, faster horses, right? You have to have a deep empathy for the customer. You've spent a lot of time talking to customers. You have a lot of time doing market research, a lot of time introspecting on whether you're succeeding at your goals. You can't forget that you are meant to be the expert. You're the one who’s there to solve problems to somebody else.

[0:44:31] J: All right, so James, I want to get started with Convex. You know my background. I like to say, I'm conceptually a good programmer. I could write fantastic algorithms in my mind. If I have to bring them down on paper, I just very slow also. I try advent of code every year and I've never gotten past A30, just because at some point, it just takes so long to implement what I have in my head. What are my next steps? You've mentioned Angela. Now, I'm an absolute crutch when it comes to anything [inaudible 0:45:04]. I've recently bought a book, literally days ago, about JavaScript, because I'm finally giving in to Atwood's Law. You know Atwood's Law?

[0:45:16] JC: I don't know Atwood's Law, but hopefully, it says that JavaScript is taking over the world.

[0:45:21] J: It sadly does. Atwood's Law aptly and correctly says, any application that can be written in JavaScript will eventually be written in JavaScript. I've decided to stop fighting it. I won't bore you with my story of learning JavaScript. I was 15. That traumatized me for the next 15 years. What are the three requisites for me to get started with Convex? I actually have a web application in mind. It's not complicated when I think about it, but I just don't know where to start. Do I learn JavaScript? Do I have to learn that to learn TypeScript? You mentioned Angular. Can I use other frameworks?

[0:46:06] JC: React. React. But yes, you could use other framework. Yeah. Okay, first thing, given that I have a have a decent sense of who you are as an engineer, right? Skip over JavaScript to TypeScript. JavaScript is going to drive you insane. There's a lot of things in JavaScript that, and in TypeScript too that are pretty inelegant. It's like a runtime type safety. Convex has this concept of input valid is where we validate at runtime that your types are what you say they are. The fact that there's –

[0:46:39] J: This is going to sound really dumb, but I hope that our listeners feel smart if they hear me say dumb things. I realize now, this wasn't found, particularly founded, but I just intuitively assumed you have to know JavaScript in order to learn TypeScript, because it's as an extension, a superset. Is it possible to just learn TypeScript?

[0:46:58] JC: Yes, they're very, very, very, very similar programming languages. TypeScript is JavaScript, plus types to a large extent. They're approximately the same thing. I would say, it's very easy for a non-JS, non-web person to come into this world. They say, the web world, and be a little disparaging about it, like, “Oh, why are they not type safety over here, or why there are no obvious built-in integer types in JavaScript.” You have to use big in if you have integer. Number in JavaScript’s afloat by the way.

Why are there a million web frameworks? Then you start realizing, no, wait. There's a lot to learn about this world and this community. A lot of very sophisticated ideas. I would just start learning TypeScript and I will start looking at something like React. React is a very, very interesting framework, because React is a reactive web development framework. It'll appeal to people want to send functional programming, the you write components and data dependencies and those components refresh when those data dependencies change. There's quite a lot of very interesting ideas in the world, where people in the web board are very familiar with concurrent programming, because there's async everywhere.

There's a lot to love about the web dev world, a lot of very sophisticated, very, very smart people who have come up with some really interesting ideas in the space. Let's say, you're a non-product developer who wants to get started with product development, I would go and you can pick up TypeScript as you go. It's not a language like Rust, which I love Rust, but very hard to learn. You can just you can just get stuck in a TypeScript, what you have for your double equals versus triple equals, which is a bit of a got you object identity, versus equivalents.

Then I would take a look at React, which is a really cool framework and then I would try the Convex tutorial. You will have a chat app, or a to-do app, or some kind of shopping cart, or something I let down in a day. You may repent for your lifetime of infrastructure work and decide that you want to build some products that people will use. It's tremendously gratifying. We have, obviously, at Convex, a cross-section of engineers who work here. There's people who have spent their life building applications. There’s people who've been their life building databases.

It's always fun just seeing how we have a build on Convex day pretty frequently, where we just take a day that everyone here builds products on Convex. Everyone loves those days. They get so excited. Like, “Wow, I built this really cool thing really fast.” I think that's great. That's a great sign for, obviously, for Convex, but for the software development community, having people be excited about building cool applications that they wouldn't otherwise build.

Especially if these folks who say stuff like, “Oh, well I would never have thought of building an app for my bookstore, my book club, or whatever it is. Now I don't find this world intimidating. I'm just going to go build something.” That's awesome, for my collective.

[0:50:22] J: Okay, it sounds as easy, as I always make it out to be in the whole area of movies and the TV series. Remind me, where do I have to go to get started with Convex?

[0:50:30] JC: Convex.dev. C-O-N-V-E-X.D-E-V.

[0:50:36] J: James, thank you very much for coming on the show.

[0:50:38] JC: Thank you, Jeff.

[END]