[0:00:00] L: John, welcome to Software Engineering Daily.

[0:00:03] JA: Hey, Lee. Thanks for having me.

[0:00:05] L: Yeah. First of all, is that the right title for you? Chief Innovation Officer?

[0:00:09] JA: Yeah. That's correct.

[0:00:12] L: Tell me the story behind that.

[0:00:13] JA: All behind the title.

[0:00:14] L: Yeah.

[0:00:15] JA: It's interesting to me. I've held a lot of positions over the course of my career. Tomas and I were trying to figure out where I would fit in. It was interesting, because as a technical co-founder, I was the person who wrote the first line of source code, and I started building the site from nothing. When we had brought Lloyd on, Lloyd Taylor, who is our COO and CTO now, it was trying to understand where my skills would fit in. Because I have always been more of an engineer, an individual contributor. I definitely have been a director of departments. I've had teams of people, but I have to have my fingers in the system. I want to be building things. That's how I keep track of where the technology is and where it's going.

Interestingly enough, we had seen this article, I believe it was on Forbes, about what the CINO title was. We looked at that, and Tomas and I were going, “That's you. That's exactly you.” I said, “Okay. Well, let's run with that.” I like that. Maybe if I'm CINO, as opposed to CTO, I won't have to deal with the customers as much. I mean, don’t get me wrong, I love our customers, but I'm definitely – I'm an engineer’s engineer.

[0:01:36] L: Yeah, that makes sense. Even the most tactical CTOs, I think get caught up in the – there's so much non-technical stuff you have to do, and it's just so hard to make it work. Something like this lets you do, the technical you like to do. That's great. Alembic uses AI, essentially to – let me know if this is not an accurate description. You use AI to correlate and report on marketing data and not – you don't generate marketing content. You don't create, write tweets, or write articles using AI and those sorts of things. But you report on marketing data. Is that a valid assumption, valid statement?

[0:02:18] JA: It is. It is. But there's a few different places where we incorporate AI into the platform. It's interesting, because I think there's a thin line in the world right now, where artificial intelligence is very, shall we say, it's being used to define a very wide series of solutions. We would have said, expert systems, or we would have said, statistics, or predictive analytics. There is a time and a place for AI and we use it in a number of different ways. One way is looking at where data is going and trying to predict what was going to happen in a time series.

Prior to that, people would have used linear regression, or other math – the standard math formulas to do that. But you can incorporate AI systems to say, okay, I have this idea of where things are going, but I know from previous data, it's going to go that way, so the algorithm says, okay, we're going in this direction. One usage is for predicting the future. The other usage is for looking at images and being able to say, “Okay, what's in the image? What keywords are in here?” I look at image, Microsoft, Amazon, there's a TensorFlow. There's a whole bunch of different systems for identifying what's in an image.

Then correlating that back to marketing data. An example would be like, you posted an image that had a dog in it, and that image got these many impressions. We know that over the course of everything you've done, the images with dogs perform better than the images with, I don't know, cats. That level of analysis into the image and then being able to correlate it back to social media data is huge for marketers, because I think frequently, they don't know what works and they don't know what does. I think that's one of the questions that Alembic tries to answer.

[0:04:06] L: One of the things you do is you take marketing information that is sent out and you categorize it and then figure out which things perform well and which categories, etc.

[0:04:19] JA: Yes, that's correct. From an engineering standpoint, you come into us, you can set up Alembic in under 10 minutes, right? Because you connect your accounts to us. The accounts we use, OAuth and other standard protocols to link your accounts across a multitude of social media and advertising platforms and analytics platforms, like Adobe Analytics, Google Analytics 4, LinkedIn, Twitter, all the usual ones. What we do is we ingest all your data and then we analyze every post you've ever made, every connection that anyone's ever done with your account. Then we take all those metrics and we're talking hundreds of thousands of metrics per day. We then apply algorithms to identify what's happening. It's a combination of AI, statistics, category algorithms, and natural language processing. All of this is used to give the marketer a picture of where they are.

[0:05:22] L: No, I was going to say, so if you look at a typical sales pipeline, is this mostly the front-end marketing piece? Or can it apply anywhere during the pipeline process?

[0:05:33] JA: Right. Just to be clear, I'm a data person, right? I'm not a marketing person. But Tomas, Tomas is an amazing marketing person. As my co-founder, he used to work for WP Engine. He's got a long history of being a CMO. There are a number of questions that come up for marketers, right? I think that, especially in the sales cycle, one of the hardest problems is attribution. You don't know where the sale came from. You don't know what works, or when. 

Another issue is content, right? Yes, on the front end, you don't know what content is actually performing. One of the rules, and you may know this from doing podcasts and other types of social media advertising is they say, keep posting, keep posting, keep posting, right?

[0:06:17] L: What do you post?

[0:06:19] JA: Well, what do you post, right? One, what do you post? Two, a lot of people don't realize that long-tail content actually drives advertising. It drives companies. It drives sales. People assume that only the newest, greatest content is bringing people in. When we go back and we have an entire screen in our application that tells people, here's your long-tail content and here's how it performed. Did you know that this post from three years ago is actually driving a lot of traffic to your site?

It's interesting, too. Recently, there's an article. I think it was an article on CNET, right? They said, CNET was deleting old posts, because they thought it was impacting their SEO and impacting traffic to their site. I guarantee you, that's not the case. I think they have bad data.

[0:07:03] L: Impacting, but impacting the wrong direction they were thinking. Yeah.

[0:07:09] JA: The wrong direction. Probably negative when they removed it. Yeah. They were deleting posts. Who knows?

[0:07:13] L: Well, I did not hear about that. As a big fan of long-tail content, I have a lot of long-tail content that I produce. I find that critical to keep going. I have –

[0:07:25] JA: How would you know?

[0:07:26] L: Here exactly. Here, yeah. I know in my business, so content creation is a big part of what I do. I have no information about which leads come from which pieces of content and how all that works. I’m trying to use UTM codes, but that's all real simplistic stuff and that doesn't really give you –

[0:07:47] JA: Yeah, and it's about to go away, too. That's another really impactful thing that's occurring, right? Apple has already announced, and Safari, that they're going to block all UTM codes. Chrome, obviously, is not going to do that, but Safari will, and that's going to impact every iOS device. Here's another aspect of my career, right? I spent a long time working in privacy and security. I was a founding member of Twitter's security team. This era of surveillance capitalism is coming to an end. Tracking the user is going to be more and more difficult.

I think, one of the things that we offer with Alembic is a way to analyze the impact of your company without directly surveilling a user and having this massive privacy issue. It's only getting harder with the – they call it the death of cookies, right? The cookies are going away. UTM codes are going away. Markers need somewhere to go that gives them an accurate representation that was occurring. Anything we can do to foster privacy and I'll still allow marketers to do their job, I think is fantastic. I think Alembic makes that happen. Yeah.

[0:08:59] L: Yeah. One of the things AI allows you to do is you can analyze the back-end data and make assumptions and correlations about who's doing what, rather than tracking the actual workflow of an individual person and seeing what's doing what. Is it fair to say that you can track – if you've got a sudden uptick in customer signups, that you can track what types of things have occurred recently?

[0:09:28] JA: Yes.

[0:09:29] L: Okay. Can you go into an example of that?

[0:09:31] JA: Yeah. Give me an example. For example, let's say – It's funny. Let's say, you made a tweet. I don't know what they're called now. I guess, Elon thinks they're called Xs, or something. You make a post on some social media website. Let's just be generic. That social media site is connected to Alembic. The user, you start to see an increase in views from a multitude of users. People are now engaging with that content, right? Then simultaneously, you see an uptick on portions of the Google Analytics pipeline. Google Analytics has goals and it has events. Most people have a sales funnel. Here's when they hit the first page, here's when they put something in their card, here's when they – there's an event for every step of that sales funnel.

We can actually see the correlation between like, here's the number of impressions that occurred on, let's say, YouTube, and here's a number of impressions that occurred when you advertise that post on Twitter. Then following that chain, we now see an uptick in the number of goal completions on a particular phase of the Google Analytics pipeline. That's an overview that's not commonly available. You don't get to see that on Google Analytics. You can't really instrument the other sites. You can only instrument sites that you have direct control over. In the case of Twitter, you may use a link shortener, or you may have a post on Facebook, but you can't really correlate that back to Google Analytics.

Being able to see that entire pipeline and correlate those spikes together, so we call this – the internal name, we call it ECD. It's an event correlation detector. We see a multitude of events across different services. We can use internal algorithms and AI to detect that spike, and then correlate the spikes across all the services together, and then give you, basically, a heat map that says, “Look, on this day, something happened. Why?” You click on it, and we show you every event that happened on that day.

An interesting aspect of our work is that when you go to competitors, like if you look at Salesforce, Hootsuite, anyone who's currently pulling social media data, there's something that we do that they don't, which is we have a system called Time Series Reconstruction. If you go today and you ask Facebook, what happened, like, let's say you made a post on Monday, and Tuesday afternoon, you got a 100 views and Wednesday, you got another a 100 views, and by the time you get to Friday, it's been 500 views. Hootsuite will report that you got 500 views on Monday, because the only data they have access to is the post ID, the date of posting, and the lifetime value, and the lifetime number of impressions.

We reconstruct that timeline, and this is how we're able to do these detections and spikes, because we can say, well, we know the delta from Monday to Tuesday was a 100, and Tuesday to Wednesday was, let's say, 300. Oh, that looks a spike. That's above the normal.

[0:12:29] L: You pull regularly to get that data, and then we use that to reconstruct?

[0:12:33] JA: Yeah, regular polls, and then cross calculations, and then using AI on top of that. It's a very interesting dynamic. I think it starts to solve the attribution problem of like, where did your sales go?

[0:12:47] L: Right, right. But that's a direct attribution. I mean, you see a spike on this day, in this type of social media, or this type of activity, or whatever, your cats, and pictures, or whatever, and then generates sales. What about, a lot of businesses have a rather long process of engagement before a customer becomes a customer, or a customer actually signs up to get a newsletter, or signs up to get a –

[0:13:16] JA: Yeah.

[0:13:17] L: They might read five pieces of content before they get to that, or they get to that –

[0:13:22] JA: Well, we've got a few different ways of tracking that, right? One is we have a Salesforce integration that's mostly done right now. We're working on that for our customers, so we can track areas of this, like the Salesforce pipeline. If you're an enterprise customer, and you're working down different portions of that, yeah, of that sales funnel, we can track there. We also offer a fairly comprehensive link shortener, so you could actually apply links along the path, so we can track links, and then we can feed that into the correlation engine with everything else.

[0:13:55] L: Without requiring specific user information that's going away.

[0:14:00] JA: Yeah, that's correct. Yeah. We like to work in the aggregate. We don't want to collect data individually, because obviously, we want to be a privacy preserving marketing solution, which is an oxymoron, but it's absolutely possible.

[0:14:16] L: You gather lots of data from lots of different sources and correlated. Social media is one of those sources. I'm assuming, traditional news sources information about?

[0:14:29] JA: Yes, we actually integrate with a number of news APIs and also, broadcast podcast APIs. We have partnerships with companies that track and provide transcripts for that area. One wonderful feature Alembic among many is this ability to track mentions. We can actually look for spikes in the news media and in podcast. If we start to see a particular phrase, or a product name, and this is a lot of AI correlation. Mention extraction NLP works. We say, “Oh you're looking for this particular mention of, I don't know, some sports team.”

[0:15:09] L: [Inaudible 0:15:10].

[0:15:10] JA: Yeah, some sports team, and then some show. Oh, look. Okay, there was this – Your technology was being talked about in these six podcasts. Then it was mentioned on the news here. We know how many impressions they were because we have Nielsen data and we have other data. We can look at that and now synthesize this whole picture of where things are going.

[0:15:29] L: You take a not only social media, traditional news sources, industry blogs. You're able to apply AI on the content as well in order to know what's in it. You mentioned the catch, but you can look for mentions, etc.

[0:15:44] JA: Yeah. We're still working on refining that work. It's very interesting. I think, one thing is that where we stand right now in the AI market is, there are a lot of good solutions out there that do things. Some of it looks like – it looks like a ChatGPT to me. Always looks like a fancy parlor trick, because it's got predictive – it's predicting words, right? Based on strengths and random numbers in previous training. At the same time, it takes so much compute power to process one article –

[0:16:19] L: One word.

[0:16:20] JA: - that it makes it really hard to use that technology right now on a regular basis. The large language models are great. They do very wonderful things. I think at the same time, we're not really there yet as far as speed, especially if you got someone that's posting millions of articles. There might be something like an NVIDIA, or a search for the word Linux, you're going to get tens of thousands of articles to process.

In some of those cases, going back to traditional computer science and NLP algorithms is sometimes more effective than trying to apply the new, shiny to all the problems. As an engineer and as – if you're a technologist, you need to really peel back the layers, right? You need to understand, where can I apply these AI technologies in a way that makes sense? Where is the traditional approach still significantly more performance and better for your company? This is going to be a hard thing for people to understand with this hype cycle we're in currently with AI.

[0:17:26] L: Once this hype cycle ends, or goes on the other side of the hype cycle, where do you think the reality of the technology is really going to be and where do you think the focus is going to be?

[0:17:37] JA: Well, I always go back. There's a wonderful book. I might screw up the title here, but there was an economist, her name is Carlota Perez, and she had a book, I think, was called Venture Capital and Technological Revolutions. It's an absolutely amazing book. She was the one that talked about the curve, okay. Where new technology adoption, the trough of disillusionment, where people forget about things and get past the hype, and then the wide acceptance of the technology, right? We're seeing this now. We're seeing this now, where there's a bit of disillusionment with AI. There's a – what's the word? People are coming against it, right? They're saying, “Oh, driverless cars are dangerous and AI is dangerous.” Yeah, all these things are true, but all to –

[0:18:25] L: I can take my job.

[0:18:26] JA: Yeah. All these things are true about all technologies, right? All technologies have a bad side and a good side. I believe that this will mellow out. I think it'll reach a steady state. The things that I'm most excited about from an optimistic perspective are, I look at assistive technologies, right? I look at translation. I look at image identification. Being able to tell someone what's in something if you're blind and you can't see. Looking at, GitHub co-pilot is a fine example, but there's a lot of – obviously, a lot of controversy around that of assistive coding. That's huge.

Augmented reality with assistance is probably something that's going to really pick up. It's already getting there. If Apple does, what I think they're going to do with the glasses, it's going to be pretty serious. Then the scary part, right? The scary part is we could go into Skynet and terminate world and we're all doomed. That's Boston Dynamics putting ChatGPT into Big Dog. Oh, okay. Now it's going to happen. I think with all technologies, there's always fear and there's always something new that's going to happen.

Honestly, the one lesson I learned very hard at Twitter was second order effects. You design something, you think it's going to have a certain purpose, or you think you're making things better. In some cases, you make things worse. Obviously, look at Facebook and Cambridge Analytica, I mean, that was a terrible use of that technology that impacted democracy and hurt things. That was an application of both statistics and AI, but with a very upsetting direction. We have to be concerned about where these things are going. I don't think we should be completely dismissive and not explore what technology has given us.

[0:20:22] L: That's a great way to think about it. I have a similar view on how all this is going to work out. I think the people who are – I've got a friend who's a screenwriter, and she's obviously out on strike right now. She's constantly worried about the effect of AI in picking over her job and taking over the screenwriting job. I keep wanting to say, you should be excited about the possibilities of what AI can do to help you write better screening. Yes, do better screenwriting, versus it taking over your job. Your job's going to change. Your job's definitely going to change, it's not going to go away.

[0:21:07] JA: The only constant is change.

[0:21:10] L: Exactly, exactly.

[0:21:13] JA: It's interesting to mention that, because as someone who – I mean, in my off time, I write music and I look at – I love working in video and things like that. I've had people ask me, what do you think? What do you think about all this AI stuff? These are people that are, they're artists. They're not technologists for the most part, or they’re musicians. Like, this is terrible. It's going to ruin everything. It's like, I think about sampling. When people could first sample audio and incorporate that into other forms of art, they said, this will ruin everything. People will just sample things. I said, “No, well, you're actually making a new – you’re making a new art form. You have a new tool.” You have a tool you didn't have before.

Perhaps, this will happen with AI. At the same time, we have to be very aware of where the AIs are trained and what does this mean in the face of things like, licensing, copyright and patent, right?

[0:22:05] L: Absolutely.

[0:22:07] JA: A comment that – I actually made this last night. I was sitting in a bar talking to a friend about this. They made a mention about, well, it was trained on everything. Isn't it just plagiarism? Isn't it just stealing from other programmers? I said, “Well, how did you learn? Where did you get your sorting and searching algorithms from? How did you learn to do those things?” “Oh, and I went to school, I read a book.” I said, “Right. You remember that vaguely. Then you wrote a new version of it.” “Oh, yeah, that's what I did. But it was original work.” I said, “Well, no. Actually, it wasn't.” I said, Don Knuth –

[0:22:47] L: So many ways to sort.

[0:22:49] JA: Yeah, he wrote so many ways to sort. He wrote, yes, there’s so many ways to sort. He wrote sorting and searching algorithms. He wrote a wonderful, The Art of Computer Programming, massive tome, that was the Bible for the computer industry. He did that back in the 60s. The thing is, is like, okay, so now you're dealing with something that can remember all the lessons that's ever been taught. Maybe not in a great way, but certainly better than you can. Who's in the right? This is an ethical dilemma that we're going to have a long time sorting out.

[0:23:21] L: Yeah, I think AI ethics is definitely a career choice that's going to be around for a long time to come.

[0:23:29] JA: Yeah, exactly.

[0:23:31] L: What's next for Alembic? I understand what you're doing now, but what's the next steps?

[0:23:38] JA: Well, I always look at – I look at where we are today, and we have written a really wonderful piece of software. It is as with all companies, right, you never want to get married to the software. What I mean by that is, I think at Twitter, I think we rewrote it three times. Once in Ruby, once in Scala, once in other languages, Go and Erlang and all kinds of things. The idea is, is that as the company evolves, every decision you make, you’re making thousands of decisions to make the startup go, and they are in a way, they're a hedge, right? They're like, I'll get this working for this amount of time, and I'll go clean it up and make it better.

I think that what we've done is built a really great system. Internally, we want to make that the best it can be. We want to make it scale. We want to make it work for anything. We want to have correlations across as many services as possible. We want to accelerate the ability for engineering and sales to add more connectors, add more sources. Obviously, always trying to improve the software. The idea is, can you get it to a point where the most basic user can get what they need out of the system, and then the most advanced user can really build on a platform that works for them?

Not trying to make any forward-looking statements, but the thing that I look at is, can we make this a universal platform for all the data that marketers need? Can we create a community around it that people can really build and grow in? I think that's where I'd like to see things going.

[0:25:14] L: New and more data sources, more advanced AI, more advanced analytics.

[0:25:19] JA: New more data sources. Obviously, they advanced analytics. Being able to offer some of the tools that we're using internally, externally. I think that that's a huge thing. Because we do use a lot of these tooling to create results for people, but it would be wonderful if people could ask for what they want and get it back, without us having to code. That'd be great.

[0:25:42] L: Makes sense. Well, thank you very much. Yeah, my guest today has been John Adams, who's the Co-Founder and Chief Innovation Officer at Alembic, an AI marketing, reporting and an analytics company. John, thank you so much for being my guest on Software Engineering Daily.

[0:25:58] JA: Yeah, thanks for having me, Lee. It's a pleasure.

[END]