Visual Search with Neel Vadoothker

If I have a picture of a dog, and I want to search the Internet for pictures that look like that dog, how can I do that?

I need to make an algorithm to build an index of all the pictures on the Internet. That index can define the different features of my images. I can find mathematical features in each image that describe that image. The mathematical features can be represented by a matrix of numbers. Then I can run the same algorithm on the picture of my dog, which will make another matrix of numbers. I can compare the matrix representing my dog picture to the matrices of all the pictures on the internet.

This is what Google and Facebook do–and we covered this topic in our episode about similarity search a few weeks ago. Today, we evaluate a similar problem: searching images within Squarespace. Squarespace is a platform where users can easily build their own website for blogging, e-commerce, or anything else.

Neel Vadoothker is a machine learning engineer at Squarespace, and he joins the show to talk about how and why he built a visual similarity search engine.

If you like this episode, we have done many other shows about machine learning. You can check out our back catalog by going to or by downloading the Software Engineering Daily app for iOS, where you can listen to all of our old episodes, and easily discover new topics that might interest you. You can upvote the episodes you like and get recommendations based on your listening history. With 600 episodes, it is hard to find the episodes that appeal to you, and we hope the app helps with that.

Software Daily

Software Daily

Subscribe to Software Daily, a curated newsletter featuring the best and newest from the software engineering community.