Are you familiar with the images shown above? What if you want to find out what these images contain? For those of you who are not familiar with Google reverse image search, do the following: Go to Google Images and click on the camera icon towards the right end of the search bar.
Now, you can either paste the URL to an image or upload your own image. You can use the two images above to check it out. Once you upload the image, click on search and see what happens! How on earth did it find out what was there in this image? We didn’t provide any kind of information and yet it was able to find out exactly what those images contain. Not only that, it provided many similar images and a lot of links providing more information about the same. This is the magic of reverse image search!
Why do we need it?
When Google first came out, it was a text based search engine. It means that people would enter text in the search bar and Google would try to find out the links which are closest to what the user wants. But this search was rather inane and unintuitive. After sometime, Google improved their service and started to understand what the users want. Whenever a user types something, it is important to understand the structure of the sentence to provide the most accurate search results. So Google heavily integrated natural language processing into the service. Then the world starting moving more and more towards visual data. If the users know what they want, they would just type it and Google would display millions of links pointing towards that. But what if I see something and I don’t know what it’s called? I just want to take a picture and find out more about it. This is where reverse image search comes into picture.
What exactly is it?
Whenever a user enters an image as the input, a reverse image search engine tries to understand the contents of the image. It extracts certain prominent features from that image and creates a compact digital signature (or ‘fingerprint’). It will then use this to compare against all the images in the database. The good thing is that it is invariant to the size or orientation of the image. It means that if a monument has horizontal orientation in one image of smaller size and vertical orientation in some other image of bigger size, the algorithm will still match these two images because the content is the same. For computer vision buffs, some of the most popular feature descriptors include SURF, SIFT (although SIFT is under heavy patent protection), FAST, BRIEF, SLF, PHOW etc. The algorithms are proprietary and companies don’t tend to reveal this information.
How is it done?
Since there are billions of images on the web, how do we look for this particular image in a fast and efficient way? We expect an accurate result really quickly along with a bunch of images and links which are similar to what we are looking for. A commonly used technique is Locality-Sensitive Hashing (LSH), which is used for indexing and searching the database. It is a method for performing probabilistic dimensionality reduction of high dimensional data. Similar items are hashed into a single bucket. Other methods are used as well, but the common idea is that the number of dimensions should be reduced so that the search speed can be increased. For example, if you are asked to spot Seville on the world map, it would be helpful if you know that it is a city in Spain. You will not waste your time in searching the entire globe. Instead, you will just limit your search to Spain. Similarly, efficient organization is required to reduce the time taken to come up with search results.
Google is not the only player in this field, but it’s certainly the biggest. There are many other services like TinEye, Byo Image Search, Gazpopa, RevIMG etc, but they don’t have Google’s resources. They have to continuously crawl the web and keep indexing the images. Google has already indexed billions of images and they do it at a rapid pace. TinEye started up before Google and was actually the first service that was actually good in reverse image search, but then Google came up with their own and gobbled up all the competition.