Image similarity search


DeepDetect Server and Platform come with image similarity search built-in. DeepDetect supports two type of image similarity analysis:

  • Image global similarity search: indexing, search & similarity over full images. This capability is simple and mainstream since the emergence of deep neural networks for images.

  • Objects in image similarity search: indexing, search & similarity over detected objects within images. This capability is less commonly made available than the one above and is more powerful: it allows to build search engines for objects that are within the images, as opposed to search of globally similar images.

Overall, this is a two step process:

  • A Deep Learning model is run over images to process the image globally or detect objects. The output are semantic vectors that represent the images or its relevant portions (i.e. objects). These vectors are indexed for later similarity search.

  • The same deep model is run over new images, and the outputs are used to search the previously built index.

In the following we’re setting up a object similarity search solution.

TODO: images of typical visual output


We start by setting up the DeepDetect Server, we assume a GPU setup with Docker that can be adapted as needed.

We then setup the required model.

We use the DeepDetect object detection model for object similarity search. Follow installation instructions, then test with the call below:

curl -X POST 'http://localhost:8080/predict' -d '{
    "data": [
    "parameters": {
        "mllib": {
            "gpu": true
        "output": {
            "confidence_threshold": 0.1,
            "rois": "rois"
    "service": "detection_201"

This yields the following results: