Chaining models via the API
Since release v0.9.3.1, DeepDetect supports chaining multiple models and actions. This allows for instance to detect objects then run other models on object crops. A typical application is detecting faces, then estimating every face’s age.
Let’s define the key elements:
/chain
is the REST resource defined by DeepDetect to execute a tree-like succession of models and actions- a model is represented by a
service
, as in regular API/predict
calls - an actions is a stateless piece of code that takes a model’s output, transforms it as needed, and pass it on to the next set of models. As an example, the
crop
action crops objects after a model has detected them. Available actions are listed at the bottom of this page.
In a chain, the results from all executed model are then merged into the final JSON response. It is important to note that a chain
is stateless, i.e. it is executed once and it’s structure does not remain into memory, only the existing services remain.
In order to use the PUT /chain
REST API, you’ll first need to create the services used in the chain.
In the following example, faces
and age_real
services are already running on DeepDetect server:
curl -X PUT 'http://localhost:8080/chain/faces-crop-age_real'
-d '{
"chain": {
"calls": [
{
"parameters": {
"input": {
"connector": "image",
"keep_orig": true
},
"mllib": {
"gpu": true
},
"output": {
"bbox": true,
"confidence_threshold": 0.3
}
},
"service": "faces",
"data": [
"https://deepdetect.com/img/examples/multiple_faces.jpg"
]
},
{
"id": "crop",
"action": {
"type": "crop"
}
},
{
"parent_id": "crop",
"parameters": {
"input": {
"connector": "image",
"keep_orig": true
},
"output": {
"confidence_threshold": 0.01,
"best": 1
}
},
"service": "age_real"
}
]
}
}'
calls
is an array of the services and actions to be executed. The call above executes a first service that detect faces in an image, then executes a cropping action, then runs the age estimation model.
Results are available in json as usual, with the age_real
service results nested inside each faces
result:
{
"status": {
"code": 200,
"msg": "OK"
},
"head": {
"method": "/chain",
"time": 3467
},
"body": {
"predictions": [
{
"uri": "https://deepdetect.com/img/examples/multiple_faces.jpg",
"classes": [
{
"age_real": {
"classes": [
{
"prob": 0.04538746178150177,
"last": true,
"cat": "17"
}
]
},
"cat": "1",
"prob": 0.9733325242996216,
"bbox": {
"xmax": 1224.7215576171875,
"ymax": 123.81903076171875,
"ymin": 308.4881896972656,
"xmin": 1007.9154663085938
}
},
...
]
}
]
}
}
Chain tree structure
Chains are designed to support tree-like structures. In practice, this means one model’s output can serve as input to many models, thus branching the execution.
Tree-structured chains require the id
attribute on every chain service and actions, so that they can be refered as parent_id
by lower level services and actions. See the example below:
curl -X PUT 'http://localhost:8080/chain/test'
-d '{
"chain": {
"calls": [
{
"id": "root",
...
},
{
"parent_id": "root",
"id": "leaf_child_1",
...
},
{
"parent_id": "root",
"id": "leaf_child_2",
...
},
{
"parent_id": "leaf_child_2",
"id": "leaf_great_child_2",
...
},
...
]
}
}
Action chain items
Between each service in a chain, action can by call with the following types:
Cropping
- crop: Crops the bounding boxes from the parent call and send each box to the children services.
- padding_ratio: extend the bounding boxes before cropping by a value that is passed as a ratio to the unpadded bounding box size
- save_crops: saves cropped images on disk
- save_path: path to where the crops are saved
Example:
curl -X PUT 'http://localhost:8080/chain/test'
-d '{
"chain": {
"calls": [
...
{
"action": {
"type": "crop",
"parameters": {
"padding_ratio": 0.05,
"save_crops": false,
"save_path": "/path/to/crops"
}
}
},
...
]
}
}
Filtering
- filter: Keeps parent results based on a list of class names
- classes: target class names to keep
Example:
Keep results for the “car”, “dog” and “cat” classes only.
curl -X PUT 'http://localhost:8080/chain/test'
-d '{
"chain": {
"calls": [
...
{
"action": {
"type": "filter",
"parameters": {
"classes": ["car", "dog", "cat"]
}
}
},
...
]
}
}
Image rotation
- rotate: Keeps parent results based on a list of class names
- orientation:
relative
(default) uses angle wrt to current position, orabsolute
that rotates to the given angle.
- orientation:
Example:
Rotate from a model that predicts the relative rotation angle to apply.
curl -X PUT 'http://localhost:8080/chain/test'
-d '{
"chain": {
"calls": [
...
{
"action": {
"type": "filter",
"parameters": {
"orientation": "relative"
}
}
},
...
]
}
}