Moving a service across servers
It is often the case that a machine learning service is used in some application, while it is being refined on the side through many witty manipulations and trials. DeepDetect allows to quickly move machine learning services across servers, typically from a development to a production server.
Below we assume we are moving a service from development to production server:
- production server is already running
dede
- target service is
service1
on production server, will be replaced byservice2
- we avoid any downtime
Steps for moving a service from development to production server:
- Copy the
repository
model dataset to the production server, in what follows this repository has value/path/to/production/service2/model
- Assuming
service1
is the running target, create a newservice2
service with aPUT /services/service2
call. Of importance:- do not use
template
parameters in themllib
object as this would erase your existing neural net model architecture - as an example a typical image classification service creation call would look like:
- do not use
curl -X PUT "http://localhost:8080/services/service2" -d '{
"mllib":"caffe",
"description":"image classification service",
"type":"supervised",
"parameters":{
"input":{
"connector":"image"
},
"mllib":{
"nclasses":1000
}
},
"model":{
"repository":"/path/to/production/service2/model"
}
}'
service2
is now ready for action, a few tips:- run one or more test
POST /predict
calls: the first call may load/reload the full model and thus be slightly less quick than subsequent calls - redirect your calls to
service2
- you can leave
service1
running or use it to compare results in real-time
- run one or more test
- phase
service1
out with aDELETE /services/service1
call, see API
Notes:
- it is not possible to rename a live service
- if you are calling development and production servers from the same endpoint, beware that
/path/to/production/service2/model
may differ from/path/to/development/service2/model
and thus you need to change thePUT
call accordingly