DeepDetect Server examples

This page lists a set of useful examples, as a piece of code is worth so many words

  • Multi-layer Neural Network (aka Perceptron / MLP) for data and for text
  • MLP for single and multi-target regression tasks
  • Convolutional Neural Network (CNN) for image classification
  • Large-scale Neural Networks from templates, such as GoogleNet, Inception with Caffe and Tensorflow
  • Finetuning GoogleNet
  • Character-based Deep Convolutional Neural Networks
  • Inner layer extraction from a Neural Network, useful for building similarity search engines typically
  • Using XGBoost to train gradient boosted trees from a CSV dataset
  • Using XGBoost to train gradient boosted trees from an SVM sparse format dataset
  • Train a logistic regression from an SVM sparse format dataset
  • Train a multilayer neural network from textual content with sparse features
  • Train an autoencoder on CSV dataset

Principles

API and clients

DeepDetect API is best used as follows:

  • using curl and calling on the server directly
  • using the Python client that builds the underlying calls to the server. Output can be either JSON or Python dict.
  • other clients, for Java, Scala, etc…

Examples below alternate the use of curl and Python client.

Machine Learning & Deep Learning libraries

DeepDetect and the examples below support:

  • Caffe for training and prediction via deep neural nets for images, text and raw data
  • Tensorflow for prediction via deep neural nets for images only at the moment
  • XGBoost for traning and prediction of gradient-boosed trees for text and raw data

Useful terms in the examples below:

  • Large-scale dataset: dataset that does not fit in memory, requiring a intermediary disk storage.
  • Categorical variables: discrete variables whose values represent categories. The distance from one category to another should be equal regardless of the number of categories. E.g.: Color: blue, grey, red, … (as opposed to low, medium, high that can be replaced by a numerical scale)
  • BoW: bag of words features for text classification are common word-based features
  • Character-based: neural net reads text character per character, and thus requires no parsing, see our section on character-based models
  • Sparse: low-memory computations on GPU and CPU dedicated to sparse data, very useful for dealing with BOW and other related features

Requirement:

  • Have the deepdetect server up & running, assumed to be listening on localhost:8080
  • For using Python client, symlink or copy deepdetect/clients/python/dd_client.py to the repository your working from or your PYTHONPATH.

Rules of thumb:

  • Dataset does not fit into memory ? Use mlp or convnet as neural network template and db:true in mllib parameter in /service creation call and again input parameter in /train call: this tells the neural network to take its data from a db and the connector to build this db as needed.
  • Not enough memory on GPU (aka CudaSuccess error) ? Lower the value of batch_size in mllib’s net object
  • Training is tool long, want to stop and resume ? Easy, use DELETE API call to terminate training, a snapshot of the model will be automatically created. Use resume:true to resume training when ready.
  • When re-creating a service after training has completed on a finetuning task, it is best to not use the template parameter for the neural net at service creation as it would replace your existing network definition and lead to an error

Multilayer Perceptron (MLP)

MLP from CSV data with scaling and label offset (Python version of of the training from CSV tutorial)

import time, sys
from dd_client import DD

host = 'localhost'
model_repo = '/path/to/models/covert'
training_repo = '/path/to/data/train.csv'

sname = 'covert'
description = 'forest classification service'
mllib = 'caffe'
dd = DD(host)
dd.set_return_format(dd.RETURN_PYTHON)

# service setting
layers = [150,150,150]
model = {'templates':'../templates/caffe/','repository':model_repo}
parameters_input = {'connector':'csv'}
parameters_mllib = {'template':'mlp','nclasses':7,'layers':layers,'activation':'prelu','dropout':0.2}
parameters_output = {}
dd.put_service(sname,model,description,mllib,
               parameters_input,parameters_mllib,parameters_output)

# training
train_data = [training_repo]
parameters_input = {'shuffle':True,'test_split':0.1,'id':'Id','label':'Cover_Type','separator':',','scale':True,'label_offset':-1}
parameters_mllib = {'gpu':True,'solver':{'iterations':10000,'test_interval':100,'base_lr':0.05},'net':{'batch_size':500}}
parameters_output = {'measure':['f1','mcll']}
dd.post_train(sname,train_data,parameters_input,parameters_mllib,parameters_output,async=True)

# deleting the service, keeping the model (use clear='lib' to clear the model as well)
time.sleep(1)
train_status = ''
while True:
    train_status = dd.get_train(sname,job=1,timeout=10)
    if train_status['head']['status'] == 'running':
        print train_status['body']['measure']
    else:
        print train_status
        break
dd.delete_service(sname)

MLP from CSV data for multi-target regression with scaling & categorical variables


curl -X PUT 'http://localhost:8080/services/regserv' -d '{
       "mllib":"caffe",
       "description":"data regression",
       "type":"supervised",
       "parameters":{
         "input":{
           "connector":"csv"
         },
         "mllib":{
           "template":"mlp",
           "regression":true,
           "ntargets":3,
           "layers":[150,150,150]
         }
       },
       "model":{
         "repository":"/home/me/models/mymodel"
       }
     }'

curl -X POST 'http://localhost:8080/train' -d '{
       "service":"regserv",
       "async":false,
       "parameters":{
         "input":{
           "test_split":0.1,
           "shuffle":true,
           "label":["c_class","m_class","x_class"],
           "separator":",",
           "scale":true,
           "categoricals":["class_code","code_spot","code_spot_distr"]
         },
         "mllib":{
           "gpu":true,
           "solver":{
             "iterations":2000,
             "test_interval":200,
             "base_lr":0.001,
             "snapshot":2000,
             "test_initialization":true
           },
           "net":{
             "batch_size":100
           }
         },
         "output":{
           "measure":["eucll"]
         }
       },
       "data":["/home/mydata/train.csv"]
     }'

curl -X POST 'http://localhost:8080/predict' -d '{
       "service":"my_service",
       "parameters":{
         "input":{
           "connector":"csv",
           "scale":true,
           "min_vals":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.0,0.0,0.0],
           "max_vals":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,2.0,3.0,3.0,2.0,2.0,2.0,2.0,8.0,5.0,2.0],
           "categoricals_mapping":{
             "class_code":{
               "E":5,
               "D":1,
               "B":2,
               "F":3,
               "C":0,
               "H":4
             },
             "code_spot":{
               "R":3,
               "K":2,
               "H":5,
               "A":1,
               "X":4,
               "S":0
             },
             "code_spot_distr":{
               "O":0,
               "C":3,
               "I":1,
               "X":2
             }
           }
         },
         "output":{}
       },
       "data":["class_code,code_spot,code_spot_distr,act,evo,prev_act,hist,reg,area,larg_area,x,y,z","B,X,O,1,2,1,1,2,1,1,0,0,0"]
     }'

Multilayer Perceptron for text

MLP + ReLU activations for BoW text classification

import time
from dd_client import DD

host = 'localhost'
model_repo = '/path/to/models/n20/'
training_repo = '/path/to/news20'

# dd global variables
sname = 'n20'
description = 'NN for n20'
mllib = 'caffe'
dd = DD(host)
dd.set_return_format(dd.RETURN_PYTHON)

# setting up the ML service
layers = [1500,800,300]
model = {'templates':'../templates/caffe/','repository':model_repo}
parameters_input = {'connector':'txt'}
parameters_mllib = {'template':'mlp','nclasses':20,'layers':layers,'activation':'relu','dropout':0.2}
parameters_output = {}
dd.put_service(sname,model,description,mllib,
               parameters_input,parameters_mllib,parameters_output)

# training
train_data = [training_repo]
parameters_input = {'shuffle':True,'test_split':0.2,'min_count':2,'min_word_length':2,'db':False,'count':False,'tfidf':False}
parameters_mllib = {'gpu':True,'solver':{'iterations':2000,'test_interval':200,'base_lr':0.05,'solver_type':'SGD'},'net':{'batch_size':300}}
parameters_output = {'measure':['mcll','f1']}
dd.post_train(sname,train_data,parameters_input,parameters_mllib,parameters_output,async=True)

# report on results every 10 seconds
time.sleep(1)
train_status = ''
while True:
    train_status = dd.get_train(sname,job=1,timeout=10)
    if train_status['head']['status'] == 'running':
        print train_status['body']['measure']
    else:
        print train_status
        break

# deleting the service, keeping the model (use clear='lib' to clear the model as well)
dd.delete_service(sname)

Sparse inputs for all neural nets

When using bag of words or bag of chars representations, DeepDetect supports sparse inputs, typically in SVM format as below.


curl -X PUT "http://localhost:8080/services/testsvm" -d '{
       "mllib":"caffe",
       "description":"classification service",
       "type":"supervised",
       "parameters":{
         "input":{
           "connector":"svm"
         },
         "mllib":{
           "gpu":true,
           "template":"mlp",
           "layers":[150,150,100],
           "activation":"relu",
           "nclasses":2,
           "db":true,
           "dropout":0.2
         }
       },
       "model":{
         "templates":"../templates/caffe/",
         "repository":"/path/to/model"
       }
     }'

curl -X POST "http://localhost:8080/train" -d '{
       "service":"testsvm",
       "async":true,
       "parameters":{
         "mllib":{
           "gpu":true,
           "solver":{
             "test_initialization":false,
             "iterations":5000,
             "test_interval":500,
             "base_lr":0.1,
             "solver_type":"RMSPROP"
           },
           "net":{
             "batch_size":512,
             "test_batch_size":32
           }
         },
         "input":{"db":true},
         "output":{
           "measure":["auc","mcll","f1"]
         }
       },
       "data":["/path/to/train.svm","/path/to/test.svm"]
     }'

Sparse inputs also work with input txt connector, simply add sparse:true to the input connector of the MLP Perceptron for text above.

Autoencoder

Typical example on MNIST data in CSV format, and also see https://github.com/jolibrain/deepdetect/pull/161#issuecomment-235875232.


curl -X PUT 'http://localhost:8080/services/mnist' -d '{
       "mllib":"caffe",
       "description":"mnist autoencoder",
       "type":"supervised",
       "parameters":{
         "input":{
           "connector":"csv"
         },
         "mllib":{
           "template":"mlp",
           "db":true,
           "layers":[500,250,30,250,500],
           "activation":"sigmoid",
           "dropout":0.0,
           "init":"gaussian",
           "init_std":0.1,
           "autoencoder":true
         }
       },
       "model":{
         "templates":"../templates/caffe/",
         "repository":"/path/to/models/autoenc"
       }
     }'
  • train:

curl -X POST 'http://localhost:8080/train' -d '{
       "service":"mnist",
       "async":true,
       "parameters":{
         "mllib":{
           "gpu":true,
           "solver":{
             "iterations":65000,
             "test_interval":5000,
             "base_lr":0.01,
             "solver_type":"NESTEROV",
             "weight_decay":0.0005
           },
           "net":{
             "batch_size":256,
             "test_batch_size":256
           }
         },
         "input":{
           "db":true,
           "autoencoder":true,
           "ignore":["label"],
           "separator":",",
           "scale":true
         },
         "output":{
           "measure":["eucll"]
         }
       },
       "data":[
         "/path/to/mnist_train.csv",
         "/path/to/mnist_test.csv"
       ]
     }'

See https://github.com/jolibrain/deepdetect/pull/161#issuecomment-235875232 for code to visualize the decoded output.

Otherwise, create an unsupervised service to access the inner codes (final layer of encoder), or a supervised service to access the final loss of the network, that reflects the quality of the encoding at prediction time.

Convolutional Neural Network (CNN) for images

3-convolutional layers (32, 64, 128) network + PReLU activations and max-pooling + fully connected layer of 1024 neurons + mirroring and rotations as data augmentation

import time
from dd_client import DD

model_repo = '/path/to/models/convnet/'
training_repo = '/path/to/data/'

height = width = 64

host = 'localhost'
sname = 'imgserv'
description = 'image classification'
mllib = 'caffe'
dd = DD(host)
dd.set_return_format(dd.RETURN_PYTHON)

# creating ML service
model = {'templates':'../templates/caffe/','repository':model_repo}
parameters_input = {'connector':'image','width':width,'height':height}
parameters_mllib = {'template':'convnet','nclasses':18,'layers':['1CR32','1CR64','1CR128','1024'],'dropout':0.2,'activation':'prelu','rotate':True,'mirror':True}
parameters_output = {}
dd.put_service(sname,model,description,mllib,
               parameters_input,parameters_mllib,parameters_output)

# training
train_data = [training_repo]
parameters_input = {'test_split':0.1,'shuffle':True}
parameters_mllib = {'gpu':True,'net':{'batch_size':128},'solver':{'test_interval':1000,'iterations':16000,'base_lr':0.001,'solver_type':'SGD'}}
parameters_output = {'measure':['mcll','f1']}
dd.post_train(sname,train_data,parameters_input,parameters_mllib,parameters_output,async=True)

# report on results every 10 seconds
time.sleep(1)
train_status = ''
while True:
    train_status = dd.get_train(sname,job=1,timeout=10)
    if train_status['head']['status'] == 'running':
        print train_status['body']['measure']
    else:
        print train_status
        break

# deleting the service, keeping the model (use clear='lib' to clear the model as well)
dd.delete_service(sname)

Large Neural Networks from templates

GoogleNet for image classification


curl -X PUT 'http://localhost:8080/services/imgserv' -d '{
       "mllib":"caffe",
       "description":"image classification",
       "type":"supervised",
       "parameters":{
         "input":{
           "connector":"image",
           "height":224,
           "width":224
         },
         "mllib":{
           "nclasses":1000
         }
       },
       "model":{
         "repository":"/home/me/models/mymodel"
       }
     }'

curl -X POST 'http://localhost:8080/predict' -d '{
       "service":"imgserv",
       "parameters":{
         "output":{
           "best":5
         }
       },
       "data":["http://example.com/image.jpg"]
     }'
from dd_client import DD

model_repo = '/home/me/models/mymodel'
height = width = 224
nclasses = 300

# setting up DD client
host = '127.0.0.1'
sname = 'imgserv'
description = 'image classification'
mllib = 'caffe'
dd = DD(host)
dd.set_return_format(dd.RETURN_PYTHON)

# creating ML service
model = {'repository':model_repo}
parameters_input = {'connector':'image','width':width,'height':height}
parameters_mllib = {'nclasses':nclasses}
parameters_output = {}
dd.put_service(sname,model,description,mllib,
               parameters_input,parameters_mllib,parameters_output)

# classifying a single image
parameters_input = {}
parameters_mllib = {}
parameters_output = {'best':5}
data = ['http://example.com/image.jpg']
classif = dd.post_predict(sname,data,parameters_input,parameters_mllib,parameters_output)
print classif

Inception network with Tensorflow

For replicating the example below, you need to build DeepDetect with Tensorflow support, see the related documentation.

Here we are using a pre-trained network with Tensorflow to classify images

Let’s first prepare a model directory and get the pre-trained model

mkdir inception
cd inception
wget https://deepdetect.com/models/tf/inception_v3.pb

Now we create a service through the API:


curl -X PUT "http://localhost:8080/services/imageserv" -d '{
  "mllib":"tensorflow",
  "description":"image classification service",
  "type":"supervised",
  "parameters":{
    "input":{
      "connector":"image",
      "height":224,
      "width":224
    },
    "mllib":{
      "nclasses":1001,
      "inputlayer":"InputImage"
    }
  },
  "model":{
    "repository":"/path/to/inception/"
  }
}'

Now to classify one or more images:


curl -X POST "http://localhost:8080/predict" -d '{
  "service":"imageserv",
  "parameters":{
    "mllib":{
      "test_batch_size":2
    },
    "output":{
      "best":1
    }
  },
  "data":[
    "https://deepdetect.com/img/cat.jpg",
    "https://deepdetect.com/img/thai-market.jpg"
  ]
}'

That’s it, the JSON output contains the predictions from the network.

Finetuning GoogleNet

The script below takes a pre-trained GoogLeNet and finetunes it on a new classification task. The pre-trained network weights file should be present in the model repository. To use ResNet or any other pre-trained network instead of GoogLeNet, change the weights file name accordingly, put the file into the model repository and either change or remove the template accordingly.

from dd_client import DD

model_repo = '/home/me/models/mymodel'
height = width = 224
nclasses = 1000

# setting up DD client
host = '127.0.0.1'
sname = 'imgserv'
description = 'image classification'
mllib = 'caffe'
dd = DD(host)
dd.set_return_format(dd.RETURN_PYTHON)

# creating ML service
model = {'repository':model_repo}
parameters_input = {'connector':'image','width':width,'height':height}
parameters_mllib = {'template':'googlenet','nclasses':nclasses,'finetuning':True,'rotate':False,'mirror':True,'weights':'bvlc_googlenet.caffemodel'}
parameters_output = {}
dd.put_service(sname,model,description,mllib,
               parameters_input,parameters_mllib,parameters_output)

# training / finetuning from pre-trained network
train_data = ['path/to/train/data']
parameters_input = {'test_split':0.1,'shuffle':True}
parameters_mllib = {'gpu':True,'net':{'batch_size':64,'test_batch_size':64},'solver':{'test_interval':4000,'iterations':50000,'snapshot':10000,'base_lr':0.01,'solver_type':'NESTEROV','test_initialization':True}}
parameters_output = {'measure':['mcll','f1','acc-5']}
dd.post_train(sname,train_data,parameters_input,parameters_mllib,parameters_output,async=True)

time.sleep(1)
train_status = ''
while True:
    train_status = dd.get_train(sname,job=1,timeout=10)
    if train_status['head']['status'] == 'running':
	print train_status['body']['measure']
    else:
	print train_status
	break

dd.delete_service(sname)

Character-Based Deep Convolutional Neural Networks

For more information on character-based CNNs, see http://deepdetect.com/applications/text_model/

from dd_client import DD

model_repo = '/home/me/models/sent_en_char'
nclasses = 2

# setting up DD client
host = '127.0.0.1'
sname = 'sent_en'
description = 'English sentiment classification'
mllib = 'caffe'
dd = DD(host)
dd.set_return_format(dd.RETURN_PYTHON)

# creating ML service
model = {'repository':model_repo}
parameters_input = {'connector':'txt','characters':True,'sequence':140,'alphabet':"abcdefghijklmnopqrstuvwxyz0123456789,;.!?'"}
parameters_mllib = {'nclasses':nclasses}
parameters_output = {}
dd.put_service(sname,model,description,mllib,
               parameters_input,parameters_mllib,parameters_output)

# classifying a single piece of text:
parameters_input = {}
parameters_mllib = {}
parameters_output = {}
data = ['Chilling in the West Indies']
classif = dd.post_predict(sname,data,parameters_input,parameters_mllib,parameters_output)
print classif

The output naturally comes in JSON form:


{
  "status":{
    "code":200,
    "msg":"OK"
  },
  "head":{
    "method":"/predict",
    "time":104.0,
    "service":"sent_en"
  },
  "body":{
    "predictions":{
      "uri":"0",
      "classes":{
        "prob":0.702814519405365,
        "last":true,
        "cat":"positive"
      }
    }
  }
}

Inner Deep Neural Network layer extraction

First it is required to setup an unsupervised service:


curl -X PUT "http://localhost:8080/services/imgserv" -d '{
       "mllib":"caffe",
       "description":"image classification",
       "type":"unsupervised",
       "parameters":{
         "input":{
           "connector":"image",
           "height":224,
           "width":224
         },
         "mllib":{
           "nclasses":304
         }
       },
       "model":{
         "repository":"/path/to/model/"
       }
     }'

Note that the service’s type is unsupervised.

Now, requesting net’s layer pool5/7x7_s1 from GoogleNet:


curl -X POST "http://localhost:8080/predict" -d '{
       "service":"imgserv",
       "parameters":{
         "mllib":{
           "extract_layer":"pool5/7x7_s1"
         }
       },
       "data":["https://i.ytimg.com/vi/4sMQ9dEpuo8/maxresdefault.jpg","https://i.ytimg.com/vi/4sMQ9dEpuo8/maxresdefault.jpg"]
     }'

Note the extract_layer parameter to the mllib object. The response is:


{
  "status":{
    "code":200,
    "msg":"OK"
  },
  "head":{
    "method":"/predict",
    "time":2695.0,
    "service":"imgserv"
  },
  "body":{
    "predictions":[{
      "last":true,
      "vals":[0.0,0.16237296164035798,0.0,2.24597430229187,0.5692943334579468,...,1.8091919422149659,0.0],
      "uri":"https://i.ytimg.com/vi/4sMQ9dEpuo8/maxresdefault.jpg"
    }]
  }
}

You can similarly try to get the last layer from the net, with identifier loss3/classifier.

Boosted Trees

Gradient Boosted Trees from SVM or CSV input format

For more information on training models with XGBoost, see http://deepdetect.com/tutorials/xgb-training/


curl -X PUT "http://localhost:8080/services/covert" -d '{
       "mllib":"xgboost",
       "description":"forest classification service",
       "type":"supervised",
       "parameters":{
         "input":{
           "connector":"csv"
         },
         "mllib":{
           "nclasses":7
         }
       },
       "model":{
         "repository":"/path/to/models/covert"
       }
     }'

curl -X POST "http://localhost:8080/train" -d '{
       "service":"covert",
       "async":false,
       "parameters":{
         "mllib":{
           "iterations":100,
           "objective":"multi:softprob"
         },
         "input":{
           "label_offset":-1,
           "label":"Cover_Type",
           "id":"Id",
           "separator":",",
           "shuffle":true,
           "test_split":0.1
         },
         "output":{
           "measure":["acc","mcll","f1"]
         }
       },
       "data":["/path/to/train.csv"]
     }'

Training from SVM format is supported and easy: use the "connector":"svm" when creating the service and leave input empty when training.

Related