TensorFlow Deep MNIST Demo

This example will take you through creating a microservice that recognizes numbers between 0 and 9, based on the CNN model from the tensorflow deep MNIST demo. In this example you will learn how to deploy the microservice from the prepackage docker image available in seldon-server.

If you want to learn how to build this image from your model, check out our TensorFlow Digit Classifier Advanced Tutorial.

Prerequisites

Create a Predictive Model

In this demo, we have created and pre-packaged a predictive model built with tensorflow. A wrapper to call this library has been added to our python library to make integrating it as a microservice easy. However you can build your own model using any machine learning library.

The image is seldonio/tensorflow_deep_mnist.

For details on building the model see the tensorflow deep mnist tutorial.

For details on building a docker image from your model see our detailed deep mnist tutorial.

Train the model

The model training can be accomplished by running the Kubernetes job in kubernetes/conf/examples/tensorflow_deep_mnist/train-tensorflow-deep-mnist.json which should have been created when you followed the install and configuration steps.

Create the kubernetes job to train the model:

cd kubernetes/conf/examples/tensorflow_deep_mnist
kubectl create -f train-tensorflow-deep-mnist.json

This job will:

This job can take several hours to complete on a machine without a CUDA enabled GPU. If you do not want to wait this long you can load a pretrained model using the Kubernetes job in kubernetes/conf/examples/tensorflow_deep_mnist/load-model-tensorflow-deep-mnist.json.

Create the kubernetes job to load the model:

cd kubernetes/conf/examples/tensorflow_deep_mnist
kubectl create -f load-model-tensorflow-deep-mnist.json

Create Client

You will need to create a Seldon Client to serve as an entry point for your microservices. You can create a client in one line using the seldon cli as below. For more details on how to use the seldon cli please refer to our seldon-cli documentation.

seldon-cli client --action setup --db-name ClientDB --client-name deep_mnist_client

Here we setup a new client called deep_mnist_client that uses the database ClientDB (created when you installed seldon on your cluster).

Serve Predictions

To serve predictions we will load the saved pipeline into a microservice. This can be accomplished by using the script start-microservice.

start-microservice --type prediction --client deep_mnist_client -p tensorflow-deep-mnist /seldon-data/seldon-models/tensorflow_deep_mnist/1/ rest 1.0

This will load the pipeline saved in /seldon-data/seldon-models/tensorflow_deep_mnist/1/ and create a single replica REST microservice called tensorflow-deep-mnist. It will activate this for the “deep-mnist-client” created in the previous step.

Testing your service

The microservice takes as input a vector of 784 floats corresponding to the pixels of a 28x28 image and returns a list of probabilities for each number between 0 and 9. In order to test it you can use the flask webapp we have created for this purpose.