Installing Seldon

Download Seldon

git clone https://github.com/seldonio/seldon-server -b v1.4.3

Create a Kubernetes Cluster

Seldon runs inside a Kubernetes cluster so you need to follow their guides to create a cluster locally, on servers or in the cloud. We support kubernetes >= 1.2.

If you are testing Seldon on a single machine you will need at least 6G of memory for your Kubernetes cluster. For single machine exploration we suggest using minikube.

To create a Kubernetes cluster on Google Cloud you can follow our guidelines.

Create Kubernetes Configuration

Once you have a Kubernetes cluster Seldon can be started as a series of containers which run within it. As a first step you have to create the required JSON Kubernetes files. A Makefile to create these can be found in kubernetes/conf You will need to configure:

Persistent Storage

Seldon uses a Kubernetes volume to store and share data between containers. The Makefile allows you to create the Kuberenetes configuration with HostPath or GlusterFS persistent volumes. You can modify it to use other possible volumes, such as NFS, as allowed by Kubernetes.

HostPath

To create the default HostPath kubernetes conf files set for /seldon-data do the following:

 cd kubernetes/conf
 make clean conf

Note : HostPath only makes sense for demo/testing where you have a Kubernetes cluster with a single minion where all containers can share the location on the host. You will need to create the host path folder on your single kubernetes minion.

GlusterFS

GlusterFS works well for a production setting. For this you will need to have setup your own GlusterFS cluster. We provide some notes to help you. Edit the Makefile and change the value of DATA_VOLUME:

DATA_VOLUME="glusterfs": {"endpoints": "glusterfs-cluster","path": "gv0","readOnly": false}

The Makefile assumes there is a volume called gs0. You will need to provide two ip addresses of two nodes in your GlusterFS cluster, e.g.:

 cd kubernetes/conf
 make clean conf GLUSTERFS_IP1=1.2.3.4 GLUSTERFS_IP2=1.2.3.5

Seldon API Endpoint

By default the Seldon API server endpoint is set to a Kubernetes NodePort at port 30015. If you run in the cloud you can change this to LoadBalancer, e.g.

 cd kubernetes/conf
 make clean conf SELDON_SERVICE_TYPE=LoadBalancer

Launch Seldon

Scripts seldon-up and seldon-down in kubernetes/bin start and stop Seldon and should be in your PATH.

To launch seldon with all components run

seldon-up

To start with GlusterFS run

SELDON_WITH_GLUSTERFS=true seldon-up

To shutdown seldon run

seldon-down

The first time you run seldon-up it may take some time to complete as it will need to download all the images from DockerHub.

On successful completion you will have a standard Seldon installation with mysql, memcache and zookeeper running within the cluster as well as a single Seldon API server and Spark cluster. The appropriate seldon-cli commands would have be run to set up the default settings and a “test” client.

Next Steps

Troubleshooting

Check the reason its not finishing using: kubectl get all and kubectl get events

If you plan to test Seldon on a non-local cluster you will need to ensure your cluster is large enough to run all the Seldon services or disable the Kubernetes LimitRanger plugin. In the current version of Kubernetes to disable this plugin do the following. Edit <kubernetes>/cluster/<provider>/config-default.sh and remove LimitRanger from the following line:

ADMISSION_CONTROL=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota,PersistentVolumeLabel

Check you have enough memory. At least 6G is needed to run everything locally on a single node. If you are using minikube then you can start a minikube kubernetes with 6G of memory with minikube start --memory=6000

In addition, the following may help:

  1. Reduce the memory allocation for mysql and seldon-server pods before running seldon-up using the following commands:
 cd kubernetes/conf
 make clean conf MYSQL_RESOURCES='"requests":{ "memory" : "2Gi" }' SELDON_SERVER_RESOURCES='"requests":{ "memory" : "2Gi" }'

This command reduces the memory allocation for the mysql and seldon-server pods from the default 3GB each to 2GB each. If you just need to run the Reuters recommendation and Iris prediction examples, even 1 GB for mysql will work.

  1. Disable Spark, in case you do not intend to use it. To do so, invoke seldon-up like this:
SELDON_WITH_SPARK=false seldon-up

If you are using a Vagrant VM to run your kubernetes cluster ensure it has 6G of memory available from the host machine.

Check you have enough memory. At least 6G is needed to run everything locally on a single node. If you are using minikube then you can start a minikube kubernetes with 6G of memory with minikube start --memory=6000

The first time you run seldon-up it will need to pull all the container images from Docker Hub. This may take some time on a slow internet connection.

If you are using minikube, first remove old nodes using the minikube delete command.