This page was generated from notebooks/server_examples.ipynb.

Example Model Servers with Seldon

Setup Seldon Core

Use the setup notebook to Setup Cluster with Ambassador Ingress and Seldon Core. Instructions also online.

Serve SKlearn Iris Model

[ ]:
!pygmentize ../servers/sklearnserver/samples/iris.yaml
[ ]:
!kubectl apply -f ../servers/sklearnserver/samples/iris.yaml
[ ]:
!kubectl rollout status deploy/iris-default-4903e3c
[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="sklearn",namespace="seldon")
[ ]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,4))
print(r)
[ ]:
!kubectl delete -f ../servers/sklearnserver/samples/iris.yaml

Serve XGBoost Iris Model

[ ]:
!pygmentize ../servers/xgboostserver/samples/iris.yaml
[ ]:
!kubectl apply -f ../servers/xgboostserver/samples/iris.yaml
[ ]:
!kubectl rollout status deploy/iris-default-af1783b
[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="xgboost",namespace="seldon")
[ ]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,4))
print(r)
[ ]:
!kubectl delete -f ../servers/xgboostserver/samples/iris.yaml

Serve Tensorflow MNIST Model

Will only work on a GCP Kubernetes Cluster

[ ]:
!pygmentize ../servers/tfserving/samples/mnist_rest.yaml
[ ]:
!kubectl apply -f ../servers/tfserving/samples/mnist_rest.yaml
[ ]:
!kubectl rollout status deploy/mnist-default-725903e
[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="tfserving",namespace="seldon")
[ ]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,784))
print(r)
[ ]:
!kubectl delete -f ../servers/tfserving/samples/mnist_rest.yaml

Serve Tensorflow MNIST Model with Tensorflow protocol

[ ]:
!pygmentize ../servers/tfserving/samples/halfplustwo_rest.yaml
[ ]:
!kubectl apply -f ../servers/tfserving/samples/halfplustwo_rest.yaml
[ ]:
!kubectl rollout status deploy/hpt-default-a79d958
[ ]:
!curl -d '{"instances": [1.0, 2.0, 5.0]}' \
   -X POST http://localhost:8003/seldon/seldon/hpt/v1/models/halfplustwo/:predict \
   -H "Content-Type: application/json"
[ ]:
!kubectl delete -f ../servers/tfserving/samples/halfplustwo_rest.yaml

Serve MLFlow Elasticnet Wines Model

[ ]:
!pygmentize ../servers/mlflowserver/samples/elasticnet_wine.yaml
[ ]:
!kubectl apply -f ../servers/mlflowserver/samples/elasticnet_wine.yaml
[ ]:
!kubectl rollout status deploy/wines-default-8c791aa
[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mlflow",namespace="seldon")
[ ]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,11))
print(r)
[ ]:
!kubectl delete -f ../servers/mlflowserver/samples/elasticnet_wine.yaml