This page was generated from notebooks/istio_example.ipynb.

Example Seldon Core Deployments using Helm

predictor with canary

Setup Cluster and Ingress

Use the setup notebook to Setup Cluster with Istio Ingress. Instructions also online.

[ ]:
!kubectl create namespace seldon
[ ]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon

Configure Istio

For this example we will create the default istio gateway for seldon which needs to be called seldon-gateway. You can supply your own gateway by adding to your SeldonDeployments resources the annotation seldon.io/istio-gateway with values the name of your istio gateway.

Create a gateway for our istio-ingress

[ ]:
%%writefile resources/seldon-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
[ ]:
!kubectl create -f resources/seldon-gateway.yaml -n istio-system

Ensure the istio ingress gatewaty is port-forwarded to localhost:8004

  • Istio: kubectl port-forward $(kubectl get pods -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -n istio-system 8004:80

[ ]:
ISTIO_GATEWAY="localhost:8004"

Start Seldon Core

Use the setup notebook to Install Seldon Core with Istio Ingress. Instructions also online.

Serve Single Model

[ ]:
!helm install mymodel ../helm-charts/seldon-single-model --set 'model.image=seldonio/mock_classifier_rest:1.3'
[ ]:
!helm template mymodel ../helm-charts/seldon-single-model --set 'model.image=seldonio/mock_classifier_rest:1.3' | pygmentize -l json
[ ]:
!kubectl rollout status deploy/mymodel-default-0-model

Get predictions

[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mymodel",namespace="seldon",gateway_endpoint=ISTIO_GATEWAY)

REST Request

[ ]:
r = sc.predict(gateway="istio",transport="rest")
assert(r.success==True)
print(r)
[ ]:
!helm delete mymodel

Serve AB Test

[ ]:
!helm install myabtest ../helm-charts/seldon-abtest
[ ]:
!helm template ../helm-charts/seldon-abtest | pygmentize -l json
[ ]:
!kubectl rollout status deploy/myabtest-default-0-classifier-1
!kubectl rollout status deploy/myabtest-default-1-classifier-2

Get predictions

[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="myabtest",namespace="seldon",gateway_endpoint=ISTIO_GATEWAY)

REST Request

[ ]:
r = sc.predict(gateway="istio",transport="rest")
assert(r.success==True)
print(r)
[ ]:
!helm delete myabtest

Serve Multi-Armed Bandit

[ ]:
!helm install mymab ../helm-charts/seldon-mab
[ ]:
!helm template ../helm-charts/seldon-mab | pygmentize -l json
[ ]:
!kubectl rollout status deploy/mymab-default-0-classifier-1
!kubectl rollout status deploy/mymab-default-1-classifier-2
!kubectl rollout status deploy/mymab-default-2-eg-router

Get predictions

[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mymab",namespace="seldon",gateway_endpoint=ISTIO_GATEWAY)

REST Request

[ ]:
r = sc.predict(gateway="istio",transport="rest")
assert(r.success==True)
print(r)
[ ]:
!helm delete mymab

Serve with Shadow

[ ]:
!pygmentize ./resources/istio_shadow.yaml
[ ]:
!kubectl apply -f ./resources/istio_shadow.yaml
[ ]:
!kubectl rollout status deploy/iris-default-0-iris-default
!kubectl rollout status deploy/iris-shadow-0-iris-shadow
[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="iris",namespace="seldon",gateway_endpoint=ISTIO_GATEWAY)
[ ]:
r = sc.predict(gateway="istio",transport="rest",shape=(1,4))
assert(r.success==True)
print(r)
[ ]:
!kubectl delete -f ./resources/istio_shadow.yaml
[ ]: