This page was generated from notebooks/istio_example.ipynb.

Example Seldon Core Deployments using Helm with Istio

Prequisites

Setup Cluster and Ingress

Use the setup notebook to Setup Cluster with Istio Ingress. Instructions also online.

[1]:
!kubectl create namespace seldon
namespace/seldon created
[2]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
Context "kind-kind" modified.

Configure Istio

For this example we will create the default istio gateway for seldon which needs to be called seldon-gateway. You can supply your own gateway by adding to your SeldonDeployments resources the annotation seldon.io/istio-gateway with values the name of your istio gateway.

Create a gateway for our istio-ingress

[3]:
%%writefile resources/seldon-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
Overwriting resources/seldon-gateway.yaml
[4]:
!kubectl create -f resources/seldon-gateway.yaml -n istio-system
gateway.networking.istio.io/seldon-gateway created

Ensure the istio ingress gatewaty is port-forwarded to localhost:8004

  • Istio: kubectl port-forward $(kubectl get pods -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -n istio-system 8004:8080

[5]:
ISTIO_GATEWAY="localhost:8004"
VERSION=!cat ../version.txt
VERSION=VERSION[0]
[6]:
VERSION
[6]:
'1.7.0-dev'

Start Seldon Core

Use the setup notebook to Install Seldon Core with Istio Ingress. Instructions also online.

Serve Single Model

[7]:
!helm install mymodel ../helm-charts/seldon-single-model --set model.image=seldonio/mock_classifier:$VERSION
NAME: mymodel
LAST DEPLOYED: Wed Mar 10 16:37:01 2021
NAMESPACE: seldon
STATUS: deployed
REVISION: 1
TEST SUITE: None
[8]:
!helm template mymodel ../helm-charts/seldon-single-model --set model.image=seldonio/mock_classifier:$VERSION | pygmentize -l json
---
# Source: seldon-single-model/templates/seldondeployment.json
{
  "kind": "SeldonDeployment",
  "apiVersion": "machinelearning.seldon.io/v1",
  "metadata": {
    "name": "mymodel",
    "namespace": "seldon",
    "labels": {}
  },
  "spec": {
      "name": "mymodel",
      "protocol": "seldon",
    "annotations": {},
    "predictors": [
      {
        "name": "default",
        "graph": {
          "name": "model",
          "type": "MODEL",
        },
        "componentSpecs": [
          {
            "spec": {
              "containers": [
                {
                  "name": "model",
                  "image": "seldonio/mock_classifier:1.7.0-dev",
                  "env": [
                      {
                        "name": "LOG_LEVEL",
                        "value": "INFO"
                      },
                    ],
                  "resources": {"requests":{"memory":"1Mi"}},
                }
              ]
            },
          }
        ],
        "replicas": 1
      }
    ]
  }
}
[9]:
!kubectl rollout status deploy/mymodel-default-0-model
Waiting for deployment "mymodel-default-0-model" rollout to finish: 0 of 1 updated replicas are available...
deployment "mymodel-default-0-model" successfully rolled out

Get predictions

[10]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mymodel",namespace="seldon",gateway_endpoint=ISTIO_GATEWAY)

REST Request

[11]:
r = sc.predict(gateway="istio",transport="rest")
assert(r.success==True)
print(r)
Success:True message:
Request:
meta {
}
data {
  tensor {
    shape: 1
    shape: 1
    values: 0.721679221744617
  }
}

Response:
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.1002015221659356]}}, 'meta': {'requestPath': {'model': 'seldonio/mock_classifier:1.7.0-dev'}}}

gRPC Request

[12]:
r = sc.predict(gateway="istio",transport="grpc")
assert(r.success==True)
print(r)
Success:True message:
Request:
{'meta': {}, 'data': {'tensor': {'shape': [1, 1], 'values': [0.17825624441824628]}}}
Response:
{'meta': {'requestPath': {'model': 'seldonio/mock_classifier:1.7.0-dev'}}, 'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.06074453279395597]}}}
[13]:
!helm delete mymodel
release "mymodel" uninstalled