This page was generated from notebooks/helm_examples.ipynb.

Example Seldon Core Deployments using Helm¶

predictor with canary

Setup Seldon Core¶

Use the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core. Instructions also online.

[1]:
!kubectl create namespace seldon
Error from server (AlreadyExists): namespaces "seldon" already exists
[2]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
Context "kind-kind" modified.

Serve Single Model¶

[3]:
!helm install mymodel ../helm-charts/seldon-single-model --set 'model.image=seldonio/mock_classifier:1.5.0-dev'
NAME: mymodel
LAST DEPLOYED: Mon Nov  2 11:18:38 2020
NAMESPACE: seldon
STATUS: deployed
REVISION: 1
TEST SUITE: None
[4]:
!helm template mymodel ../helm-charts/seldon-single-model --set 'model.image=seldonio/mock_classifier:1.5.0-dev' | pygmentize -l json
---
# Source: seldon-single-model/templates/seldondeployment.json
{
  "kind": "SeldonDeployment",
  "apiVersion": "machinelearning.seldon.io/v1",
  "metadata": {
    "name": "mymodel",
    "namespace": "seldon",
    "labels": {}
  },
  "spec": {
      "name": "mymodel",
      "protocol": "seldon",
    "annotations": {},
    "predictors": [
      {
        "name": "default",
        "graph": {
          "name": "model",
          "type": "MODEL",
        },
        "componentSpecs": [
          {
            "spec": {
              "containers": [
                {
                  "name": "model",
                  "image": "seldonio/mock_classifier:1.5.0-dev",
                  "env": [
                      {
                        "name": "LOG_LEVEL",
                        "value": "INFO"
                      },
                    ],
                  "resources": {"requests":{"memory":"1Mi"}},
                }
              ]
            },
          }
        ],
        "replicas": 1
      }
    ]
  }
}
[6]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=mymodel -o jsonpath='{.items[0].metadata.name}')
deployment "mymodel-default-0-model" successfully rolled out

Get predictions¶

[7]:
from seldon_core.seldon_client import SeldonClient

sc = SeldonClient(
    deployment_name="mymodel",
    namespace="seldon",
    gateway_endpoint="localhost:8003",
    gateway="ambassador",
)

REST Request¶

[8]:
r = sc.predict(transport="rest")
assert r.success == True
print(r)
Success:True message:
Request:
meta {
}
data {
  tensor {
    shape: 1
    shape: 1
    values: 0.0406846384836026
  }
}

Response:
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.05335370865277927]}}, 'meta': {}}

GRPC Request¶

[9]:
r = sc.predict(transport="grpc")
print(r)
Success:True message:
Request:
{'meta': {}, 'data': {'tensor': {'shape': [1, 1], 'values': [0.3321428950191112]}}}
Response:
{'meta': {}, 'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.07014111011256721]}}}
[10]:
!helm delete mymodel
release "mymodel" uninstalled

Serve REST AB Test¶

[11]:
!helm install myabtest ../helm-charts/seldon-abtest
NAME: myabtest
LAST DEPLOYED: Mon Nov  2 11:19:50 2020
NAMESPACE: seldon
STATUS: deployed
REVISION: 1
TEST SUITE: None
[12]:
!helm template ../helm-charts/seldon-abtest | pygmentize -l json
---
# Source: seldon-abtest/templates/ab_test_2pods.json
{
    "apiVersion": "machinelearning.seldon.io/v1alpha2",
    "kind": "SeldonDeployment",
    "metadata": {
        "labels": {
            "app": "seldon"
        },
        "name": "RELEASE-NAME"
    },
    "spec": {
        "name": "RELEASE-NAME",
        "predictors": [
            {
                "name": "default",
                "replicas": 1,
                "componentSpecs": [{
                    "spec": {
                        "containers": [
                            {
                                "image": "seldonio/mock_classifier:1.5.0-dev",
                                "imagePullPolicy": "IfNotPresent",
                                "name": "classifier-1",
                                "resources": {
                                    "requests": {
                                        "memory": "1Mi"
                                    }
                                }
                            }],
                        "terminationGracePeriodSeconds": 20
                    }},
                {
                    "metadata":{
                        "labels":{
                            "version":"v2"
                        }
                    },
                        "spec":{
                            "containers":[
                                {
                                "image": "seldonio/mock_classifier:1.5.0-dev",
                                "imagePullPolicy": "IfNotPresent",
                                "name": "classifier-2",
                                "resources": {
                                    "requests": {
                                        "memory": "1Mi"
                                    }
                                }
                            }
                        ],
                        "terminationGracePeriodSeconds": 20
                                   }
                                   }],
                "graph": {
                    "name": "RELEASE-NAME",
                    "implementation":"RANDOM_ABTEST",
                    "parameters": [
                        {
                            "name":"ratioA",
                            "value":"0.5",
                            "type":"FLOAT"
                        }
                    ],
                    "children": [
                        {
                            "name": "classifier-1",
                            "type":"MODEL",
                            "children":[]
                        },
                        {
                            "name": "classifier-2",
                            "type":"MODEL",
                            "children":[]
                        }
                    ]
                }
            }
        ]
    }
}
[13]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=myabtest -o jsonpath='{.items[0].metadata.name}')
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=myabtest -o jsonpath='{.items[1].metadata.name}')
Waiting for deployment "myabtest-default-0-classifier-1" rollout to finish: 0 of 1 updated replicas are available...
deployment "myabtest-default-0-classifier-1" successfully rolled out
deployment "myabtest-default-1-classifier-2" successfully rolled out

Get predictions¶

[14]:
from seldon_core.seldon_client import SeldonClient

sc = SeldonClient(
    deployment_name="myabtest",
    namespace="seldon",
    gateway_endpoint="localhost:8003",
    gateway="ambassador",
)

REST Request¶

[15]:
r = sc.predict(transport="rest")
assert r.success == True
print(r)
Success:True message:
Request:
meta {
}
data {
  tensor {
    shape: 1
    shape: 1
    values: 0.8562060281778412
  }
}

Response:
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.11299965170860979]}}, 'meta': {}}

gRPC Request¶

[16]:
r = sc.predict(transport="grpc")
print(r)
Success:True message:
Request:
{'meta': {}, 'data': {'tensor': {'shape': [1, 1], 'values': [0.45187622094165814]}}}
Response:
{'meta': {}, 'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.07836365822139986]}}}
[17]:
!helm delete myabtest
release "myabtest" uninstalled

Serve REST Multi-Armed Bandit¶

[18]:
!helm install mymab ../helm-charts/seldon-mab
NAME: mymab
LAST DEPLOYED: Mon Nov  2 11:22:19 2020
NAMESPACE: seldon
STATUS: deployed
REVISION: 1
TEST SUITE: None
[19]:
!helm template ../helm-charts/seldon-mab | pygmentize -l json
---
# Source: seldon-mab/templates/mab.json
{
    "apiVersion": "machinelearning.seldon.io/v1alpha2",
    "kind": "SeldonDeployment",
    "metadata": {
                "labels": {"app":"seldon"},
                "name": "RELEASE-NAME"
    },
    "spec": {
        "name": "RELEASE-NAME",
        "predictors": [
            {
                "name": "default",
                "replicas": 1,
                "componentSpecs": [{
                    "spec": {
                        "containers": [
                            {
                                "image": "seldonio/mock_classifier:1.5.0-dev",
                                "imagePullPolicy": "IfNotPresent",
                                "name": "classifier-1",
                                "resources": {
                                    "requests": {
                                        "memory": "1Mi"
                                    }
                                }
                            }],
                        "terminationGracePeriodSeconds": 20
                    }},
                {
                        "spec":{
                            "containers":[
                                {
                                "image": "seldonio/mock_classifier:1.5.0-dev",
                                "imagePullPolicy": "IfNotPresent",
                                "name": "classifier-2",
                                "resources": {
                                    "requests": {
                                        "memory": "1Mi"
                                    }
                                }
                            }
                        ],
                        "terminationGracePeriodSeconds": 20
                        }
                },
                {
                    "spec":{
                        "containers": [{
                            "image": "seldonio/mab_epsilon_greedy:1.5.0-dev",
                            "name": "eg-router"
                        }],
                        "terminationGracePeriodSeconds": 20
                    }}
                ],
                "graph": {
                    "name": "eg-router",
                    "type":"ROUTER",
                    "parameters": [
                        {
                            "name": "n_branches",
                            "value": "2",
                            "type": "INT"
                        },
                        {
                            "name": "epsilon",
                            "value": "0.2",
                            "type": "FLOAT"
                        },
                        {
                            "name": "verbose",
                            "value": "1",
                            "type": "BOOL"
                        }
                    ],
                    "children": [
                        {
                            "name": "classifier-1",
                            "type":"MODEL",
                            "children":[]
                        },
                        {
                            "name": "classifier-2",
                            "type":"MODEL",
                            "children":[]
                        }
                    ]
                },
                "svcOrchSpec": {
                "resources": {"requests":{"cpu":"0.1"}},
"env": [
{
"name": "SELDON_LOG_MESSAGES_EXTERNALLY",
"value": "false"
},
{
"name": "SELDON_LOG_MESSAGE_TYPE",
"value": "seldon.message.pair"
},
{
"name": "SELDON_LOG_REQUESTS",
"value": "false"
},
{
"name": "SELDON_LOG_RESPONSES",
"value": "false"
},
]
},
                "labels": {"fluentd":"true","version":"1.5.0-dev"}
            }
        ]
    }
}
[20]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=mymab -o jsonpath='{.items[0].metadata.name}')
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=mymab -o jsonpath='{.items[1].metadata.name}')
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=mymab -o jsonpath='{.items[2].metadata.name}')
Waiting for deployment "mymab-default-0-classifier-1" rollout to finish: 0 of 1 updated replicas are available...
deployment "mymab-default-0-classifier-1" successfully rolled out
deployment "mymab-default-1-classifier-2" successfully rolled out
deployment "mymab-default-2-eg-router" successfully rolled out

Get predictions¶

[21]:
from seldon_core.seldon_client import SeldonClient

sc = SeldonClient(
    deployment_name="mymab",
    namespace="seldon",
    gateway_endpoint="localhost:8003",
    gateway="ambassador",
)

REST Request¶

[22]:
r = sc.predict(transport="rest")
assert r.success == True
print(r)
Success:True message:
Request:
meta {
}
data {
  tensor {
    shape: 1
    shape: 1
    values: 0.1000299187972008
  }
}

Response:
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.05643175042558145]}}, 'meta': {}}

gRPC Request¶

[23]:
r = sc.predict(transport="grpc")
print(r)
Success:True message:
Request:
{'meta': {}, 'data': {'tensor': {'shape': [1, 1], 'values': [0.23579893772394123]}}}
Response:
{'meta': {}, 'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.0641117916962909]}}}
[24]:
!helm delete mymab
release "mymab" uninstalled
[ ]: