Basic Examples with Different Protocols


  • A kubernetes cluster with kubectl configured

  • curl

  • grpcurl

  • pygmentize

Setup Seldon Core

Use the setup notebook to Setup Cluster to setup Seldon Core with an ingress - either Ambassador or Istio.

Then port-forward to that ingress on localhost:8003 in a separate terminal either with:

  • Ambassador: kubectl port-forward $(kubectl get pods -n seldon -l -o jsonpath='{.items[0]}') -n seldon 8003:8080

  • Istio: kubectl port-forward $(kubectl get pods -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0]}') -n istio-system 8003:80

!kubectl create namespace seldon
Error from server (AlreadyExists): namespaces "seldon" already exists
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
Context "gke-ansible" modified.
import json
import time

IP=$(kubectl get pods -l seldon-deployment-id=seldon-model -n seldon -o jsonpath='{.items[0].status.podIP}')
kubectl run --quiet=true -it --rm curlmetrics --image=radial/busyboxplus:curl --restart=Never -- \
    curl -s ${IP}:6000/prometheus | grep mycounter_total

Custom Metrics

%%writefile model.yaml
kind: SeldonDeployment
  name: seldon-model
  - componentSpecs:
    - spec:
        - image: seldonio/model-with-metrics:0.2
          name: classifier
      children: []
      name: classifier
      type: MODEL
    name: example
    replicas: 1
Overwriting model.yaml
!kubectl apply -f model.yaml unchanged
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model \
                                 -o jsonpath='{.items[0]}')
Waiting for deployment "seldon-model-example-0-classifier" rollout to finish: 0 of 1 updated replicas are available...
deployment "seldon-model-example-0-classifier" successfully rolled out

Send REST request

responseRaw = !curl -s -H "Content-Type: application/json" -d '{"data": {"ndarray":[[1.0, 2.0, 5.0]]}}' http://localhost:8003/seldon/seldon/seldon-model/api/v1.0/predictions
response = json.loads(responseRaw[0])
assert len(response["meta"]["metrics"]) == 3
{'data': {'names': ['t:0', 't:1', 't:2'], 'ndarray': [[1.0, 2.0, 5.0]]}, 'meta': {'metrics': [{'key': 'mycounter', 'type': 'COUNTER', 'value': 1}, {'key': 'mygauge', 'type': 'GAUGE', 'value': 100}, {'key': 'mytimer', 'type': 'TIMER', 'value': 20.2}]}}
responseRaw = ! bash
assert len(responseRaw) == 3

Send gRPC request

responseRaw=!cd ../../../executor/proto && grpcurl -d '{"data":{"ndarray":[[1.0,2.0,5.0]]}}' \
         -rpc-header seldon:seldon-model -rpc-header namespace:seldon \
         -plaintext \
         -proto ./prediction.proto seldon.protos.Seldon/Predict
response = json.loads("".join(responseRaw))
assert len(response["meta"]["metrics"]) == 3
{'meta': {'metrics': [{'key': 'mycounter', 'value': 1}, {'key': 'mygauge', 'type': 'GAUGE', 'value': 100}, {'key': 'mytimer', 'type': 'TIMER', 'value': 20.2}]}, 'data': {'names': ['t:0', 't:1', 't:2'], 'ndarray': [[1, 2, 5]]}}
responseRaw = ! bash
assert len(responseRaw) == 6
!kubectl delete -f model.yaml "seldon-model" deleted