Basic Examples with Different Protocols

Prerequisites

  • A kubernetes cluster with kubectl configured

  • curl

  • grpcurl

  • pygmentize

Setup Seldon Core

Use the setup notebook to Setup Cluster to setup Seldon Core with an ingress - either Ambassador or Istio.

Then port-forward to that ingress on localhost:8003 in a separate terminal either with:

  • Ambassador: kubectl port-forward $(kubectl get pods -n seldon -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080

  • Istio: kubectl port-forward $(kubectl get pods -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -n istio-system 8003:80

Install Seldon Analytics

[ ]:
!helm install seldon-core-analytics ../../../helm-charts/seldon-core-analytics -n seldon-system --wait
[ ]:
!kubectl create namespace seldon
[ ]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
[ ]:
import json
import time

Custom Metrics with a REST model

[ ]:
!pygmentize model_rest.yaml
[ ]:
!kubectl create -f model_rest.yaml
[ ]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model \
                                 -o jsonpath='{.items[0].metadata.name}')
[ ]:
responseRaw=!curl -s -d '{"data": {"ndarray":[[1.0, 2.0, 5.0]]}}'    -X POST http://localhost:8003/seldon/seldon/seldon-model/api/v1.0/predictions    -H "Content-Type: application/json"
[ ]:
response=json.loads(responseRaw[0])
print(response)
assert(len(response["meta"]["metrics"]) == 3)
[ ]:
print("Waiting so metrics can be scraped")
time.sleep(3)
[ ]:
%%writefile get-metrics.sh

kubectl run --quiet=true -it --rm curlmetrics --image=tutum/curl --restart=Never -- \
    curl -s seldon-core-analytics-prometheus-seldon.seldon-system/api/v1/query?query=mycounter_total
[ ]:
responseRaw =! bash get-metrics.sh
[ ]:
responseRaw[0]
[ ]:
response=json.loads(responseRaw[0])
print(response)
assert(response['data']["result"][0]["metric"]["__name__"]=='mycounter_total')
[ ]:
!kubectl delete -f model_rest.yaml

Custom Metrics with a GRPC model

[ ]:
!pygmentize model_grpc.yaml
[ ]:
!kubectl create -f model_grpc.yaml
[ ]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model \
                                 -o jsonpath='{.items[0].metadata.name}')
[ ]:
responseRaw=!cd ../../../executor/proto && grpcurl -d '{"data":{"ndarray":[[1.0,2.0,5.0]]}}' \
         -rpc-header seldon:seldon-model -rpc-header namespace:seldon \
         -plaintext \
         -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/Predict
[ ]:
response=json.loads("".join(responseRaw))
print(response)
assert(len(response["meta"]["metrics"]) == 3)
[ ]:
print("Waiting so metrics can be scraped")
time.sleep(3)
[ ]:
responseRaw =! bash get-metrics.sh
[ ]:
response=json.loads(responseRaw[0])
print(response)
assert(response['data']["result"][0]["metric"]["__name__"]=='mycounter_total')
assert(response['data']["result"][0]["metric"]["image_name"]=='seldonio/model-with-metrics_grpc')
[ ]:
!kubectl delete -f model_grpc.yaml
[ ]:
!helm delete seldon-core-analytics --namespace seldon-system
[ ]: