Canary Rollout with Seldon and Ambassador

Setup Seldon Core

Use the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core. Instructions also online.

[ ]:
!kubectl create namespace seldon
[ ]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon

Launch main model

We will create a very simple Seldon Deployment with a dummy model image seldonio/mock_classifier:1.0. This deployment is named example.

[ ]:
!pygmentize model.json
[ ]:
!kubectl create -f model.json
[ ]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=example -o jsonpath='{.items[0].metadata.name}')

Get predictions

[ ]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="example",namespace="seldon")

REST Request

[ ]:
r = sc.predict(gateway="ambassador",transport="rest")
assert(r.success==True)
print(r)

Launch Canary

We will now extend the existing graph and add a new predictor as a canary using a new model seldonio/mock_classifier_rest:1.1. We will add traffic values to split traffic 75/25 to the main and canary.

[ ]:
!pygmentize canary.json
[ ]:
!kubectl apply -f canary.json
[ ]:
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=example -o jsonpath='{.items[0].metadata.name}')
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=example -o jsonpath='{.items[1].metadata.name}')

Show our REST requests are now split with roughly 25% going to the canary.

[ ]:
sc.predict(gateway="ambassador",transport="rest")
[ ]:
from collections import defaultdict
counts = defaultdict(int)
n = 100
for i in range(n):
    r = sc.predict(gateway="ambassador",transport="rest")
[ ]:
default_count=!kubectl logs $(kubectl get pod -lseldon-app=example-main -o jsonpath='{.items[0].metadata.name}') classifier | grep "/predict" | wc -l
[ ]:
canary_count=!kubectl logs $(kubectl get pod -lseldon-app=example-canary -o jsonpath='{.items[0].metadata.name}') classifier | grep "/predict" | wc -l
[ ]:
canary_percentage=float(canary_count[0])/float(default_count[0])
print(canary_percentage)
assert(canary_percentage > 0.1 and canary_percentage < 0.5)
[ ]:
!kubectl delete -f canary.json
[ ]: