This page was generated from notebooks/server_examples.ipynb.

Example Model Servers with Seldon

Prerequistes

You will need - Git clone of Seldon Core - A running Kubernetes cluster with kubectl authenticated - seldon-core Python package (pip install seldon-core>=0.2.6.1) - Helm client

Creating a Kubernetes Cluster

Follow the Kubernetes documentation to create a cluster.

Once created ensure kubectl is authenticated against the running cluster.

Setup

[1]:
!kubectl create namespace seldon
namespace/seldon created
[2]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
Context "minikube" modified.
[3]:
!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created

Install Helm

[4]:
!kubectl -n kube-system create sa tiller
!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
!helm init --service-account tiller
serviceaccount/tiller created
clusterrolebinding.rbac.authorization.k8s.io/tiller created
$HELM_HOME has been configured at /home/clive/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
[5]:
!kubectl rollout status deploy/tiller-deploy -n kube-system
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out

Start seldon-core

[7]:
!helm install ../helm-charts/seldon-core-operator --name seldon-core  --set image.pullPolicy=IfNotPresent --set usageMetrics.enabled=true --namespace seldon-system
NAME:   seldon-core
LAST DEPLOYED: Sat Aug 31 10:58:33 2019
NAMESPACE: seldon-system
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRole
NAME                          AGE
seldon-operator-manager-role  1s

==> v1/ClusterRoleBinding
NAME                                 AGE
seldon-operator-manager-rolebinding  1s

==> v1/ConfigMap
NAME                     DATA  AGE
seldon-config            1     1s
seldon-spartakus-config  1     1s

==> v1/Pod(related)
NAME                                         READY  STATUS             RESTARTS  AGE
seldon-operator-controller-manager-0         0/1    ContainerCreating  0         1s
seldon-spartakus-volunteer-7d6dd98f89-2jgbp  0/1    ContainerCreating  0         1s

==> v1/Secret
NAME                                   TYPE    DATA  AGE
seldon-operator-webhook-server-secret  Opaque  0     1s

==> v1/Service
NAME                                        TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)  AGE
seldon-operator-controller-manager-service  ClusterIP  10.106.64.198  <none>       443/TCP  1s
webhook-server-service                      ClusterIP  10.97.147.240  <none>       443/TCP  1s

==> v1/ServiceAccount
NAME                              SECRETS  AGE
seldon-core-seldon-core-operator  1        1s
seldon-spartakus-volunteer        1        1s

==> v1/StatefulSet
NAME                                READY  AGE
seldon-operator-controller-manager  0/1    1s

==> v1beta1/ClusterRole
NAME                        AGE
seldon-spartakus-volunteer  1s

==> v1beta1/ClusterRoleBinding
NAME                        AGE
seldon-spartakus-volunteer  1s

==> v1beta1/CustomResourceDefinition
NAME                                         AGE
seldondeployments.machinelearning.seldon.io  1s

==> v1beta1/Deployment
NAME                        READY  UP-TO-DATE  AVAILABLE  AGE
seldon-spartakus-volunteer  0/1    1           0          1s


NOTES:
NOTES: TODO


[10]:
!kubectl rollout status statefulset.apps/seldon-operator-controller-manager -n seldon-system
partitioned roll out complete: 1 new pods have been updated...

Setup Ingress

Please note: There are reported gRPC issues with ambassador (see https://github.com/SeldonIO/seldon-core/issues/473).

[6]:
!helm install stable/ambassador --name ambassador --set crds.keep=false
NAME:   ambassador
LAST DEPLOYED: Sat Aug 31 10:56:45 2019
NAMESPACE: seldon
STATUS: DEPLOYED

RESOURCES:
==> v1/Deployment
NAME        READY  UP-TO-DATE  AVAILABLE  AGE
ambassador  0/3    3           0          1s

==> v1/Pod(related)
NAME                        READY  STATUS             RESTARTS  AGE
ambassador-5f7fd859b-4bhj7  0/1    ContainerCreating  0         0s
ambassador-5f7fd859b-6br5r  0/1    ContainerCreating  0         0s
ambassador-5f7fd859b-rr6pj  0/1    ContainerCreating  0         0s

==> v1/Service
NAME              TYPE          CLUSTER-IP      EXTERNAL-IP  PORT(S)                     AGE
ambassador        LoadBalancer  10.111.228.245  <pending>    80:31947/TCP,443:32642/TCP  1s
ambassador-admin  ClusterIP     10.99.206.49    <none>       8877/TCP                    1s

==> v1/ServiceAccount
NAME        SECRETS  AGE
ambassador  1        1s

==> v1beta1/ClusterRole
NAME             AGE
ambassador       1s
ambassador-crds  1s

==> v1beta1/ClusterRoleBinding
NAME             AGE
ambassador       1s
ambassador-crds  1s

==> v1beta1/CustomResourceDefinition
NAME                                          AGE
authservices.getambassador.io                 1s
consulresolvers.getambassador.io              1s
kubernetesendpointresolvers.getambassador.io  1s
kubernetesserviceresolvers.getambassador.io   1s
mappings.getambassador.io                     1s
modules.getambassador.io                      1s
ratelimitservices.getambassador.io            1s
tcpmappings.getambassador.io                  1s
tlscontexts.getambassador.io                  1s
tracingservices.getambassador.io              1s


NOTES:
Congratuations! You've successfully installed Ambassador.

For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.

To get the IP address of Ambassador, run the following commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
     You can watch the status of by running 'kubectl get svc -w  --namespace seldon ambassador'

  On GKE/Azure:
  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

  On AWS:
  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

  echo http://$SERVICE_IP:

[43]:
!kubectl rollout status deployment.apps/ambassador
Waiting for deployment "ambassador" rollout to finish: 0 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 1 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 2 of 3 updated replicas are available...
deployment "ambassador" successfully rolled out

Port Forward to Ambassador

kubectl port-forward $(kubectl get pods -n seldon -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080

Serve SKlearn Iris Model

[44]:
!pygmentize ../servers/sklearnserver/samples/iris.yaml
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: sklearn
spec:
  name: iris
  predictors:
  - graph:
      children: []
      implementation: SKLEARN_SERVER
      modelUri: gs://seldon-models/sklearn/iris
      name: classifier
    name: default
    replicas: 1
    svcOrchSpec:
      env:
      - name: SELDON_LOG_LEVEL
        value: DEBUG
[50]:
!kubectl apply -f ../servers/sklearnserver/samples/iris.yaml
seldondeployment.machinelearning.seldon.io/sklearn created
[51]:
!kubectl rollout status deploy/iris-default-4903e3c
Waiting for deployment "iris-default-4903e3c" rollout to finish: 0 of 1 updated replicas are available...
deployment "iris-default-4903e3c" successfully rolled out
[52]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="sklearn",namespace="seldon")
[53]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,4))
print(r)
Success:True message:
Request:
data {
  tensor {
    shape: 1
    shape: 4
    values: 0.44028212923599264
    values: 0.22694244373903638
    values: 0.08693601526817618
    values: 0.721446205469061
  }
}

Response:
meta {
  puid: "2k933j6cl1sq2kgh24ck96fmi8"
  requestPath {
    key: "classifier"
    value: "seldonio/sklearnserver_rest:0.2"
  }
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  tensor {
    shape: 1
    shape: 3
    values: 0.3328333104192785
    values: 0.352243232066047
    values: 0.3149234575146744
  }
}

[49]:
!kubectl delete -f ../servers/sklearnserver/samples/iris.yaml
seldondeployment.machinelearning.seldon.io "sklearn" deleted

Serve XGBoost Iris Model

[8]:
!pygmentize ../servers/xgboostserver/samples/iris.yaml
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: xgboost
spec:
  name: iris
  predictors:
  - graph:
      children: []
      implementation: XGBOOST_SERVER
      modelUri: gs://seldon-models/xgboost/iris
      name: classifier
    name: default
    replicas: 1
[36]:
!kubectl apply -f ../servers/xgboostserver/samples/iris.yaml
seldondeployment.machinelearning.seldon.io/xgboost created
[21]:
!kubectl rollout status deploy/iris-default-af1783b
Waiting for deployment "iris-default-af1783b" rollout to finish: 0 of 1 updated replicas are available...
deployment "iris-default-af1783b" successfully rolled out
[22]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="xgboost",namespace="seldon")
[23]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,4))
print(r)
Success:True message:
Request:
data {
  tensor {
    shape: 1
    shape: 4
    values: 0.2379841226697781
    values: 0.6766416345988621
    values: 0.8055747864263723
    values: 0.020395604144540558
  }
}

Response:
meta {
  puid: "gku8gaufrl69ra8btc1n1dih97"
  requestPath {
    key: "classifier"
    value: "seldonio/xgboostserver_rest:0.2"
  }
}
data {
  tensor {
    shape: 1
    values: 0.0
  }
}

[37]:
!kubectl delete -f ../servers/xgboostserver/samples/iris.yaml
seldondeployment.machinelearning.seldon.io "xgboost" deleted

Serve Tensorflow MNIST Model

Will only work on a GCP Kubernetes Cluster

[14]:
!pygmentize ../servers/tfserving/samples/mnist_rest.yaml
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: tfserving
spec:
  name: mnist
  predictors:
  - graph:
      children: []
      implementation: TENSORFLOW_SERVER
      modelUri: gs://seldon-models/tfserving/mnist-model
      name: mnist-model
      parameters:
        - name: signature_name
          type: STRING
          value: predict_images
        - name: model_name
          type: STRING
          value: mnist-model
    name: default
    replicas: 1
[25]:
!kubectl apply -f ../servers/tfserving/samples/mnist_rest.yaml
seldondeployment.machinelearning.seldon.io/tfserving created
[27]:
!kubectl rollout status deploy/mnist-default-4f27cf3
Waiting for deployment "mnist-default-4f27cf3" rollout to finish: 0 of 1 updated replicas are available...
deployment "mnist-default-4f27cf3" successfully rolled out
[28]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="tfserving",namespace="seldon")
[29]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,784))
print(r)
Success:True message:
Request:
data {
  tensor {
    shape: 1
    shape: 784
    values: 0.6960851737435524
    values: 0.31659351175832906
    values: 0.8839475487253241
    values: 0.3088659639767263
    values: 0.8891109022304765
    values: 0.19015731671987868
    values: 0.2084631221539136
    values: 0.4544208075311512
    values: 0.8926698161003384
    values: 0.19433553241548673
    values: 0.13531711500693755
    values: 0.8101485252410934
    values: 0.357998040732349
    values: 0.8149058111283166
    values: 0.7166138223407796
    values: 0.5481713626892256
    values: 0.8970834930204503
    values: 0.8913760644394269
    values: 0.8865946422300978
    values: 0.7283231572016693
    values: 0.061513546769038197
    values: 0.8946366478131781
    values: 0.08234186387502296
    values: 0.10724101325268287
    values: 0.6018581181011512
    values: 0.2833759431809676
    values: 0.12643611214180328
    values: 0.622302479013179
    values: 0.995550102668352
    values: 0.3328497257873455
    values: 0.9302707947391331
    values: 0.4468558590780892
    values: 0.42920347680284276
    values: 0.5372591933577155
    values: 0.11270788071998872
    values: 0.28988672422908424
    values: 0.7416611600623187
    values: 0.9971065442301893
    values: 0.6929695492570231
    values: 0.7146022772722823
    values: 0.5225420262232614
    values: 0.2957470630732245
    values: 0.20180834104752754
    values: 0.027983327167379057
    values: 0.743259268294824
    values: 0.9380320728626145
    values: 0.15101646676991454
    values: 0.7409886299539135
    values: 0.8708450759891614
    values: 0.7398479612053028
    values: 0.47169861563557147
    values: 0.4758555045353844
    values: 0.051715042211538886
    values: 0.9305707812109276
    values: 0.7432651725586414
    values: 0.7451248626685253
    values: 0.9306479902282564
    values: 0.3702371078940443
    values: 0.22222707494585447
    values: 0.22510121015280415
    values: 0.6595390828716817
    values: 0.5263762785871035
    values: 0.962362278625186
    values: 0.6547519478334944
    values: 0.3838371764345332
    values: 0.021925712793501195
    values: 0.959785838899824
    values: 0.10138166375879198
    values: 0.8402941786304847
    values: 0.2617904990637009
    values: 0.8431358767809322
    values: 0.27435662005154604
    values: 0.46969436581758883
    values: 0.3298464049115838
    values: 0.23876202039200844
    values: 0.11630513443656354
    values: 0.02497429318900901
    values: 0.8851278231876147
    values: 0.5898975059337347
    values: 0.11786756715910929
    values: 0.2519907309564371
    values: 0.030904070969085207
    values: 0.7663182302598766
    values: 0.14698768033501763
    values: 0.3078100599039062
    values: 0.9122021858583312
    values: 0.9296836721719605
    values: 0.48520841913653834
    values: 0.47214943726962544
    values: 0.4697355700654081
    values: 0.6681340824357441
    values: 0.6226153217980376
    values: 0.7233315833674575
    values: 0.6801846802579725
    values: 0.7001260131784245
    values: 0.2433027123936522
    values: 0.9020985629858237
    values: 0.7951204462482829
    values: 0.00478415566826762
    values: 0.6933112743162382
    values: 0.49543836729405366
    values: 0.9370649280049107
    values: 0.8254719360253051
    values: 0.5216725429531935
    values: 0.3805339266721025
    values: 0.8063426251076562
    values: 0.31565964381890144
    values: 0.4462362590261284
    values: 0.12009061557498446
    values: 0.18647448952252488
    values: 0.054650278429649646
    values: 0.6960188942790909
    values: 0.5806706486723264
    values: 0.6363740900229701
    values: 0.1724671777720067
    values: 0.27789202161257087
    values: 0.8103112708833594
    values: 0.42002413498208346
    values: 0.05460906955805889
    values: 0.8055664468835065
    values: 0.612239544710043
    values: 0.48166416135191403
    values: 0.5384873778692566
    values: 0.858583195130647
    values: 0.32710971467043704
    values: 0.690602669025769
    values: 0.9859575511612774
    values: 0.2110689318148241
    values: 0.4894298760979662
    values: 0.9688163873790141
    values: 0.8056032882153414
    values: 0.6835773039412582
    values: 0.26122322588598335
    values: 0.8156334102575398
    values: 0.40025400630980157
    values: 0.11067678668736436
    values: 0.9496285172478437
    values: 0.4734446376613902
    values: 0.37166554642438254
    values: 0.9783823684061197
    values: 0.46758659667842317
    values: 0.8109768916933405
    values: 0.055520558757641614
    values: 0.4528453142024328
    values: 0.42985929391454825
    values: 0.42143614581918454
    values: 0.40306290301342784
    values: 0.46406332299978104
    values: 0.28975939298090014
    values: 0.28947524591540175
    values: 0.18631033413027365
    values: 0.304377522923453
    values: 0.46175693066239365
    values: 0.1150837510886189
    values: 0.6812154917443837
    values: 0.9614488756919379
    values: 0.04913394446037633
    values: 0.40737285679723745
    values: 0.4329428459912802
    values: 0.5037427062521643
    values: 0.8123848185422287
    values: 0.655191928369746
    values: 0.04527689434634741
    values: 0.0873815126142633
    values: 0.5529769822794395
    values: 0.1889822510783703
    values: 0.22165216714263847
    values: 0.7378659785922592
    values: 0.12365069730517098
    values: 0.14410012880222267
    values: 0.314434996897037
    values: 0.1703689839679695
    values: 0.6690574474130547
    values: 0.6664710564253505
    values: 0.9339641442677029
    values: 0.5409462387630555
    values: 0.26074766339540667
    values: 0.6957378315803447
    values: 0.6263815242810925
    values: 0.8000587612574093
    values: 0.6933560258476635
    values: 0.4657656293471626
    values: 0.7285362738265765
    values: 0.5888972328606239
    values: 0.6337069487682453
    values: 0.7749635685108407
    values: 0.8332180653637755
    values: 0.7220249677284571
    values: 0.9450056846774213
    values: 0.2949990524468443
    values: 0.7812177080310112
    values: 0.0007479935505044777
    values: 0.6472500190114214
    values: 0.05500446345677934
    values: 0.9273297074667317
    values: 0.29508984112297865
    values: 0.18139798598121393
    values: 0.043420646475409
    values: 0.748747838187905
    values: 0.9696210129999953
    values: 0.4444161356554087
    values: 0.34929215119538615
    values: 0.11437128360879367
    values: 0.40878396270888695
    values: 0.7525849341373413
    values: 0.6734271192794087
    values: 0.21882426715297365
    values: 0.1845465469769656
    values: 0.7646381533622951
    values: 0.928688307970282
    values: 0.5835209639658983
    values: 0.9395910841618587
    values: 0.5430694448462892
    values: 0.7316370109389899
    values: 0.7881513920529627
    values: 0.7760949104722156
    values: 0.275134393397644
    values: 0.7082774551494976
    values: 0.8747442566482749
    values: 0.5960786141658948
    values: 0.34820133974955747
    values: 0.17431717975043237
    values: 0.9897531863569138
    values: 0.4600232569300692
    values: 0.23383447908889976
    values: 0.4596112078591956
    values: 0.8360406626702981
    values: 0.7272870967626213
    values: 0.7608953726381754
    values: 0.6737369575965617
    values: 0.7731532080628881
    values: 0.348797025312249
    values: 0.9790666072921824
    values: 0.353185101375623
    values: 0.4562595538413017
    values: 0.6394333896676226
    values: 0.3851883983757808
    values: 0.751879328723079
    values: 0.749543184886521
    values: 0.877920064666115
    values: 0.658283652355672
    values: 0.36857864756244674
    values: 0.530157555200853
    values: 0.34183838767424424
    values: 0.2729533988639926
    values: 0.8156090783031779
    values: 0.2638208836938779
    values: 0.48741219981945405
    values: 0.27396550303338907
    values: 0.3570731065760536
    values: 0.6106294928408991
    values: 0.48521455473900355
    values: 0.27636567274268
    values: 0.5115665470448926
    values: 0.4296451756950952
    values: 0.07822243419838792
    values: 0.5129113065705019
    values: 0.5275964697175866
    values: 0.8680526618713468
    values: 0.9931993779512479
    values: 0.5700505374808907
    values: 0.18648636809161856
    values: 0.8126782876701942
    values: 0.5757764515674432
    values: 0.9812369649104862
    values: 0.1767305130616016
    values: 0.2332112431783091
    values: 0.37171481763124914
    values: 0.6604781534451094
    values: 0.7288104296673006
    values: 0.08867812462148938
    values: 0.7113488119471519
    values: 0.6076531448323415
    values: 0.48151791957579915
    values: 0.7657466199156793
    values: 0.7514395999437494
    values: 0.7196029464473742
    values: 0.876174802090644
    values: 0.3491125681069417
    values: 0.7093588345472498
    values: 0.7555475008164038
    values: 0.563492855525081
    values: 0.1825410644218276
    values: 0.9578608832195183
    values: 0.9429716868236594
    values: 0.7567589659306391
    values: 0.5824410312188966
    values: 0.39987288906208784
    values: 0.6871346148063908
    values: 0.07958339716779195
    values: 0.7488788906174467
    values: 0.4504334694990618
    values: 0.937154000460234
    values: 0.6161862682408318
    values: 0.14232391934240418
    values: 0.7051409704133775
    values: 0.554246978117284
    values: 0.5409135294512057
    values: 0.6232907292193071
    values: 0.7924326899453114
    values: 0.07900784626277102
    values: 0.7311328133696953
    values: 0.10879879872611475
    values: 0.6883285801280582
    values: 0.4471349275407557
    values: 0.04263383220408479
    values: 0.048483081893327395
    values: 0.9068155019549612
    values: 0.04018171506169255
    values: 0.3107078521213035
    values: 0.6881708298352383
    values: 0.9311764679633986
    values: 0.10148788622035942
    values: 0.8577863329184886
    values: 0.2819715083557457
    values: 0.8543442999842149
    values: 0.5094937033682979
    values: 0.7867208110859827
    values: 0.07775847041676798
    values: 0.3157778964700201
    values: 0.6575468146276017
    values: 0.713653422836335
    values: 0.693893431275562
    values: 0.6888791061622257
    values: 0.9936624300495328
    values: 0.15402012437652002
    values: 0.06845411988012884
    values: 0.04292562769682795
    values: 0.28978983828939153
    values: 0.04054542197846511
    values: 0.885154054310623
    values: 0.11600231965213603
    values: 0.6057493825981799
    values: 0.38917891830081686
    values: 0.9167964900708111
    values: 0.1414291928923963
    values: 0.046840221209781374
    values: 0.7879633178240502
    values: 0.9327632079422756
    values: 0.9740906447697041
    values: 0.3753163440049284
    values: 0.736786611530133
    values: 0.7519448068825569
    values: 0.5348406338310284
    values: 0.6097736767799726
    values: 0.6707171921612622
    values: 0.4850489360560779
    values: 0.06435899958314162
    values: 0.4069752865709426
    values: 0.11677526900516966
    values: 0.25485384753477425
    values: 0.1042817017080313
    values: 0.23546499511754904
    values: 0.6477076533486364
    values: 0.0969132590922418
    values: 0.18063296631981052
    values: 0.19125842188089703
    values: 0.9398630903350261
    values: 0.04917178111818554
    values: 0.07001026561293933
    values: 0.7851696552390228
    values: 0.2519985786336376
    values: 0.24578980986087484
    values: 0.8785369343912791
    values: 0.06851557237775674
    values: 0.0791052985909998
    values: 0.7607754240657496
    values: 0.3043556732133149
    values: 0.1412923208586616
    values: 0.07861697162268466
    values: 0.5855683622249785
    values: 0.015474296364465734
    values: 0.3626273627096026
    values: 0.6142006101072658
    values: 0.15317882949135564
    values: 0.4760634056296531
    values: 0.1281058951147056
    values: 0.9817978623276874
    values: 0.2822779830057438
    values: 0.18899912943255026
    values: 0.15040460340302597
    values: 0.47323549071920257
    values: 0.39294851264160224
    values: 0.5607007328691221
    values: 0.8886912337489032
    values: 0.34556280687170704
    values: 0.1995031054004417
    values: 0.19023269258849673
    values: 0.7983547694883882
    values: 0.5753693096868119
    values: 0.07589245456486815
    values: 0.9088265096402942
    values: 0.06549385727779888
    values: 0.9160717605188328
    values: 0.7150161057039807
    values: 0.9952194736289688
    values: 0.2609461687442499
    values: 0.8092499916174705
    values: 0.08873534152440599
    values: 0.14229174863771732
    values: 0.9667719130844892
    values: 0.650434896401207
    values: 0.7464119737663227
    values: 0.9302255414481282
    values: 0.07560836125116543
    values: 0.35188095761446636
    values: 0.4346279847365707
    values: 0.15909349868445533
    values: 0.3240893664304797
    values: 0.6476345237575679
    values: 0.9706901495418921
    values: 0.9313898241008745
    values: 0.735725791605063
    values: 0.5024632164435275
    values: 0.4422749063180411
    values: 0.7570735718033837
    values: 0.7043073835290968
    values: 0.8309032045222933
    values: 0.9306319143793531
    values: 0.018737570455761965
    values: 0.29705523369946696
    values: 0.9148901050717295
    values: 0.4252276286471166
    values: 0.0698628099684322
    values: 0.12942926304415958
    values: 0.9895572467028818
    values: 0.5612464392047
    values: 0.9622499724232425
    values: 0.2957794985780192
    values: 0.3123704593143455
    values: 0.521813736272192
    values: 0.15734837674333113
    values: 0.9300686063703849
    values: 0.025894431641466098
    values: 0.19989143558250877
    values: 0.8098083114545819
    values: 0.45823886699942595
    values: 0.6444181383359471
    values: 0.39598416215941235
    values: 0.8287630113507214
    values: 0.03847956709186062
    values: 0.5920790051769474
    values: 0.7060219874872269
    values: 0.57182345133701
    values: 0.34681824431555863
    values: 0.772127135080517
    values: 0.6981691534033239
    values: 0.09376915332560876
    values: 0.02413035563680399
    values: 0.8430329122570326
    values: 0.314129807956945
    values: 0.8646694892839248
    values: 0.8732961980982168
    values: 0.060043785004324834
    values: 0.8087740816427692
    values: 0.44896486902443433
    values: 0.22375067593049192
    values: 0.3579990468342229
    values: 0.6256172132546013
    values: 0.744831821030211
    values: 0.09913673838044523
    values: 0.37043901594583506
    values: 0.23965358822099825
    values: 0.1693343930184279
    values: 0.8899237831783093
    values: 0.2696736277318531
    values: 0.6409735451548131
    values: 0.008784220443631363
    values: 0.9010537138158498
    values: 0.2730262930932933
    values: 0.005027750666353303
    values: 0.31242724057477833
    values: 0.2992320102738386
    values: 0.1808866764459649
    values: 0.5015938018724175
    values: 0.8752247333851866
    values: 0.671587983854636
    values: 0.9688107094933738
    values: 0.2703009118847447
    values: 0.7728503199572744
    values: 0.03368798626783731
    values: 0.6600590631797197
    values: 0.12051474241524607
    values: 0.7896141448339796
    values: 0.33202267059904
    values: 0.5557459223563734
    values: 0.8896362802777633
    values: 0.006455206574829897
    values: 0.36775163800881294
    values: 0.5710844186942228
    values: 0.24599089823130738
    values: 0.6455246922751214
    values: 0.9629833699840081
    values: 0.8906004020972904
    values: 0.11043493825427708
    values: 0.07817133215027727
    values: 0.14883312839207086
    values: 0.18896407276173832
    values: 0.006405622112598097
    values: 0.18174074752718572
    values: 0.7084031453965034
    values: 0.6606144469797846
    values: 0.5871042427566378
    values: 0.43861253376889464
    values: 0.12669077553771302
    values: 0.19683468767832835
    values: 0.04970640434258533
    values: 0.5726524841381484
    values: 0.981169806297863
    values: 0.3209577095574365
    values: 0.4846256682190856
    values: 0.16997694913532146
    values: 0.16868811200058542
    values: 0.8963932611562712
    values: 0.2512771807214328
    values: 0.4969903234144778
    values: 0.5073810961301962
    values: 0.7048461763567072
    values: 0.45230294590520503
    values: 0.777021844693988
    values: 0.800977925138967
    values: 0.016247130659590825
    values: 0.6313971018877517
    values: 0.5624121894706194
    values: 0.0239147231347
    values: 0.9631320285887279
    values: 0.06418590190826412
    values: 0.05248658465196909
    values: 0.845250343059501
    values: 0.637831910340642
    values: 0.33922261657075126
    values: 0.6552333913452396
    values: 0.7051885178865996
    values: 0.7681770568758165
    values: 0.7989325307716076
    values: 0.019107745569871937
    values: 0.7946235532817478
    values: 0.5005678164050742
    values: 0.015315627382751495
    values: 0.8834804848065625
    values: 0.7500302014792979
    values: 0.46004316564622183
    values: 0.0731941569888912
    values: 0.03214823737700068
    values: 0.2999932019225978
    values: 0.972776229543541
    values: 0.03710282505599105
    values: 0.11482962960476517
    values: 0.2482980992380871
    values: 0.3102809224558386
    values: 0.3912102030029877
    values: 0.3262647338659558
    values: 0.5161124532548613
    values: 0.11872389458119992
    values: 0.5953730774499911
    values: 0.15000444444064087
    values: 0.20097883771091363
    values: 0.4495728961691686
    values: 0.5191506168846992
    values: 0.36064870234181834
    values: 0.542684393859647
    values: 0.6274453466242965
    values: 0.6145302438770212
    values: 0.8711863790525174
    values: 0.2351843798203309
    values: 0.9113434300193178
    values: 0.06089294763325537
    values: 0.8203517960556094
    values: 0.9180327484300594
    values: 0.34905965824462926
    values: 0.6187869948765344
    values: 0.5623947125228388
    values: 0.14231915340169954
    values: 0.1088444727013872
    values: 0.7721060893741788
    values: 0.2600628968961195
    values: 0.14196593387017487
    values: 0.6201059966784203
    values: 0.7592180251591637
    values: 0.25283286048847164
    values: 0.9978210604275419
    values: 0.7380790999425393
    values: 0.7485575725821964
    values: 0.08512223751242987
    values: 0.012579629860025587
    values: 0.4781944384297896
    values: 0.7647727693833505
    values: 0.03310836060181288
    values: 0.30474966902472633
    values: 0.5813831715028552
    values: 0.15561070625440676
    values: 0.31580173219869523
    values: 0.14144304394362517
    values: 0.6822783309708192
    values: 0.7745950463535098
    values: 0.11329166397284163
    values: 0.3765425112990788
    values: 0.7756608646870186
    values: 0.026401166859190583
    values: 0.7086109599000308
    values: 0.6683704537092324
    values: 0.898865629534389
    values: 0.5614105941976864
    values: 0.9295770629451052
    values: 0.08457971242188356
    values: 0.09731557673982782
    values: 0.717528461447746
    values: 0.4435589819450947
    values: 0.09267859954832602
    values: 0.22075050237890836
    values: 0.5440385992306206
    values: 0.28869038035142713
    values: 0.6414667331700948
    values: 0.1032554273547387
    values: 0.5624248200788803
    values: 0.7529239720921993
    values: 0.37174394855131276
    values: 0.1874762518156724
    values: 0.8450512847434836
    values: 0.4677247624980835
    values: 0.4925384698119981
    values: 0.3922452617036699
    values: 0.5754634838509465
    values: 0.8978127732090391
    values: 0.6541189224722362
    values: 0.8829980807519465
    values: 0.4164403725342146
    values: 0.27701332777131693
    values: 0.7442734060764078
    values: 0.43709317554404625
    values: 0.11069489900105622
    values: 0.04372423154286542
    values: 0.6188816320735743
    values: 0.5226056312182997
    values: 0.07721293584340905
    values: 0.447249184479529
    values: 0.7483750281727048
    values: 0.9543491190476169
    values: 0.03283751989472039
    values: 0.23753708777980254
    values: 0.597696738841465
    values: 0.07290028826935813
    values: 0.752427774864024
    values: 0.22213480132387098
    values: 0.7981321800944998
    values: 0.6992784626450497
    values: 0.6506205013874153
    values: 0.6110762048950947
    values: 0.7411405767899414
    values: 0.32300169510968435
    values: 0.3249173253006642
    values: 0.19765556725943279
    values: 0.907607727419948
    values: 0.049159598848120045
    values: 0.11813253996195405
    values: 0.5352296928708075
    values: 0.2894959603816454
    values: 0.41085713895845977
    values: 0.16505693962086831
    values: 0.9327669948182279
    values: 0.18723316817419433
    values: 0.6897462433648315
    values: 0.5541311610792786
    values: 0.3631240951438194
    values: 0.7819798670964472
    values: 0.8567068382704525
    values: 0.13741698440528027
    values: 0.8841391969091527
    values: 0.20271236056898578
    values: 0.09242464895078384
    values: 0.9766739478136849
    values: 0.7350808985634926
    values: 0.7888170664033589
    values: 0.7870899672144818
    values: 0.9998261360560717
    values: 0.8243184647258451
    values: 0.8126889766066961
    values: 0.8919538365102384
    values: 0.6908472905653973
    values: 0.8348379895761122
    values: 0.9315872393965912
    values: 0.19240208610841958
    values: 0.2917105898928204
    values: 0.9970238972236198
    values: 0.5285998562610984
    values: 0.9651870370929523
    values: 0.4773414208202188
    values: 0.7080265528775707
    values: 0.5531478401143465
    values: 0.20067402465106143
    values: 0.0910473652431597
    values: 0.5566812653679821
    values: 0.1895863277679234
    values: 0.18972017452285228
    values: 0.21573093078266958
    values: 0.28819941962670015
    values: 0.7554714567345686
    values: 0.3299833119023001
    values: 0.4823634605797319
    values: 0.7347895854119262
    values: 0.5419590120717362
    values: 0.5796473212803687
    values: 0.15895666421627253
    values: 0.27820651324314327
    values: 0.5307501316553745
    values: 0.6017282898064901
    values: 0.652350809017883
    values: 0.5866517445716298
    values: 0.33780409364726616
    values: 0.6134773116492235
    values: 0.9635631255069724
    values: 0.3155534987404389
    values: 0.8445536335795542
    values: 0.5472537300640109
    values: 0.730739888926089
    values: 0.9379887639741543
    values: 0.647185622812615
    values: 0.9525324000917615
    values: 0.12124162825886475
    values: 0.9820500980791068
    values: 0.6351761819009761
    values: 0.5096275492626993
    values: 0.7828497538276272
    values: 0.15893915471373588
    values: 0.3451990159398336
    values: 0.6475856433679662
    values: 0.36241675959479336
    values: 0.05223772741291266
    values: 0.4426143815083019
    values: 0.7483366979450781
    values: 0.15732287826242441
    values: 0.5505937863592139
    values: 0.5838898870400506
    values: 0.8015764619260546
    values: 0.36608357653743173
    values: 0.7363443956244988
    values: 0.0735287974062151
    values: 0.47041809681173274
    values: 0.21865426368912189
    values: 0.4426815449770912
    values: 0.057378160647420295
    values: 0.7586390554209108
    values: 0.26421578613496544
    values: 0.8819407711539675
    values: 0.5914772064287014
    values: 0.8567896103014099
    values: 0.8450385566992789
    values: 0.24089876508232289
    values: 0.9161708472348465
    values: 0.0018941740885223268
    values: 0.6282595126309252
    values: 0.29861601609411925
    values: 0.5601121724735366
    values: 0.6387182386062428
    values: 0.8137147846879669
    values: 0.9337864597692335
    values: 0.8414109922079155
    values: 0.982870606046976
    values: 0.9927274415841693
    values: 0.8591044305852436
    values: 0.10660635287001141
    values: 0.6986013233859512
    values: 0.025221674231743152
    values: 0.8881888704557174
    values: 0.5108908540467048
    values: 0.07870239201369944
    values: 0.26108951344586095
    values: 0.6012072043277703
    values: 0.6715219596531513
    values: 0.5119352594863543
    values: 0.7155813301808434
    values: 0.6443101319727587
    values: 0.8372477307086283
    values: 0.4821563386787143
    values: 0.7355682681078428
    values: 0.8354490487874056
    values: 0.35903954925805637
    values: 0.8342739311609206
    values: 0.17795820834568554
    values: 0.7876698854014467
    values: 0.13293124638987963
    values: 0.2618021890579171
    values: 0.8442123309423877
    values: 0.255222477754938
  }
}

Response:
meta {
  puid: "ubj03lvd5hhkivjf4isd5nhfh6"
  requestPath {
    key: "mnist-model"
    value: "seldonio/tfserving-proxy_rest:0.6"
  }
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  names: "t:3"
  names: "t:4"
  names: "t:5"
  names: "t:6"
  names: "t:7"
  names: "t:8"
  names: "t:9"
  tensor {
    shape: 1
    shape: 10
    values: 3.32885639e-23
    values: 1.21791702e-33
    values: 0.113864869
    values: 0.886001587
    values: 1.36405575e-31
    values: 9.80257901e-05
    values: 1.05495472e-14
    values: 8.12247104e-16
    values: 3.54823496e-05
    values: 1.82557789e-18
  }
}

[30]:
!kubectl delete -f ../servers/tfserving/samples/mnist_rest.yaml
seldondeployment.machinelearning.seldon.io "tfserving" deleted

Serve MLFlow Elasticnet Wines Model

[9]:
!pygmentize ../servers/mlflowserver/samples/elasticnet_wine.yaml
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: mlflow
spec:
  name: wines
  predictors:
  - graph:
      children: []
      implementation: MLFLOW_SERVER
      modelUri: gs://seldon-models/mlflow/elasticnet_wine
      name: classifier
    name: default
    replicas: 1
[3]:
!kubectl apply -f ../servers/mlflowserver/samples/elasticnet_wine.yaml
seldondeployment.machinelearning.seldon.io/mlflow created
[6]:
!kubectl rollout status deploy/wines-default-057e362
deployment "wines-default-8c791aa" successfully rolled out
[13]:
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mlflow",namespace="seldon")
[17]:
r = sc.predict(gateway="ambassador",transport="rest",shape=(1,11))
print(r)
Success:True message:
Request:
data {
  tensor {
    shape: 1
    shape: 11
    values: 0.9370144045923416
    values: 0.8142853007643482
    values: 0.19471732807241948
    values: 0.5155919467447084
    values: 0.31446447854903803
    values: 0.6282233088869803
    values: 0.7504781874678473
    values: 0.6568746664582721
    values: 0.32870680648593553
    values: 0.09666979723379854
    values: 0.64486945913671
  }
}

Response:
meta {
  puid: "j2qdunn3uebeq5oj7gunmpr81u"
  requestPath {
    key: "classifier"
  }
}
data {
  tensor {
    shape: 1
    values: 5.231826975896663
  }
}

[15]:
!kubectl delete -f ../servers/mlflowserver/samples/elasticnet_wine.yaml
seldondeployment.machinelearning.seldon.io "mlflow" deleted
[ ]: