Scikit-Learn IRIS Model

  • Wrap a scikit-learn python model for use as a prediction microservice in seldon-core
  • Run locally on Docker to test
  • Deploy on seldon-core running on minikube

Dependencies

pip install sklearn
pip install seldon-core

Train locally

[1]:
import numpy as np
import os
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.externals import joblib
from sklearn import datasets

def main():
    clf = LogisticRegression()
    p = Pipeline([('clf', clf)])
    print('Training model...')
    p.fit(X, y)
    print('Model trained!')

    filename_p = 'IrisClassifier.sav'
    print('Saving model in %s' % filename_p)
    joblib.dump(p, filename_p)
    print('Model saved!')

if __name__ == "__main__":
    print('Loading iris data set...')
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    print('Dataset loaded!')
    main()

Loading iris data set...
Dataset loaded!
Training model...
Model trained!
Saving model in IrisClassifier.sav
Model saved!
/home/clive/anaconda3/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
  FutureWarning)
/home/clive/anaconda3/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:459: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.
  "this warning.", FutureWarning)

Wrap model using s2i

REST test

[2]:
!s2i build . seldonio/seldon-core-s2i-python3:0.6 sklearn-iris:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting scikit-learn==0.19.0 (from -r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/a4/b3/209652a5d60ce4a2a8a35ad893d7565bbb0f87ce043264ba5c9e7de304cd/scikit_learn-0.19.0-cp36-cp36m-manylinux1_x86_64.whl (12.4MB)
Collecting scipy==0.18.1 (from -r requirements.txt (line 2))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/74/c0/f0bf4eaef1b6aa7bdd1ae5597ce1d9e729417b3ca085c47d0f1c640d34f8/scipy-0.18.1-cp36-cp36m-manylinux1_x86_64.whl (42.5MB)
Installing collected packages: scikit-learn, scipy
Successfully installed scikit-learn-0.19.0 scipy-0.18.1
Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
You are using pip version 19.0.3, however version 19.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Build completed successfully
[3]:
!docker run --name "iris_predictor" -d --rm -p 5000:5000 sklearn-iris:0.1
78da854832cd2bf320b01902395aece9945741a77992d16e0d975dc6111b7f9e

Send some random features that conform to the contract

[4]:
!seldon-core-tester contract.json 0.0.0.0 5000 -p
----------------------------------------
SENDING NEW REQUEST:

[[4.964 3.656 5.656 0.265]]
RECEIVED RESPONSE:
meta {
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  ndarray {
    values {
      list_value {
        values {
          number_value: 0.0025707553517767597
        }
        values {
          number_value: 0.6995056431201664
        }
        values {
          number_value: 0.2979236015280567
        }
      }
    }
  }
}


[5]:
!docker rm iris_predictor --force
iris_predictor

grpc test

[6]:
!s2i build -E .s2i/environment_grpc . seldonio/seldon-core-s2i-python3:0.6 sklearn-iris:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting scikit-learn==0.19.0 (from -r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/a4/b3/209652a5d60ce4a2a8a35ad893d7565bbb0f87ce043264ba5c9e7de304cd/scikit_learn-0.19.0-cp36-cp36m-manylinux1_x86_64.whl (12.4MB)
Collecting scipy==0.18.1 (from -r requirements.txt (line 2))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/74/c0/f0bf4eaef1b6aa7bdd1ae5597ce1d9e729417b3ca085c47d0f1c640d34f8/scipy-0.18.1-cp36-cp36m-manylinux1_x86_64.whl (42.5MB)
Installing collected packages: scikit-learn, scipy
Successfully installed scikit-learn-0.19.0 scipy-0.18.1
Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Build completed successfully
[7]:
!docker run --name "iris_predictor" -d --rm -p 5000:5000 sklearn-iris:0.1
da83bce5f091b5e3fa20b2d7a950786b7945ff5fe36a766b1ae8d2f98a5d9c4e

Test using NDArray payload

[8]:
!seldon-core-tester contract.json 0.0.0.0 5000 -p --grpc
----------------------------------------
SENDING NEW REQUEST:
RECEIVED RESPONSE:
Success:True message:
Request:
data {
  ndarray {
    values {
      list_value {
        values {
          number_value: 5.876
        }
        values {
          number_value: 2.84
        }
        values {
          number_value: 5.36
        }
        values {
          number_value: 0.988
        }
      }
    }
  }
}

Response:
meta {
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  ndarray {
    values {
      list_value {
        values {
          number_value: 0.0005719905486572722
        }
        values {
          number_value: 0.636311194209786
        }
        values {
          number_value: 0.36311681524155676
        }
      }
    }
  }
}


Test using Tensor payload

[9]:
!seldon-core-tester contract.json 0.0.0.0 5000 -p --grpc --tensor
----------------------------------------
SENDING NEW REQUEST:
RECEIVED RESPONSE:
Success:True message:
Request:
data {
  tensor {
    shape: 1
    shape: 4
    values: 4.188
    values: 3.802
    values: 7.504
    values: 0.129
  }
}

Response:
meta {
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  tensor {
    shape: 1
    shape: 3
    values: 6.558508088953225e-07
    values: 0.022355480648596854
    values: 0.9776438635005942
  }
}


[10]:
!docker rm iris_predictor --force
iris_predictor

Test using Minikube

Due to a minikube/s2i issue you will need s2i >= 1.1.13

[12]:
!minikube start --memory 4096
😄  minikube v0.34.1 on linux (amd64)
🔥  Creating virtualbox VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
📶  "minikube" IP address is 192.168.99.100
🐳  Configuring Docker as the container runtime ...
✨  Preparing Kubernetes environment ...
🚜  Pulling images required by Kubernetes v1.13.3 ...
🚀  Launching Kubernetes v1.13.3 using kubeadm ...
🔑  Configuring cluster permissions ...
🤔  Verifying component health .....
💗  kubectl is now configured to use "minikube"
🏄  Done! Thank you for using minikube!
[13]:
!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created
[14]:
!helm init
$HELM_HOME has been configured at /home/clive/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
[15]:
!kubectl rollout status deploy/tiller-deploy -n kube-system
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out
[6]:
!helm install ../../../helm-charts/seldon-core-operator --name seldon-core --set usageMetrics.enabled=true --namespace seldon-system
NAME:   seldon-core
LAST DEPLOYED: Thu Apr 25 09:02:59 2019
NAMESPACE: seldon-system
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/CustomResourceDefinition
NAME                                         AGE
seldondeployments.machinelearning.seldon.io  0s

==> v1/ClusterRole
seldon-operator-manager-role  0s

==> v1/ClusterRoleBinding
NAME                                 AGE
seldon-operator-manager-rolebinding  0s

==> v1/Service
NAME                                        TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)  AGE
seldon-operator-controller-manager-service  ClusterIP  10.106.80.138  <none>       443/TCP  0s

==> v1/StatefulSet
NAME                                DESIRED  CURRENT  AGE
seldon-operator-controller-manager  1        1        0s

==> v1/Pod(related)
NAME                                  READY  STATUS             RESTARTS  AGE
seldon-operator-controller-manager-0  0/1    ContainerCreating  0         0s

==> v1/Secret
NAME                                   TYPE    DATA  AGE
seldon-operator-webhook-server-secret  Opaque  0     0s


NOTES:
NOTES: TODO


[7]:
!kubectl rollout status statefulset.apps/seldon-operator-controller-manager -n seldon-system
partitioned roll out complete: 1 new pods have been updated...

Setup Ingress

There are gRPC issues with the latest Ambassador, so we rewcommend 0.40.2 until these are fixed.

[8]:
!helm install stable/ambassador --name ambassador --set image.tag=0.40.2
NAME:   ambassador
LAST DEPLOYED: Thu Apr 25 09:03:43 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/ClusterRoleBinding
NAME        AGE
ambassador  0s

==> v1/Service
NAME               TYPE          CLUSTER-IP     EXTERNAL-IP  PORT(S)                     AGE
ambassador-admins  ClusterIP     10.107.239.6   <none>       8877/TCP                    0s
ambassador         LoadBalancer  10.97.236.148  <pending>    80:30062/TCP,443:30447/TCP  0s

==> v1/Deployment
NAME        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
ambassador  3        3        3           0          0s

==> v1/Pod(related)
NAME                         READY  STATUS             RESTARTS  AGE
ambassador-5b89d44544-8rjms  0/1    ContainerCreating  0         0s
ambassador-5b89d44544-fvqjc  0/1    ContainerCreating  0         0s
ambassador-5b89d44544-wr55x  0/1    ContainerCreating  0         0s

==> v1/ServiceAccount
NAME        SECRETS  AGE
ambassador  1        0s

==> v1beta1/ClusterRole
NAME        AGE
ambassador  0s


NOTES:
Congratuations! You've successfully installed Ambassador.

For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.

To get the IP address of Ambassador, run the following commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
     You can watch the status of by running 'kubectl get svc -w  --namespace default ambassador'

  On GKE/Azure:
  export SERVICE_IP=$(kubectl get svc --namespace default ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

  On AWS:
  export SERVICE_IP=$(kubectl get svc --namespace default ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

  echo http://$SERVICE_IP:

[9]:
!kubectl rollout status deployment.apps/ambassador
Waiting for deployment "ambassador" rollout to finish: 0 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 1 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 2 of 3 updated replicas are available...
deployment "ambassador" successfully rolled out

Wrap Model and Test

[10]:
!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3:0.6 sklearn-iris:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting scikit-learn==0.19.0 (from -r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/a4/b3/209652a5d60ce4a2a8a35ad893d7565bbb0f87ce043264ba5c9e7de304cd/scikit_learn-0.19.0-cp36-cp36m-manylinux1_x86_64.whl (12.4MB)
Collecting scipy==0.18.1 (from -r requirements.txt (line 2))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/74/c0/f0bf4eaef1b6aa7bdd1ae5597ce1d9e729417b3ca085c47d0f1c640d34f8/scipy-0.18.1-cp36-cp36m-manylinux1_x86_64.whl (42.5MB)
Installing collected packages: scikit-learn, scipy
Successfully installed scikit-learn-0.19.0 scipy-0.18.1
Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
You are using pip version 19.0.3, however version 19.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Build completed successfully
[11]:
!kubectl create -f sklearn_iris_deployment.json
seldondeployment.machinelearning.seldon.io/seldon-deployment-example created
[12]:
!kubectl rollout status deploy/sklearn-iris-deployment-sklearn-iris-predictor-a5a7453
Waiting for deployment "sklearn-iris-deployment-sklearn-iris-predictor-a5a7453" rollout to finish: 0 of 1 updated replicas are available...
deployment "sklearn-iris-deployment-sklearn-iris-predictor-a5a7453" successfully rolled out
[13]:
!seldon-core-api-tester contract.json `minikube ip` `kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'` \
    seldon-deployment-example --namespace default -p
----------------------------------------
SENDING NEW REQUEST:

[[7.578 3.382 4.294 0.482]]
RECEIVED RESPONSE:
meta {
  puid: "u9bfl24ggtjhjpac4d60qstgqv"
  requestPath {
    key: "sklearn-iris-classifier"
    value: "sklearn-iris:0.1"
  }
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  ndarray {
    values {
      list_value {
        values {
          number_value: 0.07252377179843429
        }
        values {
          number_value: 0.9272157329207761
        }
        values {
          number_value: 0.00026049528078956984
        }
      }
    }
  }
}


[25]:
!minikube delete
🔥  Deleting "minikube" from virtualbox ...
💔  The "minikube" cluster has been deleted.
[ ]: