Scikit-Learn IRIS Model

  • Wrap a scikit-learn python model for use as a prediction microservice in seldon-core
  • Run locally on Docker to test
  • Deploy on seldon-core running on minikube

Dependencies

pip install sklearn
pip install seldon-core

Train locally

[1]:
import numpy as np
import os
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.externals import joblib
from sklearn import datasets

def main():
    clf = LogisticRegression()
    p = Pipeline([('clf', clf)])
    print('Training model...')
    p.fit(X, y)
    print('Model trained!')

    filename_p = 'IrisClassifier.sav'
    print('Saving model in %s' % filename_p)
    joblib.dump(p, filename_p)
    print('Model saved!')

if __name__ == "__main__":
    print('Loading iris data set...')
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    print('Dataset loaded!')
    main()

Loading iris data set...
Dataset loaded!
Training model...
Model trained!
Saving model in IrisClassifier.sav
Model saved!
/home/clive/anaconda3/envs/seldon-core/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=DeprecationWarning)
/home/clive/anaconda3/envs/seldon-core/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
  FutureWarning)
/home/clive/anaconda3/envs/seldon-core/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:469: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.
  "this warning.", FutureWarning)

Wrap model using s2i

REST test

[2]:
!s2i build . seldonio/seldon-core-s2i-python37:0.12 sklearn-iris:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting scikit-learn (from -r requirements.txt (line 1))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/9f/c5/e5267eb84994e9a92a2c6a6ee768514f255d036f3c8378acfa694e9f2c99/scikit_learn-0.21.3-cp37-cp37m-manylinux1_x86_64.whl (6.7MB)
Collecting scipy (from -r requirements.txt (line 2))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/5d/bd/c0feba81fb60e231cf40fc8a322ed5873c90ef7711795508692b1481a4ae/scipy-1.3.0-cp37-cp37m-manylinux1_x86_64.whl (25.2MB)
Collecting joblib>=0.11 (from scikit-learn->-r requirements.txt (line 1))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/cd/c1/50a758e8247561e58cb87305b1e90b171b8c767b15b12a1734001f41d356/joblib-0.13.2-py2.py3-none-any.whl (278kB)
Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn->-r requirements.txt (line 1)) (1.17.0)
Installing collected packages: joblib, scipy, scikit-learn
Successfully installed joblib-0.13.2 scikit-learn-0.21.3 scipy-1.3.0
WARNING: You are using pip version 19.1.1, however version 19.2.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Build completed successfully
[3]:
!docker run --name "iris_predictor" -d --rm -p 5000:5000 sklearn-iris:0.1
183e42c61200a89eae242a9400ff03f0c74a9405c2d022e7a01ecb3542d36380

Send some random features that conform to the contract

[4]:
!seldon-core-tester contract.json 0.0.0.0 5000 -p
----------------------------------------
SENDING NEW REQUEST:

[[5.964 4.006 2.081 1.031]]
RECEIVED RESPONSE:
meta {
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  ndarray {
    values {
      list_value {
        values {
          number_value: 0.9548873249364169
        }
        values {
          number_value: 0.04505474761561406
        }
        values {
          number_value: 5.7927447968952436e-05
        }
      }
    }
  }
}


[5]:
!docker rm iris_predictor --force
iris_predictor

grpc test

[6]:
!s2i build -E .s2i/environment_grpc . seldonio/seldon-core-s2i-python3:0.12 sklearn-iris:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting scikit-learn (from -r requirements.txt (line 1))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/9f/c5/e5267eb84994e9a92a2c6a6ee768514f255d036f3c8378acfa694e9f2c99/scikit_learn-0.21.3-cp37-cp37m-manylinux1_x86_64.whl (6.7MB)
Collecting scipy (from -r requirements.txt (line 2))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/5d/bd/c0feba81fb60e231cf40fc8a322ed5873c90ef7711795508692b1481a4ae/scipy-1.3.0-cp37-cp37m-manylinux1_x86_64.whl (25.2MB)
Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn->-r requirements.txt (line 1)) (1.17.0)
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Collecting joblib>=0.11 (from scikit-learn->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/cd/c1/50a758e8247561e58cb87305b1e90b171b8c767b15b12a1734001f41d356/joblib-0.13.2-py2.py3-none-any.whl (278kB)
Installing collected packages: joblib, scipy, scikit-learn
Successfully installed joblib-0.13.2 scikit-learn-0.21.3 scipy-1.3.0
WARNING: You are using pip version 19.1.1, however version 19.2.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Build completed successfully
[7]:
!docker run --name "iris_predictor" -d --rm -p 5000:5000 sklearn-iris:0.1
d84a589e03dd9c5461d913f093834b511e1593749cf8be9794985a582b8094a0

Test using NDArray payload

[8]:
!seldon-core-tester contract.json 0.0.0.0 5000 -p --grpc
----------------------------------------
SENDING NEW REQUEST:

[[6.002 3.459 7.38  0.827]]
RECEIVED RESPONSE:
meta {
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  ndarray {
    values {
      list_value {
        values {
          number_value: 3.41671271543092e-05
        }
        values {
          number_value: 0.43959527937639553
        }
        values {
          number_value: 0.5603705534964502
        }
      }
    }
  }
}


Test using Tensor payload

[9]:
!seldon-core-tester contract.json 0.0.0.0 5000 -p --grpc --tensor
----------------------------------------
SENDING NEW REQUEST:

[[7.076 2.11  5.33  2.677]]
RECEIVED RESPONSE:
meta {
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  tensor {
    shape: 1
    shape: 3
    values: 0.0001347135284905143
    values: 0.34256764312153126
    values: 0.6572976433499783
  }
}


[10]:
!docker rm iris_predictor --force
iris_predictor

Test using Minikube

Due to a minikube/s2i issue you will need s2i >= 1.1.13

[12]:
!minikube start --memory 4096
😄  minikube v0.34.1 on linux (amd64)
🔥  Creating virtualbox VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
📶  "minikube" IP address is 192.168.99.100
🐳  Configuring Docker as the container runtime ...
✨  Preparing Kubernetes environment ...
🚜  Pulling images required by Kubernetes v1.13.3 ...
🚀  Launching Kubernetes v1.13.3 using kubeadm ...
🔑  Configuring cluster permissions ...
🤔  Verifying component health .....
💗  kubectl is now configured to use "minikube"
🏄  Done! Thank you for using minikube!
[13]:
!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created
[14]:
!helm init
$HELM_HOME has been configured at /home/clive/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
[15]:
!kubectl rollout status deploy/tiller-deploy -n kube-system
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out
[6]:
!helm install ../../../helm-charts/seldon-core-operator --name seldon-core --set usageMetrics.enabled=true --namespace seldon-system
NAME:   seldon-core
LAST DEPLOYED: Thu Apr 25 09:02:59 2019
NAMESPACE: seldon-system
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/CustomResourceDefinition
NAME                                         AGE
seldondeployments.machinelearning.seldon.io  0s

==> v1/ClusterRole
seldon-operator-manager-role  0s

==> v1/ClusterRoleBinding
NAME                                 AGE
seldon-operator-manager-rolebinding  0s

==> v1/Service
NAME                                        TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)  AGE
seldon-operator-controller-manager-service  ClusterIP  10.106.80.138  <none>       443/TCP  0s

==> v1/StatefulSet
NAME                                DESIRED  CURRENT  AGE
seldon-operator-controller-manager  1        1        0s

==> v1/Pod(related)
NAME                                  READY  STATUS             RESTARTS  AGE
seldon-operator-controller-manager-0  0/1    ContainerCreating  0         0s

==> v1/Secret
NAME                                   TYPE    DATA  AGE
seldon-operator-webhook-server-secret  Opaque  0     0s


NOTES:
NOTES: TODO


[7]:
!kubectl rollout status statefulset.apps/seldon-operator-controller-manager -n seldon-system
partitioned roll out complete: 1 new pods have been updated...

Setup Ingress

Please note: There are reported gRPC issues with ambassador (see https://github.com/SeldonIO/seldon-core/issues/473).

[8]:
!helm install stable/ambassador --name ambassador --set crds.keep=false
NAME:   ambassador
LAST DEPLOYED: Thu Apr 25 09:03:43 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/ClusterRoleBinding
NAME        AGE
ambassador  0s

==> v1/Service
NAME               TYPE          CLUSTER-IP     EXTERNAL-IP  PORT(S)                     AGE
ambassador-admins  ClusterIP     10.107.239.6   <none>       8877/TCP                    0s
ambassador         LoadBalancer  10.97.236.148  <pending>    80:30062/TCP,443:30447/TCP  0s

==> v1/Deployment
NAME        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
ambassador  3        3        3           0          0s

==> v1/Pod(related)
NAME                         READY  STATUS             RESTARTS  AGE
ambassador-5b89d44544-8rjms  0/1    ContainerCreating  0         0s
ambassador-5b89d44544-fvqjc  0/1    ContainerCreating  0         0s
ambassador-5b89d44544-wr55x  0/1    ContainerCreating  0         0s

==> v1/ServiceAccount
NAME        SECRETS  AGE
ambassador  1        0s

==> v1beta1/ClusterRole
NAME        AGE
ambassador  0s


NOTES:
Congratuations! You've successfully installed Ambassador.

For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.

To get the IP address of Ambassador, run the following commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
     You can watch the status of by running 'kubectl get svc -w  --namespace default ambassador'

  On GKE/Azure:
  export SERVICE_IP=$(kubectl get svc --namespace default ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

  On AWS:
  export SERVICE_IP=$(kubectl get svc --namespace default ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

  echo http://$SERVICE_IP:

[9]:
!kubectl rollout status deployment.apps/ambassador
Waiting for deployment "ambassador" rollout to finish: 0 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 1 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 2 of 3 updated replicas are available...
deployment "ambassador" successfully rolled out

Wrap Model and Test

[11]:
!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3:0.12 sklearn-iris:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting scikit-learn (from -r requirements.txt (line 1))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/9f/c5/e5267eb84994e9a92a2c6a6ee768514f255d036f3c8378acfa694e9f2c99/scikit_learn-0.21.3-cp37-cp37m-manylinux1_x86_64.whl (6.7MB)
Collecting scipy (from -r requirements.txt (line 2))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/5d/bd/c0feba81fb60e231cf40fc8a322ed5873c90ef7711795508692b1481a4ae/scipy-1.3.0-cp37-cp37m-manylinux1_x86_64.whl (25.2MB)
Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn->-r requirements.txt (line 1)) (1.17.0)
Collecting joblib>=0.11 (from scikit-learn->-r requirements.txt (line 1))
  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/cd/c1/50a758e8247561e58cb87305b1e90b171b8c767b15b12a1734001f41d356/joblib-0.13.2-py2.py3-none-any.whl (278kB)
Installing collected packages: joblib, scipy, scikit-learn
Successfully installed joblib-0.13.2 scikit-learn-0.21.3 scipy-1.3.0
Build completed successfully
[12]:
!kubectl create -f sklearn_iris_deployment.json
seldondeployment.machinelearning.seldon.io/seldon-deployment-example created
[13]:
!kubectl rollout status deploy/sklearn-iris-deployment-sklearn-iris-predictor-a5a7453
Waiting for deployment "sklearn-iris-deployment-sklearn-iris-predictor-a5a7453" rollout to finish: 0 of 1 updated replicas are available...
deployment "sklearn-iris-deployment-sklearn-iris-predictor-a5a7453" successfully rolled out
[14]:
!seldon-core-api-tester contract.json `minikube ip` `kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'` \
    seldon-deployment-example --namespace default -p
----------------------------------------
SENDING NEW REQUEST:

[[6.956 4.036 9.426 2.103]]
RECEIVED RESPONSE:
meta {
  puid: "if585togj6ra3nb4tt5c1vrccg"
  requestPath {
    key: "sklearn-iris-classifier"
    value: "sklearn-iris:0.1"
  }
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  ndarray {
    values {
      list_value {
        values {
          number_value: 3.6116624137419636e-07
        }
        values {
          number_value: 0.3368495629835383
        }
        values {
          number_value: 0.6631500758502203
        }
      }
    }
  }
}


[25]:
!minikube delete
🔥  Deleting "minikube" from virtualbox ...
💔  The "minikube" cluster has been deleted.
[ ]: