This page was generated from examples/models/mlflow_model/mlflow.ipynb.

Run MLFlow Model in Seldon CoreΒΆ

This notebook shows how you can easily train a model using MLFlow and serve requests within Seldon Core on Kubernetes.

Dependencies

  • pip install seldon-core
  • pip install mlflow

Train Example MlFlow ModelΒΆ

[8]:
!git clone https://github.com/mlflow/mlflow
Cloning into 'mlflow'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 7786 (delta 3), reused 4 (delta 1), pack-reused 7772
Receiving objects: 100% (7786/7786), 8.15 MiB | 407.00 KiB/s, done.
Resolving deltas: 100% (4781/4781), done.
Checking connectivity... done.
[9]:
!python mlflow/examples/sklearn_elasticnet_wine/train.py
Elasticnet model (alpha=0.500000, l1_ratio=0.500000):
  RMSE: 0.82224284975954
  MAE: 0.6278761410160691
  R2: 0.12678721972772689

Test Inference LocallyΒΆ

[3]:
!pygmentize MyMlflowModel.py
from mlflow import pyfunc
import os
import pandas as pd

class MyMlflowModel(object):

    def __init__(self):
        self.pyfunc_model = pyfunc.load_pyfunc("mlruns/0/"+next(os.walk('mlruns/0'))[1][0]+"/artifacts/model")

    def predict(self,X,features_names):
        if not features_names is None and len(features_names)>0:
            df = pd.DataFrame(data=X,columns=features_names)
        else:
            df = pd.DataFrame(data=X)
        return self.pyfunc_model.predict(df)
[10]:
!s2i build -E environment_rest . seldonio/seldon-core-s2i-python3:0.6-SNAPSHOT mlflow_model:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting mlflow (from -r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/e5/f7/d349b011f4e03b06a3a0f2916647c578cd89d338f1e244ddb4e1fee600fc/mlflow-0.9.0.1-py3-none-any.whl (11.8MB)
Collecting sklearn (from -r requirements.txt (line 2))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/1e/7a/dbb3be0ce9bd5c8b7e3d87328e79063f8b263b2b1bfa4774cb1147bfcd3f/sklearn-0.0.tar.gz
Collecting pandas (from -r requirements.txt (line 3))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/19/74/e50234bc82c553fecdbd566d8650801e3fe2d6d8c8d940638e3d8a7c5522/pandas-0.24.2-cp36-cp36m-manylinux1_x86_64.whl (10.1MB)
Collecting mleap>=0.8.1 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/3b/17/486a3cf30695581f029ba618ee878252a9e9f859c48913e30a07d13c8a21/mleap-0.8.1.tar.gz
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (7.0)
Collecting gitpython>=2.1.0 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/fe/e5/fafe827507644c32d6dc553a1c435cdf882e0c28918a5bab29f7fbebfb70/GitPython-2.1.11-py2.py3-none-any.whl (448kB)
Collecting databricks-cli>=0.8.0 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/b2/9e/e2868a5560827357796f8682803bbac73d30de4e890604e61f2488ce9c03/databricks-cli-0.8.6.tar.gz (41kB)
Collecting querystring-parser (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/57/64/3086a9a991ff3aca7b769f5b0b51ff8445a06337ae2c58f215bcee48f527/querystring_parser-1.2.3.tar.gz
Requirement already satisfied: Flask in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (1.0.2)
Collecting cloudpickle (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/6e/bc/67f13115adcce4efc5e4d7f8220fb9a50aaa2b5c7ed460b26cbb76aa76ad/cloudpickle-0.8.1-py2.py3-none-any.whl
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (1.12.0)
Collecting entrypoints (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/ac/c6/44694103f8c221443ee6b0041f69e2740d89a25641e62fb4f2ee568f2f9c/entrypoints-0.3-py2.py3-none-any.whl
Collecting boto3>=1.7.12 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/c3/ef/f81d8c35c7254fe2af9a1a8e034f07b88a824a441f37955d0a07a90b8ec7/boto3-1.9.130-py2.py3-none-any.whl (128kB)
Collecting sqlparse (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.17.3 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (2.21.0)
Collecting scipy (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/7f/5f/c48860704092933bf1c4c1574a8de1ffd16bf4fde8bab190d747598844b2/scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8MB)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (5.1)
Collecting docker>=3.6.0 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/48/68/c3afca1a5aa8d2997ec3b8ee822a4d752cf85907b321f07ea86888545152/docker-3.7.2-py2.py3-none-any.whl (134kB)
Collecting scikit-learn (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/5e/82/c0de5839d613b82bddd088599ac0bbfbbbcbd8ca470680658352d2c435bd/scikit_learn-0.20.3-cp36-cp36m-manylinux1_x86_64.whl (5.4MB)
Collecting python-dateutil (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (3.7.1)
Collecting simplejson (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/e3/24/c35fb1c1c315fc0fffe61ea00d3f88e85469004713dab488dee4f35b0aff/simplejson-3.16.0.tar.gz (81kB)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (1.16.2)
Collecting gunicorn (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl (112kB)
Collecting pytz>=2011k (from pandas->-r requirements.txt (line 3))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
Collecting argparse>=1.1 (from mleap>=0.8.1->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl
Collecting nose-exclude>=0.5.0 (from mleap>=0.8.1->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/63/cf/90c4be56bf11b7bc8801086d9445baf731aa36b8e8fc5791731e8e604dcd/nose-exclude-0.5.0.tar.gz
Collecting gitdb2>=2.0.0 (from gitpython>=2.1.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/da/30/a407568aa8d8f25db817cf50121a958722f3fc5f87e3a6fba1f40c0633e3/gitdb2-2.0.5-py2.py3-none-any.whl (62kB)
Collecting tabulate>=0.7.7 (from databricks-cli>=0.8.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/c2/fd/202954b3f0eb896c53b7b6f07390851b1fd2ca84aa95880d7ae4f434c4ac/tabulate-0.8.3.tar.gz (46kB)
Collecting configparser>=0.3.5 (from databricks-cli>=0.8.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/ba/05/6c96328e92e625fc31445d24d75a2c92ef9ba34fc5b037fe69693c362a0d/configparser-3.7.4-py2.py3-none-any.whl
Requirement already satisfied: Werkzeug>=0.14 in /usr/local/lib/python3.6/site-packages (from Flask->mlflow->-r requirements.txt (line 1)) (0.15.1)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/site-packages (from Flask->mlflow->-r requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: Jinja2>=2.10 in /usr/local/lib/python3.6/site-packages (from Flask->mlflow->-r requirements.txt (line 1)) (2.10)
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/d7/de/5737f602e22073ecbded7a0c590707085e154e32b68d86545dcc31004c02/s3transfer-0.2.0-py2.py3-none-any.whl (69kB)
Collecting botocore<1.13.0,>=1.12.130 (from boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/21/80/6683916e9e1eaf867e09b81a6df6d7a2ba181e3095d2367c7a0f635ed1ed/botocore-1.12.130-py2.py3-none-any.whl (5.4MB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (2019.3.9)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (2.8)
Collecting websocket-client>=0.32.0 (from docker>=3.6.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)
Collecting docker-pycreds>=0.4.0 (from docker>=3.6.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/f5/e8/f6bd1eee09314e7e6dee49cbe2c5e22314ccdb38db16c9fc72d2fa80d054/docker_pycreds-0.4.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages (from protobuf>=3.6.0->mlflow->-r requirements.txt (line 1)) (40.6.3)
Collecting nose (from nose-exclude>=0.5.0->mleap>=0.8.1->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/15/d8/dd071918c040f50fa1cf80da16423af51ff8ce4a0f2399b7bf8de45ac3d9/nose-1.3.7-py3-none-any.whl (154kB)
Collecting smmap2>=2.0.0 (from gitdb2>=2.0.0->gitpython>=2.1.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/55/d2/866d45e3a121ee15a1dc013824d58072fd5c7799c9c34d01378eb262ca8f/smmap2-2.0.5-py2.py3-none-any.whl
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/site-packages (from Jinja2>=2.10->Flask->mlflow->-r requirements.txt (line 1)) (1.1.1)
Collecting docutils>=0.10 (from botocore<1.13.0,>=1.12.130->boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl (543kB)
Building wheels for collected packages: sklearn, mleap, databricks-cli, querystring-parser, simplejson, nose-exclude, tabulate
Building wheel for sklearn (setup.py): started
Building wheel for sklearn (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/76/03/bb/589d421d27431bcd2c6da284d5f2286c8e3b2ea3cf1594c074
Building wheel for mleap (setup.py): started
Building wheel for mleap (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/bc/a5/8d/7d021f2741ed8a7354e4fe19c1890e71ed719d681cd16cf292
Building wheel for databricks-cli (setup.py): started
Building wheel for databricks-cli (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/7a/3e/18/4770e5048530a759970f03239d88b3d32d7434c410d803cfc9
Building wheel for querystring-parser (setup.py): started
Building wheel for querystring-parser (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/ee/09/99/bf937e4f02788fa8b33dc5240842ba3977ba5c3c4ad4a115d7
Building wheel for simplejson (setup.py): started
Building wheel for simplejson (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/5d/1a/1e/0350bb3df3e74215cd91325344cc86c2c691f5306eb4d22c77
Building wheel for nose-exclude (setup.py): started
Building wheel for nose-exclude (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/fe/b9/8f/764ed47b11e2e062ae8fe5f09d1c801a600292fdd9ba3477bb
Building wheel for tabulate (setup.py): started
Building wheel for tabulate (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/2b/67/89/414471314a2d15de625d184d8be6d38a03ae1e983dbda91e84
Successfully built sklearn mleap databricks-cli querystring-parser simplejson nose-exclude tabulate
mlflow 0.9.0.1 has requirement cloudpickle==0.6.1, but you'll have cloudpickle 0.8.1 which is incompatible.
Installing collected packages: argparse, scipy, python-dateutil, pytz, pandas, scikit-learn, nose, nose-exclude, mleap, smmap2, gitdb2, gitpython, tabulate, configparser, databricks-cli, querystring-parser, cloudpickle, entrypoints, jmespath, docutils, botocore, s3transfer, boto3, sqlparse, websocket-client, docker-pycreds, docker, simplejson, gunicorn, mlflow, sklearn
Successfully installed argparse-1.4.0 boto3-1.9.130 botocore-1.12.130 cloudpickle-0.8.1 configparser-3.7.4 databricks-cli-0.8.6 docker-3.7.2 docker-pycreds-0.4.0 docutils-0.14 entrypoints-0.3 gitdb2-2.0.5 gitpython-2.1.11 gunicorn-19.9.0 jmespath-0.9.4 mleap-0.8.1 mlflow-0.9.0.1 nose-1.3.7 nose-exclude-0.5.0 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 querystring-parser-1.2.3 s3transfer-0.2.0 scikit-learn-0.20.3 scipy-1.2.1 simplejson-3.16.0 sklearn-0.0 smmap2-2.0.5 sqlparse-0.3.0 tabulate-0.8.3 websocket-client-0.56.0
Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
You are using pip version 19.0.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Build completed successfully
[11]:
!docker run --name "mlflow_model" -d --rm -p 5000:5000 mlflow_model:0.1
44ba8b3fd0999714f4a56f061275973a54564aee4e1161c79e7fbb8288adea9d
[13]:
!curl -H "Content-Type: application/x-www-form-urlencoded" -g 0.0.0.0:5000/predict -d 'json={"data":{"names":["alcohol", "chlorides", "citric acid", "density", "fixed acidity", "free sulfur dioxide", "pH", "residual sugar", "sulphates", "total sulfur dioxide", "volatile acidity"],"ndarray":[[12.8, 0.029, 0.48, 0.98, 6.2, 29, 3.33, 1.2, 0.39, 75, 0.66]]}}'
{"data":{"ndarray":[5.455573233630147]},"meta":{}}
[14]:
!curl -H "Content-Type: application/x-www-form-urlencoded" -g 0.0.0.0:5000/predict -d 'json={"data":{"ndarray":[[12.8, 0.029, 0.48, 0.98, 6.2, 29, 3.33, 1.2, 0.39, 75, 0.66]]}}'
{"data":{"ndarray":[5.455573233630147]},"meta":{}}
[15]:
!docker rm mlflow_model --force
mlflow_model

Test Inference on MinikubeΒΆ

Due to a `minikube/s2i issue <https://github.com/SeldonIO/seldon-core/issues/253>`__ you will need `s2i >= 1.1.13 <https://github.com/openshift/source-to-image/releases/tag/v1.1.13>`__

[9]:
!minikube start --memory 4096
πŸ˜„  minikube v0.34.1 on linux (amd64)
πŸ”₯  Creating virtualbox VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
πŸ“Ά  "minikube" IP address is 192.168.99.100
🐳  Configuring Docker as the container runtime ...
✨  Preparing Kubernetes environment ...
🚜  Pulling images required by Kubernetes v1.13.3 ...
πŸš€  Launching Kubernetes v1.13.3 using kubeadm ...
πŸ”‘  Configuring cluster permissions ...
πŸ€”  Verifying component health .....
πŸ’—  kubectl is now configured to use "minikube"
πŸ„  Done! Thank you for using minikube!
[10]:
!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created
[11]:
!helm init
$HELM_HOME has been configured at /home/clive/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
[12]:
!kubectl rollout status deploy/tiller-deploy -n kube-system
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out
[1]:
!helm install ../../../helm-charts/seldon-core-operator --name seldon-core --set usageMetrics.enabled=true --namespace seldon-system
NAME:   seldon-core
LAST DEPLOYED: Tue Apr 16 14:39:51 2019
NAMESPACE: seldon-system
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/ClusterRole
NAME                        AGE
seldon-spartakus-volunteer  1s

==> v1/Pod(related)
NAME                                  READY  STATUS             RESTARTS  AGE
seldon-operator-controller-manager-0  0/1    ContainerCreating  0         1s

==> v1/Secret
NAME                                   TYPE    DATA  AGE
seldon-operator-webhook-server-secret  Opaque  0     1s

==> v1/ConfigMap
NAME                     DATA  AGE
seldon-spartakus-config  3     1s

==> v1beta1/CustomResourceDefinition
NAME                                         AGE
seldondeployments.machinelearning.seldon.io  1s

==> v1/ClusterRole
seldon-operator-manager-role  1s

==> v1beta1/Deployment
NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
seldon-spartakus-volunteer  1        0        0           0          1s

==> v1/StatefulSet
NAME                                DESIRED  CURRENT  AGE
seldon-operator-controller-manager  1        1        1s

==> v1/ClusterRoleBinding
NAME                                 AGE
seldon-operator-manager-rolebinding  1s

==> v1/Service
NAME                                        TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)  AGE
seldon-operator-controller-manager-service  ClusterIP  10.96.202.141  <none>       443/TCP  1s

==> v1/ServiceAccount
NAME                        SECRETS  AGE
seldon-spartakus-volunteer  1        1s

==> v1beta1/ClusterRoleBinding
NAME                        AGE
seldon-spartakus-volunteer  1s


NOTES:
NOTES: TODO


[2]:
!kubectl rollout status statefulset.apps/seldon-operator-controller-manager -n seldon-system
partitioned roll out complete: 1 new pods have been updated...

Setup IngressΒΆ

There are gRPC issues with the latest Ambassador, so we rewcommend 0.40.2 until these are fixed.

[3]:
!helm install stable/ambassador --name ambassador --set image.tag=0.40.2
NAME:   ambassador
LAST DEPLOYED: Tue Apr 16 14:40:42 2019
NAMESPACE: seldon
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME                         READY  STATUS             RESTARTS  AGE
ambassador-5b89d44544-ph44v  0/1    ContainerCreating  0         0s
ambassador-5b89d44544-q4jfn  0/1    ContainerCreating  0         0s
ambassador-5b89d44544-qdc9t  0/1    ContainerCreating  0         0s

==> v1/ServiceAccount
NAME        SECRETS  AGE
ambassador  1        1s

==> v1beta1/ClusterRole
NAME        AGE
ambassador  1s

==> v1beta1/ClusterRoleBinding
NAME        AGE
ambassador  1s

==> v1/Service
NAME               TYPE          CLUSTER-IP      EXTERNAL-IP  PORT(S)                     AGE
ambassador-admins  ClusterIP     10.101.35.185   <none>       8877/TCP                    1s
ambassador         LoadBalancer  10.108.206.144  <pending>    80:30921/TCP,443:30522/TCP  0s

==> v1/Deployment
NAME        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
ambassador  3        3        3           0          0s


NOTES:
Congratuations! You've successfully installed Ambassador.

For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.

To get the IP address of Ambassador, run the following commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
     You can watch the status of by running 'kubectl get svc -w  --namespace seldon ambassador'

  On GKE/Azure:
  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

  On AWS:
  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

  echo http://$SERVICE_IP:

[4]:
!kubectl rollout status deployment.apps/ambassador
Waiting for deployment "ambassador" rollout to finish: 0 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 1 of 3 updated replicas are available...
Waiting for deployment "ambassador" rollout to finish: 2 of 3 updated replicas are available...
deployment "ambassador" successfully rolled out
[16]:
!eval $(minikube docker-env) && s2i build -E environment_rest . seldonio/seldon-core-s2i-python3:0.6-SNAPSHOT mlflow_model:0.1
---> Installing application source...
---> Installing dependencies ...
Looking in links: /whl
Collecting mlflow (from -r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/e5/f7/d349b011f4e03b06a3a0f2916647c578cd89d338f1e244ddb4e1fee600fc/mlflow-0.9.0.1-py3-none-any.whl (11.8MB)
Collecting sklearn (from -r requirements.txt (line 2))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/1e/7a/dbb3be0ce9bd5c8b7e3d87328e79063f8b263b2b1bfa4774cb1147bfcd3f/sklearn-0.0.tar.gz
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Collecting pandas (from -r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/19/74/e50234bc82c553fecdbd566d8650801e3fe2d6d8c8d940638e3d8a7c5522/pandas-0.24.2-cp36-cp36m-manylinux1_x86_64.whl (10.1MB)
Collecting simplejson (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/e3/24/c35fb1c1c315fc0fffe61ea00d3f88e85469004713dab488dee4f35b0aff/simplejson-3.16.0.tar.gz (81kB)
Collecting cloudpickle==0.6.1 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/fc/87/7b7ef3038b4783911e3fdecb5c566e3a817ce3e890e164fc174c088edb1e/cloudpickle-0.6.1-py2.py3-none-any.whl
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Collecting pyyaml (from mlflow->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/9f/2c/9417b5c774792634834e730932745bc09a7d36754ca00acf1ccd1ac2594d/PyYAML-5.1.tar.gz (274kB)
Collecting python-dateutil (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
Collecting gitpython>=2.1.0 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/fe/e5/fafe827507644c32d6dc553a1c435cdf882e0c28918a5bab29f7fbebfb70/GitPython-2.1.11-py2.py3-none-any.whl (448kB)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (1.16.1)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (1.12.0)
Collecting docker>=3.6.0 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/48/68/c3afca1a5aa8d2997ec3b8ee822a4d752cf85907b321f07ea86888545152/docker-3.7.2-py2.py3-none-any.whl (134kB)
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Collecting gunicorn (from mlflow->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl (112kB)
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Collecting databricks-cli>=0.8.0 (from mlflow->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/b2/9e/e2868a5560827357796f8682803bbac73d30de4e890604e61f2488ce9c03/databricks-cli-0.8.6.tar.gz (41kB)
Collecting scipy (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/7f/5f/c48860704092933bf1c4c1574a8de1ffd16bf4fde8bab190d747598844b2/scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8MB)
Requirement already satisfied: Flask in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (1.0.2)
Collecting entrypoints (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/ac/c6/44694103f8c221443ee6b0041f69e2740d89a25641e62fb4f2ee568f2f9c/entrypoints-0.3-py2.py3-none-any.whl
Collecting querystring-parser (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/57/64/3086a9a991ff3aca7b769f5b0b51ff8445a06337ae2c58f215bcee48f527/querystring_parser-1.2.3.tar.gz
Collecting mleap>=0.8.1 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/3b/17/486a3cf30695581f029ba618ee878252a9e9f859c48913e30a07d13c8a21/mleap-0.8.1.tar.gz
Collecting scikit-learn (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/5e/82/c0de5839d613b82bddd088599ac0bbfbbbcbd8ca470680658352d2c435bd/scikit_learn-0.20.3-cp36-cp36m-manylinux1_x86_64.whl (5.4MB)
Collecting boto3>=1.7.12 (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/c3/ef/f81d8c35c7254fe2af9a1a8e034f07b88a824a441f37955d0a07a90b8ec7/boto3-1.9.130-py2.py3-none-any.whl (128kB)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (3.6.1)
Requirement already satisfied: requests>=2.17.3 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (2.21.0)
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.6/site-packages (from mlflow->-r requirements.txt (line 1)) (7.0)
Collecting sqlparse (from mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
Collecting pytz>=2011k (from pandas->-r requirements.txt (line 3))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
Collecting gitdb2>=2.0.0 (from gitpython>=2.1.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/da/30/a407568aa8d8f25db817cf50121a958722f3fc5f87e3a6fba1f40c0633e3/gitdb2-2.0.5-py2.py3-none-any.whl (62kB)
Collecting websocket-client>=0.32.0 (from docker>=3.6.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)
Collecting docker-pycreds>=0.4.0 (from docker>=3.6.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/f5/e8/f6bd1eee09314e7e6dee49cbe2c5e22314ccdb38db16c9fc72d2fa80d054/docker_pycreds-0.4.0-py2.py3-none-any.whl
Collecting tabulate>=0.7.7 (from databricks-cli>=0.8.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/c2/fd/202954b3f0eb896c53b7b6f07390851b1fd2ca84aa95880d7ae4f434c4ac/tabulate-0.8.3.tar.gz (46kB)
Collecting configparser>=0.3.5 (from databricks-cli>=0.8.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/ba/05/6c96328e92e625fc31445d24d75a2c92ef9ba34fc5b037fe69693c362a0d/configparser-3.7.4-py2.py3-none-any.whl
Requirement already satisfied: Werkzeug>=0.14 in /usr/local/lib/python3.6/site-packages (from Flask->mlflow->-r requirements.txt (line 1)) (0.14.1)
Requirement already satisfied: Jinja2>=2.10 in /usr/local/lib/python3.6/site-packages (from Flask->mlflow->-r requirements.txt (line 1)) (2.10)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/site-packages (from Flask->mlflow->-r requirements.txt (line 1)) (1.1.0)
Collecting argparse>=1.1 (from mleap>=0.8.1->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl
Collecting nose-exclude>=0.5.0 (from mleap>=0.8.1->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/63/cf/90c4be56bf11b7bc8801086d9445baf731aa36b8e8fc5791731e8e604dcd/nose-exclude-0.5.0.tar.gz
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/d7/de/5737f602e22073ecbded7a0c590707085e154e32b68d86545dcc31004c02/s3transfer-0.2.0-py2.py3-none-any.whl (69kB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting botocore<1.13.0,>=1.12.130 (from boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/21/80/6683916e9e1eaf867e09b81a6df6d7a2ba181e3095d2367c7a0f635ed1ed/botocore-1.12.130-py2.py3-none-any.whl (5.4MB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages (from protobuf>=3.6.0->mlflow->-r requirements.txt (line 1)) (40.8.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (2018.11.29)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/site-packages (from requests>=2.17.3->mlflow->-r requirements.txt (line 1)) (2.8)
Collecting smmap2>=2.0.0 (from gitdb2>=2.0.0->gitpython>=2.1.0->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/55/d2/866d45e3a121ee15a1dc013824d58072fd5c7799c9c34d01378eb262ca8f/smmap2-2.0.5-py2.py3-none-any.whl
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/site-packages (from Jinja2>=2.10->Flask->mlflow->-r requirements.txt (line 1)) (1.1.0)
Collecting nose (from nose-exclude>=0.5.0->mleap>=0.8.1->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/15/d8/dd071918c040f50fa1cf80da16423af51ff8ce4a0f2399b7bf8de45ac3d9/nose-1.3.7-py3-none-any.whl (154kB)
Collecting docutils>=0.10 (from botocore<1.13.0,>=1.12.130->boto3>=1.7.12->mlflow->-r requirements.txt (line 1))
  Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Downloading https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl (543kB)
Building wheels for collected packages: sklearn, simplejson, pyyaml, databricks-cli, querystring-parser, mleap, tabulate, nose-exclude
Building wheel for sklearn (setup.py): started
Building wheel for sklearn (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/76/03/bb/589d421d27431bcd2c6da284d5f2286c8e3b2ea3cf1594c074
Building wheel for simplejson (setup.py): started
Building wheel for simplejson (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/5d/1a/1e/0350bb3df3e74215cd91325344cc86c2c691f5306eb4d22c77
Building wheel for pyyaml (setup.py): started
Building wheel for pyyaml (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/ad/56/bc/1522f864feb2a358ea6f1a92b4798d69ac783a28e80567a18b
Building wheel for databricks-cli (setup.py): started
Building wheel for databricks-cli (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/7a/3e/18/4770e5048530a759970f03239d88b3d32d7434c410d803cfc9
Building wheel for querystring-parser (setup.py): started
Building wheel for querystring-parser (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/ee/09/99/bf937e4f02788fa8b33dc5240842ba3977ba5c3c4ad4a115d7
Building wheel for mleap (setup.py): started
Building wheel for mleap (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/bc/a5/8d/7d021f2741ed8a7354e4fe19c1890e71ed719d681cd16cf292
Building wheel for tabulate (setup.py): started
Building wheel for tabulate (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/2b/67/89/414471314a2d15de625d184d8be6d38a03ae1e983dbda91e84
Building wheel for nose-exclude (setup.py): started
Building wheel for nose-exclude (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/fe/b9/8f/764ed47b11e2e062ae8fe5f09d1c801a600292fdd9ba3477bb
Successfully built sklearn simplejson pyyaml databricks-cli querystring-parser mleap tabulate nose-exclude
Installing collected packages: simplejson, cloudpickle, pyyaml, python-dateutil, smmap2, gitdb2, gitpython, websocket-client, docker-pycreds, docker, pytz, pandas, gunicorn, tabulate, configparser, databricks-cli, scipy, entrypoints, querystring-parser, argparse, scikit-learn, nose, nose-exclude, mleap, jmespath, docutils, botocore, s3transfer, boto3, sqlparse, mlflow, sklearn
Successfully installed argparse-1.4.0 boto3-1.9.130 botocore-1.12.130 cloudpickle-0.6.1 configparser-3.7.4 databricks-cli-0.8.6 docker-3.7.2 docker-pycreds-0.4.0 docutils-0.14 entrypoints-0.3 gitdb2-2.0.5 gitpython-2.1.11 gunicorn-19.9.0 jmespath-0.9.4 mleap-0.8.1 mlflow-0.9.0.1 nose-1.3.7 nose-exclude-0.5.0 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 pyyaml-5.1 querystring-parser-1.2.3 s3transfer-0.2.0 scikit-learn-0.20.3 scipy-1.2.1 simplejson-3.16.0 sklearn-0.0 smmap2-2.0.5 sqlparse-0.3.0 tabulate-0.8.3 websocket-client-0.56.0
Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.
Build completed successfully
[17]:
!kubectl create -f deployment.json
seldondeployment.machinelearning.seldon.io/mlflow-example created
[18]:
!kubectl rollout status deployment/mlflow-dep-mlflow-pred-d580056
Waiting for deployment "mlflow-dep-mlflow-pred-d580056" rollout to finish: 0 of 1 updated replicas are available...
deployment "mlflow-dep-mlflow-pred-d580056" successfully rolled out
[19]:
!seldon-core-api-tester contract.json `minikube ip` `kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'` \
    mlflow-example --namespace seldon -p
----------------------------------------
SENDING NEW REQUEST:

[[2.8040e+00 7.5600e-01 2.8000e-02 1.3500e-01 5.1000e-02 6.6000e-02
  6.8270e+00 8.6900e-01 2.0100e-01 7.5268e+01 4.2500e-01]]
RECEIVED RESPONSE:
meta {
  puid: "fjmht64oou6r5f8b8ju3aagu4u"
  requestPath {
    key: "model"
    value: "mlflow_model:0.1"
  }
}
data {
  ndarray {
    values {
      number_value: 5.1800195352034475
    }
  }
}


[ ]:
!minikube delete