This page was generated from examples/models/metadata/metadata.ipynb.

Simple Metadata Example

Prerequisites

• A kubernetes cluster with kubectl configured

• curl

• grpcurl

• pygmentize

Setup Seldon Core

Use the setup notebook to Setup Cluster to setup Seldon Core with an ingress.

!kubectl create namespace seldon

Error from server (AlreadyExists): namespaces "seldon" already exists

!kubectl config set-context $(kubectl config current-context) --namespace=seldon

Context "kind-kind" modified.

Example description

In this example we will define a following metadata

name: my-model-name
versions: [ my-model-version-01 ]
platform: seldon
inputs:
- messagetype: tensor
  schema:
    names: [a, b, c, d]
    shape: [ 4 ]
outputs:
- messagetype: tensor
  schema:
    shape: [ 1 ]

which corresponds to model taking a tensor input with four columns named a, b, c, and d.

We will define the metadata in two ways: directly in the model.py and in the deployment manifest.

When defining metadata in deployment manifest we will use different names to indicate that metadata can be overwritten.

At the end we will see what happens if one misdefined the Metadata.

import time

import requests


def getWithRetry(url, expected_code=requests.codes.ok):
    for i in range(3):
        r = requests.get(url)
        if r.status_code == expected_code:
            meta = r.json()
            return meta
        else:
            print("Failed request with status code ", r.status_code)
            time.sleep(3)

1. Directly define in model

Define Model

%%writefile models/init-metadata/Model.py

import logging


class Model:
    def predict(self, features, names=[], meta=[]):
        logging.info(f"model features: {features}")
        logging.info(f"model names: {names}")
        logging.info(f"model meta: {meta}")
        return features

    def init_metadata(self):
        logging.info("metadata method  called")

        meta = {
            "name": "my-model-name",
            "versions": ["my-model-version-01"],
            "platform": "seldon",
            "inputs": [
                {
                    "messagetype": "tensor",
                    "schema": {"names": ["a", "b", "c", "d"], "shape": [4]},
                }
            ],
            "outputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
            "custom": {
                "author": "seldon-dev"
            }
        }

        return meta

Overwriting models/init-metadata/Model.py

Build image

build image using provided Makefile

cd models/init-metadata
make build

If you are using kind you can use kind_image_install target to directly load your image into your local cluster.

Define deployment

%%writefile model-metadata/init-metadata.yaml

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-model-init-metadata
spec:
  name: test-deployment
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: seldonio/model-with-metadata:0.5
          name: my-model
          env:
          - name: SELDON_LOG_LEVEL
            value: DEBUG
    graph:
      children: []
      name: my-model
      type: MODEL
    name: example
    replicas: 1

Overwriting model-metadata/init-metadata.yaml

Deploy

!kubectl apply -f model-metadata/init-metadata.yaml

seldondeployment.machinelearning.seldon.io/seldon-model-init-metadata created

!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model-init-metadata -o jsonpath='{.items[0].metadata.name}')

Waiting for deployment "seldon-model-init-metadata-example-0-my-model" rollout to finish: 0 of 1 updated replicas are available...
deployment "seldon-model-init-metadata-example-0-my-model" successfully rolled out

Model Metadata

meta = getWithRetry(
    "http://localhost:8003/seldon/seldon/seldon-model-init-metadata/api/v1.0/metadata/my-model"
)

assert meta == {
    "name": "my-model-name",
    "versions": ["my-model-version-01"],
    "platform": "seldon",
    "inputs": [
        {
            "messagetype": "tensor",
            "schema": {"names": ["a", "b", "c", "d"], "shape": [4]},
        }
    ],
    "outputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
    "custom": {"author": "seldon-dev"},
}

meta

{'custom': {'author': 'seldon-dev'},
 'inputs': [{'messagetype': 'tensor',
   'schema': {'names': ['a', 'b', 'c', 'd'], 'shape': [4]}}],
 'name': 'my-model-name',
 'outputs': [{'messagetype': 'tensor', 'schema': {'shape': [1]}}],
 'platform': 'seldon',
 'versions': ['my-model-version-01']}

Graph Metadata

meta = getWithRetry(
    "http://localhost:8003/seldon/seldon/seldon-model-init-metadata/api/v1.0/metadata"
)

assert meta == {
    "name": "example",
    "models": {
        "my-model": {
            "name": "my-model-name",
            "platform": "seldon",
            "versions": ["my-model-version-01"],
            "inputs": [
                {
                    "messagetype": "tensor",
                    "schema": {"names": ["a", "b", "c", "d"], "shape": [4]},
                }
            ],
            "outputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
            "custom": {"author": "seldon-dev"},
        }
    },
    "graphinputs": [
        {
            "messagetype": "tensor",
            "schema": {"names": ["a", "b", "c", "d"], "shape": [4]},
        }
    ],
    "graphoutputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
}

meta

{'name': 'example',
 'models': {'my-model': {'name': 'my-model-name',
   'platform': 'seldon',
   'versions': ['my-model-version-01'],
   'inputs': [{'messagetype': 'tensor',
     'schema': {'names': ['a', 'b', 'c', 'd'], 'shape': [4]}}],
   'outputs': [{'messagetype': 'tensor', 'schema': {'shape': [1]}}],
   'custom': {'author': 'seldon-dev'}}},
 'graphinputs': [{'messagetype': 'tensor',
   'schema': {'names': ['a', 'b', 'c', 'd'], 'shape': [4]}}],
 'graphoutputs': [{'messagetype': 'tensor', 'schema': {'shape': [1]}}]}

2. Via environmental variable

Metadata defined via environmental variables will overwrite top-level fields in metadata defined directly in the models.

You can use this approach to overwrite metadata defined in the model or to provide metadata if model does not define it.

%%writefile model-metadata/environ-metadata.yaml

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-model-environ-metadata
spec:
  name: test-deployment
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: seldonio/model-with-metadata:0.5
          name: my-model
          env:
          - name: SELDON_LOG_LEVEL
            value: DEBUG
          - name: MODEL_METADATA
            value: |
              ---
              name: second-example-model-name
              versions: [ my-model-version-01 ]
              inputs:
              - messagetype: tensor
                schema:
                  names: [alpha, beta, gamma, delta]
                  shape: [4]
              custom:
                author: seldon-dev
                purpose: tutorial
    graph:
      children: []
      name: my-model
      type: MODEL
    name: example
    replicas: 1

Overwriting model-metadata/environ-metadata.yaml

!kubectl apply -f model-metadata/environ-metadata.yaml

seldondeployment.machinelearning.seldon.io/seldon-model-environ-metadata created

!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model-environ-metadata -o jsonpath='{.items[0].metadata.name}')

Waiting for deployment "seldon-model-environ-metadata-example-0-my-model" rollout to finish: 0 of 1 updated replicas are available...
deployment "seldon-model-environ-metadata-example-0-my-model" successfully rolled out

Model Metadata

meta = getWithRetry(
    "http://localhost:8003/seldon/seldon/seldon-model-environ-metadata/api/v1.0/metadata/my-model"
)


assert meta == {
    "name": "second-example-model-name",
    "versions": ["my-model-version-01"],
    "platform": "seldon",
    "inputs": [
        {
            "messagetype": "tensor",
            "schema": {"names": ["alpha", "beta", "gamma", "delta"], "shape": [4]},
        }
    ],
    "outputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
    "custom": {"author": "seldon-dev", "purpose": "tutorial"},
}

meta

{'custom': {'author': 'seldon-dev', 'purpose': 'tutorial'},
 'inputs': [{'messagetype': 'tensor',
   'schema': {'names': ['alpha', 'beta', 'gamma', 'delta'], 'shape': [4]}}],
 'name': 'second-example-model-name',
 'outputs': [{'messagetype': 'tensor', 'schema': {'shape': [1]}}],
 'platform': 'seldon',
 'versions': ['my-model-version-01']}

Graph Metadata

meta = getWithRetry(
    "http://localhost:8003/seldon/seldon/seldon-model-environ-metadata/api/v1.0/metadata"
)

assert meta == {
    "name": "example",
    "models": {
        "my-model": {
            "name": "second-example-model-name",
            "platform": "seldon",
            "versions": ["my-model-version-01"],
            "inputs": [
                {
                    "messagetype": "tensor",
                    "schema": {
                        "names": ["alpha", "beta", "gamma", "delta"],
                        "shape": [4],
                    },
                }
            ],
            "outputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
            "custom": {"author": "seldon-dev", "purpose": "tutorial"},
        }
    },
    "graphinputs": [
        {
            "messagetype": "tensor",
            "schema": {"names": ["alpha", "beta", "gamma", "delta"], "shape": [4]},
        }
    ],
    "graphoutputs": [{"messagetype": "tensor", "schema": {"shape": [1]}}],
}

meta

{'name': 'example',
 'models': {'my-model': {'name': 'second-example-model-name',
   'platform': 'seldon',
   'versions': ['my-model-version-01'],
   'inputs': [{'messagetype': 'tensor',
     'schema': {'names': ['alpha', 'beta', 'gamma', 'delta'], 'shape': [4]}}],
   'outputs': [{'messagetype': 'tensor', 'schema': {'shape': [1]}}],
   'custom': {'author': 'seldon-dev', 'purpose': 'tutorial'}}},
 'graphinputs': [{'messagetype': 'tensor',
   'schema': {'names': ['alpha', 'beta', 'gamma', 'delta'], 'shape': [4]}}],
 'graphoutputs': [{'messagetype': 'tensor', 'schema': {'shape': [1]}}]}

3. Invalid metadata definition

Here we define metadata that is invalid (versions must be a list)

%%writefile model-metadata/invalid-environ-metadata.yaml

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-model-invalid-environ-metadata
spec:
  name: test-deployment
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: seldonio/model-with-metadata:0.5
          name: my-model
          env:
          - name: SELDON_LOG_LEVEL
            value: DEBUG
          - name: MODEL_METADATA
            value: |
              ---
              name: my-model-name
              versions: my-model-version-01
    graph:
      children: []
      name: my-model
      type: MODEL
    name: example
    replicas: 1

Overwriting model-metadata/invalid-environ-metadata.yaml

!kubectl apply -f model-metadata/invalid-environ-metadata.yaml

seldondeployment.machinelearning.seldon.io/seldon-model-invalid-environ-metadata created

!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model-invalid-environ-metadata -o jsonpath='{.items[0].metadata.name}')

Waiting for deployment "seldon-model-invalid-environ-metadata-example-0-my-model" rollout to finish: 0 of 1 updated replicas are available...
deployment "seldon-model-invalid-environ-metadata-example-0-my-model" successfully rolled out

Requesting Metadata will lead to 500 error.

meta = getWithRetry(
    "http://localhost:8003/seldon/seldon/seldon-model-invalid-environ-metadata/api/v1.0/metadata/my-model",
    500,
)

assert meta == {
    "status": {
        "code": -1,
        "info": "Model metadata unavailable",
        "reason": "MICROSERVICE_BAD_METADATA",
        "status": 1,
    }
}

meta

{'status': {'code': -1,
  'info': 'Model metadata unavailable',
  'reason': 'MICROSERVICE_BAD_METADATA',
  'status': 1}}

But Model will still be serving predictions

%%bash
curl -s -H 'Content-Type: application/json' \
    -d '{"data": {"names": ["input"], "ndarray": ["data"]}}' \
    http://localhost:8003/seldon/seldon/seldon-model-invalid-environ-metadata/api/v1.0/predictions

{"data":{"names":[],"ndarray":["data"]},"meta":{}}

Cleanup resources

%%bash
kubectl delete -f model-metadata/

seldondeployment.machinelearning.seldon.io "seldon-model-environ-metadata" deleted
seldondeployment.machinelearning.seldon.io "seldon-model-init-metadata" deleted
seldondeployment.machinelearning.seldon.io "seldon-model-invalid-environ-metadata" deleted