This page was generated from examples/models/metadata/metadata_grpc.ipynb.

Metadata GRPC API example

Prerequisites

  • A kubernetes cluster with kubectl configured

  • curl

  • grpcurl

  • pygmentize

Setup Seldon Core

Use the setup notebook to Setup Cluster to setup Seldon Core with an ingress.

[1]:
!kubectl create namespace seldon
Error from server (AlreadyExists): namespaces "seldon" already exists
[2]:
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
Context "kind-kind" modified.

Example description

Simple, two-node graph that shows how Metadata is handled in GRPC API mode.

Model used in this example is defined in graph_metadata notebook.

[3]:
%%writefile example-grpc.yaml

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: graph-metadata-grpc
spec:
  name: test-deployment
  transport: grpc
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: seldonio/metadata-generic-node:0.4
          name: node-one
          env:
          - name: MODEL_METADATA
            value: |
              ---
              name: node-one
              versions: [ generic-node/v0.4 ]
              platform: seldon
              inputs:
              - messagetype: tensor
                schema:
                  names: [one-input]
              outputs:
              - messagetype: tensor
                schema:
                  names: [one-output]
              custom:
                author: seldon-dev
        - image: seldonio/metadata-generic-node:0.4
          name: node-two
          env:

          - name: MODEL_METADATA
            value: |
              ---
              name: node-two
              versions: [ generic-node/v0.4 ]
              platform: seldon
              inputs:
              - messagetype: tensor
                schema:
                  names: [two-input]
              outputs:
              - messagetype: tensor
                schema:
                  names: [two-output]
              custom:
                author: seldon-dev
    graph:
      name: node-one
      type: MODEL
      children:
      - name: node-two
        type: MODEL
        children: []
    name: example
    replicas: 1
Overwriting example-grpc.yaml
[4]:
!kubectl apply -f  example-grpc.yaml
seldondeployment.machinelearning.seldon.io/graph-metadata-grpc created
[5]:
%%bash
kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=graph-metadata-grpc -o jsonpath='{.items[0].metadata.name}')
Waiting for deployment "graph-metadata-grpc-example-0-node-one-node-two" rollout to finish: 0 of 1 updated replicas are available...
deployment "graph-metadata-grpc-example-0-node-one-node-two" successfully rolled out
[6]:
%%bash
cd ../../../executor/proto && grpcurl \
    -d '{"name": "node-one"}' \
    -rpc-header seldon:graph-metadata-grpc -rpc-header namespace:seldon \
    -plaintext -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/ModelMetadata
{
  "name": "node-one",
  "versions": [
    "generic-node/v0.4"
  ],
  "platform": "seldon",
  "inputs": [
    {
      "messagetype": "tensor",
      "schema": {
            "names": [
                  "one-input"
                ]
          }
    }
  ],
  "outputs": [
    {
      "messagetype": "tensor",
      "schema": {
            "names": [
                  "one-output"
                ]
          }
    }
  ],
  "custom": {
    "author": "seldon-dev"
  }
}
[7]:
%%bash
cd ../../../executor/proto && grpcurl \
    -d '{"name": "node-two"}' \
    -rpc-header seldon:graph-metadata-grpc -rpc-header namespace:seldon \
    -plaintext -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/ModelMetadata
{
  "name": "node-one",
  "versions": [
    "generic-node/v0.4"
  ],
  "platform": "seldon",
  "inputs": [
    {
      "messagetype": "tensor",
      "schema": {
            "names": [
                  "one-input"
                ]
          }
    }
  ],
  "outputs": [
    {
      "messagetype": "tensor",
      "schema": {
            "names": [
                  "one-output"
                ]
          }
    }
  ],
  "custom": {
    "author": "seldon-dev"
  }
}
[8]:
%%bash
cd ../../../executor/proto && grpcurl \
    -rpc-header seldon:graph-metadata-grpc -rpc-header namespace:seldon \
    -plaintext -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/GraphMetadata
{
  "name": "example",
  "models": {
    "node-one": {
      "name": "node-one",
      "versions": [
        "generic-node/v0.4"
      ],
      "platform": "seldon",
      "inputs": [
        {
          "messagetype": "tensor",
          "schema": {
                "names": [
                      "one-input"
                    ]
              }
        }
      ],
      "outputs": [
        {
          "messagetype": "tensor",
          "schema": {
                "names": [
                      "one-output"
                    ]
              }
        }
      ],
      "custom": {
        "author": "seldon-dev"
      }
    },
    "node-two": {
      "name": "node-two",
      "versions": [
        "generic-node/v0.4"
      ],
      "platform": "seldon",
      "inputs": [
        {
          "messagetype": "tensor",
          "schema": {
                "names": [
                      "two-input"
                    ]
              }
        }
      ],
      "outputs": [
        {
          "messagetype": "tensor",
          "schema": {
                "names": [
                      "two-output"
                    ]
              }
        }
      ],
      "custom": {
        "author": "seldon-dev"
      }
    }
  },
  "inputs": [
    {
      "messagetype": "tensor",
      "schema": {
            "names": [
                  "one-input"
                ]
          }
    }
  ],
  "outputs": [
    {
      "messagetype": "tensor",
      "schema": {
            "names": [
                  "two-output"
                ]
          }
    }
  ]
}
[9]:
!kubectl delete -f example-grpc.yaml
seldondeployment.machinelearning.seldon.io "graph-metadata-grpc" deleted
[ ]: