This page was generated from examples/ambassador/custom/ambassador_custom.ipynb.

Custom URL prefix with Seldon and Ambassador

This notebook shows how you can deploy Seldon Deployments with custom Ambassador configuration.

Setup Seldon Core

Uset the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core. Instructions also online.

Launch main model

We will create a very simple Seldon Deployment with a dummy model image seldonio/mock_classifier:1.0. This deployment is named example. We will add custom Ambassador config which sets the Ambassador prefix to /mycompany/ml

We must ensure we set the correct service endpoint. Seldon Core creates an endpoint of the form:

<spec.name>-<metadata.name>.<namespace>:<port>

Where

• <spec-name> is the name you give to the Seldon Deployment spec: production-model below
• <metadata.name> is the metadata name in the Seldon Deployment: example below
• <namespace> is the namespace your Seldon Deployment is deployed to
• <port> is the port either 8000 for REST or 5000 for gRPC

This will allow you to set the service value in the Ambassador config you create. So for the example below we have:

service: production-model-example.seldon:8000

!pygmentize model_custom_ambassador.json

{
    "apiVersion": "machinelearning.seldon.io/v1alpha2",
    "kind": "SeldonDeployment",
    "metadata": {
        "labels": {
            "app": "seldon"
        },
        "name": "example"
    },
    "spec": {
        "name": "production-model",
        "annotations": {
            "seldon.io/ambassador-config":"apiVersion: ambassador/v0\nkind: Mapping\nname: seldon_example_rest_mapping\nprefix: /mycompany/ml/\nservice: production-model-example.seldon:8000\ntimeout_ms: 3000"
        },
        "predictors": [
            {
                "componentSpecs": [{
                    "spec": {
                        "containers": [
                            {
                                "image": "seldonio/mock_classifier:1.0",
                                "imagePullPolicy": "IfNotPresent",
                                "name": "classifier"
                            }
                        ],
                        "terminationGracePeriodSeconds": 1
                    }}
                                  ],
                "graph":
                {
                    "children": [],
                    "name": "classifier",
                    "type": "MODEL",
                    "endpoint": {
                        "type": "REST"
                    }},
                "name": "single",
                "replicas": 1
            }
        ]
    }
}

!kubectl create -f model_custom_ambassador.json

seldondeployment.machinelearning.seldon.io/example created

!kubectl rollout status deploy/production-model-single-7cd068f

Waiting for deployment "production-model-single-7cd068f" rollout to finish: 0 of 1 updated replicas are available...
deployment "production-model-single-7cd068f" successfully rolled out

Get predictions

from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="example",namespace="seldon")

REST Request

r = sc.predict(gateway="ambassador",transport="rest",ambassador_prefix="/mycompany/ml")
print(r)

Success:True message:
Request:
data {
  tensor {
    shape: 1
    shape: 1
    values: 0.1628281752658065
  }
}

Response:
meta {
  puid: "5lb8edtgn4rhbd03g94g318io7"
  requestPath {
    key: "classifier"
    value: "seldonio/mock_classifier:1.0"
  }
}
data {
  names: "proba"
  tensor {
    shape: 1
    shape: 1
    values: 0.059870232304306846
  }
}