This page was generated from notebooks/seldon_client.ipynb.

Advanced Usage Exampes for Seldon Client

Istio Gateway Request with token over HTTPS - no SSL verification

Test against a current kubeflow cluster with Dex token authentication.

1. Install kubeflow with Dex authentication

INGRESS_HOST = !kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
ISTIO_GATEWAY = INGRESS_HOST[0]

ISTIO_GATEWAY

Get a token from the Dex gateway. At present as Dex does not support curl password credentials you will need to get it from your browser logged into the cluster. Open up a browser console and run document.cookie

TOKEN = "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1NjM2MjA0ODYsImlhdCI6MTU2MzUzNDA4NiwiaXNzIjoiMzQuNjUuNzMuMjU1IiwianRpIjoiYjllNDQxOGQtZjNmNC00NTIyLTg5ODEtNDcxOTY0ODNmODg3IiwidWlmIjoiZXlKcGMzTWlPaUpvZEhSd2N6b3ZMek0wTGpZMUxqY3pMakkxTlRvMU5UVTJMMlJsZUNJc0luTjFZaUk2SWtOcFVYZFBSMFUwVG1wbk1GbHBNV3RaYW1jMFRGUlNhVTU2VFhSUFZFSm9UMU13ZWxreVVYaE9hbGw0V21wVk1FNXFXVk5DVjNoMldUSkdjeUlzSW1GMVpDSTZJbXQxWW1WbWJHOTNMV0YxZEdoelpYSjJhV05sTFc5cFpHTWlMQ0psZUhBaU9qRTFOak0yTWpBME9EWXNJbWxoZENJNk1UVTJNelV6TkRBNE5pd2lZWFJmYUdGemFDSTZJbE5OWlZWRGJUQmFOVkZoUTNCdVNHTndRMWgwTVZFaUxDSmxiV0ZwYkNJNkltRmtiV2x1UUhObGJHUnZiaTVwYnlJc0ltVnRZV2xzWDNabGNtbG1hV1ZrSWpwMGNuVmxMQ0p1WVcxbElqb2lZV1J0YVc0aWZRPT0ifQ.7CQIz4A1s9m6lJeWTqpz_JKGArGX4e_zpRCOXXjVRJgguB3z48rSfei_KL7niMCWpruhU11c8UIw9E79PwHNNw"

Start Seldon Core

Use the setup notebook to Install Seldon Core with Istio Ingress. Instructions also online.

Note When running helm install for this example you will need to set the istio.gateway flag to kubeflow-gateway (--set istio.gateway=kubeflow-gateway).

deployment_name = "test1"
namespace = "default"

from seldon_core.seldon_client import (
    SeldonCallCredentials,
    SeldonChannelCredentials,
    SeldonClient,
)

sc = SeldonClient(
    deployment_name=deployment_name,
    namespace=namespace,
    gateway_endpoint=ISTIO_GATEWAY,
    debug=True,
    channel_credentials=SeldonChannelCredentials(verify=False),
    call_credentials=SeldonCallCredentials(token=TOKEN),
)

r = sc.predict(gateway="istio", transport="rest", shape=(1, 4))
print(r)

Its not presently possible to use gRPC without getting access to the certificates. We will update this once its clear how to obtain them from a Kubeflow cluser setup.

Istio - SSL Endpoint - Client Side Verification - No Authentication

1. First run through the Istio Secure Gateway SDS example and make sure this works for you.

   • This will create certificates for httpbin.example.com and test them out.

2. Update your /etc/hosts file to include an entry for the ingress gateway for httpbin.example.com e.g. add a line like: 10.107.247.132  httpbin.example.com replacing the ip address with your ingress gateway ip address.

# Set to folder where the httpbin certificates are
ISTIO_HTTPBIN_CERT_FOLDER = "/home/clive/work/istio/httpbin.example.com"

Start Seldon Core

Use the setup notebook to Install Seldon Core with Istio Ingress. Instructions also online.

Note When running helm install for this example you will need to set the istio.gateway flag to mygateway (--set istio.gateway=mygateway) used in the example.

deployment_name = "mymodel"
namespace = "default"

from seldon_core.seldon_client import (
    SeldonCallCredentials,
    SeldonChannelCredentials,
    SeldonClient,
)

sc = SeldonClient(
    deployment_name=deployment_name,
    namespace=namespace,
    gateway_endpoint="httpbin.example.com",
    debug=True,
    channel_credentials=SeldonChannelCredentials(
        certificate_chain_file=ISTIO_HTTPBIN_CERT_FOLDER
        + "/2_intermediate/certs/ca-chain.cert.pem",
        root_certificates_file=ISTIO_HTTPBIN_CERT_FOLDER
        + "/4_client/certs/httpbin.example.com.cert.pem",
        private_key_file=ISTIO_HTTPBIN_CERT_FOLDER
        + "/4_client/private/httpbin.example.com.key.pem",
    ),
)

r = sc.predict(gateway="istio", transport="rest", shape=(1, 4))
print(r)

r = sc.predict(gateway="istio", transport="grpc", shape=(1, 4))
print(r)