Seldon Core Workflow

There are 4 steps to using seldon-core.

  1. Install seldon-core onto a Kubernetes cluster
  2. Wrap your components (usually runtime model servers) as Docker containers that respect the internal Seldon microservice API.
  3. Define your runtime service graph as a SeldonDeployment resource and deploy your model and serve predictions
  4. Deploy and request predictions

At the end of this page you will find a set of suggested tutorials you can follow to get started with Seldon.

steps

1. Install Seldon Core

To install seldon-core follow the installation guide.

2. Wrap Your Model

The components you want to run in production need to be wrapped as Docker containers that respect the Seldon microservice API. You can create models that serve predictions, routers that decide on where requests go, such as A-B Tests, Combiners that combine responses and transformers that provide generic components that can transform requests and/or responses.

To allow users to easily wrap machine learning components built using different languages and toolkits we provide wrappers that allow you easily to build a docker container from your code that can be run inside seldon-core. Our current recommended tool is RedHat’s Source-to-Image. More detail can be found in Wrapping your models docs.

3. Define Runtime Service Graph

To run your machine learning graph on Kubernetes you need to define how the components you created in the last step fit together to represent a service graph. This is defined inside a SeldonDeployment Kubernetes Custom resource. A guide to constructing this inference graph is provided.

graph

4. Deploy and Serve Predictions

You can use kubectl to deploy your ML service like any other Kubernetes resource. This is discussed here. Once deployed you can get predictions by calling the exposed API.

Next Steps

If you have a saved model for SKLearn, XGBoost or Tensorflow then you can use one of our prepackaged model servers.

If you want to wrap your custom inference code then follow one of our starter tutorials below for the framework you are using.

Prepacked Model Servers

Other sources