Annotation Based Configuration

You can configure aspects of Seldon Core via annotations in the SeldonDeployment resource and also the optional API OAuth Gateway. Please create an issue if you would like some configuration added.

SeldonDeployment Annotations

gRPC API Control

  • seldon.io/grpc-max-message-size : Maximum gRPC message size (bytes)
  • seldon.io/grpc-read-timeout : gRPC read timeout (msecs)

REST API Control

Service Orchestrator

  • seldon.io/engine-separate-pod : Use a separate pod for the service orchestrator
  • seldon.io/headless-svc : Run main endpoint as headless kubernetes service. This is required for gRPC load balancing via Ambassador.

Otherwise any annotations starting with seldon.io/engine- will be interpreted as specifying environment variables for the engine container. These include:

  • seldon.io/engine-java-opts : Java Opts for Service Orchestrator
    • Locations : SeldonDeployment.spec.annotations
    • Java Opts example
    • Translates to the environment variable JAVA_OPTS
  • seldon.io/engine-seldon-log-requests : Whether to log raw requests from engine
    • Locations : SeldonDeployment.spec.annotations
    • Translates to the environment variable SELDON_LOG_REQUESTS
  • seldon.io/engine-seldon-log-responses : Whether to log raw responses from engine
    • Locations : SeldonDeployment.spec.annotations
    • Translates to the environment variable SELDON_LOG_RESPONSES
  • seldon.io/engine-seldon-log-messages-externally : Option to turn on logging of requests via a logging service
    • Locations : SeldonDeployment.spec.annotations
    • Translates to the environment variable SELDON_LOG_MESSAGES_EXTERNALLY
  • seldon.io/engine-seldon-log-message-type : Option to override type set on messages when sending to logging service. Used to determine which logger impl
    • Locations : SeldonDeployment.spec.annotations
    • Translates to the environment variable SELDON_LOG_MESSAGE_TYPE
  • seldon.io/engine-seldon-message-logging-service : Option to override url to broker that sends to logging service
    • Locations : SeldonDeployment.spec.annotations
    • Translates to the environment variable SELDON_MESSAGE_LOGGING_SERVICE

More details on logging-related variables can be seen in the request-logging example.

Environment variables for the engine can also be set in the svcOrchSpec section of the SeldonDeployment, alongside engine resources. For examples see the helm charts or the distributed tracing example.

If both annotations and svcOrchSpec environment variables are used to set an environment variable for the engine container then svcOrchSpec environment variables take priority.

The above are the key engine env vars. For a full listing of engine env vars see the application.properties file of the engine source code.

Misc

  • seldon.io/svc-name : Custom service name for predictor. You will be responsible that it doesn’t clash with any existing service name in the namespace of the deployed SeldonDeployment.