KFserving Storage Initializer¶
Prior to Seldon Core 1.8 seldon core was using by default
kfserving/storage-initializer for its pre-packaged model servers. This can be still used by configuring a following helm value:
storageInitializer: image: gcr.io/kfserving/storage-initializer:v0.4.0
NOTE: Current default storage initializer is
seldonio/rclone-storage-initializer:1.10.0-devis described here.
kfserving/storage-initializer is used
modeluri supports the following four object storage providers:
Google Cloud Storage (using
Azure Blob storage (using
A Kubernetes PersistentVolume can be used instead of a bucket using
In order to handle credentials you must make available a secret with the environment variables that will be added into the
Init Containers. For this you need to perform the following actions:
Understand which environment variables you need to set
Create a secret containing the environment variables
Provide the Seldon Core Controller or Seldon Deployment with the name of the secret
1. Understand which Environment Variables you need to set¶
In order to understand what are the environment variables required, you can have a look directly into our Storage.py library that we use in our
AWS Required Variables¶
Minio Required Variables¶
Azure Required Variables¶
Google Cloud Required Variables¶
Currently for Google Cloud it is required to follow a slightly more complex method given that it requires the secret to be mounted as a file. For this please follow the example at the Google Cloud Section.
If application cretentials are not set, the client will use an Anonymous client.
2. Create a secret containing the environment variables¶
You can now create a secret, below we show what the env variables would look like for the AWS credentials.
apiVersion: v1 kind: Secret metadata: name: seldon-init-container-secret type: Opaque data: AWS_ACCESS_KEY_ID: XXXX AWS_SECRET_ACCESS_KEY: XXXX AWS_ENDPOINT_URL: XXXX USE_SSL: XXXX
It is also possible to create a
Secret object from the command line:
kubectl create secret generic seldon-init-container-secret \ --from-literal=AWS_ENDPOINT_URL='XXXX' \ --from-literal=AWS_ACCESS_KEY_ID='XXXX' \ --from-literal=AWS_SECRET_ACCESS_KEY='XXXX' \ --from-literal=USE_SSL=false
You can read the documentation of Kubernetes to learn more about Kubernetes Secrets.
3. Ensure your SeldonDeployment has access to the secret¶
In order for your SeldonDeployment to know what is the name of the secret, we have to specify the name of the secret we created - in the example above we named the secret
Option 1: Default Seldon Core Manager Controller value¶
You can set a global default when you install Seldon Core through the Helm chart through the
executor.defaultEnvSecretRefName. You can see all the variables available in the Advanced Helm Installation Page.
# ... other variables predictiveUnit: defaultEnvSecretRefName: seldon-init-container-secret # ... other variables
Option 2: Override through SeldonDeployment config¶
It is also possible to provide an override value when you deploy your model using the SeldonDeploymen YAML. You can do this through the
apiVersion: machinelearning.seldon.io/v1alpha2 kind: SeldonDeployment metadata: name: sklearn spec: name: iris predictors: - graph: children:  implementation: SKLEARN_SERVER modelUri: s3://seldon-models/sklearn/iris envSecretRefName: seldon-init-container-secret name: classifier name: default replicas: 1
MinIO running inside same Kubernetes cluster¶
Assuming that you have MinIO instance running on port
9000 avaible at
minio.minio-system.svc.cluster.local and you want to reference bucket
mymodel you would set
modelUri being set as
For full example please see this notebook.
Adding Credentials for Google Cloud¶
Currently the Google Credentials require a file to be set up so the process required involves creation of a service account as outlined below.
You can also create a
ServiceAccount and attach a differently formatted
Secret to it similar to how kfserving does it. See kfserving documentation on this topic. Supported annotation prefix includes
For GCP/GKE, go to gcloud console and create a key as json and export as a file. Then create a secret from the file using:
kubectl create secret generic user-gcp-sa --from-file=gcloud-application-credentials.json=<LOCALFILE>
The file in the secret needs to be called
gcloud-application-credentials.json (the name can be configured in the seldon configmap, visible in
kubectl get cm -n seldon-system seldon-config -o yaml).
Then create a service account to reference the secret:
apiVersion: v1 kind: ServiceAccount metadata: name: user-gcp-sa secrets: - name: user-gcp-sa
This can then be referenced in the SeldonDeployment manifest by setting
serviceAccountName: user-gcp-sa at the same level as
apiVersion: machinelearning.seldon.io/v1alpha2 kind: SeldonDeployment metadata: name: sklearn spec: name: iris predictors: - graph: children:  implementation: SKLEARN_SERVER modelUri: gs://seldon-models/sklearn/iris serviceAccountName: user-gcp-sa name: classifier name: default replicas: 1