Knative Eventing Integration¶
Seldon has an integration with Knative eventing that allows for real time processing.
This allow Seldon Core users to connect SeldonDeployments through triggers that will receive any relevant Cloudevents.
Triggers¶
The way that Knative Eventing works is by creating triggers that send any relevant Cloudevents that match a specific setup into the relevant addressable location.
Seldon Core implements the Knative Eventing Duck Typing requirements which allows users to create triggers that reference specific SeldonDeployments.
An example of a trigger for a SeldonDeployment named “iris-deployment” can be created with the following format:
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: seldon-eventing-sklearn-trigger
spec:
broker: default
filter:
attributes:
type: seldon.iris-deployment.default.request
subscriber:
ref:
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
name: iris-deployment
This means that any Cloudevents of type “seldon.iris-deployment.default.request
“ will be sent to the SeldonDeployment with the name iris-deployment
.
The URL path is inferred through our implementation of the Duck Typing from Knative, which automatically extracts the URL from the Kubernetes resource status, specifically from the attribute status.addressable.url
.
In the case of every Seldon Deployment, the status.addressable.url
is always the serviceName, port and path for the first predictor. You can see the Addressable type in our CRD definition.
Overriding URI¶
In the case of multiple predictors, or in the case that you want to send the Cloudevent through your ingress, you can actually create a trigger that overrides the URI. An example of this would be the following:
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: seldon-eventing-sklearn-trigger
spec:
broker: default
filter:
attributes:
type: seldon.iris-deployment.default.request
subscriber:
uri: http://istio-ingressgateway/seldon/default/iris-deployment/api/v1.0/predicions
Which would then create a trigger that would forward the messages of that type into that URI.
Sending Test Requests¶
In production you would have multiple services creating cloudevents from various different sources. However for testing, it’s possible to send requests directly from your terminal to the Knative Eventing broker, by using the following curl
docker image locally:
kubectl run --quiet=true -it --rm curl --image=radial/busyboxplus:curl --restart=Never -- \
curl -v "default-broker.default.svc.cluster.local" \
-H "Ce-Id: 536808d3-88be-4077-9d7a-a3f162705f79" \
-H "Ce-specversion: 0.3" \
-H "Ce-Type: seldon.iris-deployment.default.request" \
-H "Ce-Source: seldon.examples.streaming.curl" \
-H "Content-Type: application/json" \
-d '{"data": { "ndarray": [[1,2,3,4]]}}'
This will be sending a cloud event with the type seldon.iris-deployment.default.request
into the default broker in the default namespace, which would actually match the example provided in the trigger above.
Seldon Cloudevent Response¶
When receiving a Cloudevent, SeldonDeployments will return a Cloudevent-enabled response, which is available for other services to consume.
What this means is that you can create further triggers that could perform other actions with the resulting data.
The triggers will have to match the cloudevent headers, which are standardised by the SeldonDeployment, and are of the following format:
Ce-Id: SeldonDeployment unique request ID
Ce-Specversion: Version of the specversion used (default: 0.3)
Ce-Type: seldon.<sdep_name>.<namespace>.response
Ce-Source: seldon.<sdep_name>.<namespace>
This means that you could create a trigger for a seldon deployment of name iris-deployment
in namespace default
, such as the following example which would create an event-display
pod that will print out the contents of that cloudevent:
---
# Trigger to send events to service above
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: event-display
spec:
broker: default
filter:
attributes:
type: seldon.iris-deployment.default.response
source: seldon.iris-deployment
subscriber:
ref:
apiVersion: v1
kind: Service
name: event-display
---
# event-display app deploment
apiVersion: apps/v1
kind: Deployment
metadata:
n
ame: event-display
spec:
replicas: 1
selector:
matchLabels: &labels
app: event-display
template:
metadata:
labels: *labels
spec:
containers:
- name: helloworld-python
image: gcr.io/knative-releases/github.com/knative/eventing-sources/cmd/event_display
---
# Service that exposes event-display app.
# This will be the subscriber for the Trigger
kind: Service
apiVersion: v1
metadata:
name: event-display
spec:
selector:
app: event-display
ports:
- protocol: TCP
port: 80
targetPort: 8080
This will show all the cloudevents that are sent with that are processed by the SeldonDeployment.
Hands on Example¶
We have a fully worked notebook where we showcase these capabilities end to end.
You can try it yourself through the Seldon Core Real Time Stream Processing with Knative Eventing page