Internal Microservice API¶
To add microservice components to a runtime prediction graph users need to create service that respects the internal API. The API provides a default service for each type of component within the system:
Router
Output_Transformer
See full proto definition.
Model¶
A service to return predictions.
REST API¶
Endpoint |
POST /predict |
Request |
JSON representation of SeldonMessage |
Response |
JSON representation of SeldonMessage |
Example request payload:
{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}
Example response payload
gRPC¶
service Model {
rpc Predict(SeldonMessage) returns (SeldonMessage) {};
rpc SendFeedback(Feedback) returns (SeldonMessage) {};
rpc Metadata(google.protobuf.Empty) returns (SeldonModelMetadata) {};
}
Route¶
A service to route requests to one of its children and receive feedback rewards for them.
REST API¶
Endpoint |
POST /route |
Request |
JSON representation of SeldonMessage |
Response |
JSON representation of SeldonMessage |
Example request payload:
{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}
Example response payload:
{"data":{"ndarray":[1]}}
gRPC¶
service Router {
rpc Route(SeldonMessage) returns (SeldonMessage) {};
}
Send Feedback¶
Endpoint |
POST /send-feedback |
Request |
JSON representation of Feedback |
Response |
JSON representation of SeldonMessage |
Example request payload:
{
"request": {
"data": {
"names": ["a", "b"],
"tensor": {
"shape": [1, 2],
"values": [0, 1]
}
}
},
"response": {
"data": {
"names": ["a", "b"],
"tensor": {
"shape": [1, 1],
"values": [0.9]
}
}
},
"reward": 1.0
}
gRPC¶
service Router {
rpc SendFeedback(Feedback) returns (SeldonMessage) {};
}
Combiner¶
A service to combine responses from its children into a single response.
REST API¶
Endpoint |
POST /combine |
Request |
JSON representation of SeldonMessageList |
Response |
JSON representation of SeldonMessage |
gRPC¶
service Combiner {
rpc Aggregate(SeldonMessageList) returns (SeldonMessage) {};
}
Transformer¶
A service to transform its input.
REST API¶
Endpoint |
POST /transform-input |
Request |
JSON representation of SeldonMessage |
Response |
JSON representation of SeldonMessage |
Example request payload:
{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}
gRPC¶
service Transformer {
rpc TransformInput(SeldonMessage) returns (SeldonMessage) {};
}
Output_Transformer¶
A service to transform the response from its child.
REST API¶
Endpoint |
POST /transform-output |
Request |
JSON representation of SeldonMessage |
Response |
JSON representation of SeldonMessage |
Example request payload:
{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}
gRPC¶
service OutputTransformer {
rpc TransformOutput(SeldonMessage) returns (SeldonMessage) {};
}