Internal Microservice API

graph

To add microservice components to a runtime prediction graph users need to create service that respects the internal API. The API provides a default service for each type of component within the system:

See full proto definition.

Model

A service to return predictions.

REST API

Predict

   
Endpoint POST /predict
Request JSON representation of SeldonMessage
Response JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

gRPC

service Model {
  rpc Predict(SeldonMessage) returns (SeldonMessage) {};
 }

Router

A service to route requests to one of its children and receive feedback rewards for them.

REST API

Route

   
Endpoint POST /route
Request JSON representation of SeldonMessage
Response JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

Send Feedback

   
Endpoint POST /send-feedback
Request JSON representation of Feedback
Response JSON representation of SeldonMessage

Example request payload:

{
    "request": {
        "data": {
            "names": ["a", "b"],
            "tensor": {
                "shape": [1, 2],
                "values": [0, 1]
            }
        }
    },
    "response": {
        "data": {
            "names": ["a", "b"],
            "tensor": {
                "shape": [1, 1],
                "values": [0.9]
            }
        }
    },
    "reward": 1.0
}

gRPC

service Router {
  rpc Route(SeldonMessage) returns (SeldonMessage) {};
  rpc SendFeedback(Feedback) returns (SeldonMessage) {};
 }

Combiner

A service to combine responses from its children into a single response.

REST API

Combine

   
Endpoint POST /combine
Request JSON representation of SeldonMessageList
Response JSON representation of SeldonMessage

gRPC

service Combiner {
  rpc Aggregate(SeldonMessageList) returns (SeldonMessage) {};
}

Transformer

A service to transform its input.

REST API

Transform

   
Endpoint POST /transform-input
Request JSON representation of SeldonMessage
Response JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

gRPC

service Transformer {
  rpc TransformInput(SeldonMessage) returns (SeldonMessage) {};
}

Output_Transformer

A service to transform the response from its child.

REST API

Transform

   
Endpoint POST /transform-output
Request JSON representation of SeldonMessage
Response JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

gRPC

service OutputTransformer {
  rpc TransformOutput(SeldonMessage) returns (SeldonMessage) {};
}