Internal Microservice API

graph

To add microservice components to a runtime prediction graph users need to create service that respects the internal API. The API provides a default service for each type of component within the system:

See full proto definition.

Model

A service to return predictions.

REST API

Endpoint

POST /predict

Request

JSON representation of SeldonMessage

Response

JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

Example response payload

gRPC

service Model {
  rpc Predict(SeldonMessage) returns (SeldonMessage) {};
  rpc SendFeedback(Feedback) returns (SeldonMessage) {};
  rpc Metadata(google.protobuf.Empty) returns (SeldonModelMetadata) {};
}

Route

A service to route requests to one of its children and receive feedback rewards for them.

REST API

Endpoint

POST /route

Request

JSON representation of SeldonMessage

Response

JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

Example response payload:

{"data":{"ndarray":[1]}}

gRPC

service Router {
  rpc Route(SeldonMessage) returns (SeldonMessage) {};
 }

Send Feedback

Endpoint

POST /send-feedback

Request

JSON representation of Feedback

Response

JSON representation of SeldonMessage

Example request payload:

{
    "request": {
        "data": {
            "names": ["a", "b"],
            "tensor": {
                "shape": [1, 2],
                "values": [0, 1]
            }
        }
    },
    "response": {
        "data": {
            "names": ["a", "b"],
            "tensor": {
                "shape": [1, 1],
                "values": [0.9]
            }
        }
    },
    "reward": 1.0
}

gRPC

service Router {
  rpc SendFeedback(Feedback) returns (SeldonMessage) {};
 }

Combiner

A service to combine responses from its children into a single response.

REST API

Endpoint

POST /combine

Request

JSON representation of SeldonMessageList

Response

JSON representation of SeldonMessage

gRPC

service Combiner {
  rpc Aggregate(SeldonMessageList) returns (SeldonMessage) {};
}

Transformer

A service to transform its input.

REST API

Endpoint

POST /transform-input

Request

JSON representation of SeldonMessage

Response

JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

gRPC

service Transformer {
  rpc TransformInput(SeldonMessage) returns (SeldonMessage) {};
}

Output_Transformer

A service to transform the response from its child.

REST API

Endpoint

POST /transform-output

Request

JSON representation of SeldonMessage

Response

JSON representation of SeldonMessage

Example request payload:

{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}

gRPC

service OutputTransformer {
  rpc TransformOutput(SeldonMessage) returns (SeldonMessage) {};
}