seldon_core package

Submodules

seldon_core.api_tester module

seldon_core.api_tester.get_seldon_client(args)[source]

Get the appropriate Seldon Client based on args

Parameters:args – Command line args
Return type:SeldonClient
Returns:A Seldon Client
seldon_core.api_tester.main()[source]
seldon_core.api_tester.run_predict(args)[source]

Make a prediction call to the Seldon API

Parameters:args – Command line args
seldon_core.api_tester.run_send_feedback(args)[source]

Do a semd-feedback call to the Seldon API

Parameters:args – Command line args

seldon_core.flask_utils module

exception seldon_core.flask_utils.SeldonMicroserviceException(message, status_code=None, payload=None, reason='MICROSERVICE_BAD_DATA')[source]

Bases: Exception

status_code = 400
to_dict()[source]
seldon_core.flask_utils.get_multi_form_data_request()[source]

Parses a request submitted with Content-type:multipart/form-data all the keys under SeldonMessage are accepted as form input binData can only be passed as file input strData can be passed as file or text input the file input is base64 encoded

Return type:Dict[~KT, ~VT]
Returns:JSON Dict
seldon_core.flask_utils.get_request()[source]

Parse a request to get JSON dict

Return type:Dict[~KT, ~VT]
Returns:JSON Dict

seldon_core.metrics module

seldon_core.metrics.create_counter(key, value)[source]

Utility method to create a counter metric :type key: str :param key: Counter name :type value: float :param value: Counter value

Returns:Valid counter metric dict
seldon_core.metrics.create_gauge(key, value)[source]

Utility method to create a guage metric :type key: str :param key: Guage name :type value: float :param value: Guage value

Return type:Dict[~KT, ~VT]
Returns:Valid Guage metric dict
seldon_core.metrics.create_timer(key, value)[source]

Utility mehtod to create a timer metric :type key: str :param key: Name of metric :type value: float :param value: Value for metric

Return type:Dict[~KT, ~VT]
Returns:Valid timer metric dict
seldon_core.metrics.validate_metrics(metrics)[source]

Validate a list of metrics :type metrics: List[Dict[~KT, ~VT]] :param metrics: List of metrics

Return type:bool

seldon_core.microservice module

class seldon_core.microservice.StandaloneApplication(app, user_object, options=None)[source]

Bases: gunicorn.app.base.BaseApplication

load()[source]
load_config()[source]

This method is used to load the configuration from one or several input(s). Custom Command line, configuration file. You have to override this method in your class.

seldon_core.microservice.load_annotations()[source]

Attempt to load annotations

Return type:Dict[~KT, ~VT]
seldon_core.microservice.main()[source]
seldon_core.microservice.parse_parameters(parameters)[source]

Parse the user object parameters

Parameters:parameters (Dict[~KT, ~VT]) –
Return type:Dict[~KT, ~VT]
seldon_core.microservice.setup_tracing(interface_name)[source]
Return type:object
seldon_core.microservice.start_servers(target1, target2)[source]

Start servers

Parameters:
  • target1 (Callable) – Main flask process
  • target2 (Callable) – Auxilary flask process
Return type:

None

seldon_core.microservice_tester module

exception seldon_core.microservice_tester.SeldonTesterException(message)[source]

Bases: Exception

seldon_core.microservice_tester.gen_categorical(values, n)[source]

Generate a random categorical feature

Parameters:
  • values (List[str]) – The list of categorical values
  • n (List[int]) – The number of features to create
Return type:

ndarray

Returns:

Numpy array of values

seldon_core.microservice_tester.gen_continuous(f_range, n)[source]

Create a continuous feature based on given range

Parameters:
Return type:

ndarray

seldon_core.microservice_tester.generate_batch(contract, n, field)[source]
Return type:ndarray
seldon_core.microservice_tester.get_class_names(contract)[source]
Return type:List[str]
seldon_core.microservice_tester.main()[source]
seldon_core.microservice_tester.reconciliate_cont_type(feature, dtype)[source]

Ensure numpy arrays are always of type float

Parameters:
  • feature (ndarray) – Feature values
  • dtype (str) – The required type in the contract
Return type:

ndarray

Returns:

Numpy array

seldon_core.microservice_tester.run_predict(args)[source]

Make a predict call to microservice

Parameters:args – Command line args
seldon_core.microservice_tester.run_send_feedback(args)[source]

Make a feedback call to microservice

Parameters:args – Command line args
seldon_core.microservice_tester.unfold_contract(contract)[source]

Expand contract to full version

Parameters:contract (Dict[~KT, ~VT]) – The Seldon Contract
Return type:Dict[~KT, ~VT]
Returns:Full contract

seldon_core.persistence module

class seldon_core.persistence.PersistenceThread(user_object, push_frequency)[source]

Bases: threading.Thread

run()[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

stop()[source]
seldon_core.persistence.persist(user_object, push_frequency=None)[source]

Start a thread to persist a user class to Redis

Parameters:
seldon_core.persistence.restore(user_class, parameters)[source]

Restore saved state from Redis

Parameters:
Return type:

SeldonComponent

Returns:

A restored class or a new one

seldon_core.seldon_client module

class seldon_core.seldon_client.SeldonCallCredentials(token=None)[source]

Bases: object

Credentials for each call

class seldon_core.seldon_client.SeldonChannelCredentials(verify=True, root_certificates_file=None, private_key_file=None, certificate_chain_file=None)[source]

Bases: object

Channel credentials Presently just denotes an SSL connection.

class seldon_core.seldon_client.SeldonClient(gateway='ambassador', transport='rest', namespace=None, deployment_name=None, payload_type='tensor', oauth_key=None, oauth_secret=None, seldon_rest_endpoint='localhost:8002', seldon_grpc_endpoint='localhost:8004', gateway_endpoint='localhost:8003', microservice_endpoint='localhost:5000', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, channel_credentials=None, call_credentials=None, debug=False)[source]

Bases: object

A reference Seldon API Client

__init__(gateway='ambassador', transport='rest', namespace=None, deployment_name=None, payload_type='tensor', oauth_key=None, oauth_secret=None, seldon_rest_endpoint='localhost:8002', seldon_grpc_endpoint='localhost:8004', gateway_endpoint='localhost:8003', microservice_endpoint='localhost:5000', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, channel_credentials=None, call_credentials=None, debug=False)[source]
Parameters:
  • gateway (str) – API Gateway - either ambassador, istio or seldon
  • transport (str) – API transport - grpc or rest
  • namespace (Optional[str]) – k8s namespace of running deployment
  • deployment_name (Optional[str]) – name of seldon deployment
  • payload_type (str) – type of payload - tensor, ndarray or tftensor
  • oauth_key (Optional[str]) – OAUTH key (if using seldon api server)
  • oauth_secret (Optional[str]) – OAUTH secret (if using seldon api server)
  • seldon_rest_endpoint (str) – REST endpoint to seldon api server
  • seldon_grpc_endpoint (str) – gRPC endpoint to seldon api server
  • gateway_endpoint (str) – Gateway endpoint
  • microservice_endpoint (str) – Running microservice endpoint
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
explain(gateway=None, transport=None, deployment_name=None, payload_type=None, seldon_rest_endpoint=None, seldon_grpc_endpoint=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, data=None, bin_data=None, str_data=None, names=None, gateway_prefix=None, headers=None, http_path=None)[source]
Parameters:
  • gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon
  • transport (Optional[str]) – API transport - grpc or rest
  • namespace (Optional[str]) – k8s namespace of running deployment
  • deployment_name (Optional[str]) – name of seldon deployment
  • payload_type (Optional[str]) – type of payload - tensor, ndarray or tftensor
  • seldon_rest_endpoint (Optional[str]) – REST endpoint to seldon api server
  • seldon_grpc_endpoint (Optional[str]) – gRPC endpoint to seldon api server
  • gateway_endpoint (Optional[str]) – Gateway endpoint
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • grpc_max_send_message_length – Max grpc send message size in bytes
  • grpc_max_receive_message_length – Max grpc receive message size in bytes
  • data (Optional[ndarray]) – Numpy Array Payload to send
  • bin_data (Union[bytes, bytearray, None]) – Binary payload to send - will override data
  • str_data (Optional[str]) – String payload to send - will override data
  • names (Optional[Iterable[str]]) – Column names
  • gateway_prefix (Optional[str]) – prefix path for gateway URL endpoint
  • headers (Optional[Dict[~KT, ~VT]]) – Headers to add to request
  • http_path (Optional[str]) – Custom http path for predict call to use
Return type:

Dict[~KT, ~VT]

feedback(prediction_request=None, prediction_response=None, reward=0, gateway=None, transport=None, deployment_name=None, payload_type=None, oauth_key=None, oauth_secret=None, seldon_rest_endpoint=None, seldon_grpc_endpoint=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, gateway_prefix=None)[source]
Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon
  • transport (Optional[str]) – API transport - grpc or rest
  • deployment_name (Optional[str]) – name of seldon deployment
  • payload_type (Optional[str]) – payload - tensor, ndarray or tftensor
  • oauth_key (Optional[str]) – OAUTH key (if using seldon api server)
  • oauth_secret (Optional[str]) – OAUTH secret (if using seldon api server)
  • seldon_rest_endpoint (Optional[str]) – REST endpoint to seldon api server
  • seldon_grpc_endpoint (Optional[str]) – gRPC endpoint to seldon api server
  • gateway_endpoint (Optional[str]) – Gateway endpoint
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • grpc_max_send_message_length – Max grpc send message size in bytes
  • grpc_max_receive_message_length – Max grpc receive message size in bytes
  • method (Optional[str]) – The microservice method to call
  • shape (Tuple) – The shape of the data to send
  • namespace (Optional[str]) – k8s namespace of running deployment
Return type:

SeldonClientFeedback

microservice(gateway=None, transport=None, deployment_name=None, payload_type=None, oauth_key=None, oauth_secret=None, seldon_rest_endpoint=None, seldon_grpc_endpoint=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, data=None, datas=None, ndatas=None, bin_data=None, str_data=None, names=None)[source]
Parameters:
  • gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon
  • transport (Optional[str]) – API transport - grpc or rest
  • deployment_name (Optional[str]) – name of seldon deployment
  • payload_type (Optional[str]) – payload - tensor, ndarray or tftensor
  • oauth_key (Optional[str]) – OAUTH key (if using seldon api server)
  • oauth_secret (Optional[str]) – OAUTH secret (if using seldon api server)
  • seldon_rest_endpoint (Optional[str]) – REST endpoint to seldon api server
  • seldon_grpc_endpoint (Optional[str]) – gRPC endpoint to seldon api server
  • gateway_endpoint (Optional[str]) – Gateway endpoint
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • grpc_max_send_message_length – Max grpc send message size in bytes
  • grpc_max_receive_message_length – Max grpc receive message size in bytes
  • method (Optional[str]) – The microservice method to call
  • shape (Tuple) – The shape of the data to send
  • namespace (Optional[str]) – k8s namespace of running deployment
  • data (Optional[ndarray]) – Numpy Array Payload to send
  • bin_data (Union[bytes, bytearray, None]) – Binary payload to send - will override data
  • str_data (Optional[str]) – String payload to send - will override data
  • ndatas (Optional[int]) – Multiple numpy arrays to send for aggregation
  • bin_data – Binary data payload
  • str_data – String data payload
  • names (Optional[Iterable[str]]) – Column names
Return type:

Union[SeldonClientPrediction, SeldonClientCombine]

Returns:

A prediction result

microservice_feedback(prediction_request=None, prediction_response=None, reward=0, gateway=None, transport=None, deployment_name=None, payload_type=None, oauth_key=None, oauth_secret=None, seldon_rest_endpoint=None, seldon_grpc_endpoint=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None)[source]
Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • gateway (Optional[str]) – API Gateway - either Gateway or seldon
  • transport (Optional[str]) – API transport - grpc or rest
  • deployment_name (Optional[str]) – name of seldon deployment
  • payload_type (Optional[str]) – payload - tensor, ndarray or tftensor
  • oauth_key (Optional[str]) – OAUTH key (if using seldon api server)
  • oauth_secret (Optional[str]) – OAUTH secret (if using seldon api server)
  • seldon_rest_endpoint (Optional[str]) – REST endpoint to seldon api server
  • seldon_grpc_endpoint (Optional[str]) – gRPC endpoint to seldon api server
  • gateway_endpoint (Optional[str]) – Gateway endpoint
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • grpc_max_send_message_length – Max grpc send message size in bytes
  • grpc_max_receive_message_length – Max grpc receive message size in bytes
  • method (Optional[str]) – The microservice method to call
  • shape (Tuple) – The shape of the data to send
  • namespace (Optional[str]) – k8s namespace of running deployment
Return type:

SeldonClientFeedback

Returns:

A client response

predict(gateway=None, transport=None, deployment_name=None, payload_type=None, oauth_key=None, oauth_secret=None, seldon_rest_endpoint=None, seldon_grpc_endpoint=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, data=None, bin_data=None, str_data=None, names=None, gateway_prefix=None, headers=None, http_path=None)[source]
Parameters:
  • gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon
  • transport (Optional[str]) – API transport - grpc or rest
  • namespace (Optional[str]) – k8s namespace of running deployment
  • deployment_name (Optional[str]) – name of seldon deployment
  • payload_type (Optional[str]) – type of payload - tensor, ndarray or tftensor
  • oauth_key (Optional[str]) – OAUTH key (if using seldon api server)
  • oauth_secret (Optional[str]) – OAUTH secret (if using seldon api server)
  • seldon_rest_endpoint (Optional[str]) – REST endpoint to seldon api server
  • seldon_grpc_endpoint (Optional[str]) – gRPC endpoint to seldon api server
  • gateway_endpoint (Optional[str]) – Gateway endpoint
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • grpc_max_send_message_length – Max grpc send message size in bytes
  • grpc_max_receive_message_length – Max grpc receive message size in bytes
  • data (Optional[ndarray]) – Numpy Array Payload to send
  • bin_data (Union[bytes, bytearray, None]) – Binary payload to send - will override data
  • str_data (Optional[str]) – String payload to send - will override data
  • names (Optional[Iterable[str]]) – Column names
  • gateway_prefix (Optional[str]) – prefix path for gateway URL endpoint
  • headers (Optional[Dict[~KT, ~VT]]) – Headers to add to request
  • http_path (Optional[str]) – Custom http path for predict call to use
Return type:

SeldonClientPrediction

class seldon_core.seldon_client.SeldonClientCombine(request, response, success=True, msg='')[source]

Bases: object

Data class to return from Seldon Client for aggregate calls

exception seldon_core.seldon_client.SeldonClientException(message)[source]

Bases: Exception

Seldon Client Exception

status_code = 400
class seldon_core.seldon_client.SeldonClientFeedback(request, response, success=True, msg='')[source]

Bases: object

Data class to return from Seldon Client for feedback calls

class seldon_core.seldon_client.SeldonClientPrediction(request, response, success=True, msg='')[source]

Bases: object

Data class to return from Seldon Client

seldon_core.seldon_client.explain_predict_gateway(deployment_name, namespace=None, gateway_endpoint='localhost:8003', shape=(1, 1), data=None, headers=None, gateway_prefix=None, payload_type='tensor', bin_data=None, str_data=None, names=None, call_credentials=None, channel_credentials=None, http_path=None, **kwargs)[source]

REST explain request to Gateway Ingress

Parameters:
Return type:

Dict[~KT, ~VT]

Returns:

A JSON Dict

seldon_core.seldon_client.get_token(oauth_key='', oauth_secret='', namespace=None, endpoint='localhost:8002')[source]

Get an OAUTH key from the Seldon Gateway

Parameters:
  • oauth_key (str) – OAUTH key
  • oauth_secret (str) – OAUTH secret
  • namespace (Optional[str]) – k8s namespace of running deployment
  • endpoint (str) – The host:port of the endpoint for the OAUTH API server
Return type:

str

Returns:

The OAUTH token

seldon_core.seldon_client.grpc_feedback_gateway(prediction_request=None, prediction_response=None, reward=0, deployment_name='', namespace=None, gateway_endpoint='localhost:8003', headers=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, **kwargs)[source]
Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • deployment_name (str) – The name of the running Seldon deployment
  • namespace (Optional[str]) – k8s namespace of running deployment
  • gateway_endpoint (str) – The gateway host:port endpoint
  • headers (Optional[Dict[~KT, ~VT]]) – Headers to add to the request
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
  • kwargs
Return type:

SeldonClientFeedback

seldon_core.seldon_client.grpc_feedback_seldon_oauth(prediction_request=None, prediction_response=None, reward=0, oauth_key='', oauth_secret='', namespace=None, seldon_rest_endpoint='localhost:8002', seldon_grpc_endpoint='localhost:8004', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, **kwargs)[source]

Send feedback to Seldon API gateway via gRPC

Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • oauth_key (str) – OAUTH key
  • oauth_secret (str) – OAUTH secret
  • namespace (Optional[str]) – k8s namespace of running deployment
  • seldon_rest_endpoint (str) – Endpoint of REST endpoint
  • seldon_grpc_endpoint (str) – Endpoint for Seldon grpc
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
  • kwargs
Return type:

SeldonClientFeedback

seldon_core.seldon_client.grpc_predict_gateway(deployment_name, namespace=None, gateway_endpoint='localhost:8003', shape=(1, 1), data=None, headers=None, payload_type='tensor', bin_data=None, str_data=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, call_credentials=None, channel_credentials=None, **kwargs)[source]

gRPC request to Gateway Ingress

Parameters:
  • deployment_name (str) – Deployment name of Seldon Deployment
  • namespace (Optional[str]) – The namespace the Seldon Deployment is running in
  • gateway_endpoint (str) – The endpoint for gateway
  • shape (Tuple[int, int]) – The shape of the data
  • data (Optional[ndarray]) – The numpy array data to send
  • headers (Optional[Dict[~KT, ~VT]]) – A Dict of key value pairs to add to gRPC HTTP Headers
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • bin_data (Union[bytes, bytearray, None]) – Binary data to send
  • str_data (Optional[str]) – String data to send
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
  • names (Optional[Iterable[str]]) – Column names
Return type:

SeldonClientPrediction

Returns:

A SeldonMessage proto response

seldon_core.seldon_client.grpc_predict_seldon_oauth(oauth_key, oauth_secret, namespace=None, seldon_rest_endpoint='localhost:8002', seldon_grpc_endpoint='localhost:8004', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, **kwargs)[source]

Call Seldon gRPC API Gateway endpoint

Parameters:
  • oauth_key (str) – OAUTH key
  • oauth_secret (str) – OAUTH secret
  • namespace (Optional[str]) – k8s namespace of running deployment
  • seldon_rest_endpoint (str) – Endpoint of REST endpoint
  • shape (Tuple[int, int]) – Shape of endpoint
  • data (Optional[ndarray]) – Data to send
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • bin_data (Union[bytes, bytearray, None]) – Binary data to send
  • str_data (Optional[str]) – String data to send
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
  • names (Optional[Iterable[str]]) – Column names
  • kwargs
Return type:

SeldonClientPrediction

Returns:

A SeldonMessage proto

seldon_core.seldon_client.microservice_api_grpc_aggregate(microservice_endpoint='localhost:5000', shape=(1, 1), datas=None, ndatas=None, payload_type='tensor', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, **kwargs)[source]

Call Seldon microservice gRPC API aggregate

Parameters:
  • microservice_endpoint (str) – Microservice API endpoint
  • shape (Tuple) – Shape of the data to send
  • datas (Optional[List[ndarray]]) – List of Numpy array data to send
  • ndatas (Optional[int]) – Multiple numpy arrays to send for aggregation
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
  • names (Optional[Iterable[str]]) – Column names
  • kwargs
Return type:

SeldonClientCombine

Returns:

SeldonClientCombine

seldon_core.seldon_client.microservice_api_grpc_feedback(prediction_request=None, prediction_response=None, reward=0, microservice_endpoint=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, **kwargs)[source]

Call Seldon gRPC

Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • kwargs
Return type:

SeldonClientFeedback

seldon_core.seldon_client.microservice_api_grpc_seldon_message(method='predict', microservice_endpoint='localhost:5000', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, **kwargs)[source]

Call Seldon microservice gRPC API

Parameters:
  • method (str) – Method to call
  • microservice_endpoint (str) – Running microservice endpoint
  • shape (Tuple) – The shape of the data to send
  • data (object) – Numpy array data to send
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • bin_data (Union[bytes, bytearray, None]) – Binary data to send
  • str_data (Optional[str]) – String data to send
  • grpc_max_send_message_length (int) – Max grpc send message size in bytes
  • grpc_max_receive_message_length (int) – Max grpc receive message size in bytes
  • names (Optional[Iterable[str]]) – column names
  • kwargs
Return type:

SeldonClientPrediction

Returns:

SeldonClientPrediction

seldon_core.seldon_client.microservice_api_rest_aggregate(microservice_endpoint='localhost:5000', shape=(1, 1), datas=None, ndatas=None, payload_type='tensor', names=None, **kwargs)[source]

Call Seldon microservice REST API aggregate endpoint

Parameters:
  • microservice_endpoint (str) – Running microservice endpoint
  • shape (Tuple) – The shape of the data to send
  • datas (Optional[List[ndarray]]) – List of Numpy array data to send
  • ndatas (Optional[int]) – Multiple numpy arrays to send for aggregation
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • names (Optional[Iterable[str]]) – Column names
  • kwargs
Return type:

SeldonClientCombine

Returns:

A SeldonClientPrediction

seldon_core.seldon_client.microservice_api_rest_feedback(prediction_request=None, prediction_response=None, reward=0, microservice_endpoint=None, **kwargs)[source]

Call Seldon microserice REST API to send feedback

Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • microservice_endpoint (Optional[str]) – Running microservice endpoint
  • kwargs
Return type:

SeldonClientFeedback

Returns:

A SeldonClientFeedback

seldon_core.seldon_client.microservice_api_rest_seldon_message(method='predict', microservice_endpoint='localhost:5000', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, names=None, **kwargs)[source]

Call Seldon microservice REST API

Parameters:
  • method (str) – The microservice method to call
  • microservice_endpoint (str) – Running microservice endpoint
  • grpc_max_send_message_length – Max grpc send message size in bytes
  • grpc_max_receive_message_length – Max grpc receive message size in bytes
  • method – The microservice method to call
  • shape (Tuple) – The shape of the data to send
  • namespace – k8s namespace of running deployment
  • shape – Shape of the data to send
  • data (object) – Numpy array data to send
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • bin_data (Union[bytes, bytearray, None]) – Binary data payload
  • str_data (Optional[str]) – String data payload
  • names (Optional[Iterable[str]]) – Column names
  • kwargs
Return type:

SeldonClientPrediction

Returns:

A SeldonClientPrediction data response

seldon_core.seldon_client.rest_feedback_gateway(prediction_request=None, prediction_response=None, reward=0, deployment_name='', namespace=None, gateway_endpoint='localhost:8003', headers=None, gateway_prefix=None, **kwargs)[source]

Send Feedback to Seldon via gateway using REST

Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • deployment_name (str) – The name of the running Seldon deployment
  • namespace (Optional[str]) – k8s namespace of running deployment
  • gateway_endpoint (str) – The gateway host:port endpoint
  • headers (Optional[Dict[~KT, ~VT]]) – Headers to add to the request
  • gateway_prefix (Optional[str]) – The prefix to add to the request path for gateway
  • kwargs
Return type:

SeldonClientFeedback

Returns:

A Seldon Feedback Response

seldon_core.seldon_client.rest_feedback_seldon_oauth(prediction_request=None, prediction_response=None, reward=0, oauth_key='', oauth_secret='', namespace=None, seldon_rest_endpoint='localhost:8002', **kwargs)[source]

Send Feedback to Seldon API Gateway using REST

Parameters:
  • prediction_request (Optional[SeldonMessage]) – Previous prediction request
  • prediction_response (Optional[SeldonMessage]) – Previous prediction response
  • reward (float) – A reward to send in feedback
  • oauth_key (str) – OAUTH key
  • oauth_secret (str) – OAUTH secret
  • namespace (Optional[str]) – k8s namespace of running deployment
  • seldon_rest_endpoint (str) – Endpoint of REST endpoint
  • kwargs
Return type:

SeldonClientFeedback

seldon_core.seldon_client.rest_predict_gateway(deployment_name, namespace=None, gateway_endpoint='localhost:8003', shape=(1, 1), data=None, headers=None, gateway_prefix=None, payload_type='tensor', bin_data=None, str_data=None, names=None, call_credentials=None, channel_credentials=None, http_path=None, **kwargs)[source]

REST request to Gateway Ingress

Parameters:
Return type:

SeldonClientPrediction

Returns:

A requests Response object

seldon_core.seldon_client.rest_predict_seldon_oauth(oauth_key, oauth_secret, namespace=None, seldon_rest_endpoint='localhost:8002', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, names=None, **kwargs)[source]

Call Seldon API Gateway using REST

Parameters:
  • oauth_key (str) – OAUTH key
  • oauth_secret (str) – OAUTH secret
  • namespace (Optional[str]) – k8s namespace of running deployment
  • seldon_rest_endpoint (str) – Endpoint of REST endpoint
  • shape (Tuple) – Shape of endpoint
  • data (object) – Data to send
  • payload_type (str) – payload - tensor, ndarray or tftensor
  • bin_data (Union[bytes, bytearray, None]) – Binary data to send
  • str_data (Optional[str]) – String data to send
  • names (Optional[Iterable[str]]) – column names
  • kwargs
Return type:

SeldonClientPrediction

Returns:

Seldon Client Prediction

seldon_core.seldon_methods module

seldon_core.seldon_methods.aggregate(user_model, request)[source]

Aggregate a list of payloads

Parameters:
  • user_model (Any) – A Seldon user model
  • request (SeldonMessageList) – SeldonMessage proto
Return type:

SeldonMessage

Returns:

Aggregated SeldonMessage proto

seldon_core.seldon_methods.predict(user_model, request)[source]

Call the user model to get a prediction and package the response

Parameters:
  • user_model (Any) – User defined class instance
  • request (Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]) – The incoming request
Return type:

Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]

Returns:

The prediction

seldon_core.seldon_methods.route(user_model, request)[source]
Parameters:
  • user_model (Any) – A Seldon user model
  • request (Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]) – A SelodonMessage proto
Return type:

Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]

seldon_core.seldon_methods.send_feedback(user_model, request, predictive_unit_id)[source]
Parameters:
  • user_model (Any) – A Seldon user model
  • request (Feedback) – SeldonMesage proto
  • predictive_unit_id (str) – The ID of the enclosing container predictive unit. Will be taken from environment.
Return type:

SeldonMessage

seldon_core.seldon_methods.transform_input(user_model, request)[source]
Parameters:
  • user_model (Any) – User defined class to handle transform input
  • request (Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]) – The incoming request
Return type:

Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]

Returns:

The transformed request

seldon_core.seldon_methods.transform_output(user_model, request)[source]
Parameters:
  • user_model (Any) – User defined class to handle transform input
  • request (Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]) – The incoming request
Return type:

Union[SeldonMessage, List[~T], Dict[~KT, ~VT]]

Returns:

The transformed request

seldon_core.serving_test_gen module

seldon_core.storage module

class seldon_core.storage.Storage[source]

Bases: object

static download(uri, out_dir=None)[source]
Return type:str

seldon_core.user_model module

class seldon_core.user_model.SeldonComponent(**kwargs)[source]

Bases: object

aggregate(features_list, feature_names_list)[source]
Return type:Union[ndarray, List[~T], str, bytes]
aggregate_raw(msgs)[source]
Return type:SeldonMessage
class_names()[source]
Return type:Iterable[str]
feature_names()[source]
Return type:Iterable[str]
load()[source]
metrics()[source]
Return type:List[Dict[~KT, ~VT]]
predict(X, names, meta=None)[source]
Return type:Union[ndarray, List[~T], str, bytes]
predict_raw(msg)[source]
Return type:SeldonMessage
route(features, feature_names)[source]
Return type:int
route_raw(msg)[source]
Return type:SeldonMessage
send_feedback(features, feature_names, reward, truth, routing)[source]
Return type:Union[ndarray, List[~T], str, bytes, None]
send_feedback_raw(feedback)[source]
Return type:SeldonMessage
tags()[source]
Return type:Dict[~KT, ~VT]
transform_input(X, names, meta=None)[source]
Return type:Union[ndarray, List[~T], str, bytes]
transform_input_raw(msg)[source]
Return type:SeldonMessage
transform_output(X, names, meta=None)[source]
Return type:Union[ndarray, List[~T], str, bytes]
transform_output_raw(msg)[source]
Return type:SeldonMessage
exception seldon_core.user_model.SeldonNotImplementedError(message)[source]

Bases: seldon_core.flask_utils.SeldonMicroserviceException

status_code = 400
seldon_core.user_model.client_aggregate(user_model, features_list, feature_names_list)[source]

Aggregate payloads

Parameters:
  • user_model (SeldonComponent) – A Seldon user model
  • features_list (List[Union[ndarray, str, bytes]]) – A list of payloads
  • feature_names_list (List[~T]) – Column names for payloads
Return type:

Union[ndarray, List[~T], str, bytes]

Returns:

An aggregated payload

seldon_core.user_model.client_class_names(user_model, predictions)[source]

Get class names from user model

Parameters:
  • user_model (SeldonComponent) – User defined class instance
  • predictions (ndarray) – Prediction results
Return type:

Iterable[str]

Returns:

Class names

seldon_core.user_model.client_custom_metrics(user_model)[source]

Get custom metrics

Parameters:user_model (SeldonComponent) – A Seldon user model
Return type:List[Dict[~KT, ~VT]]
Returns:A list of custom metrics
seldon_core.user_model.client_custom_tags(user_model)[source]

Get tags from user model

Parameters:user_model (SeldonComponent) –
Return type:Dict[~KT, ~VT]
Returns:Dictionary of key value pairs
seldon_core.user_model.client_feature_names(user_model, original)[source]

Get feature names for user model

Parameters:
Return type:

Iterable[str]

Returns:

A list if feature names

seldon_core.user_model.client_predict(user_model, features, feature_names, **kwargs)[source]

Get prediction from user model

Parameters:
  • user_model (SeldonComponent) – A seldon user model
  • features (Union[ndarray, str, bytes]) – The data payload
  • feature_names (Iterable[str]) – The feature names in the payload
  • kwargs (Dict[~KT, ~VT]) – Optional keyword arguments
Return type:

Union[ndarray, List[~T], str, bytes]

Returns:

A prediction from the user model

seldon_core.user_model.client_route(user_model, features, feature_names)[source]

Get routing from user model

Parameters:
Return type:

int

Returns:

Routing index for one of children

seldon_core.user_model.client_send_feedback(user_model, features, feature_names, reward, truth, routing)[source]

Feedback to user model

Parameters:
Return type:

Union[ndarray, List[~T], str, bytes, None]

Returns:

Optional payload

seldon_core.user_model.client_transform_input(user_model, features, feature_names, **kwargs)[source]

Transform data with user model

Parameters:
  • user_model (SeldonComponent) – A Seldon user model
  • features (Union[ndarray, str, bytes]) – Data payload
  • feature_names (Iterable[str]) – Data payload column names
  • kwargs (Dict[~KT, ~VT]) – Optional keyword args
Return type:

Union[ndarray, List[~T], str, bytes]

Returns:

Transformed data

seldon_core.user_model.client_transform_output(user_model, features, feature_names, **kwargs)[source]

Transform output

Parameters:
  • user_model (SeldonComponent) – A Seldon user model
  • features (Union[ndarray, str, bytes]) – Data payload
  • feature_names (Iterable[str]) – Data payload column names
  • kwargs (Dict[~KT, ~VT]) – Optional keyword args
Return type:

Union[ndarray, List[~T], str, bytes]

Returns:

Transformed data

seldon_core.utils module

seldon_core.utils.array_to_grpc_datadef(data_type, array, names=[])[source]

Convert numpy array and optional column names into a SeldonMessage DefaultData proto

Parameters:
  • array (ndarray) – numpy array
  • names (Optional[Iterable[str]]) – column names
  • data_type (str) – The SeldonMessage type to convert to
Return type:

DefaultData

Returns:

SeldonMessage DefaultData

seldon_core.utils.array_to_list_value(array, lv=None)[source]

Construct a proto ListValue from numpy array

Parameters:
  • array (ndarray) – Numpy array
  • lv (Optional[ListValue]) – Proto buffer ListValue to extend
Return type:

ListValue

seldon_core.utils.array_to_rest_datadef(data_type, array, names=[])[source]

Construct a payload Dict from a numpy array

Parameters:
Return type:

Dict[~KT, ~VT]

Returns:

Dict representing Seldon payload

seldon_core.utils.construct_response(user_model, is_request, client_request, client_raw_response)[source]
Parameters:
  • user_model (SeldonComponent) – Client user class
  • is_request (bool) – Whether this is part of the request flow as opposed to the response flow
  • client_request (SeldonMessage) – The request received
  • client_raw_response (Union[ndarray, str, bytes, dict]) – The raw client response from their model
Return type:

SeldonMessage

Returns:

A SeldonMessage proto response

seldon_core.utils.construct_response_json(user_model, is_request, client_request_raw, client_raw_response)[source]

This class converts a raw REST response into a JSON object that has the same structure as the SeldonMessage proto. This is necessary as the conversion using the SeldonMessage proto changes the Numeric types of all ints in a JSON into Floats.

Parameters:
  • user_model (SeldonComponent) – Client user class
  • is_request (bool) – Whether this is part of the request flow as opposed to the response flow
  • client_request_raw (Union[List[~T], Dict[~KT, ~VT]]) – The request received in JSON format
  • client_raw_response (Union[ndarray, str, bytes, dict]) – The raw client response from their model
Return type:

Union[List[~T], Dict[~KT, ~VT]]

Returns:

A SeldonMessage JSON response

seldon_core.utils.extract_feedback_request_parts(request)[source]

Extract key parts of the Feedback Message

Parameters:request (Feedback) – Feedback proto
Return type:Tuple[DefaultData, ndarray, ndarray, float]
Returns:Tuple of parts including extracted payloads
seldon_core.utils.extract_request_parts(request)[source]
Parameters:request (SeldonMessage) – Input request
Return type:Tuple[Union[ndarray, str, bytes, dict], Dict[~KT, ~VT], DefaultData, str]
Returns:Key parts of the request extracted
seldon_core.utils.extract_request_parts_json(request)[source]
Parameters:request (Union[Dict[~KT, ~VT], List[~T]]) – Input request in JSON format
Return type:Tuple[Union[ndarray, str, bytes, Dict[~KT, ~VT], List[~T]], Optional[Dict[~KT, ~VT]], Union[ndarray, str, bytes, Dict[~KT, ~VT], List[~T], None], str]
Returns:Key parts of the request extracted
seldon_core.utils.feedback_to_json(message_proto)[source]

Convert a SeldonMessage proto to JSON Dict

Parameters:message_proto (Feedback) – SeldonMessage proto
Return type:Dict[~KT, ~VT]
Returns:JSON Dict
seldon_core.utils.get_data_from_proto(request)[source]

Extract the data payload from the SeldonMessage

Parameters:request (SeldonMessage) – SeldonMessage
Return type:Union[ndarray, str, bytes, dict]
Returns:Data payload as numpy array or the raw message format. Numpy array will be returned if the “data” field was used.
seldon_core.utils.get_meta_from_proto(request)[source]

Convert SeldonMessage proto meta into Dict

Parameters:request (SeldonMessage) – SeldonMessage proto
Return type:Dict[~KT, ~VT]
Returns:Dict
seldon_core.utils.grpc_datadef_to_array(datadef)[source]

Convert a SeldonMessage DefaultData to a numpy array.

Parameters:datadef (DefaultData) – SeldonMessage DefaultData
Return type:ndarray
Returns:A numpy array
seldon_core.utils.json_to_feedback(message_json)[source]

Parse a JSON message to a Feedback proto

Parameters:message_json (Dict[~KT, ~VT]) – Input json message
Return type:Feedback
Returns:A SeldonMessage
seldon_core.utils.json_to_seldon_message(message_json)[source]

Parses JSON input to a SeldonMessage proto

Parameters:message_json (Union[List[~T], Dict[~KT, ~VT]]) – JSON input
Return type:SeldonMessage
Returns:SeldonMessage
seldon_core.utils.json_to_seldon_messages(message_json)[source]
Return type:SeldonMessageList
seldon_core.utils.seldon_message_to_json(message_proto)[source]

Convert a SeldonMessage proto to JSON Dict

Parameters:message_proto (SeldonMessage) – SeldonMessage proto
Return type:Dict[~KT, ~VT]
Returns:JSON Dict
seldon_core.utils.seldon_messages_to_json(message_protos)[source]

Convert a SeldonMessage proto list to JSON Dict

Parameters:message_protos (SeldonMessageList) – SeldonMessage protos
Return type:Dict[~KT, ~VT]
Returns:JSON Dict

seldon_core.version module

seldon_core.wrapper module

class seldon_core.wrapper.SeldonModelGRPC(user_model)[source]

Bases: object

Aggregate(request_grpc, context)[source]
Predict(request_grpc, context)[source]
Route(request_grpc, context)[source]
SendFeedback(feedback_grpc, context)[source]
TransformInput(request_grpc, context)[source]
TransformOutput(request_grpc, context)[source]
seldon_core.wrapper.get_grpc_server(user_model, annotations={}, trace_interceptor=None)[source]
seldon_core.wrapper.get_rest_microservice(user_model)[source]