seldon_core package

Subpackages

• seldon_core.proto package
  • Submodules
  • seldon_core.proto.prediction_pb2 module
  • seldon_core.proto.prediction_pb2_grpc module
    • Combiner
      • Combiner.Aggregate()
    • CombinerServicer
      • CombinerServicer.Aggregate()
    • CombinerStub
      • CombinerStub.__init__()
    • Generic
      • Generic.Aggregate()
      • Generic.Route()
      • Generic.SendFeedback()
      • Generic.TransformInput()
      • Generic.TransformOutput()
    • GenericServicer
      • GenericServicer.Aggregate()
      • GenericServicer.Route()
      • GenericServicer.SendFeedback()
      • GenericServicer.TransformInput()
      • GenericServicer.TransformOutput()
    • GenericStub
      • GenericStub.__init__()
    • Model
      • Model.Metadata()
      • Model.Predict()
      • Model.SendFeedback()
    • ModelServicer
      • ModelServicer.Metadata()
      • ModelServicer.Predict()
      • ModelServicer.SendFeedback()
    • ModelStub
      • ModelStub.__init__()
    • OutputTransformer
      • OutputTransformer.TransformOutput()
    • OutputTransformerServicer
      • OutputTransformerServicer.TransformOutput()
    • OutputTransformerStub
      • OutputTransformerStub.__init__()
    • Router
      • Router.Route()
      • Router.SendFeedback()
    • RouterServicer
      • RouterServicer.Route()
      • RouterServicer.SendFeedback()
    • RouterStub
      • RouterStub.__init__()
    • Seldon
      • Seldon.GraphMetadata()
      • Seldon.ModelMetadata()
      • Seldon.Predict()
      • Seldon.SendFeedback()
    • SeldonServicer
      • SeldonServicer.GraphMetadata()
      • SeldonServicer.ModelMetadata()
      • SeldonServicer.Predict()
      • SeldonServicer.SendFeedback()
    • SeldonStub
      • SeldonStub.__init__()
    • Transformer
      • Transformer.TransformInput()
    • TransformerServicer
      • TransformerServicer.TransformInput()
    • TransformerStub
      • TransformerStub.__init__()
    • add_CombinerServicer_to_server()
    • add_GenericServicer_to_server()
    • add_ModelServicer_to_server()
    • add_OutputTransformerServicer_to_server()
    • add_RouterServicer_to_server()
    • add_SeldonServicer_to_server()
    • add_TransformerServicer_to_server()

Submodules

seldon_core.api_tester module

seldon_core.api_tester.get_seldon_client(args)

Get the appropriate Seldon Client based on args

Parameters:

args – Command line args

Return type:

SeldonClient

Returns:

A Seldon Client

seldon_core.api_tester.main()

seldon_core.api_tester.run_predict(args)

Make a prediction call to the Seldon API

Parameters:

args – Command line args

seldon_core.api_tester.run_send_feedback(args)

Do a semd-feedback call to the Seldon API

Parameters:

args – Command line args

seldon_core.batch_processor module

seldon_core.batch_processor.setup_logging(log_level)

seldon_core.batch_processor.start_multithreaded_batch_worker(deployment_name, gateway_type, namespace, host, transport, data_type, payload_type, workers, retries, batch_size, input_data_path, output_data_path, method, log_level, benchmark, batch_id, batch_interval, call_credentials_token, use_ssl, ssl_verify)

Starts the multithreaded batch worker which consists of three worker types and two queues; the input_file_worker which reads a file and puts all lines in an input queue, which are then read by the multiple request_processor_workers (the number of parallel workers is specified by the workers param), which puts the output in the output queue and then the output_file_worker which puts all the outputs in the output file in a thread-safe approach.

All parameters are defined and explained in detail in the run_cli function.

Return type:

None

seldon_core.env_utils module

Utilities to deal with Environment variables

seldon_core.env_utils.get_deployment_name(default_val='NOT_IMPLEMENTED')

Get deployment name from ENV_SELDON_DEPLOYMENT_NAME environment variable. If not set return default_val

Parameters:

default_val (str) – Default value to return if the environment variable is not set

Return type:

str

Returns:

str

seldon_core.env_utils.get_image_name(default_val='NOT_IMPLEMENTED:NOT_IMPLEMENTED')

Get model image name from ENV_MODEL_IMAGE environment variable. If not set return default_val

Parameters:

default_val (str) – Default value to return if the environment variable is not set

Return type:

str

Returns:

str

seldon_core.env_utils.get_model_name(default_val='NOT_IMPLEMENTED')

Get model name from ENV_MODEL_NAME environment variable. If not set return default_val

Parameters:

default_val (str) – Default value to return if the environment variable is not set

Return type:

str

Returns:

str

seldon_core.env_utils.get_predictor_name(default_val='NOT_IMPLEMENTED')

Get predictor name from ENV_PREDICTOR_NAME environment variable. If not set return default_val

Parameters:

default_val (str) – Default value to return if the environment variable is not set

Return type:

str

Returns:

str

seldon_core.env_utils.get_predictor_version(default_val='NOT_IMPLEMENTED')

Get predictor version from ENV_PREDICTOR_LABELS environment variable. If not set return default_val

Parameters:

default_val (str) – Default value to return if the environment variable is not set

Return type:

str

Returns:

str

seldon_core.flask_utils module

exception seldon_core.flask_utils.SeldonMicroserviceException(message, status_code=None, payload=None, reason='MICROSERVICE_BAD_DATA')

Bases: Exception

status_code = 400

to_dict()

seldon_core.flask_utils.get_multi_form_data_request()

Parses a request submitted with Content-type:multipart/form-data all the keys under SeldonMessage are accepted as form input binData can only be passed as file input strData can be passed as file or text input the file input is base64 encoded

Return type:

Dict

Returns:

JSON Dict

seldon_core.flask_utils.get_request(skip_decoding=False)

Parse a request to get JSON dict

Return type:

Union[Dict, bytes]

Returns:

JSON Dict

seldon_core.flask_utils.jsonify(response, skip_encoding=False)

seldon_core.gunicorn_utils module

class seldon_core.gunicorn_utils.StandaloneApplication(app, options=None)

Bases: BaseApplication

Standalone Application to run a Flask app in Gunicorn.

load()

load_config()

This method is used to load the configuration from one or several input(s). Custom Command line, configuration file. You have to override this method in your class.

class seldon_core.gunicorn_utils.UserModelApplication(app, user_object, tracing, jaeger_extra_tags, interface_name, options=None)

Bases: StandaloneApplication

Gunicorn application to run a Flask app in Gunicorn loading first the user’s model.

load()

seldon_core.gunicorn_utils.accesslog(flag)

Enable / disable access log in Gunicorn depending on the flag.

Return type:

Optional[str]

seldon_core.gunicorn_utils.post_worker_init(worker)

seldon_core.gunicorn_utils.threads(threads, single_threaded)

Number of threads to run in each Gunicorn worker.

Return type:

int

seldon_core.gunicorn_utils.worker_exit(server, worker, seldon_metrics)

seldon_core.imports_helper module

seldon_core.metadata module

exception seldon_core.metadata.SeldonInvalidMetadataError

Bases: Exception

seldon_core.metadata.validate_model_metadata(data)

Validate metadata.

Parameters:

data (Dict) – User defined model metadata (json)

Return type:

Dict

Returns:

Validated model metadata (json)

Raises:

SeldonInvalidMetadataError – if data cannot be properly validated:

Notes

Read data from json and validate against v1 or v2 metadata schema. SeldonInvalidMetadataError exception will be raised if validation fails.

seldon_core.metrics module

class seldon_core.metrics.SeldonMetrics(worker_id_func=<built-in function getpid>, extra_default_labels={})

Bases: object

Class to manage custom metrics stored in shared memory.

clear()

Clear all metrics from current worker.

collect()

generate_metrics()

update(custom_metrics, method)

update_reward(reward)

Update metrics key corresponding to feedback reward counter.

seldon_core.metrics.create_counter(key, value)

Utility method to create a counter metric :type key: str :param key: Counter name :type value: float :param value: Counter value

Returns:

Valid counter metric dict

seldon_core.metrics.create_gauge(key, value)

Utility method to create a gauge metric :type key: str :param key: Gauge name :type value: float :param value: Gauge value

Return type:

Dict

Returns:

Valid Gauge metric dict

seldon_core.metrics.create_timer(key, value)

Utility mehtod to create a timer metric :type key: str :param key: Name of metric :type value: float :param value: Value for metric

Return type:

Dict

Returns:

Valid timer metric dict

seldon_core.metrics.split_image_tag(tag)

Extract image name and version from an image tag.

Parameters:

tag (str) – Fully qualified docker image tag. Eg. seldonio/sklearn-iris:0.1

Return type:

Tuple[str]

Returns:

Image name, image version tuple

seldon_core.metrics.validate_metrics(metrics)

Validate a list of metrics :type metrics: List[Dict] :param metrics: List of metrics

Return type:

bool

seldon_core.microservice module

class seldon_core.microservice.MetricsEndpointFilter(name='')

Bases: Filter

filter(record)

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

seldon_core.microservice.load_annotations()

Attempt to load annotations

Return type:

Dict[str, str]

seldon_core.microservice.main()

seldon_core.microservice.parse_args()

Return type:

Tuple[Namespace, List[str]]

seldon_core.microservice.parse_parameters(parameters)

Parse the user object parameters

Parameters:

parameters (Dict) –

Return type:

Dict

seldon_core.microservice.setup_logger(log_level, debug_mode)

Return type:

Logger

seldon_core.microservice.start_servers(target1, target2, target3, metrics_target)

Start servers

Parameters:

• target1 (Callable) – Main flask process

• target2 (Callable) – Auxiliary flask process

Return type:

None

seldon_core.microservice_tester module

exception seldon_core.microservice_tester.SeldonTesterException(message)

Bases: Exception

seldon_core.microservice_tester.gen_categorical(values, n)

Generate a random categorical feature

Parameters:

• values (List[str]) – The list of categorical values

• n (List[int]) – The number of features to create

Return type:

ndarray

Returns:

Numpy array of values

seldon_core.microservice_tester.gen_continuous(f_range, n)

Create a continuous feature based on given range

Parameters:

• f_range (Tuple[Union[float, str], Union[float, str]]) – A range tuple

• n (int) – The number of feature values to create

Return type:

ndarray

seldon_core.microservice_tester.generate_batch(contract, n, field)

Return type:

ndarray

seldon_core.microservice_tester.get_class_names(contract)

Return type:

List[str]

seldon_core.microservice_tester.main()

seldon_core.microservice_tester.reconciliate_cont_type(feature, dtype)

Ensure numpy arrays are always of type float

Parameters:

• feature (ndarray) – Feature values

• dtype (str) – The required type in the contract

Return type:

ndarray

Returns:

Numpy array

seldon_core.microservice_tester.run_method(args, method)

Make a predict call to microservice

Parameters:

args – Command line args

seldon_core.microservice_tester.run_send_feedback(args)

Make a feedback call to microservice

Parameters:

args – Command line args

seldon_core.microservice_tester.unfold_contract(contract)

Expand contract to full version

Parameters:

contract (Dict) – The Seldon Contract

Return type:

Dict

Returns:

Full contract

seldon_core.mlmd_utils module

seldon_core.seldon_client module

class seldon_core.seldon_client.SeldonCallCredentials(token=None)

Bases: object

Credentials for each call, currently implements the ability to provide

an OAuth token which is currently made available through REST via the X-Auth-Token header, and via GRPC via the metadata call creds.

class seldon_core.seldon_client.SeldonChannelCredentials(verify=True, root_certificates_file=None, private_key_file=None, certificate_chain_file=None)

Bases: object

Channel credentials.

Presently just denotes an SSL connection. For GRPC in order to be properly implemented, you need to provide either the root_certificate_files, or all the file paths. The verify attribute currently is used to avoid SSL verification in REST however for GRPC it is recommended that you provide a path at least for the root_certificates_file otherwise it may not work as expected.

class seldon_core.seldon_client.SeldonClient(gateway='ambassador', transport='rest', namespace=None, deployment_name=None, payload_type='tensor', gateway_endpoint='localhost:8003', microservice_endpoint='localhost:5000', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, channel_credentials=None, call_credentials=None, debug=False, client_return_type='dict', ssl=None)

Bases: object

A reference Seldon API Client

__init__(gateway='ambassador', transport='rest', namespace=None, deployment_name=None, payload_type='tensor', gateway_endpoint='localhost:8003', microservice_endpoint='localhost:5000', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, channel_credentials=None, call_credentials=None, debug=False, client_return_type='dict', ssl=None)

Parameters:

• gateway (str) – API Gateway - either ambassador, istio or seldon

• transport (str) – API transport - grpc or rest

• namespace (Optional[str]) – k8s namespace of running deployment

• deployment_name (Optional[str]) – name of seldon deployment

• payload_type (str) – type of payload - tensor, ndarray or tftensor

• gateway_endpoint (str) – Gateway endpoint

• microservice_endpoint (str) – Running microservice endpoint

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• client_return_type (str) – the return type of all functions can be either dict or proto

explain(gateway=None, transport=None, deployment_name=None, payload_type=None, gateway_endpoint=None, shape=(1, 1), namespace=None, data=None, bin_data=None, str_data=None, json_data=None, names=None, gateway_prefix=None, headers=None, http_path=None, client_return_type=None, predictor=None, ssl=None)

Parameters:

• gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon

• transport (Optional[str]) – API transport - grpc or rest

• namespace (Optional[str]) – k8s namespace of running deployment

• deployment_name (Optional[str]) – name of seldon deployment

• payload_type (Optional[str]) – type of payload - tensor, ndarray or tftensor

• gateway_endpoint (Optional[str]) – Gateway endpoint

• microservice_endpoint – Running microservice endpoint

• grpc_max_send_message_length – Max grpc send message size in bytes

• grpc_max_receive_message_length – Max grpc receive message size in bytes

• data (Optional[ndarray]) – Numpy Array Payload to send

• bin_data (Union[bytes, bytearray, None]) – Binary payload to send - will override data

• str_data (Optional[str]) – String payload to send - will override data

• json_data (Optional[str]) – JSON payload to send - will override data

• names (Optional[Iterable[str]]) – Column names

• gateway_prefix (Optional[str]) – prefix path for gateway URL endpoint

• headers (Optional[Dict]) – Headers to add to request

• http_path (Optional[str]) – Custom http path for predict call to use

• client_return_type (Optional[str]) – the return type of all functions can be either dict or proto

• predictor (Optional[str]) – The name of the predictor to send the explanations to

Return type:

Dict

feedback(prediction_request=None, prediction_response=None, prediction_truth=None, reward=0, gateway=None, transport=None, deployment_name=None, payload_type=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, gateway_prefix=None, client_return_type=None, raw_request=None, ssl=None)

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon

• transport (Optional[str]) – API transport - grpc or rest

• deployment_name (Optional[str]) – name of seldon deployment

• payload_type (Optional[str]) – payload - tensor, ndarray or tftensor

• gateway_endpoint (Optional[str]) – Gateway endpoint

• microservice_endpoint (Optional[str]) – Running microservice endpoint

• grpc_max_send_message_length – Max grpc send message size in bytes

• grpc_max_receive_message_length – Max grpc receive message size in bytes

• method (Optional[str]) – The microservice method to call

• shape (Tuple) – The shape of the data to send

• namespace (Optional[str]) – k8s namespace of running deployment

• client_return_type (Optional[str]) – the return type of all functions can be either dict or proto

Return type:

SeldonClientFeedback

microservice(gateway=None, transport=None, deployment_name=None, payload_type=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, data=None, datas=None, ndatas=None, bin_data=None, str_data=None, json_data=None, custom_data=None, names=None)

Parameters:

• gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon

• transport (Optional[str]) – API transport - grpc or rest

• deployment_name (Optional[str]) – name of seldon deployment

• payload_type (Optional[str]) – payload - tensor, ndarray or tftensor

• gateway_endpoint (Optional[str]) – Gateway endpoint

• microservice_endpoint (Optional[str]) – Running microservice endpoint

• grpc_max_send_message_length – Max grpc send message size in bytes

• grpc_max_receive_message_length – Max grpc receive message size in bytes

• method (Optional[str]) – The microservice method to call

• shape (Tuple) – The shape of the data to send

• namespace (Optional[str]) – k8s namespace of running deployment

• data (Optional[ndarray]) – Numpy Array Payload to send

• bin_data (Union[bytes, bytearray, None]) – Binary payload to send - will override data

• str_data (Optional[str]) – String payload to send - will override data

• json_data (Union[str, List, Dict, None]) – String payload to send - will override data

• custom_data (Optional[Any]) – Custom payload to send - will override data

• ndatas (Optional[int]) – Multiple numpy arrays to send for aggregation

• bin_data – Binary data payload

• str_data – String data payload

• names (Optional[Iterable[str]]) – Column names

Return type:

Union[SeldonClientPrediction, SeldonClientCombine]

Returns:

A prediction result

microservice_feedback(prediction_request=None, prediction_response=None, reward=0, gateway=None, transport=None, deployment_name=None, payload_type=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None)

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• gateway (Optional[str]) – API Gateway - either Gateway or seldon

• transport (Optional[str]) – API transport - grpc or rest

• deployment_name (Optional[str]) – name of seldon deployment

• payload_type (Optional[str]) – payload - tensor, ndarray or tftensor

• gateway_endpoint (Optional[str]) – Gateway endpoint

• microservice_endpoint (Optional[str]) – Running microservice endpoint

• grpc_max_send_message_length – Max grpc send message size in bytes

• grpc_max_receive_message_length – Max grpc receive message size in bytes

• method (Optional[str]) – The microservice method to call

• shape (Tuple) – The shape of the data to send

• namespace (Optional[str]) – k8s namespace of running deployment

Return type:

SeldonClientFeedback

Returns:

A client response

predict(gateway=None, transport=None, deployment_name=None, payload_type=None, gateway_endpoint=None, microservice_endpoint=None, method=None, shape=(1, 1), namespace=None, data=None, bin_data=None, str_data=None, json_data=None, custom_data=None, names=None, gateway_prefix=None, headers=None, http_path=None, meta=None, client_return_type=None, raw_data=None, ssl=None)

Parameters:

• gateway (Optional[str]) – API Gateway - either ambassador, istio or seldon

• transport (Optional[str]) – API transport - grpc or rest

• namespace (Optional[str]) – k8s namespace of running deployment

• deployment_name (Optional[str]) – name of seldon deployment

• payload_type (Optional[str]) – type of payload - tensor, ndarray or tftensor

• gateway_endpoint (Optional[str]) – Gateway endpoint

• microservice_endpoint (Optional[str]) – Running microservice endpoint

• grpc_max_send_message_length – Max grpc send message size in bytes

• grpc_max_receive_message_length – Max grpc receive message size in bytes

• data (Optional[ndarray]) – Numpy Array Payload to send

• bin_data (Union[bytes, bytearray, None]) – Binary payload to send - will override data

• str_data (Optional[str]) – String payload to send - will override data

• json_data (Union[str, List, Dict, None]) – JSON payload to send - will override data

• custom_data (Optional[Any]) – Custom payload to send - will override data

• names (Optional[Iterable[str]]) – Column names

• gateway_prefix (Optional[str]) – prefix path for gateway URL endpoint

• headers (Optional[Dict]) – Headers to add to request

• http_path (Optional[str]) – Custom http path for predict call to use

• meta (Optional[Dict]) – Custom meta map, supplied as tags

• client_return_type (Optional[str]) – the return type of all functions can be either dict or proto

• raw_data (Optional[Dict]) – Raw payload, a dictionary representing the json request or the raw grpc proto

Return type:

SeldonClientPrediction

class seldon_core.seldon_client.SeldonClientCombine(request, response, success=True, msg='')

Bases: object

Data class to return from Seldon Client for aggregate calls

exception seldon_core.seldon_client.SeldonClientException(message)

Bases: Exception

Seldon Client Exception

status_code = 400

class seldon_core.seldon_client.SeldonClientFeedback(request, response, success=True, msg='')

Bases: object

Data class to return from Seldon Client for feedback calls

class seldon_core.seldon_client.SeldonClientPrediction(request, response, success=True, msg='')

Bases: object

Data class to return from Seldon Client

seldon_core.seldon_client.explain_predict_gateway(deployment_name, namespace=None, gateway_endpoint='localhost:8003', gateway=None, transport='rest', shape=(1, 1), data=None, headers=None, gateway_prefix=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, names=None, call_credentials=None, channel_credentials=None, http_path=None, client_return_type='dict', predictor=None, ssl=None, **kwargs)

REST explain request to Gateway Ingress

Parameters:

• deployment_name (str) – The name of the Seldon Deployment

• namespace (Optional[str]) – k8s namespace of running deployment

• gateway_endpoint (str) – The host:port of gateway

• gateway (Optional[str]) – The type of gateway which can be seldon or ambassador/istio

• transport (str) – The type of transport, in this case only rest is supported

• shape (Tuple[int, int]) – The shape of the data to send

• data (Optional[ndarray]) – The numpy data to send

• headers (Optional[Dict]) – Headers to add to request

• gateway_prefix (Optional[str]) – The prefix path to add to the request

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data to send

• str_data (Optional[str]) – String data to send

• json_data (Union[str, List, Dict, None]) – JSON data to send

• names (Optional[Iterable[str]]) – Column names

• call_credentials (Optional[SeldonCallCredentials]) – Call credentials - see SeldonCallCredentials

• channel_credentials (Optional[SeldonChannelCredentials]) – Channel credentials - see SeldonChannelCredentials

• http_path (Optional[str]) – Custom http path

• client_return_type (str) – the return type of all functions can be either dict or proto

Return type:

SeldonClientPrediction

Returns:

A JSON Dict

seldon_core.seldon_client.grpc_feedback_gateway(prediction_request=None, prediction_response=None, prediction_truth=None, reward=0, deployment_name='', namespace=None, gateway_endpoint='localhost:8003', headers=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, client_return_type='proto', raw_request=None, **kwargs)

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• deployment_name (str) – The name of the running Seldon deployment

• namespace (Optional[str]) – k8s namespace of running deployment

• gateway_endpoint (str) – The gateway host:port endpoint

• headers (Optional[Dict]) – Headers to add to the request

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• client_return_type (str) – the return type of all functions can be either dict or proto

• kwargs –

Return type:

SeldonClientFeedback

seldon_core.seldon_client.grpc_feedback_seldon(prediction_request=None, prediction_response=None, prediction_truth=None, reward=0, namespace=None, gateway_endpoint='localhost:8004', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, client_return_type='proto', raw_request=None, **kwargs)

Send feedback to Seldon API gateway via gRPC

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• namespace (Optional[str]) – k8s namespace of running deployment

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• client_return_type (str) – the return type of all functions can be either dict or proto

• kwargs –

Return type:

SeldonClientFeedback

seldon_core.seldon_client.grpc_predict_gateway(deployment_name, namespace=None, gateway_endpoint='localhost:8003', shape=(1, 1), data=None, headers=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, custom_data=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, call_credentials=None, channel_credentials=None, meta={}, client_return_type='proto', raw_data=None, ssl=None, **kwargs)

gRPC request to Gateway Ingress

Parameters:

• deployment_name (str) – Deployment name of Seldon Deployment

• namespace (Optional[str]) – The namespace the Seldon Deployment is running in

• gateway_endpoint (str) – The endpoint for gateway

• shape (Tuple[int, int]) – The shape of the data

• data (Optional[ndarray]) – The numpy array data to send

• headers (Optional[Dict]) – A Dict of key value pairs to add to gRPC HTTP Headers

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data to send

• str_data (Optional[str]) – String data to send

• json_data (Union[str, List, Dict, None]) – JSON data to send

• custom_data (Optional[Any]) – Custom data to send

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• names (Optional[Iterable[str]]) – Column names

• call_credentials (Optional[SeldonCallCredentials]) – Call credentials - see SeldonCallCredentials

• channel_credentials (Optional[SeldonChannelCredentials]) – Channel credentials - see SeldonChannelCredentials

• meta (Dict) – Custom meta data map, supplied as tags

• client_return_type (str) – the return type of all functions can be either dict or proto

• raw_data (Optional[Dict]) – Raw payload (dictionary or proto) given by the user

Return type:

SeldonClientPrediction

Returns:

A SeldonMessage proto response

seldon_core.seldon_client.grpc_predict_seldon(namespace=None, gateway_endpoint='localhost:8004', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, custom_data=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, client_return_type='proto', raw_data=None, **kwargs)

Call Seldon gRPC API Gateway endpoint

Parameters:

• namespace (Optional[str]) – k8s namespace of running deployment

• shape (Tuple[int, int]) – Shape of endpoint

• data (Optional[ndarray]) – Data to send

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data to send

• str_data (Optional[str]) – String data to send

• json_data (Union[str, List, Dict, None]) – JSON data to send

• custom_data (Optional[Any]) – Custom data to send

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• names (Optional[Iterable[str]]) – Column names

• client_return_type (str) – the return type of all functions can be either dict or proto

• raw_data (Optional[Dict]) – Raw payload (dictionary or proto) given by the user

• kwargs –

Return type:

SeldonClientPrediction

Returns:

A SeldonMessage proto

seldon_core.seldon_client.microservice_api_grpc_aggregate(microservice_endpoint='localhost:5000', shape=(1, 1), datas=None, ndatas=None, payload_type='tensor', grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, **kwargs)

Call Seldon microservice gRPC API aggregate

Parameters:

• microservice_endpoint (str) – Microservice API endpoint

• shape (Tuple) – Shape of the data to send

• datas (Optional[List[ndarray]]) – List of Numpy array data to send

• ndatas (Optional[int]) – Multiple numpy arrays to send for aggregation

• payload_type (str) – payload - tensor, ndarray or tftensor

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• names (Optional[Iterable[str]]) – Column names

• kwargs –

Return type:

SeldonClientCombine

Returns:

SeldonClientCombine

seldon_core.seldon_client.microservice_api_grpc_feedback(prediction_request=None, prediction_response=None, reward=0, microservice_endpoint=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, **kwargs)

Call Seldon gRPC

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• microservice_endpoint (Optional[str]) – Running microservice endpoint

• kwargs –

Return type:

SeldonClientFeedback

seldon_core.seldon_client.microservice_api_grpc_seldon_message(method='predict', microservice_endpoint='localhost:5000', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, custom_data=None, grpc_max_send_message_length=4194304, grpc_max_receive_message_length=4194304, names=None, **kwargs)

Call Seldon microservice gRPC API

Parameters:

• method (str) – Method to call

• microservice_endpoint (str) – Running microservice endpoint

• shape (Tuple) – The shape of the data to send

• data (Optional[object]) – Numpy array data to send

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data to send

• str_data (Optional[str]) – String data to send

• json_data (Union[str, List, Dict, None]) – JSON data to send

• custom_data (Optional[Any]) – Custom data to send

• grpc_max_send_message_length (int) – Max grpc send message size in bytes

• grpc_max_receive_message_length (int) – Max grpc receive message size in bytes

• names (Optional[Iterable[str]]) – column names

• kwargs –

Return type:

SeldonClientPrediction

Returns:

SeldonClientPrediction

seldon_core.seldon_client.microservice_api_rest_aggregate(microservice_endpoint='localhost:5000', shape=(1, 1), datas=None, ndatas=None, payload_type='tensor', names=None, **kwargs)

Call Seldon microservice REST API aggregate endpoint

Parameters:

• microservice_endpoint (str) – Running microservice endpoint

• shape (Tuple) – The shape of the data to send

• datas (Optional[List[ndarray]]) – List of Numpy array data to send

• ndatas (Optional[int]) – Multiple numpy arrays to send for aggregation

• payload_type (str) – payload - tensor, ndarray or tftensor

• names (Optional[Iterable[str]]) – Column names

• kwargs –

Return type:

SeldonClientCombine

Returns:

A SeldonClientPrediction

seldon_core.seldon_client.microservice_api_rest_feedback(prediction_request=None, prediction_response=None, reward=0, microservice_endpoint=None, **kwargs)

Call Seldon microserice REST API to send feedback

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• microservice_endpoint (Optional[str]) – Running microservice endpoint

• kwargs –

Return type:

SeldonClientFeedback

Returns:

A SeldonClientFeedback

seldon_core.seldon_client.microservice_api_rest_seldon_message(method='predict', microservice_endpoint='localhost:5000', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, names=None, **kwargs)

Call Seldon microservice REST API

Parameters:

• method (str) – The microservice method to call

• microservice_endpoint (str) – Running microservice endpoint

• grpc_max_send_message_length – Max grpc send message size in bytes

• grpc_max_receive_message_length – Max grpc receive message size in bytes

• method – The microservice method to call

• shape (Tuple) – The shape of the data to send

• namespace – k8s namespace of running deployment

• shape – Shape of the data to send

• data (Optional[object]) – Numpy array data to send

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data payload

• str_data (Optional[str]) – String data payload

• json_data (Union[str, List, Dict, None]) – JSON data payload

• names (Optional[Iterable[str]]) – Column names

• kwargs –

Return type:

SeldonClientPrediction

Returns:

A SeldonClientPrediction data response

seldon_core.seldon_client.rest_feedback_gateway(prediction_request=None, prediction_response=None, prediction_truth=None, reward=0, deployment_name='', namespace=None, gateway_endpoint='localhost:8003', headers=None, gateway_prefix=None, client_return_type='proto', raw_request=None, **kwargs)

Send Feedback to Seldon via gateway using REST

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• deployment_name (str) – The name of the running Seldon deployment

• namespace (Optional[str]) – k8s namespace of running deployment

• gateway_endpoint (str) – The gateway host:port endpoint

• headers (Optional[Dict]) – Headers to add to the request

• gateway_prefix (Optional[str]) – The prefix to add to the request path for gateway

• client_return_type (str) – the return type of all functions can be either dict or proto

• kwargs –

Return type:

SeldonClientFeedback

Returns:

A Seldon Feedback Response

seldon_core.seldon_client.rest_feedback_seldon(prediction_request=None, prediction_response=None, prediction_truth=None, reward=0, namespace=None, gateway_endpoint='localhost:8002', client_return_type='proto', raw_request=None, **kwargs)

Send Feedback to Seldon API Gateway using REST

Parameters:

• prediction_request (Optional[SeldonMessage]) – Previous prediction request

• prediction_response (Optional[SeldonMessage]) – Previous prediction response

• reward (float) – A reward to send in feedback

• namespace (Optional[str]) – k8s namespace of running deployment

• client_return_type (str) – the return type of all functions can be either dict or proto

• kwargs –

Return type:

SeldonClientFeedback

seldon_core.seldon_client.rest_predict_gateway(deployment_name, namespace=None, gateway_endpoint='localhost:8003', shape=(1, 1), data=None, headers=None, gateway_prefix=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, names=None, call_credentials=None, channel_credentials=None, http_path=None, meta={}, client_return_type='proto', raw_data=None, ssl=None, **kwargs)

REST request to Gateway Ingress

Parameters:

• deployment_name (str) – The name of the Seldon Deployment

• namespace (Optional[str]) – k8s namespace of running deployment

• gateway_endpoint (str) – The host:port of gateway

• shape (Tuple[int, int]) – The shape of the data to send

• data (Optional[ndarray]) – The numpy data to send

• headers (Optional[Dict]) – Headers to add to request

• gateway_prefix (Optional[str]) – The prefix path to add to the request

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data to send

• str_data (Optional[str]) – String data to send

• json_data (Union[str, List, Dict, None]) – JSON data to send as str, dict or list

• names (Optional[Iterable[str]]) – Column names

• call_credentials (Optional[SeldonCallCredentials]) – Call credentials - see SeldonCallCredentials

• channel_credentials (Optional[SeldonChannelCredentials]) – Channel credentials - see SeldonChannelCredentials

• http_path (Optional[str]) – Custom http path

• meta (Dict) – Custom meta map

• client_return_type (str) – the return type of all functions can be either dict or proto

• raw_data (Optional[Dict]) – Raw payload (dictionary) given by the user

Return type:

SeldonClientPrediction

Returns:

A requests Response object

seldon_core.seldon_client.rest_predict_seldon(namespace=None, gateway_endpoint='localhost:8002', shape=(1, 1), data=None, payload_type='tensor', bin_data=None, str_data=None, json_data=None, names=None, client_return_type='proto', raw_data=None, meta={}, **kwargs)

Call Seldon API Gateway using REST

Parameters:

• namespace (Optional[str]) – k8s namespace of running deployment

• shape (Tuple) – Shape of endpoint

• data (Optional[object]) – Data to send

• payload_type (str) – payload - tensor, ndarray or tftensor

• bin_data (Union[bytes, bytearray, None]) – Binary data to send

• str_data (Optional[str]) – String data to send

• json_data (Union[str, List, Dict, None]) – JSON data to send

• names (Optional[Iterable[str]]) – column names

• client_return_type (str) – the return type of all functions can be either dict or proto

• raw_data (Optional[Dict]) – Raw payload (dictionary) given by the user

• meta (Dict) – Custom meta data map, supplied as tags

• kwargs –

Return type:

SeldonClientPrediction

Returns:

Seldon Client Prediction

seldon_core.seldon_methods module

seldon_core.seldon_methods.aggregate(user_model, request, seldon_metrics)

Aggregate a list of payloads

Parameters:

• user_model (Any) – A Seldon user model

• request (Union[SeldonMessageList, List, Dict]) – SeldonMessage proto

• seldon_metrics (SeldonMetrics) – A SeldonMetrics instance

Return type:

Union[SeldonMessage, List, Dict]

Returns:

Aggregated SeldonMessage proto

seldon_core.seldon_methods.handle_raw_custom_metrics(msg, seldon_metrics, is_proto, method)

Update SeldonMetrics object with custom metrics from raw methods. If INCLUDE_METRICS_IN_CLIENT_RESPONSE environmental variable is set to “true” metrics will be dropped from msg.

seldon_core.seldon_methods.health_status(user_model, seldon_metrics)

Call the user model to check the health of the model

Parameters:

• user_model (Any) – User defined class instance

• seldon_metrics (SeldonMetrics) – A SeldonMetrics instance

Return type:

Union[SeldonMessage, List, Dict]

Returns:

Health check output

seldon_core.seldon_methods.init_metadata(user_model)

Call the user model to get the model init_metadata

Parameters:

user_model (Any) – User defined class instance

Return type:

Dict

Returns:

Validated model metadata

seldon_core.seldon_methods.predict(user_model, request, seldon_metrics)

Call the user model to get a prediction and package the response

Parameters:

• user_model (Any) – User defined class instance

• request (Union[SeldonMessage, List, Dict, bytes]) – The incoming request

Return type:

Union[SeldonMessage, List, Dict, bytes]

Returns:

The prediction

seldon_core.seldon_methods.route(user_model, request, seldon_metrics)

Parameters:

• user_model (Any) – A Seldon user model

• request (Union[SeldonMessage, List, Dict]) – A SelodonMessage proto

• seldon_metrics (SeldonMetrics) – A SeldonMetrics instance

Return type:

Union[SeldonMessage, List, Dict]

seldon_core.seldon_methods.send_feedback(user_model, request, predictive_unit_id, seldon_metrics)

Parameters:

• user_model (Any) – A Seldon user model

• request (Feedback) – SeldonMesage proto

• predictive_unit_id (str) – The ID of the enclosing container predictive unit. Will be taken from environment.

Return type:

SeldonMessage

seldon_core.seldon_methods.transform_input(user_model, request, seldon_metrics)

Parameters:

• user_model (Any) – User defined class to handle transform input

• request (Union[SeldonMessage, List, Dict]) – The incoming request

Return type:

Union[SeldonMessage, List, Dict]

Returns:

The transformed request

seldon_core.seldon_methods.transform_output(user_model, request, seldon_metrics)

Parameters:

• user_model (Any) – User defined class to handle transform input

• request (Union[SeldonMessage, List, Dict]) – The incoming request

Return type:

Union[SeldonMessage, List, Dict]

Returns:

The transformed request

seldon_core.serving_test_gen module

seldon_core.storage module

class seldon_core.storage.Storage

Bases: object

static download(uri, out_dir=None)

Return type:

str

seldon_core.user_model module

class seldon_core.user_model.SeldonComponent(**kwargs)

Bases: object

aggregate(features_list, feature_names_list)

Return type:

Union[ndarray, List, Dict, str, bytes, SeldonResponse]

aggregate_raw(msgs)

Return type:

Union[SeldonMessage, Dict]

class_names()

Return type:

Iterable[str]

feature_names()

Return type:

Iterable[str]

health_status()

Return type:

Union[ndarray, List, str, bytes]

health_status_raw()

Return type:

SeldonMessage

init_metadata()

Return type:

Dict

load()

metadata()

Return type:

Dict

metrics()

Return type:

List[Dict]

predict(X, names, meta=None)

Return type:

Union[ndarray, List, Dict, str, bytes, SeldonResponse]

predict_raw(msg)

Return type:

Union[SeldonMessage, Dict]

route(features, feature_names)

Return type:

Union[int, SeldonResponse]

route_raw(msg)

Return type:

Union[SeldonMessage, Dict]

send_feedback(features, feature_names, reward, truth, routing)

Return type:

Union[ndarray, List, Dict, str, bytes, None, SeldonResponse]

send_feedback_raw(feedback)

Return type:

Union[SeldonMessage, Dict]

tags()

Return type:

Dict

transform_input(X, names, meta=None)

Return type:

Union[ndarray, List, Dict, str, bytes, SeldonResponse]

transform_input_raw(msg)

Return type:

Union[SeldonMessage, Dict]

transform_output(X, names, meta=None)

Return type:

Union[ndarray, List, Dict, str, bytes, SeldonResponse]

transform_output_raw(msg)

Return type:

Union[SeldonMessage, Dict]

exception seldon_core.user_model.SeldonNotImplementedError(message)

Bases: SeldonMicroserviceException

status_code = 400

class seldon_core.user_model.SeldonResponse(data, tags=None, metrics=None)

Bases: object

Simple class to store data returned by SeldonComponent methods together with relevant runtime metrics and runtime request tags.

classmethod create(data)

Return type:

SeldonResponse

seldon_core.user_model.client_aggregate(user_model, features_list, feature_names_list)

Aggregate payloads

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• features_list (List[Union[ndarray, str, bytes]]) – A list of payloads

• feature_names_list (List) – Column names for payloads

Return type:

SeldonResponse

Returns:

An aggregated payload

seldon_core.user_model.client_class_names(user_model, predictions)

Get class names from user model

Parameters:

• user_model (SeldonComponent) – User defined class instance

• predictions (ndarray) – Prediction results

Return type:

Iterable[str]

Returns:

Class names

seldon_core.user_model.client_custom_metrics(user_model, seldon_metrics, method, runtime_metrics=[])

Get custom metrics for client and update SeldonMetrics.

This function will return empty list if INCLUDE_METRICS_IN_CLIENT_RESPONSE environmental variable is NOT set to “true” or “True”.

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• seldon_metrics (SeldonMetrics) – A SeldonMetrics instance

• method (str) – tag of a method that collected the metrics

• runtime_metrics (List[Dict]) – metrics that were defined on runtime

Return type:

List[Dict]

Returns:

A list of custom metrics

seldon_core.user_model.client_custom_tags(user_model)

Get tags from user model

Parameters:

user_model (SeldonComponent) –

Return type:

Dict

Returns:

Dictionary of key value pairs

seldon_core.user_model.client_feature_names(user_model, original)

Get feature names for user model

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• original (Iterable[str]) – Original feature names

Return type:

Iterable[str]

Returns:

A list if feature names

seldon_core.user_model.client_health_status(user_model)

Perform a health check

Parameters:

user_model (SeldonComponent) – A Seldon user model

Return type:

Union[ndarray, List, str, bytes]

Returns:

Health check results

seldon_core.user_model.client_predict(user_model, features, feature_names, **kwargs)

Get prediction from user model

Parameters:

• user_model (SeldonComponent) – A seldon user model

• features (Union[ndarray, str, bytes]) – The data payload

• feature_names (Iterable[str]) – The feature names in the payload

• kwargs (Dict) – Optional keyword arguments

Return type:

SeldonResponse

Returns:

A prediction from the user model

seldon_core.user_model.client_route(user_model, features, feature_names, **kwargs)

Get routing from user model

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• features (Union[ndarray, str, bytes]) – Payload

• feature_names (Iterable[str]) – Columns for payload

Return type:

SeldonResponse

Returns:

Routing index for one of children

seldon_core.user_model.client_send_feedback(user_model, features, feature_names, reward, truth, routing)

Feedback to user model

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• features (Union[ndarray, str, bytes]) – A payload

• feature_names (Iterable[str]) – Payload column names

• reward (float) – Reward

• truth (Union[ndarray, str, bytes]) – True outcome

• routing (Optional[int]) – Optional routing

Return type:

SeldonResponse

Returns:

Optional payload

seldon_core.user_model.client_transform_input(user_model, features, feature_names, **kwargs)

Transform data with user model

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• features (Union[ndarray, str, bytes]) – Data payload

• feature_names (Iterable[str]) – Data payload column names

• kwargs (Dict) – Optional keyword args

Return type:

SeldonResponse

Returns:

Transformed data

seldon_core.user_model.client_transform_output(user_model, features, feature_names, **kwargs)

Transform output

Parameters:

• user_model (SeldonComponent) – A Seldon user model

• features (Union[ndarray, str, bytes]) – Data payload

• feature_names (Iterable[str]) – Data payload column names

• kwargs (Dict) – Optional keyword args

Return type:

SeldonResponse

Returns:

Transformed data

seldon_core.utils module

seldon_core.utils.array_to_grpc_datadef(data_type, array, names=[])

Convert numpy array and optional column names into a SeldonMessage DefaultData proto

Parameters:

• array (ndarray) – numpy array

• names (Optional[Iterable[str]]) – column names

• data_type (str) – The SeldonMessage type to convert to

Return type:

DefaultData

Returns:

SeldonMessage DefaultData

seldon_core.utils.array_to_list_value(array, lv=None)

Construct a proto ListValue from numpy array

Parameters:

• array (ndarray) – Numpy array

• lv (Optional[ListValue]) – Proto buffer ListValue to extend

Return type:

ListValue

Returns:

ListValue protobuf

seldon_core.utils.array_to_rest_datadef(data_type, array, names=[])

Construct a payload Dict from a numpy array

Parameters:

• data_type (str) –

• array (ndarray) –

• names (Optional[List[str]]) –

Return type:

Dict

Returns:

Dict representing Seldon payload

seldon_core.utils.construct_response(user_model, is_request, client_request, client_raw_response, meta=None, custom_metrics=None, runtime_tags=None)

Parameters:

• user_model (SeldonComponent) – Client user class

• is_request (bool) – Whether this is part of the request flow as opposed to the response flow

• client_request (SeldonMessage) – The request received

• client_raw_response (Union[ndarray, str, bytes, dict, Any]) – The raw client response from their model

Return type:

SeldonMessage

Returns:

A SeldonMessage proto response

seldon_core.utils.construct_response_json(user_model, is_request, client_request_raw, client_raw_response, meta=None, custom_metrics=None, runtime_tags=None)

This class converts a raw REST response into a JSON object that has the same structure as the SeldonMessage proto. This is necessary as the conversion using the SeldonMessage proto changes the Numeric types of all ints in a JSON into Floats.

Parameters:

• user_model (SeldonComponent) – Client user class

• is_request (bool) – Whether this is part of the request flow as opposed to the response flow

• client_request_raw (Union[List, Dict]) – The request received in JSON format

• client_raw_response (Union[ndarray, str, bytes, dict]) – The raw client response from their model

Return type:

Union[List, Dict]

Returns:

A SeldonMessage JSON response

seldon_core.utils.extract_feedback_request_parts(request)

Extract key parts of the Feedback Message

Parameters:

request (Feedback) – Feedback proto

Return type:

Tuple[DefaultData, ndarray, ndarray, float]

Returns:

Tuple of parts including extracted payloads

seldon_core.utils.extract_request_parts(request)

Parameters:

request (SeldonMessage) – Input request

Return type:

Tuple[Union[ndarray, str, bytes, dict], Dict, DefaultData, str]

Returns:

Key parts of the request extracted

seldon_core.utils.extract_request_parts_json(request)

Parameters:

request (Union[Dict, List]) – Input request in JSON format

Return type:

Tuple[Union[ndarray, str, bytes, Dict, List], Optional[Dict], Union[ndarray, str, bytes, Dict, List, None], str]

Returns:

Key parts of the request extracted

seldon_core.utils.feedback_to_json(message_proto)

Convert a SeldonMessage proto to JSON Dict

Parameters:

message_proto (Feedback) – SeldonMessage proto

Return type:

Dict

Returns:

JSON Dict

seldon_core.utils.get_data_from_proto(request)

Extract the data payload from the SeldonMessage

Parameters:

request (SeldonMessage) – SeldonMessage

Return type:

Union[ndarray, str, bytes, dict]

Returns:

Data payload as numpy array or the raw message format. Numpy array will be returned if the “data” field was used.

seldon_core.utils.get_meta_from_proto(request)

Convert SeldonMessage proto meta into Dict

Parameters:

request (SeldonMessage) – SeldonMessage proto

Return type:

Dict

Returns:

Dict

seldon_core.utils.get_request_path()

seldon_core.utils.getenv(*env_vars, default=None)

Overload of os.getenv() to allow falling back through multiple environment variables. The environment variables will be checked sequentially until one of them is found.

Parameters:

• *env_vars – Variadic list of environment variable names to check.

• default – Default value to return if none of the environment variables exist.

Returns:

Value of the first environment variable set or default.

seldon_core.utils.getenv_as_bool(*env_vars, default=False)

Read environment variable, parsing it to a boolean.

seldon_core.utils.grpc_datadef_to_array(datadef)

Convert a SeldonMessage DefaultData to a numpy array.

Parameters:

datadef (DefaultData) – SeldonMessage DefaultData

Return type:

ndarray

Returns:

A numpy array

seldon_core.utils.json_to_feedback(message_json)

Parse a JSON message to a Feedback proto

Parameters:

message_json (Dict) – Input json message

Return type:

Feedback

Returns:

A SeldonMessage

seldon_core.utils.json_to_seldon_message(message_json)

Parses JSON input to a SeldonMessage proto

Parameters:

message_json (Union[List, Dict]) – JSON input

Return type:

SeldonMessage

Returns:

SeldonMessage

seldon_core.utils.json_to_seldon_messages(message_json)

Return type:

SeldonMessageList

seldon_core.utils.json_to_seldon_model_metadata(metadata_json)

Parses JSON input to SeldonModelMetadata proto

Parameters:

metadata_json (Dict) – JSON input

Return type:

SeldonModelMetadata

Returns:

SeldonModelMetadata

seldon_core.utils.seldon_message_to_json(message_proto)

Convert a SeldonMessage proto to JSON Dict

Parameters:

message_proto (SeldonMessage) – SeldonMessage proto

Return type:

Dict

Returns:

JSON Dict

seldon_core.utils.seldon_messages_to_json(message_protos)

Convert a SeldonMessage proto list to JSON Dict

Parameters:

message_protos (SeldonMessageList) – SeldonMessage protos

Return type:

Dict

Returns:

JSON Dict

seldon_core.utils.setup_tracing(interface_name)

Return type:

object

seldon_core.version module

seldon_core.wrapper module

class seldon_core.wrapper.SeldonModelGRPC(user_model, seldon_metrics)

Bases: object

Aggregate(request_grpc, context)

GraphMetadata(request_grpc, context)

GraphMetadata method of rpc Seldon service

Metadata(request_grpc, context)

Metadata method of rpc Model service

ModelMetadata(request_grpc, context)

ModelMetadata method of rpc Seldon service

Predict(request_grpc, context)

Route(request_grpc, context)

SendFeedback(feedback_grpc, context)

TransformInput(request_grpc, context)

TransformOutput(request_grpc, context)

seldon_core.wrapper.get_grpc_server(user_model, seldon_metrics, annotations={}, trace_interceptor=None, num_threads=1)

seldon_core.wrapper.get_metrics_microservice(seldon_metrics)

seldon_core.wrapper.get_rest_microservice(user_model, seldon_metrics)