Testing Your Seldon Components¶
Whether you have wrapped your component using our S2I wrappers or created your own wrapper you will want to test the Docker container standalone and also quickly within a running cluster. We have provided two python console scripts within the seldon-core Python package to allow you to easily do this:
seldon-core-microservice-tester- Allows you to test a docker component to check it respects the Seldon internal microservice API.
seldon-core-api-tester- Allows you to test the external endpoints for a running Seldon Deployment graph.
To use these, install the seldon-core package with pip install seldon-core.
Run your Wrapped Model¶
To test your model microservice you need to run it. If you have wrapped your model into a Docker container then you should run it and expose the ports. There are many examples in the notebooks in the examples folders but essential if your model is wrapped in an image myimage:0.1 then run:
docker run --name "my_model" -d --rm -p 5000:5000 myimage:0.1
Alternatively, if your component is a Python module you can run it directly from python using the core tool seldon-core-microservice (installed as part of the pip package seldon-core). This tool takes the name of the Python module as first argument and the API type REST or GRPC as second argument, for example if you have a file IrisClassifier.py in the current folder you could run:
seldon-core-microservice IrisClassifier REST
To get full details about this tool run seldon-core-microservice --help.
Next either use the Microservce API tester or testdirectly via curl.
Microservice API Tester¶
Use the seldon-core-microservice-tester script to test a packaged Docker microservice Seldon component.
usage: seldon-core-microservice-tester [-h] [--endpoint {predict,send-feedback}]
[-b BATCH_SIZE] [-n N_REQUESTS] [--grpc] [--fbs]
[-t] [-p]
contract host port
positional arguments:
contract File that contains the data contract
host
port
optional arguments:
-h, --help show this help message and exit
--endpoint {predict,send-feedback}
-b BATCH_SIZE, --batch-size BATCH_SIZE
-n N_REQUESTS, --n-requests N_REQUESTS
--grpc
--fbs
-t, --tensor
-p, --prnt Prints requests and responses
Example:
seldon-core-microservice-tester contract.json 0.0.0.0 5000 -p --grpc
The above sends a predict call to a gRPC component exposed at 0.0.0.0:5000 using the contract.json to create a random request.
You can find more examples in the example notebooks list.
To understand the format of the contract.json see details below.
Microservice API Test via Curl¶
You can also test your component if run via Docker or from the command line via curl. An example for Iris Classifier might be:
curl -g http://localhost:5000/predict --data-urlencode 'json={"data": {"names": ["sepal_length", "sepal_width", "petal_length", "petal_width"], "ndarray": [[7.233, 4.652, 7.39, 0.324]]}}'
Seldon-Core API Tester for the External API¶
Use the seldon-core-api-tester script to test a Seldon graph deployed to a kubernetes cluster.
usage: seldon-core-api-tester [-h] [--endpoint {predict,send-feedback}]
[-b BATCH_SIZE] [-n N_REQUESTS] [--grpc] [-t]
[-p] [--log-level {DEBUG,INFO,ERROR}]
[--namespace NAMESPACE]
[--oauth-port OAUTH_PORT]
[--oauth-key OAUTH_KEY]
[--oauth-secret OAUTH_SECRET]
contract host port [deployment]
positional arguments:
contract File that contains the data contract
host
port
deployment
optional arguments:
-h, --help show this help message and exit
--endpoint {predict,send-feedback}
-b BATCH_SIZE, --batch-size BATCH_SIZE
-n N_REQUESTS, --n-requests N_REQUESTS
--grpc
-t, --tensor
-p, --prnt Prints requests and responses
--log-level {DEBUG,INFO,ERROR}
--namespace NAMESPACE
--oauth-port OAUTH_PORT
--oauth-key OAUTH_KEY
--oauth-secret OAUTH_SECRET
Example:
seldon-core-api-tester contract.json 0.0.0.0 8003 --oauth-key oauth-key --oauth-secret oauth-secret -p --grpc --oauth-port 8002 --endpoint send-feedback
The above sends a gRPC send-feedback request to 0.0.0.0:8003 using the given oauth key/secret (assumes you are using the Seldon API Gateway) with the REST oauth-port at 8002 and use the contract.json file to create a random request. In this example you would have port-forwarded the Seldon api-server to local ports.
You can find more examples in the example notebooks list.
To understand the format of the contract.json see details below.
API Contract¶
Both tester scripts require you to provide a contract.json file defining the data you intend to send in a request and the response you expect back.
An example for the example Iris classification model is shown below:
{
"features":[
{
"name":"sepal_length",
"dtype":"FLOAT",
"ftype":"continuous",
"range":[4,8]
},
{
"name":"sepal_width",
"dtype":"FLOAT",
"ftype":"continuous",
"range":[2,5]
},
{
"name":"petal_length",
"dtype":"FLOAT",
"ftype":"continuous",
"range":[1,10]
},
{
"name":"petal_width",
"dtype":"FLOAT",
"ftype":"continuous",
"range":[0,3]
}
],
"targets":[
{
"name":"class",
"dtype":"FLOAT",
"ftype":"continuous",
"range":[0,1],
"repeat":3
}
]
}
Here we have 4 input features each of which is continuous in certain ranges. The response targets will be a repeated set of floats in the 0-1 range.
Definition¶
There are two sections:
features: The types of the feature array that will be in the requesttargets: The types of the feature array that will be in the response
Each section has a list of definitions. Each definition consists of:
name: String : The name of the featureftype: one of CONTINUOUS, CATEGORICAL : the type of the featuredtype: One of FLOAT, INT : Required for ftype CONTINUOUS : What type of feature to createvalues: list of Strings : Required for ftype CATEGORICAL : The possible categorical valuesrange: list of two numbers : Optional for ftype CONTINUOUS : The range of values (inclusive) that a continuous value can takerepeat: integer : Optional value for how many times to repeat this valueshape: array of integers : Optional value for the shape of array to coerce the values