Seldon makes it easy to extend the set of available algorithms. At runtime you can deploy a microservice to serve as a runtime scorer for your model for both content recommendation and general prediction scenarios using a REST interface for recommendation microservices and either REST or gRPC for prediction microservices..
Recommendation Microservices REST API
- client : client name
- user_id : user id of the user to provide recommendations
- recent_interactions : a list of item ids of recent items the user has interacted with
- exclusion_items : a list of item ids that should be excluded from scoring
- data_key : a key for memcache to get the items to score
- limit : a max number of recommendations to return
The external algorithm serving the request should return JSON with a list of items with scores with the form shown in the example below. The scores should be normalized to the range 0-1 where 1 is best.
Below is an example internal call the seldon server would make for a client test1, for user with id 1 who has recently interacted with content with item ids 16 and 260 requesting recommendations with no items that need to be excluded and a key for memcache to get the items to score which is RecentItems:test1:0:100000
Recommendation microservices can easily be created in python. The recommender should extend
seldon.Recommender and provide
save methods to store its model. A wrapper is provided to create a Flask based app given a recommender. For example if the recommender model is saved into “recommender_folder” you can start a microservice for this with:
- client : client name
- json : JSON representation of the features to score
The external algorithm serving the request should return JSON with an array of objects containing the score, classId and confidence. For example:
Below is an example internal call the seldon server would make for a client test1
Any Pipeline built using this package can easily be deployed as a microservice as shown below, where we assume a pipeline has been saved to “./pipeline” and we ish to call the loaded model “test_model”:
Seldon allows predictive microservies to be written using gRPC. An overview and worked example can be found here.
Any Pipeline built using this package can easily be deployed as a microservice using gRPC. The user needs to provide a class to convert a Seldon RPC request with custom data into a Dataframe. Below is an example client docker/examples/iris/xgboost_rpc/python/iris_rpc_client.py for the Iris prediction task which has a custom class IrisCustomDataHandler which inherits from CustomDataHandler and provides a method getData which converts the proto buffer Any type which is passed in the RPC request into a Pandas Dataframe.