A worked step-by-step example using the Movielens 100K dataset is provided here. The dataset is in reality small enough that Spark based models are not the best solution but the small size makes it easy to run and test the steps you would use for larger datasets.
You haved added seldon-server/kubernetes/bin to you shell PATH environment variable.
The entire set of steps can be executed by running the Kubernetes deployment in kubernetes/conf/examples/ml100k/ml100k-import.json which should have been created when you followed the install and configuration steps.
Create the kubernetes job:
The job may take a few minutes to fully run. You can check its status with kubectl get jobs -l name=ml100k-import.
This job will:
Download the movielens 100k data. (N.B. make sure your Kubernetes cluster has access to the internet)
Create a new client “ml100k”, create the item meta-data schema and import user, and items.
Create a historical actions dataset from the movie ratings
Run a matrix factorization Spark job to create a model
Setup a matrix factorization runtime scorer
You can then test the recommendations by doing:
The above gets recommendations based on a recent action history for a user being movie 50 which is “Star Wars”. The result should look something like:
Below are the deatiled steps which can be found here.
Hack to ensure we have a namesever for external DNS (seems to be required for local Docker running of Kubernetes)
Download and unzip Movielens data
Convert to UTF-8 the item meta-data
Create Historical Data Files
Create item, user and action CSV files from raw data
We will use a item schema to hold the title, release data and IMDB URL of the movies, as show below
The steps to setup and import the data can be done via the Seldon CLI
Create a new ml100k client
Setup the schema using above JSON
Import the users and items as defined by CSV files
Create actions file from CSV file
Build a Recommendation Model
We will build a matrix factorization model using Spark via luigi.
Setup Runtime Scorer
We setup a runtime matrix factorization scorer that takes recent activity for users to make recommendations.