Content Recommendation Steps

concepts –> setup server –> logging –> configure data –> realtime activity –> offline model –> runtime configuration –> microservices –> recommendations

Offline Recommendation Model Creation

Seldon provides a variety of item recommendation models that can be created and makes it easy for new custom models to be added.

The current integrated models are:

Confguration is either passed on the command line to the offline jobs or set in zookeeper.

Offline Data Store

The Seldon modelling and data manipulation jobs assume a structure for the data storage. This structure allows easy integration into a production environment where models are created periodically, usually each day. The directory structure is of the form

    seldon-models/${CLIENT}/${MODEL}/${DAY}
 

e.g. for a matrix_factorization model created for client client1 on 27 Jan 2014 (unix epoch day 16461) would be

    seldon-models/client1/matrix_factorization/16461
 

You can use a network file store, AWS S3 or soon HDFS for the actual store.

The jobs that require activity data will use a start day and a number of days to collect from the filesystem the data they need. They will gather data from folders of the form:

${input-path}/${client}/actions/start-day
${input-path}/${client}/actions/start-day-1
${input-path}/${client}/actions/start-day-2
.
.
${input-path}/${client}/actions/start-day-(num-days)

For example:

/seldon-models/client1/actions/16461
/seldon-models/client1/actions/16460
/seldon-models/client1/actions/16459

The output path will be of the form:

${output-path}/${client}/${model}/start-day

For example:

s3://seldon-models/client1/matrix-factorization/16461

Configuration

Configuration is held in zookeeper as JSON in nodes of the form:

/<client>/offline/<model-name>

For example:

/clientname1/offline/similar-items

All jobs usually have a set of basic parameters they need including

An example:

{
  "inputPath":"/seldon-models",
  "outputPath":"/seldon-models",
  "startDay" : 1,
  "days" : 1,
  "itemType":-1,
  "activate":true
}