Advanced Recommender Configuration

The basic configuration for providing recommendation is discused here. However, Seldon allows much more complex scenarios to be configured which are discussed in the following sections:

Cascading Algorithms

Not all recommender algorithms may be applicable to all users. For example a matrix factorization model may only be created for users with previous actions. Therefore its good practise to have a set of algorithms that can be cascaded through until one succeeds for the current user. This can be easily handled within Seldon by provided the ordered list of algorithms in the configuration, for example:

{
    "defaultStrategy": {
        "algorithms": [
            {
                "config": [],
                "filters": [],
                "includers": [],
                "name": "mfRecommender"
            },
            {
                "config": [],
                "filters": [],
                "includers": [],
                "name": "recentItemsRecommender"
            }
        ],
        "combiner": "firstSuccessfulCombiner"
    },
    "recTagToStrategy": {}
}

In the above example, we will try the mfRecommender and if that fails to return recommendations we will return recent items.

Another scenario is that you may wish to combine the results of several recommenders. This can be accomplished by changing the combiner to one of the following choices:

You can apply the configuration to a client using the seldon-cli.

Multi Variant Tests

It is common practice when testing algorithms in live environments to run A/B tests to evaluate the success of different strategies.

The configuration should provide the algorithm settings for each variant. This includes:

A template for an A/B test is shown below, with the algorithm sections missing:

{
    "defaultStrategy": {
        "variations": [
            {
                "config": {
                    "algorithms": [],
                    "combiner": "firstSuccessfulCombiner",
                    "diversityLevel": 3
                },
                "label": "A",
                "ratio": 0.5
            },
            {
                "config": {
                    "algorithms": [],
                    "combiner": "firstSuccessfulCombiner",
                    "diversityLevel": 3
                },
                "label": "B",
                "ratio": 0.5
            }
        ]
    },
    "recTagToStrategy": {}
}

You can apply the A/B test configuration to a client using the seldon-cli.

Multivariate tests can be set up in the same manner by simply extending the number of variations and changing the ratios as appropriate. The results of impressions and clicks will appear in the ctr-alg.log in the Seldon log folder. Seldon’s codebase contains Spark algorithms to process these logs once they have been pushed to a central location by fluentd and produce CTR analytics.

API Controlled Recommendion Variants and Tests

There are many situations where you want to control the algorithms run via the API itself, for example:

These cases can be handled by passing a recommendation tag in the API call. The added parameter is recTag and it should contain a string keyword. This tag is matched against the configuration for a client and the apporpriate set of algorithms is run. If no match is found then a default set of algorihtms is run. An example configuration for this is shown in outline below:

{
    "defaultStrategy": {},
    "recTagToStrategy": {
        "category": {},
        "mobile": {},
        "sitewide": {}
    }
}    

In this example there is a default strategy in case a rec tag is not found and 3 other strategies, one for “mobile”, one for “sitewide” and one for “insection”. The config (not shown) within each of these can be an algorithms configuration or a test variant. This means quite complex configurations can be created. For example, a baseline and normal test running for in-section and sitewide recommendations but a fixed set of algorithms always for mobile. To expand this example out some more this would look like:

{
    "defaultStrategy": {
    },
    "recTagToStrategy": {
        "category": {
            "variations": [
                {
                    "config": {
                    },
                    "label": "baseline",
                    "ratio": 0.05
                },
                {
                    "config": {
                    },
                    "label": "normal",
                    "ratio": 0.95
                }
            ]
        },
        "mobile": {
	 },
        "sitewide": {
            "variations": [
                {
                    "config": {
                    },
                    "label": "baseline",
                    "ratio": 0.05
                },
                {
                    "config": {
                    },
                    "label": "normal",
                    "ratio": 0.95
                }
            ]
        }
    }
}

In the above example we have baseline algorithms running for 5% of the traffic for sitewide and insection recommendations while 95% of the traffic for these recommandation tags gets the “normal” recommendations. For mobile there is a fixed set of algorithms.

You can apply the configuration to a client using the seldon-cli.