Outlier Detection in Seldon Core

Machine learning models do not extrapolate well outside of the training data distribution. In order to trust and reliably act on model predictions, it is crucial to monitor the distribution of incoming requests via different types of detectors. Outlier detectors aim to flag individual instances which do not follow the original training distribution.

Package

Version

alibi-detect

0.9.0

A worked example with using the CIFAR10 task is available. This example focuses on the serving infrastructure and discusses it in details.

The general framework shown in this example is to use the Seldon Core payload logger to pass requests to components that process them asynchronously. The results can be passed onwards to alterting systems.

Example architecture

Creating your own detector

For Alibi Detect outlier detectors that need to be trained you should

  1. Use python 3.7 as the Seldon Alibi Detect Server also runs in python 3.7.10 when it loads your detector.

  2. Follow the Alibi Detect docs for your particular desired detector.

  3. Save your detector using save_detector method and store in the object store or PVC in your cluster. We support various cloud storage solutions through our init container.

The runtime environment in our Alibi Explain Detector is locked using Poetry. See our e2e example here on how to use that definition to train your detector.