alibi_detect.cd.tensorflow.spot_the_diff module

class alibi_detect.cd.tensorflow.spot_the_diff.SpotTheDiffDriftTF(x_ref, p_val=0.05, preprocess_fn=None, kernel=None, n_diffs=1, initial_diffs=None, l1_reg=0.01, binarize_preds=False, train_size=0.75, n_folds=None, retrain_from_scratch=True, seed=0, optimizer=tensorflow.keras.optimizers.Adam, learning_rate=0.001, batch_size=32, preprocess_batch_fn=None, epochs=3, verbose=0, train_kwargs=None, dataset=<class 'alibi_detect.utils.tensorflow.data.TFDataset'>, data_type=None)[source]

Bases: object

class InterpretableClf(kernel, x_ref, initial_diffs)[source]

Bases: tensorflow.keras.Model

call(x)[source]
Return type

Tensor

classmethod from_config(config)[source]
get_config()[source]
Return type

dict

__init__(x_ref, p_val=0.05, preprocess_fn=None, kernel=None, n_diffs=1, initial_diffs=None, l1_reg=0.01, binarize_preds=False, train_size=0.75, n_folds=None, retrain_from_scratch=True, seed=0, optimizer=tensorflow.keras.optimizers.Adam, learning_rate=0.001, batch_size=32, preprocess_batch_fn=None, epochs=3, verbose=0, train_kwargs=None, dataset=<class 'alibi_detect.utils.tensorflow.data.TFDataset'>, data_type=None)[source]

Classifier-based drift detector with a classifier of form y = a + b_1*k(x,w_1) + … + b_J*k(x,w_J), where k is a kernel and w_1,…,w_J are learnable test locations. If drift has occured the test locations learn to be more/less (given by sign of b_i) similar to test instances than reference instances. The test locations are regularised to be close to the average reference instance such that the difference is then interpretable as the transformation required for each feature to make the average instance more/less like a test instance than a reference instance.

The classifier is trained on a fraction of the combined reference and test data and drift is detected on the remaining data. To use all the data to detect drift, a stratified cross-validation scheme can be chosen.

Parameters
  • x_ref (ndarray) – Data used as reference distribution.

  • p_val (float) – p-value used for the significance of the test.

  • preprocess_fn (Optional[Callable]) – Function to preprocess the data before computing the data drift metrics.

  • kernel (Optional[Model]) – Differentiable TensorFlow model used to define similarity between instances, defaults to Gaussian RBF.

  • n_diffs (int) – The number of test locations to use, each corresponding to an interpretable difference.

  • initial_diffs (Optional[ndarray]) – Array used to initialise the diffs that will be learned. Defaults to Gaussian for each feature with equal variance to that of reference data.

  • l1_reg (float) – Strength of l1 regularisation to apply to the differences.

  • binarize_preds (bool) – Whether to test for discrepency on soft (e.g. probs/logits) model predictions directly with a K-S test or binarise to 0-1 prediction errors and apply a binomial test.

  • train_size (Optional[float]) – Optional fraction (float between 0 and 1) of the dataset used to train the classifier. The drift is detected on 1 - train_size. Cannot be used in combination with n_folds.

  • n_folds (Optional[int]) – Optional number of stratified folds used for training. The model preds are then calculated on all the out-of-fold instances. This allows to leverage all the reference and test data for drift detection at the expense of longer computation. If both train_size and n_folds are specified, n_folds is prioritized.

  • retrain_from_scratch (bool) – Whether the classifier should be retrained from scratch for each set of test data or whether it should instead continue training from where it left off on the previous set.

  • seed (int) – Optional random seed for fold selection.

  • optimizer (MockModule) – Optimizer used during training of the classifier.

  • learning_rate (float) – Learning rate used by optimizer.

  • batch_size (int) – Batch size used during training of the classifier.

  • preprocess_batch_fn (Optional[Callable]) – Optional batch preprocessing function. For example to convert a list of objects to a batch which can be processed by the model.

  • epochs (int) – Number of training epochs for the classifier for each (optional) fold.

  • verbose (int) – Verbosity level during the training of the classifier. 0 is silent, 1 a progress bar.

  • train_kwargs (Optional[dict]) – Optional additional kwargs when fitting the classifier.

  • dataset (Callable) – Dataset object used during training.

  • data_type (Optional[str]) – Optionally specify the data type (tabular, image or time-series). Added to metadata.

Return type

None

predict(x, return_p_val=True, return_distance=True, return_probs=True, return_model=False)[source]

Predict whether a batch of data has drifted from the reference data.

Parameters
  • x (ndarray) – Batch of instances.

  • return_p_val (bool) – Whether to return the p-value of the test.

  • return_distance (bool) – Whether to return a notion of strength of the drift. K-S test stat if binarize_preds=False, otherwise relative error reduction.

  • return_probs (bool) – Whether to return the instance level classifier probabilities for the reference and test data (0=reference data, 1=test data).

  • return_model (bool) – Whether to return the updated model trained to discriminate reference and test instances.

Return type

Dict[str, Dict[str, Union[str, int, float, Callable]]]

Returns

  • Dictionary containing ‘meta’ and ‘data’ dictionaries.

  • ’meta’ has the detector’s metadata.

  • ’data’ contains the drift prediction, the diffs used to distinguish reference from test instances,

  • and optionally the p-value, performance of the classifier relative to its expectation under the

  • no-change null, the out-of-fold classifier model prediction probabilities on the reference and test

  • data, and the trained model.