alibi_detect.cd.preprocess module
- alibi_detect.cd.preprocess.classifier_uncertainty(x, model_fn, preds_type='probs', uncertainty_type='entropy', margin_width=0.1)[source]
Evaluate model_fn on x and transform predictions to prediction uncertainties.
- Parameters:
model_fn (
Callable) – Function that evaluates a classification model on x in a single call (contains batching logic if necessary).preds_type (
str) – Type of prediction output by the model. Options are ‘probs’ (in [0,1]) or ‘logits’ (in [-inf,inf]).uncertainty_type (
str) – Method for determining the model’s uncertainty for a given instance. Options are ‘entropy’ or ‘margin’.margin_width (
float) – Width of the margin if uncertainty_type = ‘margin’. The model is considered uncertain on an instance if the highest two class probabilities it assigns to the instance differ by less than margin_width.
- Return type:
ndarray- Returns:
A scalar indication of uncertainty of the model on each instance in x.
- alibi_detect.cd.preprocess.regressor_uncertainty(x, model_fn, uncertainty_type='mc_dropout', n_evals=25)[source]
Evaluate model_fn on x and transform predictions to prediction uncertainties.
- Parameters:
model_fn (
Callable) – Function that evaluates a regression model on x in a single call (contains batching logic if necessary).uncertainty_type (
str) – Method for determining the model’s uncertainty for a given instance. Options are ‘mc_dropout’ or ‘ensemble’. The former should output a scalar per instance. The latter should output a vector of predictions per instance.n_evals (
int) – The number of times to evaluate the model under different dropout configurations. Only relavent when using the ‘mc_dropout’ uncertainty type.
- Return type:
ndarray- Returns:
A scalar indication of uncertainty of the model on each instance in x.