alibi.confidence package

The ‘alibi.confidence’ module includes trust scores.

alibi.confidence.linearity_measure(predict_fn, x, feature_range=None, method='grid', X_train=None, epsilon=0.04, nb_samples=10, res=100, alphas=None, agg='global', model_type='classifier')[source]

Calculate the linearity measure of the model around an instance of interest x.

Parameters
  • predict_fn (Callable) – Predict function.

  • x (ndarray) – Instance of interest.

  • feature_range (Union[List, ndarray, None]) – Array with min and max values for each feature.

  • method (str) – Method for sampling. Supported values ‘knn’ or ‘grid’.

  • X_train (Optional[ndarray]) – Training set.

  • epsilon (float) – Size of the sampling region as a percentage of the feature range.

  • nb_samples (int) – Number of samples to generate.

  • res (int) – Resolution of the grid. Number of intervals in which the features range is discretized.

  • alphas (Optional[ndarray]) – Coefficients in the superposition.

  • agg (str) – Aggregation method. Supported values ‘global’ or ‘pairwise’.

  • model_type (str) – Type of task. Supported values ‘regressor’ or ‘classifier’.

Return type

ndarray

Returns

Linearity measure

class alibi.confidence.LinearityMeasure(method='grid', epsilon=0.04, nb_samples=10, res=100, alphas=None, model_type='classifier', agg='pairwise', verbose=False)[source]

Bases: object

__init__(method='grid', epsilon=0.04, nb_samples=10, res=100, alphas=None, model_type='classifier', agg='pairwise', verbose=False)[source]
Parameters
  • method (str) – Method for sampling. Supported methods are ‘knn’ or ‘grid’.

  • epsilon (float) – Size of the sampling region around the central instance as a percentage of the features range.

  • nb_samples (int) – Number of samples to generate.

  • res (int) – Resolution of the grid. Number of intervals in which the feature range is discretized.

  • alphas (Optional[ndarray]) – Coefficients in the superposition.

  • agg (str) – Aggregation method. Supported values are ‘global’ or ‘pairwise’.

  • model_type (str) – Type of task. Supported values are ‘regressor’ or ‘classifier’.

Return type

None

fit(X_train)[source]
Parameters

X_train (ndarray) – Training set

Return type

None

Returns

None

score(predict_fn, x)[source]
Parameters
  • predict_fn (Callable) – Prediction function

  • x (ndarray) – Instance of interest

Return type

ndarray

Returns

Linearity measure

class alibi.confidence.TrustScore(k_filter=10, alpha=0.0, filter_type=None, leaf_size=40, metric='euclidean', dist_filter_type='point')[source]

Bases: object

__init__(k_filter=10, alpha=0.0, filter_type=None, leaf_size=40, metric='euclidean', dist_filter_type='point')[source]

Initialize trust scores.

Parameters
  • k_filter (int) – Number of neighbors used during either kNN distance or probability filtering.

  • alpha (float) – Fraction of instances to filter out to reduce impact of outliers.

  • filter_type (Optional[str]) – Filter method; either ‘distance_knn’ or ‘probability_knn’

  • leaf_size (int) – Number of points at which to switch to brute-force. Affects speed and memory required to build trees. Memory to store the tree scales with n_samples / leaf_size.

  • metric (str) – Distance metric used for the tree. See sklearn’s DistanceMetric class for a list of available metrics.

  • dist_filter_type (str) – Use either the distance to the k-nearest point (dist_filter_type = ‘point’) or the average distance from the first to the k-nearest point in the data (dist_filter_type = ‘mean’).

Return type

None

filter_by_distance_knn(X)[source]

Filter out instances with low kNN density. Calculate distance to k-nearest point in the data for each instance and remove instances above a cutoff distance.

Parameters

X (ndarray) – Data

Return type

ndarray

Returns

Filtered data.

filter_by_probability_knn(X, Y)[source]

Filter out instances with high label disagreement amongst its k nearest neighbors.

Parameters
  • X (ndarray) – Data

  • Y (ndarray) – Predicted class labels

Return type

Tuple[ndarray, ndarray]

Returns

Filtered data and labels.

fit(X, Y, classes=None)[source]

Build KDTrees for each prediction class.

Parameters
  • X (ndarray) – Data

  • Y (ndarray) – Target labels, either one-hot encoded or the actual class label.

  • classes (Optional[int]) – Number of prediction classes, needs to be provided if Y equals the predicted class.

Return type

None

score(X, Y, k=2, dist_type='point')[source]

Calculate trust scores = ratio of distance to closest class other than the predicted class to distance to predicted class.

Parameters
  • X (ndarray) – Instances to calculate trust score for.

  • Y (ndarray) – Either prediction probabilities for each class or the predicted class.

  • k (int) – Number of nearest neighbors used for distance calculation.

  • dist_type (str) – Use either the distance to the k-nearest point (dist_type = ‘point’) or the average distance from the first to the k-nearest point in the data (dist_type = ‘mean’).

Return type

Tuple[ndarray, ndarray]

Returns

Batch with trust scores and the closest not predicted class.