alibi.confidence package
The ‘alibi.confidence’ module includes trust scores.
- class alibi.confidence.LinearityMeasure(method='grid', epsilon=0.04, nb_samples=10, res=100, alphas=None, model_type='classifier', agg='pairwise', verbose=False)[source]
Bases:
object
- __init__(method='grid', epsilon=0.04, nb_samples=10, res=100, alphas=None, model_type='classifier', agg='pairwise', verbose=False)[source]
- Parameters:
method (
str
) – Method for sampling. Supported methods:'knn'
|'grid'
.epsilon (
float
) – Size of the sampling region around the central instance as a percentage of the features range.nb_samples (
int
) – Number of samples to generate.res (
int
) – Resolution of the grid. Number of intervals in which the feature range is discretized.alphas (
Optional
[ndarray
]) – Coefficients in the superposition.agg (
str
) – Aggregation method. Supported values:'global'
|'pairwise'
.model_type (
str
) – Type of task. Supported values:'regressor'
|'classifier'
.
- class alibi.confidence.TrustScore(k_filter=10, alpha=0.0, filter_type=None, leaf_size=40, metric='euclidean', dist_filter_type='point')[source]
Bases:
object
- __init__(k_filter=10, alpha=0.0, filter_type=None, leaf_size=40, metric='euclidean', dist_filter_type='point')[source]
Initialize trust scores.
- Parameters:
k_filter (
int
) – Number of neighbors used during either kNN distance or probability filtering.alpha (
float
) – Fraction of instances to filter out to reduce impact of outliers.filter_type (
Optional
[str
]) – Filter method:'distance_knn'
|'probability_knn'
.leaf_size (
int
) – Number of points at which to switch to brute-force. Affects speed and memory required to build trees. Memory to store the tree scales with n_samples / leaf_size.metric (
str
) – Distance metric used for the tree. See sklearn DistanceMetric class for a list of available metrics.dist_filter_type (
str
) – Use either the distance to the k-nearest point (dist_filter_type = 'point'
) or the average distance from the first to the k-nearest point in the data (dist_filter_type = 'mean'
).
- filter_by_distance_knn(X)[source]
Filter out instances with low kNN density. Calculate distance to k-nearest point in the data for each instance and remove instances above a cutoff distance.
- Parameters:
X (
ndarray
) – Data.- Return type:
ndarray
- Returns:
Filtered data.
- filter_by_probability_knn(X, Y)[source]
Filter out instances with high label disagreement amongst its k nearest neighbors.
- Parameters:
X (
ndarray
) – Data.Y (
ndarray
) – Predicted class labels.
- Return type:
Tuple
[ndarray
,ndarray
]- Returns:
Filtered data and labels.
- score(X, Y, k=2, dist_type='point')[source]
Calculate trust scores = ratio of distance to closest class other than the predicted class to distance to predicted class.
- Parameters:
X (
ndarray
) – Instances to calculate trust score for.Y (
ndarray
) – Either prediction probabilities for each class or the predicted class.k (
int
) – Number of nearest neighbors used for distance calculation.dist_type (
str
) – Use either the distance to the k-nearest point (dist_type = 'point'
) or the average distance from the first to the k-nearest point in the data (dist_type = 'mean'
).
- Return type:
Tuple
[ndarray
,ndarray
]- Returns:
Batch with trust scores and the closest not predicted class.
- alibi.confidence.linearity_measure(predict_fn, x, feature_range=None, method='grid', X_train=None, epsilon=0.04, nb_samples=10, res=100, alphas=None, agg='global', model_type='classifier')[source]
Calculate the linearity measure of the model around an instance of interest x.
- Parameters:
predict_fn (
Callable
) – Predict function.x (
ndarray
) – Instance of interest.feature_range (
Union
[List
,ndarray
,None
]) – Array with min and max values for each feature.method (
str
) – Method for sampling. Supported values:'knn'
|'grid'
.X_train (
Optional
[ndarray
]) – Training set.epsilon (
float
) – Size of the sampling region as a percentage of the feature range.nb_samples (
int
) – Number of samples to generate.res (
int
) – Resolution of the grid. Number of intervals in which the features range is discretized.alphas (
Optional
[ndarray
]) – Coefficients in the superposition.agg (
str
) – Aggregation method. Supported values:'global'
|'pairwise'
.model_type (
str
) – Type of task. Supported values:'regressor'
|'classifier'
.
- Return type:
ndarray
- Returns:
Linearity measure.