alibi_detect.utils.tensorflow package

class alibi_detect.utils.tensorflow.DeepKernel(proj, kernel_a='rbf', kernel_b='rbf', eps='trainable')

Bases: Model

Computes similarities as k(x,y) = (1-eps)*k_a(proj(x), proj(y)) + eps*k_b(x,y). A forward pass takes a batch of instances x [Nx, features] and y [Ny, features] and returns the kernel matrix [Nx, Ny].

Parameters:

• proj (Model) – The projection to be applied to the inputs before applying kernel_a

• kernel_a (Union[Model, str]) – The kernel to apply to the projected inputs. Defaults to a Gaussian RBF with trainable bandwidth.

• kernel_b (Union[Model, str, None]) – The kernel to apply to the raw inputs. Defaults to a Gaussian RBF with trainable bandwidth. Set to None in order to use only the deep component (i.e. eps=0).

• eps (Union[float, str]) – The proportion (in [0,1]) of weight to assign to the kernel applied to raw inputs. This can be either specified or set to ‘trainable’. Only relavent is kernel_b is not None.

call(x, y)

Return type:

Tensor

property eps: tensorflow.Tensor

Return type:

Tensor

classmethod from_config(config)

get_config()

Return type:

dict

class alibi_detect.utils.tensorflow.GaussianRBF(sigma=None, init_sigma_fn=None, trainable=False)

Bases: Model

__init__(sigma=None, init_sigma_fn=None, trainable=False)

Gaussian RBF kernel: k(x,y) = exp(-(1/(2*sigma^2)||x-y||^2). A forward pass takes a batch of instances x [Nx, features] and y [Ny, features] and returns the kernel matrix [Nx, Ny].

Parameters:

• sigma (Optional[Tensor]) – Bandwidth used for the kernel. Needn’t be specified if being inferred or trained. Can pass multiple values to eval kernel with and then average.

• init_sigma_fn (Optional[Callable]) – Function used to compute the bandwidth sigma. Used when sigma is to be inferred. The function’s signature should match sigma_median(), meaning that it should take in the tensors x, y and dist and return sigma. If None, it is set to sigma_median().

• trainable (bool) – Whether or not to track gradients w.r.t. sigma to allow it to be trained.

call(x, y, infer_sigma=False)

Return type:

Tensor

classmethod from_config(config)

Instantiates a kernel from a config dictionary.

Parameters:

config – A kernel config dictionary.

get_config()

Returns a serializable config dict (excluding the input_sigma_fn, which is serialized in alibi_detect.saving).

Return type:

dict

property sigma: tensorflow.Tensor

Return type:

Tensor

class alibi_detect.utils.tensorflow.TFDataset(*indexables, batch_size=10000000000, shuffle=True)

Bases: Sequence

on_epoch_end()

Return type:

None

alibi_detect.utils.tensorflow.batch_compute_kernel_matrix(x, y, kernel, batch_size=10000000000, preprocess_fn=None)

Compute the kernel matrix between x and y by filling in blocks of size batch_size x batch_size at a time.

Parameters:

• x (Union[list, ndarray, Tensor]) – Reference set.

• y (Union[list, ndarray, Tensor]) – Test set.

• kernel (Union[Callable, Model]) – tf.keras model

• batch_size (int) – Batch size used during prediction.

• preprocess_fn (Optional[Callable]) – Optional preprocessing function for each batch.

Return type:

Tensor

Returns:

Kernel matrix in the form of a tensorflow tensor

alibi_detect.utils.tensorflow.mmd2(x, y, kernel)

Compute MMD^2 between 2 samples.

Parameters:

• x (Tensor) – Batch of instances of shape [Nx, features].

• y (Tensor) – Batch of instances of shape [Ny, features].

• kernel (Callable) – Kernel function.

Return type:

float

Returns:

MMD^2 between the samples x and y.

alibi_detect.utils.tensorflow.mmd2_from_kernel_matrix(kernel_mat, m, permute=False, zero_diag=True)

Compute maximum mean discrepancy (MMD^2) between 2 samples x and y from the full kernel matrix between the samples.

Parameters:

• kernel_mat (Tensor) – Kernel matrix between samples x and y.

• m (int) – Number of instances in y.

• permute (bool) – Whether to permute the row indices. Used for permutation tests.

• zero_diag (bool) – Whether to zero out the diagonal of the kernel matrix.

Return type:

Tensor

Returns:

MMD^2 between the samples from the kernel matrix.

alibi_detect.utils.tensorflow.mutate_categorical(X, rate=None, seed=0, feature_range=(0, 255))

Randomly change integer feature values to values within a set range with a specified permutation rate.

Parameters:

• X (ndarray) – Batch of data to be perturbed.

• rate (Optional[float]) – Permutation rate (between 0 and 1).

• seed (int) – Random seed.

• feature_range (tuple) – Min and max range for perturbed features.

Return type:

Tensor

Returns:

Array with perturbed data.

alibi_detect.utils.tensorflow.permed_lsdds(k_all_c, x_perms, y_perms, H, H_lam_inv=None, lam_rd_max=0.2, return_unpermed=False)

Compute LSDD estimates from kernel matrix across various ref and test window samples

Parameters:

• k_all_c (Tensor) – Kernel matrix of similarities between all samples and the kernel centers.

• x_perms (List[Tensor]) – List of B reference window index vectors

• y_perms (List[Tensor]) – List of B test window index vectors

• H (Tensor) – Special (scaled) kernel matrix of similarities between kernel centers

• H_lam_inv (Optional[Tensor]) – Function of H corresponding to a particular regulariation parameter lambda. See Eqn 11 of Bu et al. (2017)

• lam_rd_max (float) – The maximum relative difference between two estimates of LSDD that the regularization parameter lambda is allowed to cause. Defaults to 0.2. Only relavent if H_lam_inv is not supplied.

• return_unpermed (bool) – Whether or not to return value corresponding to unpermed order defined by k_all_c

Return type:

Union[Tuple[Tensor, Tensor], Tuple[Tensor, Tensor, Tensor]]

Returns:

Vector of B LSDD estimates for each permutation, H_lam_inv which may have been inferred, and optionally the unpermed LSDD estimate.

alibi_detect.utils.tensorflow.predict_batch(x, model, batch_size=10000000000, preprocess_fn=None, dtype=<class 'numpy.float32'>)

Make batch predictions on a model.

Parameters:

• x (Union[list, ndarray, Tensor]) – Batch of instances.

• model (Union[Callable, Model]) – tf.keras model or one of the other permitted types defined in Data.

• batch_size (int) – Batch size used during prediction.

• preprocess_fn (Optional[Callable]) – Optional preprocessing function for each batch.

• dtype (Union[Type[generic], DType]) – Model output type, e.g. np.float32 or tf.float32.

Return type:

Union[ndarray, Tensor, tuple]

Returns:

Numpy array, tensorflow tensor or tuples of those with model outputs.

alibi_detect.utils.tensorflow.predict_batch_transformer(x, model, tokenizer, max_len, batch_size=10000000000, dtype=<class 'numpy.float32'>)

Make batch predictions using a transformers tokenizer and model.

Parameters:

• x (Union[list, ndarray]) – Batch of instances.

• model (Model) – Transformer model.

• tokenizer (Callable) – Tokenizer for model.

• max_len (int) – Max token length.

• batch_size (int) – Batch size.

• dtype (Union[Type[generic], DType]) – Model output type, e.g. np.float32 or tf.float32.

Return type:

Union[ndarray, Tensor]

Returns:

Numpy array or tensorflow tensor with model outputs.

alibi_detect.utils.tensorflow.quantile(sample, p, type=7, sorted=False)

Estimate a desired quantile of a univariate distribution from a vector of samples

Parameters:

• sample (Tensor) – A 1D vector of values

• p (float) – The desired quantile in (0,1)

• type (int) – The method for computing the quantile. See https://wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample

• sorted (bool) – Whether or not the vector is already sorted into ascending order

Return type:

float

Returns:

An estimate of the quantile

alibi_detect.utils.tensorflow.relative_euclidean_distance(x, y, eps=1e-12, axis=-1)

Relative Euclidean distance.

Parameters:

• x (Tensor) – Tensor used in distance computation.

• y (Tensor) – Tensor used in distance computation.

• eps (float) – Epsilon added to denominator for numerical stability.

• axis (int) – Axis used to compute distance.

Return type:

Tensor

Returns:

Tensor with relative Euclidean distance across specified axis.

alibi_detect.utils.tensorflow.squared_pairwise_distance(x, y, a_min=1e-30, a_max=1e+30)

TensorFlow pairwise squared Euclidean distance between samples x and y.

Parameters:

• x (Tensor) – Batch of instances of shape [Nx, features].

• y (Tensor) – Batch of instances of shape [Ny, features].

• a_min (float) – Lower bound to clip distance values.

• a_max (float) – Upper bound to clip distance values.

Return type:

Tensor

Returns:

Pairwise squared Euclidean distance [Nx, Ny].

alibi_detect.utils.tensorflow.subset_matrix(mat, inds_0, inds_1)

Take a matrix and return the submatrix correspond to provided row and column indices

Parameters:

• mat (Tensor) – A 2D matrix

• inds_0 (Tensor) – A vector of row indices

• inds_1 (Tensor) – A vector of column indices

Return type:

Tensor

Returns:

A submatrix of shape (len(inds_0), len(inds_1))

alibi_detect.utils.tensorflow.zero_diag(mat)

Set the diagonal of a matrix to 0

Parameters:

mat (Tensor) – A 2D square matrix

Return type:

Tensor

Returns:

A 2D square matrix with zeros along the diagonal

Submodules

• alibi_detect.utils.tensorflow.data module
  • TFDataset
    • TFDataset.on_epoch_end()
• alibi_detect.utils.tensorflow.distance module
  • batch_compute_kernel_matrix()
  • mmd2()
  • mmd2_from_kernel_matrix()
  • permed_lsdds()
  • relative_euclidean_distance()
  • squared_pairwise_distance()
• alibi_detect.utils.tensorflow.kernels module
  • DeepKernel
    • DeepKernel.call()
    • DeepKernel.eps
    • DeepKernel.from_config()
    • DeepKernel.get_config()
  • GaussianRBF
    • GaussianRBF.__init__()
    • GaussianRBF.call()
    • GaussianRBF.from_config()
    • GaussianRBF.get_config()
    • GaussianRBF.sigma
  • sigma_median()
• alibi_detect.utils.tensorflow.misc module
  • clone_model()
  • quantile()
  • subset_matrix()
  • zero_diag()
• alibi_detect.utils.tensorflow.perturbation module
  • mutate_categorical()
• alibi_detect.utils.tensorflow.prediction module
  • predict_batch()
  • predict_batch_transformer()