Statistics

Generic Statistic

class fairret.statistic.base.Statistic

Bases: abc.ABC, torch.nn.modules.module.Module

Abstract base class for a statistic.

As a subclass of torch.nn.Module, it should implement the forward() method with the forward(self, pred, sens, *stat_args, **stat_kwargs)() signature.

abstract forward(pred, sens, *stat_args, **stat_kwargs)

Compute the statistic for a batch of N samples for each sensitive feature.

Parameters:

• pred (torch.Tensor) – Predictions of shape \((N, 1)\), as we assume to be performing binary classification or regression.

• sens (torch.Tensor) – Sensitive features of shape \((N, S)\) with S the number of sensitive features.

• *stat_args (Any) – Any further arguments used to compute the statistic.

• **stat_kwargs (Any) – Any keyword arguments used to compute the statistic.

Returns:

Shape \((S)\).

Return type:

torch.Tensor

Linear-fractional Statistics

class fairret.statistic.linear_fractional.LinearFractionalStatistic

Bases: fairret.statistic.base.Statistic

Absract base class for a linear-fractional Statistic. This is a Statistic that is computed as the ratio between two linear functions over the predictions.

A LinearFractionalStatistic is constructed in a canonical form by defining the intercept and slope of the numerator and denominator linear functions, i.e. the functions num_intercept(), num_slope(), denom_intercept(), and denom_slope(). Each subclass must implement these functions (using any signature).

The statistic is then computed as \(\frac{num\_intercept + num\_slope * pred}{denom\_intercept + denom\_slope * pred}\).

abstract num_intercept(*args, **kwargs)

Compute the intercept of the numerator in the canonical form of the linear-fractional statistic.

Parameters:

• *args (Any) – Any arguments.

• **kwargs (Any) – Any keyword arguments.

Returns:

Shape of the predictions tensor or a shape suitable for broadcasting (see https://pytorch.org/docs/stable/notes/broadcasting.html).

Return type:

torch.Tensor

abstract num_slope(*args, **kwargs)

Compute the slope of the numerator in the canonical form of the linear-fractional statistic. This will be multiplied by the prediction tensor.

Parameters:

• *args (Any) – Any arguments.

• **kwargs (Any) – Any keyword arguments.

Returns:

Shape of the predictions tensor or a shape suitable for broadcasting (see https://pytorch.org/docs/stable/notes/broadcasting.html).

Return type:

torch.Tensor

abstract denom_intercept(*args, **kwargs)

Compute the intercept of the denominator in the canonical form of the linear-fractional statistic.

Parameters:

• *args (Any) – Any arguments.

• **kwargs (Any) – Any keyword arguments.

Returns:

Shape of the predictions tensor or a shape suitable for broadcasting (see https://pytorch.org/docs/stable/notes/broadcasting.html).

Return type:

torch.Tensor

abstract denom_slope(*args, **kwargs)

Compute the slope of the denominator in the canonical form of the linear-fractional statistic. This will be multiplied by the prediction tensor.

Parameters:

• *args (Any) – Any arguments.

• **kwargs (Any) – Any keyword arguments.

Returns:

Shape of the predictions tensor or a shape suitable for broadcasting (see https://pytorch.org/docs/stable/notes/broadcasting.html).

Return type:

torch.Tensor

num(pred, sens, *stat_args, **stat_kwargs)

Intermediate function to compute the numerator of the linear-fractional statistic.

Parameters:

• pred (torch.Tensor) – Predictions of shape \((N, 1)\), as we assume to be performing binary classification or regression.

• sens (torch.Tensor | None) – Sensitive features of shape \((N, S)\) with S the number of sensitive features. If None, the statistic is computed over all samples, ignoring the sensitive feature.

• *stat_args (Any) – Any further arguments used to compute the statistic.

• **stat_kwargs (Any) – Any keyword arguments used to compute the statistic.

Returns:

Shape \((S)\).

Return type:

torch.Tensor

denom(pred, sens, *stat_args, **stat_kwargs)

Intermediate function to compute the denominator of the linear-fractional statistic.

Parameters:

• pred (torch.Tensor) – Predictions of shape \((N, 1)\), as we assume to be performing binary classification or regression.

• sens (torch.Tensor | None) – Sensitive features of shape \((N, S)\) with S the number of sensitive features. If None, the statistic is computed over all samples, ignoring the sensitive feature.

• *stat_args (Any) – Any further arguments used to compute the statistic.

• **stat_kwargs (Any) – Any keyword arguments used to compute the statistic.

Returns:

Shape \((S)\).

Return type:

torch.Tensor

forward(pred, sens, *stat_args, **stat_kwargs)

Compute the statistic for a batch of N samples for each sensitive feature.

Parameters:

• pred (torch.Tensor) – Predictions of shape \((N, 1)\), as we assume to be performing binary classification or regression.

• sens (torch.Tensor | None) – Sensitive features of shape \((N, S)\) with S the number of sensitive features.

• *stat_args (Any) – Any further arguments used to compute the statistic.

• **stat_kwargs (Any) – Any keyword arguments used to compute the statistic.

Returns:

Shape \((S)\).

Return type:

torch.Tensor

overall_statistic(pred, *stat_args, **stat_kwargs)

Compute the overall statistic for the predictions. This is the value of the statistic when the sensitive feature is ignored, i.e. None. The predictions are typically considered fair (with respect to the statistic) if and only if the statistic computed for every sensitive feature equals this value.

Parameters:

• pred (torch.Tensor) – Predictions of shape \((N, 1)\), as we assume to be performing binary classification or regression.

• *stat_args (Any) – Any further arguments used to compute the statistic.

• **stat_kwargs (Any) – Any keyword arguments used to compute the statistic.

Returns:

The overall statistic as a scalar tensor.

Return type:

torch.Tensor

fixed_constraint(fix_value, sens, *stat_args, **stat_kwargs)

Reformulate the fairness definition for this linear-fractional statistic as a linear constraint.

This is done by fixing the intended values of the statistic for every sensitive feature to a given value fix_value. The linear expressions that make up the numerator and denominator are then combined into a single linear constraint.

Parameters:

• fix_value (torch.Tensor) – The intended value of the statistic for every sensitive feature. Typically, this is the overall statistic.

• sens (torch.Tensor) – Sensitive features of shape \((N, S)\) with S the number of sensitive features.

• *stat_args (Any) – Any further arguments used to compute the statistic.

• **stat_kwargs (Any) – Any keyword arguments used to compute the statistic.

Returns:

The intercept and slope of the linear constraint defined as \(intercept + slope * pred = 0\).

Return type:

Tuple[torch.Tensor, torch.Tensor]

class fairret.statistic.linear_fractional.PositiveRate

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

PositiveRate computes the average rate at which positive predictions are made in binary classification.

Formulated as a probability, it computes \(P(\hat{Y} = 1 | S)\) for categorical sensitive features \(S\), with the predicted label \(\hat{Y}\) sampled according to the model’s (probabilistic) prediction.

The functions of its canonical form take no arguments.

num_intercept()

\(= 0\)

num_slope()

\(= 1\)

denom_intercept()

\(= 1\)

denom_slope()

\(= 0\)

expected_signature = <Signature (self) -> torch.Tensor>

class fairret.statistic.linear_fractional.TruePositiveRate

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

TruePositiveRate computes the average rate at which positives are actually predicted as positives, also known as the recall.

Formulated as a probability, it computes \(P(\hat{Y} = 1 | Y = 1, S)\) for categorical sensitive features \(S\) and only for samples where the target label \(Y\) is positive, with the predicted label \(\hat{Y}\) sampled according to the model’s (probabilistic) prediction.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

num_intercept(label)

\(= 0\)

num_slope(label)

\(= Y\)

denom_intercept(label)

\(= Y\)

denom_slope(label)

\(= 0\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.FalsePositiveRate

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

FalsePositiveRate computes the average rate at which negatives are actually predicted as positives.

Formulated as a probability, it computes \(P(\hat{Y} = 1 | Y = 0, S)\) for categorical sensitive features \(S\) and only for samples where the target label \(Y\) is negative, with the predicted label \(\hat{Y}\) sampled according to the model’s (probabilistic) prediction.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

num_intercept(label)

\(= 0\)

num_slope(label)

\(= 1 - Y\)

denom_intercept(label)

\(= 1 - Y\)

denom_slope(label)

\(= 0\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.PositivePredictiveValue

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

PositivePredictiveValue computes the average rate at which the predicted positives were actually labeled positive, also known as the precision.

Formulated as a probability, it computes \(P(Y = 1 | \hat{Y} = 1, S)\) for categorical sensitive features \(S\) and only for samples where the predicted label (sampled according to the model’s (probabilistic) prediction) is positive, with \(Y\) the target label.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

num_intercept(label)

\(= 0\)

num_slope(label)

\(= Y\)

denom_intercept(label)

\(= 0\)

denom_slope(label)

\(= 1\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.FalseOmissionRate

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

FalseOmissionRate computes the average rate at which the predicted negatives were actually labeled positive, also known as the precision.

Formulated as a probability, it computes \(P(Y = 1 | \hat{Y} = 0, S)\) for categorical sensitive features \(S\) and only for samples where the predicted label (sampled according to the model’s (probabilistic) prediction) is negative, with \(Y\) the target label.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

num_intercept(label)

\(= Y\)

num_slope(label)

\(= 1 - Y\)

denom_intercept(label)

\(= 1\)

denom_slope(label)

\(= -1\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.Accuracy

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

Accuracy computes the average rate at which predictions match the actual target labels.

Formulated as a probability, it computes \(P(\hat{Y} = Y | S)\) for categorical sensitive features \(S\), with the predicted label \(\hat{Y}\) sampled according to the model’s (probabilistic) prediction and with \(Y\) the target label.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

num_intercept(label)

\(= 1 - Y\)

num_slope(label)

\(= 2Y - 1\)

denom_intercept(label)

\(= 1\)

denom_slope(label)

\(= 0\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.FalseNegativeFalsePositiveFraction

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

FalseNegativeFalsePositiveFraction computes the ratio between false negatives and false positives.

The statistic cannot be formulated as a single probability.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

num_intercept(label)

\(= Y\)

num_slope(label)

\(= -Y\)

denom_intercept(label)

\(= 0\)

denom_slope(label)

\(= 1 - Y\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.FScore

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

FScore computes the \(F_β\)-score, the weighted mean of precision and recall. The \(F_1\)-score is the harmonic mean of precision and recall.

The statistic cannot be formulated as a single probability.

The functions of its canonical form require that the tensor of target labels is provided with the same shape as the predictions.

__init__(beta=1.)

Parameters:

beta (float) – The weight of recall in the harmonic mean. Default is 1.

num_intercept(label)

\(= 0\)

num_slope(label)

\(= 1 + β^2\)

denom_intercept(label)

\(= β^2Y\)

denom_slope(label)

\(= 1\)

expected_signature = <Signature (self, label: torch.Tensor) -> torch.Tensor>

class fairret.statistic.linear_fractional.StackedLinearFractionalStatistic

Bases: fairret.statistic.linear_fractional.LinearFractionalStatistic

A vector-valued statistic that combines the outputs of K LinearFractionalStatistic ‘s into a single statistic with output (K, S) by stacking all outputs in the second-to-last dimension (dim=-2).

__init__(*statistics)

Parameters:

*statistics (fairret.statistic.linear_fractional.LinearFractionalStatistic) – The LinearFractionalStatistic ‘s to be stacked.

num_intercept(*args, **kwargs)

Stack the numerator intercepts of all statistics.

num_slope(*args, **kwargs)

Stack the numerator slopes of all statistics.

denom_intercept(*args, **kwargs)

Stack the denominator intercepts of all statistics.

denom_slope(*args, **kwargs)

Stack the denominator slope of all statistics.

expected_signature = <Signature (self, *args: Any, **kwargs: Any) -> torch.Tensor>