Reference / binference.Response

binference.Response

class binference.Response(matrix, truth_bins, measured_bins=None, *, efficiency=None, open_ended=False)

The probability of each measured bin, and of being observed, given the true bin.

A response tallied from events (from_counts() and the constructors built on it) records the events generated per true bin, generated, which measure how well each column is known. It is uncertain and is inferred jointly with the population; fixed() takes it as known.

Parameters:

matrixarray_like (J, K)

The response matrix,

\[R_{jk} = P(\text{measured } j, \text{observed} \mid \text{true } k).\]

Each column sums to the efficiency of its true bin, at most one. With efficiency it is instead a migration matrix \(P(\text{measured } j \mid \text{true } k, \text{observed})\) whose columns sum to one.

truth_binsBins

The bins of the true value, with their real geometry (K bins).

measured_binsBins, optional

The bins of the measured quantity (J bins); Bins.categories(J) by default.

efficiencyarray_like (K,), optional

Probability that an object in each true bin is observed; multiplies the columns of a migration matrix.

open_endedbool

Whether the outer two of 1-D measured_bins also take every value below or above them (index()).

__init__(matrix, truth_bins, measured_bins=None, *, efficiency=None, open_ended=False)

Methods

__init__(matrix, truth_bins[, ...])
draw(n[, rng])	(n, J, K) posterior draws of the matrix given the tallies alone.
fixed()	This response taken as known at its tallied matrix, a plug-in that ignores the tallies' uncertainty in the fit (the predictive check still uses them).
from_counts(counts, generated, truth_bins[, ...])	A response tallied from events: counts[j, k] of generated[k] measured in j.
from_labelled(scores, labels[, ...])	Tally the response of a classifier from a labelled validation set.
from_simulation(true_values, ...[, weights, ...])	Tally the response from simulated events.
index(values)	Measured bin of each value (-1 outside the measured bins, or for NaN).
measured_fractions(rho)	Expected share of observed objects in each measured bin, for true fractions rho.
predictive_pvalue(counts, rho_draws[, ...])	Posterior predictive p-value of counts per measured bin, over draws of the fractions.

Attributes

generated
measured_bins
open_ended
origin
prior
truth_bins
counts	Tallied events per measured and true bin (J, K), or None for a given matrix.
efficiency	Probability that an object in each true bin is observed (column sums).
matrix	The response \(R_{jk}\), shape (J, K).
uncertain	Whether the response is tallied from events and inferred with the population.

property counts

Tallied events per measured and true bin (J, K), or None for a given matrix.

draw(n, rng=None)

(n, J, K) posterior draws of the matrix given the tallies alone.

Parameters:

nint

Number of draws.

rngint, numpy.random.Generator, or None

Random number generator.

property efficiency

Probability that an object in each true bin is observed (column sums).

fixed()

This response taken as known at its tallied matrix, a plug-in that ignores the tallies’ uncertainty in the fit (the predictive check still uses them).

classmethod from_counts(counts, generated, truth_bins, measured_bins=None, *, open_ended=False, origin='counts', prior=1.0)

A response tallied from events: counts[j, k] of generated[k] measured in j.

Parameters:

countsarray_like (J, K)

Events generated in true bin k and observed in measured bin j.

generatedarray_like (K,)

Events generated in each true bin, observed or not.

truth_binsBins

The bins of the true value, as for Response.

measured_binsBins, optional

The bins of the measured quantity, as for Response.

open_endedbool, optional

Whether the outer measured bins also take every value beyond them, as for Response.

originstr, optional

Where the tallies come from, as PopulationResult.summary() reports it.

priorfloat, optional

Concentration \(\beta\) of the Dirichlet prior on each cell of a column (the J measured bins and the lost share). Column \(k\) has the posterior \(\mathrm{Dir}(\beta + \text{counts}_{\cdot k},\ \beta + \text{lost}_k)\). The default 1 is uniform; below 1, cells with no events give NUTS long tails (DECISIONS #81).

classmethod from_labelled(scores, labels, measured_bins=None, *, classes=None, n_bins=10, prior=1.0)

Tally the response of a classifier from a labelled validation set.

Under label shift (the score distribution of each class is the same in the validation and the target set) the likelihood of a score given the class is valid for any score, calibrated or not (DECISIONS #29, route C).

Parameters:

scoresarray_like (n,)

A real-valued score per validation object (for example the classifier’s probability of one class), or a predicted label.

labelsarray_like (n,)

True class of each validation object.

measured_binsBins, optional

Bins of the score. By default real-valued scores get n_bins bins holding equal numbers of validation objects, the outer two open-ended; predicted labels (integers or strings) get one category each, which gives the confusion matrix.

classessequence, optional

The classes, in order (default: the sorted distinct labels).

n_binsint

Number of equal-count score bins when measured_bins is not given.

priorfloat, optional

Concentration of the Dirichlet prior on each cell of a tallied column, as for from_counts().

classmethod from_simulation(true_values, measured_values, truth_bins, measured_bins, *, weights=None, open_ended=False, origin='simulation', prior=1.0)

Tally the response from simulated events.

Events whose true value lies outside truth_bins are left out; a warning says how many of them were observed, since measurements coming from outside the truth range are not modelled.

Parameters:

true_valuesarray_like

True value of every generated event, observed or not.

measured_valuesarray_like

Measured value of each event, aligned with true_values; NaN (or, unless open_ended, a value outside measured_bins) marks an event that was not observed.

truth_binsBins

The bins of the true value, as for Response.

measured_binsBins

The bins of the measured quantity, as for Response.

open_endedbool, optional

Whether the outer measured bins also take every value beyond them, as for Response.

weightsarray_like, optional

Simulation weight of each event. The tallies are then expressed in effective events, \((\sum w)^2/\sum w^2\) per true bin, so that they measure how well each column is known whatever the scale of the weights.

originstr, optional

Where the tallies come from, as PopulationResult.summary() reports it.

priorfloat, optional

Concentration of the Dirichlet prior on each cell of a tallied column, as for from_counts().

index(values)

Measured bin of each value (-1 outside the measured bins, or for NaN).

Parameters:

valuesarray_like

Measured values.

property matrix

The response \(R_{jk}\), shape (J, K).

measured_fractions(rho)

Expected share of observed objects in each measured bin, for true fractions rho.

Parameters:

rhoarray_like (K,)

True fractions.

predictive_pvalue(counts, rho_draws, response_draws=None)

Posterior predictive p-value of counts per measured bin, over draws of the fractions.

For each draw, the discrepancy \(D = d^\top \Sigma^{-} d\) of the counts from those the draw predicts, \(d = n - N q\), under the multinomial covariance of the counts plus, for a tallied response, that of its estimated columns; the p-value is the mean of \(P(\chi^2_{J-1} \ge D)\) over draws (DECISIONS #68). A small value means the response does not describe the counts; for a classifier, that label shift is violated. With response_draws (n_draws, J, K), drawn with the fractions when the response is inferred jointly, each draw uses its own response and the tallies’ covariance is left out (DECISIONS #81).

Parameters:

countsarray_like (J,)

Observed objects per measured bin.

rho_drawsarray_like (n_draws, K)

Draws of the fractions.

response_drawsarray_like (n_draws, J, K), optional

Draws of an inferred response, one per draw of the fractions.

property uncertain

Whether the response is tallied from events and inferred with the population.