Reference / binference.validation.SBCResult

binference.validation.SBCResult

class binference.validation.SBCResult(ranks, n_draws, names, min_ess=<factory>)

Ranks from simulation-based calibration.

Attributes:

ranks(n_simulations, n_quantities) int array

Number of posterior draws below the true value.

n_drawsint

Draws per simulation used for ranking (ranks lie in 0..n_draws).

nameslist of str

Quantity names ("rho[k]", "lambda[k]" for rates or "w[k]" for the component weights of a Basis, and "loglike").

min_ess(n_simulations,) array

Smallest bulk effective sample size of each fit (NaN without chains); sbc() warns when it falls below n_draws.

__init__(ranks, n_draws, names, min_ess=<factory>)

Methods

__init__(ranks, n_draws, names[, min_ess])
expected([n_bins])	Expected count per histogram bin under uniform ranks, (n_bins,).
histogram([n_bins])	Rank histograms, shape (n_quantities, n_bins).
passed([alpha, n_bins, method])	True if no quantity rejects uniformity at Bonferroni-corrected level alpha.
plot(**kwargs)
pvalues([n_bins, method])	P-values of rank uniformity, one per quantity.

Attributes

ranks
n_draws
names
min_ess

expected(n_bins=None)

Expected count per histogram bin under uniform ranks, (n_bins,).

Bins hold different numbers of the n_draws + 1 possible ranks unless n_bins divides n_draws + 1 (audit G-1).

Parameters:

n_binsint, optional

Histogram bins; None takes the smallest of SBC_MAX_BINS, n_simulations // SBC_SIMULATIONS_PER_BIN and n_draws + 1, at least 2.

histogram(n_bins=None)

Rank histograms, shape (n_quantities, n_bins).

Parameters:

n_binsint, optional

Histogram bins; None takes the smallest of SBC_MAX_BINS, n_simulations // SBC_SIMULATIONS_PER_BIN and n_draws + 1, at least 2.

passed(alpha=0.01, n_bins=None, *, method='chisquare')

True if no quantity rejects uniformity at Bonferroni-corrected level alpha.

Each p-value from pvalues() must lie above alpha / n_quantities.

Parameters:

alphafloat, optional

Significance level, Bonferroni-corrected over the quantities.

n_binsint, optional

Histogram bins for method="chisquare", as for pvalues().

method{“chisquare”, “ecdf”}, optional

Chi-square test of the rank histogram or ECDF test of the ranks.

pvalues(n_bins=None, *, method='chisquare')

P-values of rank uniformity, one per quantity.

Parameters:

n_binsint, optional

Histogram bins for method="chisquare"; None as for histogram().

method{“chisquare”, “ecdf”}, optional

Chi-square test of the rank histogram or ECDF test of the ranks. "chisquare": Pearson’s chi-square test of the rank histogram. "ecdf": the ECDF test of Säilynoja, Bürkner & Vehtari (2022). With \(S\) simulations and \(M\) = n_draws, the count \(C_j\) of ranks at most \(j\) is \(\mathrm{Binomial}(S, (j+1)/(M+1))\) under uniformity. The statistic is the smallest two-sided binomial tail probability

\[\hat\gamma = \min_j 2\min\{P(C \le C_j), P(C \ge C_j)\}\]

over \(j = 0, \dots, M-1\), and the p-value is \(P(\gamma \le \hat\gamma)\) for uniform ranks, computed exactly from the joint (multinomial) law of the counts: the smallest \(\alpha\) for which the rank ECDF leaves the simultaneous \(1 - \alpha\) band. It takes no n_bins.

Returns:

(n_quantities,) array