Reference / binference.PopulationResult

binference.PopulationResult

class binference.PopulationResult(model, draws, *, method, kind=None, bins=None, x=None, log_density_unconstrained=None, approximate=False, info=None, labels=None, label_index=None, prior_repr=None, prior_key=None, basis=None, total=None, selection=None)

Posterior draws of a population (fractions and, for rate models, rates).

Every inference engine returns this object (MAP returns a PointEstimate). Draws are stored per chain; flattened views are available as samples (fractions) and rates.

Attributes:

methodstr

Engine that produced the draws.

approximatebool

True for approximations: VB, bootstrap, and Laplace draws that are uncorrected, fail their Pareto k-hat check, or (under method="auto") have an importance ESS below target_ess.

infodict

Engine statistics (e.g. NUTS tree depths, divergences, step sizes).

labelsndarray or None

Joint label draws stored by Gibbs, (chains, draws, objects) for the objects in label_index: bin indices, or basis-component indices when the likelihoods use a Basis; sample_assignments() returns bins either way.

__init__(model, draws, *, method, kind=None, bins=None, x=None, log_density_unconstrained=None, approximate=False, info=None, labels=None, label_index=None, prior_repr=None, prior_key=None, basis=None, total=None, selection=None)

Methods

__init__(model, draws, *, method[, kind, ...])
assignment_probabilities([idx, n_draws, chunk])	Posterior probability that each object lies in each bin, \(P(z_i = k \mid D)\).
assignment_probabilities_best([idx])	assignment_probabilities() at the best fit, best.
assignment_probabilities_loo([idx, n_draws, ...])	Each object's posterior over bins under the population fitted without it, \(P(z_i = k \mid D_{-i})\): leave-one-out, for calibration checks (DECISIONS #52).
attach(likelihoods_or_model)	Attach likelihoods (or a model) to a loaded result to enable per-object methods.
binning_bias([build, order, resolution, ...])	Check the flat-within-bin assumption: refit the same data on a B-spline basis over these bins and compare the bin masses (DECISIONS #6, #20).
check([strict, rhat_max, ess_min, ...])	Check diagnostics and warn (or raise with strict=True) about problems.
corr([which])	Posterior correlation matrix (K, K) of the fractions (or which=...).
cov([which])	Posterior covariance (K, K) of the fractions (or which="rates", ...).
density()	Draws of the population density per unit bin volume, (n_draws, K).
from_samples(model, samples, *[, method, ...])	Wrap draws from an external sampler, (n_chains, n_draws, K) or (n_draws, K): fractions for a fractions model (rows summing to one), rates for a rate model.
functional(func[, vectorized, which])	Draws of a derived quantity func(rho) (e.g. the mean redshift), or of the rates or densities with which.
interval([level, kind, which])	Per-bin credible intervals (lo, hi), each (K,).
load(path)	Load a result saved with save().
log_predictive(likelihoods[, n_draws, chunk])	Log posterior-predictive density of new objects, \(\log E[L_i\cdot\rho]\).
log_predictive_best(likelihoods, **kwargs)	log_predictive() at the best fit, best.
plot(**kwargs)	Plot the population posterior (see binference.plotting.plot_population()).
plot_corner([idx])	Corner plot of selected bins (see binference.plotting.plot_corner()).
plot_individual(i, **kwargs)	Object i's own likelihood against its hierarchical posterior (see binference.plotting.plot_individual()).
plot_trace([idx])	Trace plots of selected bins (see binference.plotting.plot_trace()).
predict_proba(likelihoods[, n_draws, chunk])	Posterior over bins for new objects under the inferred population.
predict_proba_best(likelihoods, **kwargs)	predict_proba() at the best fit, best.
prior_information_ratio([h])	Efron's (2016) ratio of prior to data information, \(\mathrm{tr}(-H_\text{prior}) / \mathrm{tr}(-H_\text{data})\).
quantile(q[, which])	Posterior quantiles, shape (*np.shape(q), K).
realized_counts([n_draws, rng, full])	Draws of the realised histogram of the observed objects' true bins.
realized_counts_best(**kwargs)	realized_counts() at the best fit, best.
rebin(groups[, which])	Fractions (or rates, with which="rates") aggregated into coarser groups.
sample_assignments([idx, n_draws, rng])	Joint draws of the bin labels of objects idx (all by default).
sample_assignments_best([idx])	sample_assignments() at the best fit, best.
sample_values([idx, n_draws, rng, loglike, ...])	Joint draws of the continuous true values of objects idx.
sample_values_best([idx])	sample_values() at the best fit, best.
save(path)	Save draws, metadata and bins to a compressed .npz file.
summary([level, which])	Per-bin posterior summary table (mean, sd, median, interval, ESS, R-hat).
to_datatree()	Convert to an xarray.DataTree (ArviZ >= 1.0 layout; needs xarray).
to_dict()	Plain-dictionary representation (arrays and metadata).

Attributes

K
best	The best fit: MAP with its defaults, which is the mode of \(\rho\) on the simplex for conjugate priors whose density is bounded there (e.g. Dirichlet concentrations of at least one), and the mode in the unconstrained coordinates otherwise, and always with a jointly inferred selection or response (DECISIONS #15, #35).
chains	Fraction draws per chain, shape (n_chains, n_per_chain, K).
component_draws	Draws of the model's parameters, (n_chains, n_per_chain, C): the component weights with a Basis, else the same as the per-bin draws.
data_support	Objects' worth of likelihood in each bin, \(\sum_i w_i \ell_{ik}\) with each object's likelihood normalised over the bins: below about one, a bin's fraction is set by the prior (DECISIONS #41).
diagnostics	Convergence diagnostics (computed lazily and cached).
mean	Posterior mean fractions (K,).
median	Posterior median fractions (K,).
n_chains
n_draws	Total number of draws (over all chains).
rates	Rate draws (n_draws, K): \(\lambda = \Lambda\rho\) per draw for fractions (see total), the sampled rates for rate models; None when the total is not available.
samples	Population fractions, shape (n_draws, K) (chains concatenated).
selected	Draws (n_draws, K) of the selected population \(\phi = S\circ\rho / (S\cdot\rho)\), the bins of the observed objects; \(\rho\) itself without a selection (DECISIONS #49).
selection	The selection function of the fit, kept through save() and load().
std	Posterior standard deviation of the fractions (K,).
total	Draws of the total rate \(\Lambda\) (n_draws,), or None.
basis	the Basis of the model's components, or None
log_density_unconstrained	log posterior density of each draw in the unconstrained coordinates (with the Jacobian; DECISIONS #15), for samplers that compute it
prior_key	a digest of the prior's arguments in full, which attach compares (the repr shortens arrays)
model
bins
kind
method
info

attach(likelihoods_or_model)

Attach likelihoods (or a model) to a loaded result to enable per-object methods.

Bare likelihoods take the fit’s selection; likelihoods or a model with another selection, or on other bins, are refused (audit O-1, O-2).

Parameters:

likelihoods_or_modelLikelihoods or PopulationModel

The fit’s likelihoods, or a model built on them.

basis

the Basis of the model’s components, or None

property best

The best fit: MAP with its defaults, which is the mode of \(\rho\) on the simplex for conjugate priors whose density is bounded there (e.g. Dirichlet concentrations of at least one), and the mode in the unconstrained coordinates otherwise, and always with a jointly inferred selection or response (DECISIONS #15, #35). Computed on first use and kept; per-object outputs at it carry a _best suffix (DECISIONS #51). After load(), it needs the fit’s prior: attach a model, not bare likelihoods.

binning_bias(build=None, *, order=1, resolution=1, threshold=2.0, prior=None, method=None, n_samples=None, rng=None, **engine_kwargs)

Check the flat-within-bin assumption: refit the same data on a B-spline basis over these bins and compare the bin masses (DECISIONS #6, #20).

build(bins_or_basis) returns the likelihoods of the fit’s data on what it is given, e.g. lambda b: Likelihoods.from_gaussian(mean, sigma, b); likelihoods from Likelihoods.from_gaussian() rebuild themselves without it. The reference is fitted on Basis(self.bins, order, resolution) with this fit’s prior (or prior), selection, engine (or method), chains and draws (or n_samples), and a Gibbs fit’s warm-up and thinning when the engine is the same; engine_kwargs go to the engine.

The fit’s selection keeps only its bin averages, so the reference sees it flat within bins; to test a selection’s shape within bins as well, attach it to the likelihoods build returns, which then take precedence (a selection from injections must be attached that way, fixed).

Returns a table per bin: shift, the reference’s mean fraction minus this fit’s; sd, this fit’s posterior s.d.; z = shift / sd (NaN where the fit has no spread); and, where \(|z|\) exceeds threshold, n_crit \(= W/z^2\), the number of objects at which the shift equals the statistical error (which falls as \(1/\sqrt{W}\)): from a pilot fit, the size above which these bins bias the result (inf elsewhere). Warns when \(|z|\) exceeds threshold in any bin: two models of the same data differ by up to about one s.d. even when the population is flat within bins (DECISIONS #86), so the default is 2.

Parameters:

buildcallable, optional

Function that rebuilds the likelihoods on a basis; None works for Gaussian inputs.

orderint, optional

B-spline order of the reference fit’s basis.

resolutionint, optional

Components per bin of the reference fit’s basis.

thresholdfloat, optional

Shift, in posterior standard deviations, above which a bin is flagged.

priorPopulationPrior, optional

Prior of the reference fit; None reuses this fit’s.

methodstr, optional

Engine of the reference fit; None reuses this fit’s.

n_samplesint, optional

Draws of the reference fit; None reuses this fit’s, except that method="auto" sets its own run length.

rngint, Generator or None

Random number generator.

**engine_kwargs

Options for the reference fit’s engine.

property chains

Fraction draws per chain, shape (n_chains, n_per_chain, K).

check(strict=False, rhat_max=1.01, ess_min=100.0, predictive_p_min=0.001, predictive_draws=200, support_min=None)

Check diagnostics and warn (or raise with strict=True) about problems.

Sampler problems raise a ConvergenceWarning; problems with the inputs (Monte Carlo noise of sample likelihoods, binned counts the response does not describe) an InputWarning. For binned inputs with more measured than truth bins, the counts are tested with Response.predictive_pvalue() over predictive_draws draws, and flagged below predictive_p_min. With support_min, bins whose data_support is below it (in objects) are flagged as set by the prior.

Returns a list of messages (empty if everything looks fine).

Parameters:

strictbool, optional

Raise DiagnosticsError instead of warning.

rhat_maxfloat, optional

Largest R-hat that passes.

ess_minfloat, optional

Smallest bulk or tail ESS per chain that passes.

predictive_p_minfloat, optional

Smallest posterior predictive p-value of binned counts that passes.

predictive_drawsint, optional

Draws used for the predictive p-value.

support_minfloat, optional

Flag bins with less data support than this; None skips the test.

property component_draws

Draws of the model’s parameters, (n_chains, n_per_chain, C): the component weights with a Basis, else the same as the per-bin draws.

corr(which='fractions')

Posterior correlation matrix (K, K) of the fractions (or which=...).

Parameters:

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

cov(which='fractions')

Posterior covariance (K, K) of the fractions (or which="rates", …).

Parameters:

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

density()

Draws of the population density per unit bin volume, (n_draws, K).

Fractions divided by bin volumes (rates divided by volumes for rate models): the quantity to plot as n(z).

property diagnostics

Convergence diagnostics (computed lazily and cached).

classmethod from_samples(model, samples, *, method='external', approximate=False, info=None)

Wrap draws from an external sampler, (n_chains, n_draws, K) or (n_draws, K): fractions for a fractions model (rows summing to one), rates for a rate model.

A model that infers its selection or response with the population needs their draws too, as info["selection_draws"] or info["response_draws"] (the layout the engines store).

Parameters:

modelPopulationModel

The model the draws are of.

samplesarray_like (n_chains, n_draws, K) or (n_draws, K)

The draws.

methodstr, optional

Engine name recorded in the result.

approximatebool, optional

Flag the draws as approximate.

infodict, optional

Extra information stored with the draws.

functional(func, vectorized=False, which='fractions')

Draws of a derived quantity func(rho) (e.g. the mean redshift), or of the rates or densities with which.

Parameters:

funccallable

Function of one draw (K,).

vectorizedbool, optional

Pass all draws to func at once, as an (n_draws, K) array.

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

Examples

>>> zbar = result.functional(lambda r: r @ result.bins.centers)

interval(level=0.68, kind='central', which='fractions')

Per-bin credible intervals (lo, hi), each (K,).

Parameters:

levelfloat, optional

Probability inside each interval.

kind{“central”, “hdi”}, optional

Equal-tailed ("central") or highest-density ("hdi") intervals.

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

classmethod load(path)

Load a result saved with save().

Re-attach() the likelihoods to use the per-object methods.

Parameters:

pathstr

File written by save().

log_density_unconstrained

log posterior density of each draw in the unconstrained coordinates (with the Jacobian; DECISIONS #15), for samplers that compute it

property mean

Posterior mean fractions (K,).

property median

Posterior median fractions (K,).

property n_draws

Total number of draws (over all chains).

plot(**kwargs)

Plot the population posterior (see binference.plotting.plot_population()).

Parameters:

**kwargs

Passed to binference.plotting.plot_population().

plot_corner(idx=None, **kwargs)

Corner plot of selected bins (see binference.plotting.plot_corner()).

Parameters:

idxsequence of int, optional

Bins to show; None chooses the most populated.

**kwargs

Further arguments of plot_corner().

plot_trace(idx=None, **kwargs)

Trace plots of selected bins (see binference.plotting.plot_trace()).

Parameters:

idxsequence of int, optional

Bins to show; None chooses the most populated.

**kwargs

Further arguments of plot_trace().

prior_key

a digest of the prior’s arguments in full, which attach compares (the repr shortens arrays)

quantile(q, which='fractions')

Posterior quantiles, shape (*np.shape(q), K).

Parameters:

qfloat or array_like

Quantiles, in [0, 1].

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

property rates

Rate draws (n_draws, K): \(\lambda = \Lambda\rho\) per draw for fractions (see total), the sampled rates for rate models; None when the total is not available.

rebin(groups, which='fractions')

Fractions (or rates, with which="rates") aggregated into coarser groups.

groups is either a length-K array of integers mapping bins to groups (e.g. from Bins.coarsen()) or a sequence of lists of bin indices. Returns (n_draws, n_groups).

Parameters:

groupsarray_like (K,) or sequence of lists

Group of each bin, or the bins of each group.

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

property samples

Population fractions, shape (n_draws, K) (chains concatenated).

save(path)

Save draws, metadata and bins to a compressed .npz file.

Parameters:

pathstr

File to write.

property selected

Draws (n_draws, K) of the selected population \(\phi = S\circ\rho / (S\cdot\rho)\), the bins of the observed objects; \(\rho\) itself without a selection (DECISIONS #49).

property selection

The selection function of the fit, kept through save() and load().

property std

Posterior standard deviation of the fractions (K,).

summary(level=0.68, which='fractions')

Per-bin posterior summary table (mean, sd, median, interval, ESS, R-hat).

Parameters:

levelfloat, optional

Probability inside the central intervals shown.

which{“fractions”, “rates”, “density”, “selected”}, optional

Quantity used: fractions, rates, densities or the selected population.

to_datatree()

Convert to an xarray.DataTree (ArviZ >= 1.0 layout; needs xarray).

to_dict()

Plain-dictionary representation (arrays and metadata).

property total

Draws of the total rate \(\Lambda\) (n_draws,), or None.

For fractions, each draw is \(\Lambda \sim \mathrm{Gamma}(W, S\cdot\rho)\) given its \(\rho\): the exact posterior of the total under the prior \(p(\Lambda) \propto 1/\Lambda\), with \(W\) the number of objects and \(S\) the detection probabilities or exposures (one without a selection). None when the selection is only relative (DECISIONS #9, #46, #78) or the result was loaded without its total. For rate models, the sum of the rates.

The Gamma variates come from a stream seeded by the fraction draws themselves, so the totals are reproducible for any result (also wrapped external draws, and loaded results after attach()), independent across different draws, and leave the caller’s generator untouched.