How it works / How auto chooses

How auto chooses

How does method="auto" pick an engine, and what evidence sets its rules? This page states the rules and describes the benchmark behind them. Auto gives the defaults of their options. The engines guide covers when to override the choice.

The rules

Auto tries three engines in order and keeps the first one that passes its check. Engine options passed to infer rule out the engines that do not take them.

1. Gibbs, where it mixes well. Gibbs needs a conjugate prior and integer weights. It is chosen in two cases.

   • A selection or response is inferred with the population. Gibbs is exact and fast for these joint models. Gibbs refuses a selection smoothed across bins, which is not conjugate, so auto then tries Laplace and NUTS.

   • The missing information \(f\) at the most probable population is at most gibbs_max_missing. Auto computes that population first, with MAP.

   In the second case Gibbs runs \(\lceil n_{\rm ESS}\,(1+f)/(1-f)/n_{\rm chains}\rceil\) draws per chain, and at least its usual length. Here \(n_{\rm ESS}\) is target_ess, the smallest bulk ESS the run aims for. If that would exceed max_gibbs_draws per chain, auto skips Gibbs.

2. Laplace, if its importance check passes. Auto keeps the importance-corrected Laplace result when its Pareto k-hat passes and its importance ESS reaches target_ess. A first pass with fewer proposals stops the trial early when it projects too small an ESS (defaults).

3. NUTS otherwise.

Without n_samples, auto extends a Gibbs run until every bulk ESS reaches target_ess and the run passes the \(\hat R\) and ESS limits of check(). Each extension aims a little above target_ess and is bounded in size (defaults). If the extensions would need more than max_gibbs_draws draws per chain, auto discards the run and tries the next engine. This catches a jointly inferred selection or response that mixes much more slowly than \(f\) predicts. Gibbs may be the last candidate, because no other engine takes the options given. Auto then keeps it even if it did not mix, and the reason says so. Two kinds of run are never extended. An explicit n_samples fixes the length: auto then keeps Gibbs whatever length it would need. A run that stores labels (store_labels) keeps the length it was given.

Auto keeps Laplace even when its check fails in two cases. One is likelihoods with more than nnz_limit stored entries. The other is options that no later engine takes. Such a result is flagged approximate. The result records the choice and its reason in result.info["auto_method"] and result.info["auto_reason"]. Where auto computed the most probable population without a warning, the result keeps it as result.best.

The benchmark behind the rules

The rules come from benchmarks/matrix.py, which compares every engine with a long reference run on a matrix of mock catalogues. The numbers below are computed from its committed results; the full table is matrix.md.

Catalogues. There are 18 catalogues, each varying one factor around a baseline. The baseline has one dimension, 20 bins and 1,000 objects with Gaussian errors one bin wide. Its population is bimodal, and true values are uniform within each bin. The other catalogues are:

• width-0.3: narrow likelihoods

• width-3: broad likelihoods

• n-100: few objects

• n-10000: many objects

• k-10: few bins

• k-50: many bins

• pop-sparse: half the bins empty

• pop-spiky: half the population in one bin

• sel-truth: selection on the truth

• sel-measured: selection on the measurement

• interim-correct: posteriors, interim divided out

• interim-wrong: posteriors, wrong interim divided out

• samples-20: 20 posterior samples per object

• samples-200: 200 posterior samples per object

• grid-8x8: 2-D grid, width 1 cell

• categorical: classifier outputs, 5 classes

• rail-like: photo-z-like, K=30, N=5000

Fits. Each catalogue was fitted with the priors that apply to it, from the flat Dirichlet, random walk, α = 0.1 Dirichlet, Gaussian process and Pólya tree priors: 43 combinations in all. Each engine ran at its defaults. The reference was NUTS with 4 chains of 5,000 draws after 2,000 warm-up iterations. Errors are in reference standard deviations: the largest error of a bin’s mean, and the ratio of each bin’s standard deviation to the reference’s.

Setup. binference 0.1.0.dev0, NumPy 2.5.4 and Python 3.13.16 on x86_64, started on 26 October 2026. Each fit used one thread, with 4 fits at a time.

engine	largest mean error (reference s.d.)	s.d. ratio	smallest bulk ESS per second
NUTS (43)	0.082	0.69–1.19	0.089–339, median 80
Gibbs (26)	0.94	0.24–1.75	0.62–791, median 14
Laplace, check passed (4)	0.054	0.96–1.04	1751–6607
Laplace, check failed (39)	8.1	0.05–7.28
mean-field VB (26)	2.6	0.00–1.05

For Laplace the last column is the importance ESS per second, the effective number of independent draws among the weighted proposals, not a bulk ESS of a chain.

What the table shows:

• NUTS was within 0.08 reference standard deviations in every mean. Its ratios outside 0.9–1.1 all come from the random walk and α = 0.1 Dirichlet priors. It is slow with Dirichlet concentrations below one. At 0.1 it managed 0.089 to 5 ESS per second, and its slowest fit took 859 s.

• Gibbs beat NUTS in 5 of the 5 combinations with \(f\) at most gibbs_max_missing (the largest was 0.976). There it reached 103–791 ESS per second, against 1.8–336 for NUTS.

• Above that threshold (from \(f=\) 0.994), Gibbs beat NUTS in only 3 of 21 combinations. It reached 0.62–67 ESS per second, against 0.089–339. The benchmark ran Gibbs at its default length. Below the threshold, auto’s run length from \(f\) is designed to make up for the autocorrelation; the benchmark did not measure it.

• Laplace passed its check only with the Pólya tree prior. There it agreed with the reference to 0.05 standard deviations, in 0.6 to 1.0 s. Where the check failed, its errors reached 8 standard deviations.

• The ESS requirement rejected 2 more Laplace fits, whose \(\hat k\) passed with fewer effective draws than target_ess. Their quantiles were up to 0.9 standard deviations off.

• Mean-field VB was fast, but its standard deviation ratios went down to 0.00, so auto never chooses it.

The slow test suite re-checks four of these combinations against stored reference posteriors, including the engine auto picks in each (the test suite).