Concepts
Prediction
gspice.pixelwise() and
gspice.infill() evaluate the conditional distribution
of The model and return predictions in flux units, differing in how they partition the
set of pixels that are not conditioned on.
Pixelwise prediction
Section titled “Pixelwise prediction”To find outliers, you need a prediction of
every pixel that does not use the pixel itself. pixelwise predicts each pixel from
the pixels beyond its guard window
, clipped at the ends of the spectrum:
- pixel i, predicted
- its guard window: not conditioned on
- conditioning pixels O
The conditioning pixels are all the pixels outside . With the precision matrix of Numerics,
The other pixels of are marginalized.
The guard window
Section titled “The guard window”The guard window keeps the neighbors of pixel out of its prediction, for two reasons. First, neighboring pixels are correlated through the line-spread function and the upstream processing. The prediction should come from the correlations of the spectra instead (Finkbeiner et al., 2025). Second, an artifact a few pixels wide also changes the neighbors of , and a prediction from them would follow the artifact.
The default nguard=20 is the paper’s choice, set by the size of the smoothing kernel
of its continuum normalization. nguard >= npix - 1 gives the
prior. The guard window
measures the effect of nguard on simulated artifacts.
Bad pixels and the z-score
Section titled “Bad pixels and the z-score”pixelwise conditions on the bad pixels
through their interpolated values. Their z-score is NaN, and the variance omits the
error of the interpolated values. Next to a long run of bad pixels, the z-scores of the
good pixels are therefore not calibrated. An interpolated bad pixel beyond the guard
window can also carry the flux of a good pixel inside it into the prediction.
The z-score of pixel of spectrum is
with and the conditional mean and variance in flux
units. gspice.Prediction holds the three
arrays.
Infill
Section titled “Infill”To replace pixels, you predict them from all the pixels that you trust. infill
divides the pixels of each spectrum into four kinds:
- replaced pixels M
- ignored pixel: not conditioned on, keeps its value
- bad pixel: never conditioned on
- conditioning pixels O
- the replaced pixels , given by
replace, the bad pixels by default; - the ignored pixels, given by
ignore, which are neither conditioned on nor replaced, such as a guard window around a range of pixels; - the bad pixels, which are never conditioned on;
- the conditioning pixels , all the other pixels.
is the union of replace, ignore and the bad pixels:
A pixel in both replace and ignore is replaced. The pixels of outside keep
their input values and are marginalized. Unlike pixelwise, infill never conditions
on a bad pixel, and no value in enters the mean, so an outlier in cannot change
it.
A draw is a random sample of from this distribution. Fill known bad pixels shows the mean, the variance, the covariance and the draws, and when to use each. What you get back lists the fields of the result.
χ² and its degrees of freedom
Section titled “χ² and its degrees of freedom”A prediction of several pixels needs one number that says whether the observed values
agree with it. Repaired.chi2 compares the input values with their prediction at ,
the replaced pixels that are good in the input:
with the input values minus the conditional mean, and their conditional
covariance. A bad pixel has no input value to compare, so it is in but not in .
Repaired.dof, the number of degrees of freedom, is .
The χ² uses the full conditional covariance, so
the correlated errors of neighboring pixels do not count twice. Suppose that the mean and
the covariance were exact, and that the pixel sets did not depend on the data. Then
would follow the χ² distribution with degrees of freedom, whose mean is
. A fitted model predicts a
held-out spectrum less well:
the mean of is then the finite-sample factor of
Held-out uncertainty, which Repaired.fs holds.
conditional distribution
The Gaussian distribution of the pixels of a spectrum, given the values of its conditioning pixels . Its mean is the prediction, and its covariance the uncertainty of the prediction.
outlier
A pixel whose flux is many standard deviations from its prediction: a large .
guard window
The pixels within nguard of a predicted pixel or block. A prediction does not
condition on them, and reports only the predicted pixels among them.
conditioning pixels
The pixels whose values a prediction uses, : all pixels outside . The paper calls this set , and counts it.
precision matrix
The inverse of the covariance, . GSPICE computes the conditional distribution from its blocks and .
marginalize
To leave pixels out of a prediction without conditioning on them. Their values do not enter, and the covariance of the reported pixels includes their uncertainty.
prior
The distribution of the working vector, without conditioning pixels. A prediction whose holds all pixels returns it.
bad pixel
A pixel that the input marks: a nonzero mask or ivar == 0. GSPICE never reads
flux or ivar there, and never conditions on it, except in the pixelwise
prediction, which uses its interpolated value.
z-score
The difference between the flux of a pixel and its prediction, divided by the square root of the conditional variance, .
replaced pixel
A pixel of the set , whose value an infill replaces by its conditional mean.
ignored pixel
A pixel in that is not replaced: it is not conditioned on, and keeps its input
value. repair reports its ignored pixels as .
draw
A random sample of the replaced pixels from their conditional distribution, in
Repaired.draws. It scatters like an observed spectrum, unlike the conditional mean.
χ²
One number that compares several observed values with their prediction:
, with the residuals and their conditional covariance.
It counts correlated errors once. Its number of degrees of freedom is the number of
values, Repaired.dof (Prediction).
held-out spectrum
A spectrum that the model was not fitted to.