Concepts
The model
GSPICE models an ensemble of spectra on a shared wavelength grid as draws from a multivariate Gaussian distribution and predicts masked or corrupted pixels from the remaining pixels by Gaussian conditioning (Finkbeiner et al., 2025).
A spectrum as one draw from a Gaussian
Section titled “A spectrum as one draw from a Gaussian”A model must say which spectra are likely and which are not. Take many stars observed on one wavelength grid. At pixel , the flux differs from star to star, so pixel is one variable, measured once in every spectrum. A spectrum of npix pixels is then one point in npix dimensions.
GSPICE describes where these points lie with a
multivariate Gaussian
distribution. It has two parameters: a mean spectrum, and a
covariance between every pair of
pixels. gspice.fit() estimates both from the
training spectra, and
Fitting explains how.
What GSPICE needs lists what this asks of your
spectra.
The spectra need not be Gaussian. A population of stars is far from a Gaussian distribution. The conditional estimates of a Gaussian fitted to it still predict its pixels well (Finkbeiner et al., 2025).
Covariance: lines vary together
Section titled “Covariance: lines vary together”One pixel can tell you about another only if the two vary together. Take two lines of one element. A star with a deep line at pixel also tends to have a deep line at pixel . Across the spectra, the fluxes at and then rise and fall together.
The covariance measures this. It is positive where the two fluxes tend to lie above their means together. It is negative where one tends to lie above its mean while the other lies below, and 0 where they are unrelated. is the variance at pixel . The correlation puts this on a scale from −1 to 1.
The explainer below maps the correlation of a fitted model, and plots the training spectra at any two of its pixels against each other.
Point at a cell of the map, or tap it, to choose two pixels. With the map focused, the arrow keys move the cell, and Shift with an arrow key moves it 10 pixels.
Pixels 12 and 83: correlation −0.98.
- −1
- −0.5
- 0
- 0.5
- 1
- training spectrum
- 1σ and 2σ ellipses of the model
- mean spectrum
The map shows the correlation of each pair of pixels in a model fitted to 4000 simulated spectra of 120 pixels, seed 11. The scatter shows 100 of those training spectra at the two pixels.
Conditioning on two pixels, then on many
Section titled “Conditioning on two pixels, then on many”A prediction answers this question: given the flux that you observed at some pixels, what is the flux at the others? Start with two pixels, and , and measure each flux from its mean, so that both means are 0. If you observe at pixel , the flux at pixel has a Gaussian distribution again (Rasmussen & Williams, 2006):
Its mean moves away from 0 in proportion to . Its variance is smaller than , the more so the stronger the correlation. In the explainer below, you fix the flux of one pixel and see the distribution of the other.
Move the slider, or drag across the left plot, to fix the flux of one pixel. The plot on the right shows the distribution of the other pixel.
Pixel 14 fixed at 0.825. Pixel 12: 0.713 ± 0.031, against 0.541 ± 0.118 without pixel 14.
- 1σ and 2σ ellipses; distribution without the fixed pixel
- fixed flux
- conditional mean and distribution
- ±1σ
- ±2σ
The figure compares two pixels of a model fitted to 4000 simulated spectra of 120 pixels, seed 11. Pixel 12 has the largest variance, and the 4 pairs span correlations from about 1 to about −1. The prediction is that of gspice.infill from the fixed pixel alone.
The same holds for any number of pixels. Split the pixels into two sets. holds the pixels to predict or to leave out, and holds the conditioning pixels, whose values the prediction uses. With measured from the mean again, the conditional distribution of is
is the block of with the rows and the columns , and the other blocks follow the same rule. The conditional mean is the prediction. It is the average of all spectra in the model that agree with the conditioning pixels, and a linear function of . Where holds all pixels, the result is the prior .
In the explainer below, the masked block is , and the square beside the spectrum marks the rows and columns of in the correlation of .
Drag the gray block, or drag its edges to resize it. With the plot focused, the arrow keys move the block, and Shift with an arrow key resizes it.
Pixels 50 to 69 masked (20). Mean σ 0.019. RMS error 0.003.
- observed flux
- prediction
- ±1σ
- ±2σ
- flux without noise
- −1
- −0.5
- 0
- 0.5
- 1
The prediction is that of gspice.infill with a model fitted to 4000 simulated spectra of 120 pixels. The ±1σ and ±2σ bands include the noise of the observed flux. The square shows the correlation of each pair of pixels. The prediction reads the marked rows and columns of the masked pixels.
A prediction can report only some pixels of . The others are marginalized: their values do not enter, and the conditional covariance of the reported pixels includes the uncertainty of the others. Prediction shows how each function chooses . The code computes the same distribution from the inverse of , as The precision form explains.
The uncertainty of a prediction
Section titled “The uncertainty of a prediction”A prediction needs an uncertainty. The conditional covariance provides it. It depends on which pixels you observed, but not on their values. Its diagonal holds the conditional variance of each predicted pixel.
The training spectra carry their noise, so holds the noise as well as the variation between the spectra. Every predicted variance therefore includes the noise of the predicted pixel. The prediction describes the observed flux at the pixel, not its noise-free flux.
Two checks compare the predicted uncertainty with the data:
- The z-score of one pixel is its flux minus the prediction, divided by the square root of the conditional variance.
- The χ² of several pixels uses their full conditional covariance (χ² and its degrees of freedom).
Both have a known distribution only if the model is right. A model estimated from a finite number of spectra predicts new spectra less well than it says (Held-out uncertainty). Diagnostics measures both effects on your spectra.
Working units
Section titled “Working units”Spectra differ in brightness and in signal-to-noise ratio, and some pixels are bad. One covariance of the raw flux would mix these differences with the variation that predicts pixels. The model therefore describes the working vector of spectrum , not its flux. It computes in three steps:
1. Interpolate over the bad pixels: f̃s (flux)
2. Multiply by as = 34.88: as f̃s (scaled units)
3. Subtract the mean of the training spectra: xs = as f̃s − mb (scaled units)
- observed flux
- interpolated flux
- mean of the training spectra
- bad pixels
The panels trace a simulated spectrum of 100 pixels at a signal-to-noise ratio of 31, with pixels 56 to 68 marked bad. The model is fitted to 4000 simulated spectra. Step 1 replaces the bad pixels by a straight line. Step 2 brings the noise to a variance near 1. Step 3 leaves the variation about the mean.
- Interpolate the flux linearly over the bad pixels, and hold it constant beyond the first and the last good pixel. This gives . A bad pixel is never read, so its flux may be NaN.
- Multiply by , the square root of the mean of
ivarover the good pixels of the spectrum, in flux⁻¹. This gives scaled units. - Subtract the mean of the training spectra in the S/N bin of the spectrum. There is one bin by default.
So
In scaled units, the noise at pixel has the variance .
That variance is 1 where ivar is equal at every pixel of the spectrum, whatever its
S/N. A variance in scaled units divided by is a variance in flux units.
Continuum normalization says what the
scaling means for your spectra.
Two kinds of model use other working vectors:
| Model | Working vector | Units |
|---|---|---|
| Default, or with S/N bins | scaled | |
| Constant center | scaled | |
| Noise-aware | flux |
Fitting describes both. The public functions take flux and
return flux, flux² or unit-free values. The working units appear only in
gspice.scale(),
gspice.unscale() and the fields of
gspice.Model.
multivariate Gaussian
A Gaussian distribution of several variables at once, here the fluxes of all pixels. A mean vector and a covariance matrix define it.
covariance
How the fluxes of two pixels vary together across the spectra. It is positive if they tend to rise together, and negative if one tends to rise as the other falls. The covariance matrix holds it for every pair of pixels, and its diagonal holds the variances.
training spectra
The spectra that gspice.fit() estimates the
model from.
correlation
The covariance of two pixels divided by the square root of the product of their variances, : a number from −1 to 1.
conditioning pixels
The pixels whose values a prediction uses, : all pixels outside . The paper calls this set , and counts it.
conditional distribution
The Gaussian distribution of the pixels of a spectrum, given the values of its conditioning pixels . Its mean is the prediction, and its covariance the uncertainty of the prediction.
conditional mean
The mean of the conditional distribution: the average of all spectra in the model that
agree with the conditioning pixels. It is the prediction, and it is smoother than any
one spectrum. infill returns it in Repaired.flux.
prior
The distribution of the working vector, without conditioning pixels. A prediction whose holds all pixels returns it.
marginalize
To leave pixels out of a prediction without conditioning on them. Their values do not enter, and the covariance of the reported pixels includes their uncertainty.
conditional variance
The variance of the conditional distribution at one pixel: the uncertainty of its
prediction, noise included. pixelwise and infill return it in var, in flux².
z-score
The difference between the flux of a pixel and its prediction, divided by the square root of the conditional variance, .
χ²
One number that compares several observed values with their prediction:
, with the residuals and their conditional covariance.
It counts correlated errors once. Its number of degrees of freedom is the number of
values, Repaired.dof (Prediction).
working vector
The vector that the model describes: the interpolated flux of spectrum in working units. The mean of its S/N bin, or a constant center, is subtracted from it.
bad pixel
A pixel that the input marks: a nonzero mask or ivar == 0. GSPICE never reads
flux or ivar there, and never conditions on it, except in the pixelwise
prediction, which uses its interpolated value.
scaled units
Flux multiplied by , the square root of the mean inverse variance of the good
pixels of the spectrum. The noise of a spectrum with equal ivar at every pixel then
has the variance 1.
S/N bin
A range of S/N with its own mean and covariance (fit(snr_bins=...)), with the index
.
constant center
A constant , in flux units, that a model subtracts from the flux instead of the
mean of the training spectra (fit(center=c)).
noise-aware model
A model of the noise-free flux, in flux units, whose prediction adds the noise of each
spectrum from its ivar (fit(noise=True)). Its opposite is a scaled model.