src/core/psd.js
Detrending diagnostics for light-curve residuals: is the noise white, and does the flux follow the star's
position on the detector?
fft complex FFT of any length (radix-2; Bluestein's chirp-z for the other lengths)
periodogramPsd one-sided power spectral density of an evenly sampled series, = scipy.signal.periodogram
welchPsd averaged over overlapping windows, = scipy.signal.welch; windows can be kept inside
gap-free segments (splitSegments) for space-based series with gaps
lombScarglePsd the same quantity for uneven sampling, from periodogram.js's lombScargle
whiteNoiseLevel / photonNoiseLevel / photonNoise / scintillation
the horizontal lines to compare a PSD with: error bars, CCD-equation noise
logBinPsd log-spaced averaging for display and for the fit
fitRedNoise P(f) = W + A f^-gamma by least squares on ln P, and the knee where the two terms cross
rmsVsBin "rms versus bin size" (time averaging: Pont et al. 2006; Winn et al. 2008), in the exact
convention of MC3's mc3.stats.time_avg (Cubillos et al. 2017, AJ 153, 3), and beta
pixelMap mean residual binned on the (x, y) centroid, = scipy.stats.binned_statistic_2d
polySurface 2-D polynomial of total order 1..3 by least squares (Householder QR), = numpy lstsq
surfaceOnGrid / gpSurface any function, or the Gaussian process of gp.js, sampled on the map's grid
syntheticRollSeries a K2-like test series with a known intrapixel sensitivity (used by the test and the harness)
PSD convention (all three estimators). One-sided density: the integral of P over 0 < f < Nyquist is the
variance of the series. Units are y^2 per unit of frequency, the frequency being in cycles per unit of dt
(dt in days: f in 1/day and P in y^2 day; dt in seconds: Hz and y^2/Hz). Independent ("white") noise of
variance s^2 sampled every dt has the flat level P = 2 s^2 dt (s^2 spread evenly over a band 1/(2 dt)).
Correlated ("red") noise is whatever rises above that level towards low frequency.
No DOM. Checked against scipy, astropy, numpy and MC3 in test/psd.test.mjs (reference: test/make_psd_reference.py).Import: import { … } from './src/core/psd.js' — no DOM, runs in Node and the browser.
fft
function fft(re, im)Forward discrete Fourier transform X_k = sum_j x_j exp(-2 pi i j k / n), in place, any n.
Powers of two go straight to the radix-2 transform; other lengths use Bluestein's identity
jk = (j^2 + k^2 - (k - j)^2) / 2, which turns the DFT into a convolution done with three radix-2 FFTs of
length >= 2n - 1 (the chirp phases use j^2 mod 2n, so they stay exact for any n). Cost O(n log n) always.periodogramPsd
function periodogramPsd(y, dt = 1, opts = {})Power spectral density of an evenly sampled series: the same numbers as
scipy.signal.periodogram(y, fs = 1/dt, window, detrend, scaling = 'density').- param y samples (no NaN), spacing dt
- param dt sampling interval. Frequencies come out in cycles per unit of dt and the density in y^2 times that unit (dt in days: 1/day and y^2 day; dt in seconds: Hz and y^2/Hz)
- param opts { detrend: 'constant' (default) | 'linear' | 'none', window: 'boxcar' (default) | 'hann' }
- returns { f, p, df, n }: f_k = k / (n dt) for k = 0 .. floor(n/2); sum(p) df = variance for the boxcar. f[0] = 0 is included (as scipy does); with a detrend its power is zero, so skip it on a log axis.
splitSegments
function splitSegments(t, opts = {})Split a time series at its gaps. A gap is a step larger than gapFactor (default 1.5) times the median step.
- returns { segments: [[i0, i1), ...] (index ranges, in order), dt (the median step) }
welchPsd
function welchPsd(y, dt = 1, opts = {})Welch's averaged periodogram: the same numbers as
scipy.signal.welch(y, fs = 1/dt, window = 'hann', nperseg, noverlap, detrend = 'constant', scaling = 'density', average).- param opts { nperseg = 256 (reduced to the series length if longer), noverlap = floor(nperseg / 2), window = 'hann' | 'boxcar', detrend = 'constant' | 'linear' | 'none', average = 'mean' | 'median', segments: [[i0, i1), ...] from splitSegments } With
segments, windows are laid out inside each gap-free segment separately (each exactly as scipy would on that segment alone) and all windows are averaged together; segments shorter than nperseg are skipped. Without it nperseg is reduced to the series length when the series is shorter, as scipy does. - returns { f, p, df, nperseg, nWindows }. Resolution df = 1 / (nperseg dt); the scatter of each ordinate is about 1/sqrt(nWindows) of its value (less gain from the overlap), instead of 100% for the periodogram.
lombScarglePsd
function lombScarglePsd(t, y, freqs = null, opts = {})Power spectral density of an unevenly sampled series, from the generalised Lomb-Scargle periodogram of
periodogram.js (floating mean), scaled to the convention at the top of this file:
P(f) = 2 dt (n / sum w) P_psd(f), w_i = 1 / dy_i^2 (w_i = 1 and sum w = n without error bars),
where P_psd = (chi2_ref - chi2(f)) / 2 is exactly astropy.timeseries.LombScargle(t, y[, dy]).power(f,
normalization = 'psd'), and dt = (t_max - t_min) / (n - 1) is the mean spacing. Without error bars this is
P = dt p_standard(f) sum (y - mean)^2. Why this is the right scale: for white noise of variance s^2 the
drop in chi-square from fitting one sinusoid is s^2 chi^2 with 2 degrees of freedom, so E[P_psd] = s^2 and
E[P] = 2 s^2 dt; and for evenly sampled data at the Fourier frequencies k / (n dt), 0 < k < n/2, P_psd =
|FFT|^2 / n, so P is identical to periodogramPsd(y, dt) (boxcar, detrend 'constant') ordinate by ordinate.- param freqs cycles per unit of t; null for the Fourier-like grid k / (n dt), k = 1 .. floor((n - 1) / 2) (the "average Nyquist" 1 / (2 dt) is only a guide for uneven sampling)
- param opts { dy (1-sigma errors: weights), dt (override the mean spacing), fitMean = true }
- returns { f, p, dt }. Cost: freqs.length * n sine/cosine pairs: fine for a ground-based night (n ~ 1000: a few ms per 100 frequencies); for n ~ 20,000 use periodogramPsd / welchPsd, or a short list of freqs.
whiteNoiseLevel
function whiteNoiseLevel(err, dt)Expected one-sided PSD level of independent noise with the given error bars: 2 mean(err^2) dt. err: array or number.
photonNoiseLevel
const photonNoiseLevelPSD level of the photon-noise limit: 2 mean(sigma^2) dt for a per-point relative noise sigma (number or array, e.g. photonNoise(...).rel, in the units of the residuals: relative flux if the residuals are relative flux).
scintillation
const scintillationScintillation noise as a fraction of the flux (Young 1967; the form in Dravins et al. 1998):
sigma = 0.09 D^(-2/3) X^(7/4) (2 t)^(-1/2) exp(-h / 8000)- param D diameter of the telescope in cm; airmass X; exptime t in seconds; altitude h of the site in metres.
photonNoise
function photonNoise(o)Noise of one aperture-photometry measurement from the "CCD equation" in the form of Merline & Howell (1995),
in electrons:
sigma^2 = S + npix (1 + npix / nsky) (B + D + R^2 [+ g^2 sf^2]) [+ (sigma_scint S)^2]
S = gain * flux (star, sky-subtracted), B = gain * sky (per pixel), D the dark charge per pixel, R the read
noise per pixel; npix pixels in the aperture; the factor (1 + npix / nsky) is the cost of estimating the sky
from nsky pixels (the npix^2 / nsky term). With `digit` the digitisation term g^2 sf^2 (sf = 0.289) is added.- param o { flux (ADU, in the aperture, sky-subtracted), sky = 0 (ADU per pixel), npix = 0, nsky = Infinity, gain = 1 (e-/ADU), readNoise = 0 (e- rms per pixel), dark = 0 (e- per pixel per exposure), scint: a fraction, or { D, airmass, exptime, altitude } for scintillation(), digit = false } flux and sky may be arrays (one value per frame); the result is then arrays.
- returns { sigma (e-), sigmaAdu, rel (sigma / S: the per-point relative noise), snr, photon (sqrt(S), the source-only limit, e-), relPhoton (1 / sqrt(S)) }
logBinPsd
function logBinPsd(f, p, nbins = 30)Average a PSD in nbins logarithmically spaced frequency bins (for display, and to feed fitRedNoise).
Edges e_j = exp(ln fmin + j (ln fmax - ln fmin) / nbins) over the positive frequencies; a point belongs to
the last bin whose lower edge is <= f (fmax goes in the last bin). Empty bins are dropped.- returns { f (geometric mean of the frequencies in the bin), p (arithmetic mean of the power, which keeps the level of a flat spectrum unbiased), n (points averaged: pass it to fitRedNoise as nAvg), lo, hi (bin edges) }
digamma
function digamma(x)Digamma function psi(x), x > 0 (recurrence to x >= 12, then the asymptotic series; error below 1e-14).trigamma
function trigamma(x)Trigamma function psi'(x), x > 0 (same method; relative error below 1e-13).
fitRedNoise
function fitRedNoise(f, p, opts = {})Fit P(f) = W + A f^-gamma (white level plus a power law) by least squares on the logarithm of the PSD.
A periodogram ordinate is its expectation times an exponential variate (chi-square with 2 degrees of freedom,
halved), and the mean of m of them is a Gamma(m)/m variate, whose logarithm has mean psi(m) - ln m (-0.5772
for m = 1) and variance psi'(m) (pi^2 / 6 for m = 1), independent of the level: so the logarithm is the right
space for least squares, but a plain fit would sit a factor exp(psi(m) - ln m) too low (0.56 for a raw
periodogram). Minimised here:
sum_i [ ln p_i - (psi(m_i) - ln m_i) - ln(W + A f_i^-gamma) ]^2 / psi'(m_i)
with m_i = opts.nAvg (a number, or an array such as logBinPsd().n; default 1: a raw periodogram; for welchPsd
roughly nWindows). opts.correctBias = false drops the psi term (a plain log-space fit). The minimiser is
Levenberg-Marquardt in (ln W, ln A, gamma) from four starting slopes, with frequencies scaled by their
geometric mean; gamma is kept inside opts.gammaRange (default [0, 8]: a spectrum that rises towards low
frequency). Points with f <= 0 or p <= 0 are ignored. When the residuals are white the power law is not
constrained (gamma can end anywhere in its range, with the knee below the lowest frequency; at gamma = 0 the
two terms are the same flat level and are returned merged into W with A = 0). `red` says whether the power
law is needed: the fit must beat a single flat level (`whiteOnly`) by deltaCost > 9.21 (the 99% point of
chi-square with the 2 extra parameters; the weights make each term of the sum approximately unit variance)
and the knee must lie above the lowest frequency. red = false means "no red noise detected".- returns { W, A, gamma, knee (the frequency where A f^-gamma = W, = (A / W)^(1/gamma); may lie outside the data), model(f), cost (the minimised sum), rmsLog (rms of the weighted log residuals: about 1 for a good fit), red, deltaCost, whiteOnly (see above), n, iterations } -- A is in the PSD's units times (frequency unit)^gamma.
rmsVsBin
function rmsVsBin(resid, opts = {})"rms versus bin size": the scatter of the residuals after averaging N consecutive points, against what
independent noise would give (Pont, Zucker & Queloz 2006; Winn et al. 2008), in the convention of MC3's
mc3.stats.time_avg (Cubillos et al. 2017), for each bin size N:
M = floor(n / N) non-overlapping bins of N consecutive points from the start (the remainder is dropped)
rms = sqrt( mean_j( binmean_j^2 ) ) (about zero: the residuals are not re-centred)
expected = sigma1 sqrt( M / (N (M - 1)) ), sigma1 = the standard deviation of the unbinned
residuals about their mean (divisor n), i.e. sigma1 / sqrt(N) times sqrt(M / (M - 1))
rmsErr = rms / sqrt(2 M) (asymptotic; MC3 replaces it below M = 36 by an
asymmetric interval from an inverse-gamma distribution, which is not reproduced here)
beta = rms / expected (the factor by which Winn et al. inflate the errors)- param resid residuals in time order (finite values)
- param opts { maxBin = floor(n / 2), binStep = 1 (MC3's maxbins and binstep: sizes 1, 1 + binStep, ...), binSizes (explicit list, overrides), logBins (that many log-spaced sizes from 1 to maxBin; the default for n > 5000 is 200 of them, otherwise every size as MC3 does), minBins = 10 (beta's maximum is taken over sizes with M >= minBins: with fewer bins the rms itself is uncertain by more than 1/sqrt(20) = 22%), t (times) and gapFactor, or segments [[i0, i1), ...]: bins are formed inside gap-free segments only (from the start of each, remainders dropped), M is the total over segments and the formulae are unchanged; without gaps this is identical to MC3, dt (kept in the result for the duration axis) }
- returns { n (bin sizes), M (bins used), rms, expected, rmsErr, betaCurve (rms / expected), sigma1, dt, beta (max of betaCurve over sizes with M >= minBins), nBeta (its bin size), betaAt(N) (rms / expected computed for exactly that size), betaMax(nLo, nHi) -> { beta, n } }
percentile
function percentile(v, q)The q-th percentile (0..100) of the finite values of v, linear interpolation between order statistics (numpy's default).
pixelMap
function pixelMap(x, y, resid, opts = {})Mean residual binned on the centroid: how the measured flux depends on where the star sits on the detector. Equal bins over [x0, x1] x [y0, y1]; edges x0 + i (x1 - x0) / nx; a point on an inner edge goes to the upper bin and one on the last edge to the last bin (numpy.histogram2d / scipy.stats.binned_statistic_2d rule); points outside the range are left out.
- param opts { nx = 20, ny = 20, range: [x0, x1, y0, y1] (default: the clip percentiles of the kept centroids), clip = [1, 99] (percent; [0, 100] for the full range), keep (Uint8Array: points to use) }
- returns { grid: { x0, x1, y0, y1, nx, ny }, mean, count, std (sample standard deviation, divisor count - 1), sem (std / sqrt(count)), all Float64Array(ny * nx) indexed [iy * nx + ix] with NaN where undefined (mean: empty bin; std, sem: fewer than 2 points), xc, yc (bin centres), nUsed, nOutside }
surfaceOnGrid
function surfaceOnGrid(fn, grid, mask = null)Sample fn(x, y) at the bin centres of a pixelMap grid. mask (e.g. the map's count): bins where it is 0 give NaN.
polyTerms
function polyTerms(order)Exponents (i, j) of x^i y^j with i + j <= order, by total degree: 1, x, y, x^2, xy, y^2, x^3, ...polySurface
function polySurface(x, y, resid, order = 2, keep = null, opts = {})Least-squares 2-D polynomial through the residuals as a function of the centroid:
s(x, y) = sum_k coef_k (x - xc)^i_k (y - yc)^j_k, i + j <= order (1, 2 or 3; terms in polyTerms order),
with (xc, yc) the mean kept centroid (or opts.centre = [xc, yc]). Solved by Householder QR of the design
matrix in coordinates scaled to unit scatter (no normal equations), then converted back to pixels: the same
solution as numpy.linalg.lstsq on the columns (x - xc)^i (y - yc)^j.- param keep optional Uint8Array: points to fit (the surface is evaluated at all points)
- param opts { centre: [xc, yc], grid (a pixelMap grid: adds
surface), mask (see surfaceOnGrid) } - returns { coef, coefErr (formal 1-sigma, from the residual variance with n - terms degrees of freedom), terms, xc, yc, order, evaluate(x, y), fitted (Float64Array, at every point), surface (on opts.grid), rmsBefore (kept residuals about their mean), rmsAfter (kept residuals minus the surface), n } or null when there are fewer kept points than terms or the centroids do not span a plane.
gpSurface
function gpSurface(inputsXY, resid, keep, grid, opts = {})The Gaussian-process trend of gp.js (detrendWithCentroids: squared-exponential kernel in x and y, hyperparameters by maximum marginal likelihood) sampled on a pixelMap grid, so the removed surface can be drawn next to the binned data. The grid centres are appended to the inputs as points that are predicted but not fitted, so the surface is conditioned on exactly the same points as the trend that is subtracted.
- param inputsXY [x, y] centroid arrays (two inputs only: with time as a third input the trend is not a function of position alone and has no 2-D map)
- param keep optional Uint8Array of points to fit
- param grid { x0, x1, y0, y1, nx, ny } (pixelMap(...).grid)
- param opts gp.js options { maxHyper, maxCond, maxEvals } and mask (see surfaceOnGrid)
- returns { surface (ny * nx, includes the mean residual like
trend), trend (GP mean at every point), evaluateGrid: the same as surface, hyper, lnL, rmsBefore, rmsAfter, nHyper, nCond, n, evals } or null (fewer than 10 usable points). Far from any centroid the surface returns to the mean residual.
syntheticRollSeries
function syntheticRollSeries(o = {})A K2-like series with a known intrapixel sensitivity, for the test and the plotting harness:
flux = 1 - a ((x - xs)^2 + (y - ys)^2) + white noise
while the star drifts across the pixel and is reset every rollPeriod (sawtooth of amplitude `amp` pixels in x,
-0.35 amp in y), with a slow sinusoidal drift and centroid jitter on top.- param o { n = 2880, dt = 1/48 (days: 30-minute cadence), rollPeriod = 5.88 / 24 (days), amp = 1, a = 0.012 (per pixel^2), sigma = 2e-4, seed = 7, x0 = 10.3, y0 = 20.1 }
- returns { t, x, y, flux, resid (flux - 1), sys (the noiseless systematic, flux - 1), err, a, xs, ys, sigma, dt, rollPeriod }