perf-series.mjs

Judges the newest run in a committed performance series against its baseline, and appends new runs to that series.

Contributor tooling › perf-series.mjs

What it does

Reads a committed JSONL series of measurement runs and compares the newest one against its baseline, figure by figure. Reports each figure as passing, risen beyond tolerance, or excluded, and reports every hand-set bound crossed or not, independent of the tolerance results. Appends one new run, read from a file, the only way a line is added to the series.

Usage

Two subcommands, check and append. The npm aliases:

npm run perf:series:check

npm run perf:series:append -- --file <run.json>

Direct invocation, naming a series file:

node tools/perf-series.mjs check --series docs/research/perf-series/runs-250.jsonl

Options

Flag	Value	What it does	Default
--file	path	The run JSON append reads and adds to the series. check does not use it.	required for append; check ignores it
--series	path	The series JSONL file, resolved from the repository root.	docs/research/perf-series/runs.jsonl (deliberately not committed)
--tolerance	fraction	On check, the fraction of the baseline a figure may rise by and still pass (0.1 == 10%).	0.1

Inputs and outputs

check reads the series file. append reads that same file and the run JSON at --file, checking it carries timestamp, commit, machine, load, params, and figures.

append is the only subcommand that writes: it rewrites the series file with the new run as its last line. check writes nothing.

check exits 1 when the newest run's machine differs from the baseline's, when a figure rose beyond tolerance, and when a hand-set bound is crossed. It prints "no runs yet" and exits 0 on an empty series, and exits 0 in every other case.

append exits 1 when --file is missing, unreadable, not valid JSON, or missing a required field, and 0 once the run is appended. Neither subcommand given, or an unrecognized one, prints the usage line to stderr and exits 1.

Examples

Check a fresh measurement against both tracked roster-size series, the way the nightly schedule does:

npm run perf:series:check

Record a run by hand, reproducing what a scheduled measurement found:

npm run perf:series:append -- --file run.json

Notes

A run whose machine differs from the series' baseline machine fails check outright. A comparison across hardware is worse than no comparison.

append is the only path that grows the series. The nightly schedule runs check only, because an appended run carries a machine name and a load judgment only a person appending it by hand can make.

A bound crossing (maxSubscribers, clockRate) never turns a figure's own result from pass to fail. It fails the run on its own account instead, so the two results stay separable in the output while either one is enough to refuse a landing.

Source

tools/perf-series.mjs — part of Contributor tooling