Skip to main content

maintenance

This page says how the repository stays current. It tracks the Matrix specification, the proposals that Spindle serves ahead of the spec, and the clients and suites that test it. It also says what a maintainer does when one of them moves.

The principle throughout: a gate holds every claim to the code, and a job watches every upstream. Nothing here is a checklist that a person must remember. Each item is a script. It runs in CI or on a schedule, prints what it found, and fails or files an issue. A person still makes the judgement calls. These are whether to bump a pin, whether a new endpoint is in scope, and what to do when a proposal dies.

1. The gates, per pull request​

ci.yml runs these on every push. Each one answers "does a generated artefact still tell the truth about the code":

GateHoldsRegenerate with
coverage-dashboard.py --checkdocs/dashboard.md: the endpoints the router serves, per milestonejust regen
spec-drift.py --checkdocs/spec-gaps.md: what the pinned spec defines that the router does not servejust regen
msc-ledger.py --checkdocs/mscs.md, and that /versions' flags and the unstable/ routes are all owned by a served entry in contrib/msc/ledger.toml with a test naming the MSCjust regen after editing the ledger
upstream-pins.py --checkthat every hand-pinned upstream can still be read from its filefix the regex or the file
ruma-drift.py --checkvendor/ruma-state-res is its crates.io base plus exactly the hunks SPINDLE-PATCH.md lists§8; ruma-drift.py --hunks prints the rows
openapi-check.pyevery response the scripted client gets validates against the pinned spec's schemascripts/openapi-allowlist.txt for an explained divergence
readme-numbers.py --checkthe README's headline counts are the gated onesjust regen
config-drift.pythe example config names every settingedit spindle.example.toml
untested-routes.py(report) routes nothing names in a test, rig or evidence pagewrite the test
tests/surface.rs/versions advertises nothing unbuiltbuild it, then advertise it

Run just lint, just test and just regen before you push. Together they reproduce all of it.

2. The ratchets, nightly​

Three external suites run against a fresh build every night from compliance.yml. Element Call's suite runs twice: against two Spindles, and against one Spindle and upstream's Synapse (SPINDLE_PEER=synapse). Each run has an allowlist that may only grow:

SuitePinAllowlistCheck
Complementscripts/complement.shcomplement/allowlist.txtscripts/complement-check.py
matrix-rust-sdk integration testscontrib/rust-sdk/run.shcontrib/rust-sdk/allowlist.txtscripts/rust-sdk-check.py
Element Call Playwrightcontrib/element-call/run.shcontrib/element-call/allowlist.txtscripts/element-call-check.py
Element Call Playwright, Synapse peercontrib/element-call/run.shcontrib/element-call/allowlist-synapse-peer.txtscripts/element-call-check.py

The job prints a test that passes but is not on the list as a candidate. A reviewed commit promotes it, and the commit names the run it passed in. Promotion is never automatic, so people do not learn to ignore the gate because of one flaky run. A listed test that fails is a regression and fails the job. The procedure:

  1. Read the job's tail: the candidate and REGRESSED lines.
  2. For a regression, fix the code first. Do not edit the allowlist to make the test pass. (A test that upstream changed under the same name is the one exception, and the commit says so.)
  3. For candidates, add them under a header naming the run id and the Spindle commit, one pull request per run.

The maintainers triage the remaining failures in each suite on one issue per suite (#112 for the SDK suite). Each is a real gap or a real defect; the sdk_findings tests in crates/spindle-server/tests/ are the unit-sized reproductions of what the suite found.

3. The weekly upkeep report​

upkeep.yml runs on Mondays and edits one issue with the label upkeep. Locally, just upkeep prints the same four reports:

Pins behind upstream (upstream-pins.py --upstream). Every upstream this repository pins by hand, read from the file that uses it, against the newest tag or the branch head. Dependabot covers Cargo, the actions, the Docker bases and the Go module; this covers the rest.

MSCs against the proposals (msc-ledger.py --upstream). Each ledger entry's pull request on matrix-spec-proposals: open, merged or closed, with its labels. Two findings matter. A served MSC that has landed upstream while the ledger records no stable version means the stable spelling exists and clients will start sending it — see §5. A planned MSC closed unmerged is one to stop planning for.

What a spec pin bump would bring (spec-drift.py against matrix-spec main). The operations that differ between the pinned spec and today's main, so the size of the next release's work is known before the bump.

The vendored state resolver against upstream (ruma-drift.py --upstream). vendor/ruma-state-res is a patched copy of a release. The patch has no upstream path (ADR 0006). Dependabot cannot see this copy, so no other job tells us about a ruma fix to state resolution.

The report compares the src/ of the newest crates.io release with the vendored base, hunk by hunk. It marks the files that Spindle's patch also changes. A released change is drift: the report says so and the script exits 1. Rebase per §8. The report also compares ruma's main, as a warning only: main moves daily, and an unreleased change can change again before release.

The report does not start an action. A person decides; §4 tells how to bump a pin.

4. Bumping a pin​

Every pin bump is its own pull request, and the pull request's job is to show what the bump changed. What to expect from each:

PinWhat moves with itWhat to read before merging
SPEC_PIN in scripts/openapi-check.pydocs/spec-gaps.md is rewritten; openapi-check validates against new schemasThe diff of spec-gaps.md is the release's new work. File an issue per group that is in scope. A schema failure is a real divergence: fix it or allowlist it with a reason.
RUST_SDK_REVThe nightly suite runs the new testsRun compliance.yml by dispatch on the branch and read the tail: new candidates are promoted with the bump, regressions block it.
COMPLEMENT_REVSame, for ComplementSame; the protected subset runs on the pull request, the whole suite on main.
ELEMENT_TAG, ELEMENT_CALL_REVThe browser and call suitesThe E2E jobs on the pull request. A new spec Element Call requires appears as a failing spec, and the fix goes in before or with the bump.
SYNAPSE_VERSION and the other benchmark versionsThe next sitting's fieldNothing gates it; the sitting's sidecar records the versions, and the page keeps groups apart, so an older sitting is never compared against a newer field.

Two pins are deliberately not automated: rust-toolchain.toml (the MSRV moves on its own decision, #193) and ruma (ADR-0002: the room-version, redaction and authorization rules are taken from ruma and a bump is read as a protocol change, not a dependency update). The weekly report covers the vendored ruma-state-res, and a person rebases it by hand, §8.

5. Absorbing a spec release​

A Matrix spec release (v1.N) reaches this repository as a SPEC_PIN bump, and the bump's spec-gaps.md diff lists what it added under ### Not served, added in v1.N. For each group:

  1. Decide scope. A homeserver serves what it serves; the page's job is to be honest about it. Third-party identity lookups and MSISDN tokens, for instance, are listed and not planned. File an issue for what is in scope, and say why the rest is not in the issue that tracks the release.
  2. Serve it. A route arrives with the test that would have failed without it (CONTRIBUTING). just regen moves it from spec-gaps.md to dashboard.md; the OpenAPI check validates its responses on the next run.
  3. Claim it. crates/spindle-server/src/surface.rs lists the spec versions /versions advertises, each with the routes that make the claim honest; tests/surface.rs fails if one is missing. Advertising v1.N is the last step, after everything the version requires is served — a client plans against /versions, and a longer list buys nothing but failures further from the cause.
  4. Retire the unstable spelling of anything the release stabilised (§6).

6. Adding, stabilising and retiring an MSC​

Spindle serves proposals ahead of the spec when a client it targets needs them: Element X's sliding sync, MatrixRTC's delayed and sticky events. Each one goes through the same life:

  1. Add. Serve it under the unstable/org.matrix.mscNNNN prefix the proposal names, advertise the flag in surface::UNSTABLE_FEATURES only if the endpoint answers, and add the entry to contrib/msc/ledger.toml with status = "served" and the test that proves it. msc-ledger.py --check refuses a flag or a route the ledger does not own, and a served entry with no test naming the MSC.
  2. Stabilise. When the weekly report says the proposal merged, the stable spelling exists. Serve both — the stable path and the unstable one, with the same handler, as auth_metadata does — record the spec version in the entry's stable field, and keep the flag until the clients that check it have moved. A proposal that has merged but that no spec release names yet gets merged = true instead: there is no stable spelling to adopt, and the report stops asking until there is.
  3. Retire. The unstable route and flag go once no pinned client sends them; the entry stays, with its stable version, so the page still says what was served and when.

A proposal that dies upstream is marked declined or superseded with a note saying what replaced it, never deleted: the ledger is a record, not a list of the present.

7. What is deliberately not automated​

  • Bumps. Every one above is a reviewed pull request. The suites are how a newer client tells this server what it now expects, and that is a thing to read, not to merge.
  • Promotion. A passing test enters an allowlist by a commit naming the run, so the ratchet cannot learn to ignore a flake.
  • Claims. /versions, the dashboard, the MSC page and the spec-gaps page are generated from or checked against the router; none is typed.
  • Scope. The spec-gaps page lists everything unserved and gates on nothing but its own accuracy. Which of it to build is the roadmap's decision, made in an issue where the reasons are written down.

8. Rebase vendor/ruma-state-res​

vendor/ruma-state-res is a release of ruma-state-res plus the hunks its SPINDLE-PATCH.md lists, each with the reason it exists (ADR 0006 says why the patch cannot go upstream). ruma-drift.py --check holds the tree to that list on every pull request, so the patch cannot grow without a reviewed edit to the table. When the upkeep report says a release changed src/, the rebase is its own pull request:

  1. Read the drift. The report's hunks are upstream's change to state resolution. Read each file that the report marks "Spindle patches this file" first. Upstream's change can make a hunk unnecessary. Drop that hunk; do not carry it.

  2. Rebase in git, not by eye. Commit the old base. Commit the vendored tree on top of it, and the new release beside it. Then let git rebase do the three-way merge. From the repository root:

    repo=$PWD old=0.18.0 new=0.19.0 # the base, and the release the report names
    work=$(mktemp -d)
    for v in $old $new; do
    curl -sSfL -A spindle -o "$work/$v.crate" \
    "https://static.crates.io/crates/ruma-state-res/ruma-state-res-$v.crate"
    done
    sha256sum "$work"/*.crate # $old: SPINDLE-PATCH.md's; $new: crates.io's `checksum` for it
    git init -q -b base "$work/tree" && cd "$work/tree"
    tar xzf "../$old.crate" --strip-components=1 && git add -A && git commit -qm "ruma-state-res $old"
    git switch -qc spindle
    git rm -rq . && cp -a "$repo/vendor/ruma-state-res/." . && git add -A && git commit -qm "Spindle patch"
    git switch -q base
    git rm -rq . && tar xzf "../$new.crate" --strip-components=1 && git add -A && git commit -qm "ruma-state-res $new"
    git switch -q spindle && git rebase base # a conflict is a hunk upstream touched

    Resolve each conflict by the intent of its table row, not by line. For a Cargo.lock conflict, take upstream's file. The test build in step 5 makes it again. Then copy the result back: rm -rf "$repo/vendor/ruma-state-res" && mkdir "$repo/vendor/ruma-state-res" && git archive HEAD | tar x -C "$repo/vendor/ruma-state-res".

  3. Record the new base. In SPINDLE-PATCH.md: the version, the crate checksum, and the upstream commit from the new crate's .cargo_vcs_info.json. The vendored Cargo.toml carries the version, which is where the script reads it from.

  4. Rewrite the table. ruma-drift.py --hunks prints the rows for the new tree. It keeps the reason of each row that did not change. A row with an empty reason moved or is new: write its reason. Then ruma-drift.py --check passes and --upstream reports no drift.

  5. Test. Run upstream's tests with the patch (cargo test --manifest-path vendor/ruma-state-res/Cargo.toml). Then run Spindle's tests for state resolution, imported rejections and room recovery, the gates ADR 0006 names. Then run cargo update -p ruma-state-res to update the workspace lock. Put the drift section of the report in the pull request description. The reviewer then reads the upstream change next to the rebased patch.