lans.cloud is ~87 free single-purpose web tools, and on Friday I was shipping a nice little feature: cross-listing the review games onto the classroom tools hub, so teachers browsing timers and seating charts would find the whack-a-mole quiz game too. Twelve files, seven new tests, the usual routine: commit, push, deploy.
The push ran for fourteen minutes. The pre-push hook — our full vitest suite — did its job beautifully:
Test Files 165 passed (165)
Tests 1616 passed (1616)
Duration 818.50s
And then the push failed. Exit code 141.
Sixteen hundred green tests, and nothing landed on GitHub. The best part: the evidence that it was always going to fail had scrolled past ten minutes earlier, and I'd read right over it.
The line in the middle of the log
Here's the actual output, in order:
> vitest run
RUN v4.1.10 /home/sonny/tool-sites
Connection to github.com closed by remote host.
Test Files 165 passed (165)
Tests 1616 passed (1616)
Look at where that SSH message sits. Not after the tests — during them. The connection to GitHub died somewhere around minute two or three of a thirteen-minute test run. Everything after that line was the suite diligently proving the code was fine, so it could hand a perfect report to a socket that no longer existed.
Exit 141 is the confession: 141 = 128 + 13, and signal 13 is SIGPIPE. Git tried to write the pack down the connection and found nobody listening.
The part I had backwards
My mental model of git push was: run the pre-push hook, and if it passes, connect and push. Reasonable. Wrong.
The actual order for SSH transport:
git pushopens the SSH connection first and fetches the remote's ref advertisement — it has to, because the pre-push hook's whole contract is receiving<local ref> <local sha> <remote ref> <remote sha>on stdin, and the remote SHAs come from the server.- Then it runs your pre-push hook. The connection sits open. Idle.
- Then, hook willing, it sends the pack over that same connection.
So every second your pre-push hook runs, it's inside an open, silent SSH session to GitHub — and GitHub, like any sane operator, reaps idle connections after a few minutes. A long hook isn't slow. It's a countdown. The tests weren't gating the push; they were outliving it.
This is the kind of failure that hides well, too: on a fast day (--changed mode picking up three test files, say) the hook finishes inside the timeout and everything works. The failure only appears when the suite is long enough — which for us meant "whenever it mattered most."
The false fix
First response: make the hook fast. I switched pre-push from the full suite to vitest run --changed origin/master — only run tests related to files that actually changed. Seconds instead of minutes. Problem solved.
The very next push died the same death, after another full 14-minute suite.
Why? That push changed package.json. When a config file changes, vitest's --changed can't trace the blast radius through the module graph — a dependency bump could affect anything — so it does the only safe thing and runs everything. Completely correct behavior. And it means "my pre-push hook is fast now" is a statement about the average case, while the SSH timeout only cares about the worst case. Any push that touches package.json, a lockfile, or shared config re-arms the bomb.
The real fix: get the gate out of the SSH window
The insight, once I stopped patching symptoms: the problem was never the suite's duration. It was where the suite runs. Nothing about "run tests before pushing" requires the tests to run between SSH-connect and pack-send.
Our pushes go through a small ship script (it pushes, then reads the remote ref back to prove the push actually landed — piped git output has lied to us before). The gate moved into the script, before the push:
# Test gate BEFORE the push, not in the pre-push hook: git opens the SSH
# connection to GitHub before running pre-push, and GitHub drops the idle
# socket during a long test run. Running the gate here keeps tests out of
# the SSH window entirely; the push below skips the hook because this IS
# that gate.
npx vitest run --changed origin/master --passWithNoTests
git push --no-verify
Same tests, same rigor, same failure behavior — a red suite still stops the push, because set -euo pipefail halts the script before git push ever runs. The only difference is sequencing: the tests run with no connection open, and the push happens on a fresh socket that's alive for the two seconds it actually needs.
Two supporting layers, because one fix is a fix and two is a policy:
The hook stays, unchanged, for anyone who runs a raw git push and bypasses the script. In the common case it's seconds of --changed tests; in the package-json case it's risky again — which is fine, because the script is the documented path.
SSH keepalives for GitHub, as the belt to the script's braces:
# ~/.ssh/config
Host github.com
ServerAliveInterval 30
ServerAliveCountMax 40
Now even a hook that does take minutes sends a keepalive probe every 30 seconds, and the "idle" connection is never idle. With this alone, the original 13-minute hook would probably have survived. But "probably survives the timeout" is a bet; "never enters the race" is an answer.
The numbers
The same investigation forced a look at the whole local pipeline (the pre-commit hook was running a full production build on every commit — a story of its own). After rebalancing — cached lint and incremental typecheck in the hook, the build dropped from it (every deploy rebuilds anyway, and a periodic full check runs the rest), the test gate relocated as above:
| Step | Before | After |
|---|---|---|
git commit (hooks) |
~10 min | 1m 15s |
git push (ship script) |
~14 min, failing | 33s |
From "start the push, go make coffee, come back to a SIGPIPE" to under a minute round-trip.
What I'd tell past me
- A pre-push hook runs inside an open SSH connection to the remote. Its duration is bounded by an idle timeout you don't control and can't see. Budget seconds, not minutes.
- Exit 141 means SIGPIPE. When a push "fails" with 141 after your hook succeeded, stop reading test output and start looking for the connection message buried mid-log. The socket died first; everything after was theater.
- "Fast in the average case" doesn't defuse a worst-case bomb.
vitest --changedis the right tool, and it will still legitimately run your entire suite the moment shared config changes. Design for that push, because that's the push that kills you. - Sequencing beats speed. The durable fix wasn't making the gate faster — it was moving it outside the window entirely. If a check doesn't need an open connection, don't run it inside one.
The review games made it to the classroom hub about thirty seconds after the fix landed. The suite passed again. This time, something was listening.