Skip to content

Commit 6a70cf0

Browse files
reid-spencerclaude
andcommitted
ship: never sbt clean test publish -- the gate could skip most of the suite
Found while shipping 2.0.0. Step 9 prescribed `sbt clean test publish`, and bare `test` in sbt 2 resolves to testQuick, which skips suites it judges unaffected. That judgement SURVIVES clean, because the action cache does. The build's own alias comments record what this cost once: the JS row of a GREEN run executing 109 of 567 tests, with languageJS, passesJS and testkitJS never running at all. So the release gate that was supposed to be the last line of defence was the one command in the procedure that could report success without running. The tJVM/tJS/tNative aliases already exist to defeat exactly this -- they are `testOnly *`, which ignores incremental state -- and the skill simply did not use them. Step 9 is now `sbt "; clean; tJVM; tJS; tNative; publish"`, with the reason attached so it does not get "simplified" back, plus the instruction to count `Suites: completed` lines and check for `No tests to run`. Also adds the major-release rule Reid set while shipping 2.0.0: certify from a genuinely COLD cache first, since clean does not clear the shared store. That means shutdown (the -D applies only at server boot), a throwaway localcache, and verifying afterwards that the directory grew -- an empty one means nothing was actually cold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 64f402d commit 6a70cf0

1 file changed

Lines changed: 29 additions & 2 deletions

File tree

.claude/skills/ship/SKILL.md

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,8 +94,35 @@ not provided:
9494

9595
9. Run the full test suite and publish all modules:
9696
```
97-
sbt clean test publish
98-
```
97+
sbt "; clean; tJVM; tJS; tNative; publish"
98+
```
99+
**NEVER `sbt clean test publish`.** That was prescribed here until
100+
2026-08-27 and it is a gate that can skip most of the suite while
101+
reporting success. In sbt 2 bare `test` resolves to **`testQuick`**,
102+
which skips suites it judges unaffected — and **that judgement
103+
survives `clean`, because the action cache does**. It once left the
104+
JS row of a GREEN run executing 109 of 567 tests, with `languageJS`,
105+
`passesJS` and `testkitJS` never running at all. `tJVM`/`tJS`/`tNative`
106+
exist precisely to defeat this: they are `testOnly *`, which ignores
107+
incremental state. See the alias comments in `build.sbt`.
108+
109+
**Count what ran.** One `Suites: completed N` line per module in each
110+
alias, and zero `No tests to run`. A short count means a module was
111+
skipped or the `;` chain aborted at a failure — either way, look.
112+
113+
**For a MAJOR release, certify from a genuinely cold cache first**
114+
(Reid, 2026-08-27, shipping 2.0.0). `clean` removes `target/` but NOT
115+
the shared store at `~/.cache/sbt/v2` (or `~/Library/Caches/sbt/v2`),
116+
so results can still be replayed:
117+
```
118+
sbt -batch shutdown # a warm server IGNORES the -D
119+
sbt --server -Dsbt.global.localcache=/tmp/sbt-verify-<ver> -batch "; clean; tJVM; tJS; tNative"
120+
```
121+
The property applies only at **server boot**, so the shutdown is not
122+
optional. Afterwards the throwaway directory must EXIST and have GROWN
123+
(`du -sh`) — an empty one means the run was served from the shared cache
124+
and certified nothing.
125+
99126
Because the tag is on HEAD and the tree is clean,
100127
BuildInfo and all published artifacts will carry the
101128
clean `<VERSION>`. Verify in the sbt output.

0 commit comments

Comments
 (0)