The first hour is the whole job

The page you are reading is new. Building it was a small piece of work with a lot of surface — text in a file at one end, something a person has to want to read at the other — and what it taught us has nothing to do with typography.

It is that the things which went wrong went wrong quietly. Each of the three below was, at the moment we found it, sitting behind a full set of green checks.

An ordered list that lost its numbers

The first time we opened a page built with it, the lists had no markers. Not the wrong markers. None.

Tailwind, the CSS framework this site is built on, ships a base layer that clears browser defaults so that everything starts from the same blank slate, and one of the things it clears is list markers. Our stylesheet for article text never put them back.

On a bulleted list that is cosmetic — you get a stack of indented lines and the meaning survives. On a numbered list it is not cosmetic. Step three stops being step three. The steps still run in order down the page, but nothing says that they are steps, and a reader skimming for "the fourth point" has nothing to find. That is a change to what the page says, not to how it looks.

Every check we had was green, and they were green for an honest reason. The ones that covered the stylesheet read it as text, and the text was doing exactly what it said. What was missing was a rule nobody had written, and no test asserts the absence of a rule nobody thought of.

The evidence had been sitting in the same file the whole time, a little further down: a line setting the colour of the list marker. We had been carefully styling something that could never be drawn.

A check that cannot fail

A check that cannot fail is worse than no check at all. With no check you keep looking; a check that cannot fail tells you to stop, and says so in green.

The design of this site is held in named values — a colour, a step on the spacing scale, a size on the type scale. The rule for the article stylesheet was that it must use those names rather than hard-coding numbers into itself, so that when the design moves, the writing moves with it. We wrote a check to enforce the rule. It scanned the stylesheet for pixel values and hex colour codes.

Those are not the units the file is written in. It is relative units and named colours nearly all the way down. So the check ran on every commit, passed every time, and would have gone on passing if someone had typed a font size straight into the file in the units the file actually uses — which is the one thing it existed to prevent.

It is worth being precise about how that happens, because it was not carelessness. We wrote the check against a claim: that not one value in the stylesheet is typed by hand. The claim was not true, and had never been true. A few values genuinely are typed by hand — the width a wide image is allowed to reach, the length of a horizontal rule — because those are structure rather than design. The check failed on them, so we narrowed it slightly to let them through. It failed again, and we narrowed it again. That happened three times. Each narrowing was individually reasonable, and by the end the check could no longer catch the mistake it had been written to catch.

The rule we took from it: when a check gets narrower so that it will pass, the narrowing is the finding. Either the code is wrong or the claim behind the check is wrong, and both are worth knowing. What you cannot do is treat a green you did not earn the same as one you did.

Then check the instrument

The habit that catches this is to break the code on purpose and watch the check go red. Type the wrong value in, run the check, confirm it fails, put the value back. A check that has never failed has never been tested. There is a name for this — mutation testing — and tools that do it at scale. We do it by hand, one check at a time, on the things we would be embarrassed to get wrong.

Then it happened to us anyway. One of these deliberate breakages came back green, and we were one step away from writing down that we had found a blind spot in the tests. We had not. The script that introduced the error had matched nothing — it was looking for a line indented with six spaces, and the file uses four — so the run had measured the original, unbroken file and passed for the most boring reason available.

A break that never happened looks exactly like a check that never fires. Both are green, and nothing on the screen tells them apart. The fix was small: the script now confirms the file actually changed before it runs anything, and stops loudly if it did not. That lesson is worth more than the defect it was chasing: every habit we had for testing our code quietly assumed the thing doing the testing was doing what we told it.

What we do now

None of this is sophisticated, and none of it is fast. The first hour on a new check goes into proving it can fail, and that hour is what makes every green after it worth anything. The rest is a short list.

• Write the check, then break the code and watch it fail. Green on the first run means nothing yet.
• Confirm the break actually landed before believing the colour it produced.
• When a check has to be narrowed in order to pass, stop and write down why.
• Open the page. In a browser, on a screen, with eyes. Every time.

The last one keeps earning its place. The missing list markers were found by somebody opening the page and reading it. Nothing we had written could have caught them: every file involved was individually correct, and the defect only existed once a browser had put them together.

It is the same discipline behind the numbers we publish for this site. Our performance page carries real Lighthouse results, the conditions they were taken under, and the punishing mobile profile where we score worst — not only the desktop run where we score best. A number you can go and check yourself is worth more than a number we assert, for the same reason a check that can fail is worth more than one that cannot.

These notes are what we are learning while we build the thing: the defects, the wrong turns, and the occasional lesson worth keeping. It is the working record, published because we would rather be checkable than impressive.