Your Palette Validator Only Validates the Pairs You Run

Two lines on our dashboard's dark mode sat at a color distance of ΔE 1.6 under deutan simulation. For a red-green colorblind reader — the most common kind, several percent of men — those two series were the same color. Same chart, crossing freely, distinguishable only by which one you happened to follow from the legend.

The uncomfortable part is not that this shipped. It's that the palette was validated, by a checker we trust, wired into CI, green on every run. The validator wasn't wrong. It was answering a different question than the one the chart was asking.

How a categorical palette gets validated

If you haven't wired this up before, the standard checks for a categorical (series) palette look something like:

• Lightness band — every color inside a luminance range that works on your chart surface, per mode.
• Chroma floor — nothing so desaturated it reads as gray.
• CVD separation — simulate protanopia and deuteranopia, then compute a perceptual color distance (OKLab ΔE, scaled ×100 in our tooling) for each pair of colors. Our thresholds: ΔE ≥ 8 is a pass, 6–8 is a floor band that's only acceptable with a second encoding (direct labels, texture, position), below 6 is a hard fail.
• Normal-vision floor — the same pairwise distance under unsimulated vision, with a higher bar (ΔE ≥ 15). Secondary encoding does not excuse this one: if full-color readers can't tell two lines apart, the chart is broken for everyone.

Any tool that gives you per-pair ΔE under CVD simulation can do this; ours is a ~250-line vendored script so the check runs in CI without a network.

Note the load-bearing phrase in the middle: each pair. A palette isn't validated in the abstract. It's validated relative to a pair-list, and the default pair-list — in our tooling and in most palette guidance — is adjacent pairs: color 1 vs 2, 2 vs 3, and so on down the ordered set.

Adjacent-pairs is a sensible default for the artifact it was designed around. In a stacked bar, segment 1 touches segment 2. In an ordered legend, slot order is the safety mechanism: assign series to slots in fixed order, and neighboring series get validated separation. We leaned on that mechanism hard — our palette is eight slots, assigned in order, the whole set CI-checked in light and dark mode. "The palette is validated, not chosen" is written in the repo as a rule.

Where the pair-list and the chart disagree

Here is the gap, and it has two doors.

Door one: charts that skip slots. The moment one chart draws slot 2 and slot 8 together — because slot 8 is your "this number going up is bad" color and slot 2 was the next free identity — you are rendering a pair the adjacent-pairs run never measured. We had exactly that: a sent-vs-failed mail chart on slots 2 and 8, with a code comment calling the pairing "validated-adjacent." When an adversarial review finally ran the validator on that specific pair, it failed both modes: CVD ΔE 5.6 in light mode — below even the floor band — and a normal-vision ΔE of 7.1 against a floor of 15. The comment was folklore. Nobody had ever run those two hexes against each other.

Door two: line charts. A stacked bar has a geometric notion of adjacency. Crossing lines do not. Every series in a line chart is adjacent to every other series at some x — at the crossover, the pair is all there is. So for a five-series line chart, the honest pair-list isn't four adjacent pairs. It's all ten.

Our five-series chart was the audience view: five population counts, slots 1 through 5, a textbook consecutive run — the exact pattern the adjacent-pairs doctrine blesses. Slots 1–2, 2–3, 3–4, 4–5: all validated, all green. Slots 3 and 5, which the doctrine never checks? Dark-mode ΔE 1.6. The worst pair anywhere in the app, sitting inside the most by-the-book palette usage we had.

What's the actual series ceiling of a validated palette?

Once you accept that a line chart needs all-pairs validation, a follow-up question becomes measurable: how many series can your palette honestly co-chart at all?

We brute-forced it. Eight colors, choose five: 56 combinations. Validate each with --pairs all, in both light and dark mode, against the real chart surfaces:

import { validate } from "./validate-palette.mjs";

const slots = [1, 2, 3, 4, 5, 6, 7, 8];
const passing = [];

for (const combo of choose(slots, 5)) {
  const ok = ["light", "dark"].every((mode) =>
    validate(combo.map((s) => hex(mode, s)), {
      mode,
      surface: surfaceOf(mode),
      pairs: "all",
    }).ok,
  );
  if (ok) passing.push(combo);
}
console.log(passing.length); // 0

Zero. No five colors of our validated eight can share a line chart and pass every pair in both modes.

Widening the search:

So for this palette, the honest ceiling is three crossing series per chart — four if you can genuinely lean on a second encoding. That is a property of the palette, as measurable as any contrast ratio, and we had never measured it. I'd bet most validated palettes have a similar number, and most of their owners don't know it.

The five-series chart didn't get better colors, because better colors don't exist in the set. It got split — two charts along a semantic seam (one system's populations on each), a two-series chart and a three-series chart, each passing all pairs with room to spare. Faceting is not a defeat. Below the ceiling it's the only honest option, which is presumably why small multiples keep being the answer in every serious dataviz text.

The pairs that survived got measured, not eyeballed: the two-series "good-thing vs bad-thing" charts all moved onto one blue-vs-red pairing at CVD ΔE 21.6/19.2 (light/dark), normal-vision 32.3/29.0. The one three-series chart that keeps a floor-band pair (ΔE 6.9/6.5) carries it consciously — the two series differ by an order of magnitude, so the lines rarely neighbor, and the numbers live in an adjacent table.

Gate the combinations your charts actually draw

The last mistake is the one that would have brought the whole thing back: we initially extended CI with a hand-written list of co-charted slot numbers. Which means a future re-assignment in the slot table would leave the gate validating the old pair while the charts drew the new one — the exact failure mode that produced the "validated-adjacent" folklore comment in the first place.

The fix is structural: the gate's list names series identifiers, and resolves them to colors through the same slot table the renderer reads.

const CO_CHARTED = [
  ["mail volume sent/failed", ["mail-sent", "mail-failed"]],
  ["orders completed/open/refunded",
    ["shop-orders", "shop-orders-open", "shop-orders-refunded"]],
  // ...one entry per chart that draws >1 series
] as const;

for (const [label, ids] of CO_CHARTED) {
  const palette = ids.map((id) => hexOf(seriesSlot(id), mode));
  runValidator(palette, { mode, surface, pairs: "all" }); // exit 1 on fail
}

Then prove the gate has teeth the same way you'd prove a test can fail: mutate one slot assignment back to the bad value and watch CI exit 1. Ours does. That mutation check took thirty seconds and is the difference between a gate and a decoration.

The stack-agnostic version of all of this fits in two sentences: a palette is only ever validated relative to a pair-list, and your charts define their own pair-lists — every co-rendered pair for bars, all pairs for anything that crosses. Validate the combinations you actually draw, resolve them from the same source of truth your renderer uses, and re-run the check in CI, because a validator you don't run on the combination in front of the reader validates nothing.

What found it, and what we still don't know

Worth admitting: no unit test, no reviewer reading the code, and no amount of palette discipline caught this. A 35-agent adversarial review of the change caught three other bad pairings by running the validator on them — and still waved the five-series chart through, because its slots were consecutive and consecutive meant safe. The ΔE 1.6 pair surfaced in a second, separate review of the fixes, when a fresh reviewer asked the one question nobody had: "run it all-pairs." The lesson generalizes past color: the checks that matter are the ones aimed at what ships, not at what the design intended.

Open edges, honestly held: our thresholds are one tool's calibration (OKLab ΔE ×100; CVD floor 6, target 8; normal-vision floor 15) — different CVD simulation models move borderline pairs by a point or two, so treat the bands as guidance and the method as the portable part. Tritanopia is reported but not gated, on rarity grounds. And the ceiling of three is a fact about our eight colors, not about eights in general — which is exactly the point. Measure yours. It's 56 function calls.