Tier resolvers verified against a 6,190-row labeled sample: 99.40% precision on the priority stratum.

The full M2 deliverable is a 16-section report that names the client's repo, commits, code paths, card catalog rows, and real marketplace listings row-by-row. Publishing that would publish the client's private data, so this page carries the verification method and every aggregate result instead, with identifying details removed or generalized (the flagship game's strata are labeled flagship_*; set names are replaced with "set A / set B"). The metrics are carried unchanged from the private report.

The milestone verb was "tier 1/2 resolvers verified against sample data." The weak reading is a hand-picked test set. The strong reading, and the one delivered: the platform's production matcher (the full tiered cascade, card-code tier through shrinking-window fallback, plus the set-contradiction guard) scored against all 6,190 labeled listings of the client-provided evaluation sample: 6,190 anonymized eBay listings, a 103,038-row catalog snapshot, 132,827 price rows, and a ground-truth label file.

How it was run: a scoring harness built as a parallel of the platform's own test harness, changing only the data source (the 6,190-row sample instead of the small golden fixtures), driving the real title parser and the real tiered matcher per listing, with no live database in the loop. The scorer writes a human-readable report plus a per-listing JSONL, and every figure below traces to those evidence files. The 20-test hand-picked regression gate was kept, but as the gate, not the verification.

One property of the sample is stated before any number: per the sample's own README, the ground truth is the platform's own match labels, mostly generated before the fixes under test. It is not an independent oracle. A matcher-vs-ground-truth disagreement can mean the matcher is wrong or the pre-fix label is wrong, and the disagreement analysis below separates those cases instead of counting every disagreement as a matcher error.

Scored 6,190 listings; commit threshold 0.70.

Per-stratum:

flagship_high is the headline: 99.40% precision on the 3,000-listing throughput stratum, the product-priority game and 89% of the sample.

Per-tier (committed rows; precision within tier counted against GT-committed rows only):

An honest scope note rather than a buried one: the sample export has the eBay item-specific columns all null, so the exact-aspect and fuzzy-aspect tiers almost never fire on it (1 commit between them). Those tiers are exercised by the hand-picked gate with constructed aspect-bearing inputs; the sample exercises the code tier and the title tiers at scale. The report says this plainly and reads the contract verb as the tiered resolver as a whole.

567 of 6,190 rows (9.16%) disagree with ground truth. Classified:

The 20 flagship-game conflicts were read row by row, not averaged: roughly 2 are ground-truth bugs the matcher actually fixed (the pre-fix labels still carry a bug that was since repaired), roughly 13 are the matcher at confidence 1.00 against weak fuzzy GT labels (confidence 0.71-0.78) on one set's prefixed titles, an attribution question for the next milestone rather than a demonstrated error, and roughly 1 is a genuine matcher miss (a set-namesake collision: a card sharing its name with its own set's sealed product). Read plainly: the matcher commits the same id as ground truth on 5,497 of 5,517 flagship-game listings.

The fix under test demotes a match to the review band when the listing title names one set and the matched catalog row belongs to a different one. The same scorer was run against a git worktree of the pre-fix revision; identical harness, identical 6,190 listings.

The honest reading, and the one shipped to the client: the fix raises precision on the priority stratum, costs measured coverage concentrated in the mid stratum, and is roughly flat overall. The 124 newly-abstained rows route to the near-miss review queue, reviewable instead of auto-committed. Whether the trade is net-positive cannot be settled against pre-fix ground truth; the report says so and routes the question to the next milestone instead of declaring victory.

An earlier draft of the deliverable claimed the fix raised overall precision and reduced conflicts. Both claims came from a stale scorer run. The shipped report re-derived every before/after figure from the regenerated evidence files, corrected the direction, and recorded the lesson: a before/after claim is re-derived from the evidence file every time the matcher or scorer moves, never carried forward as prose.

Both were found on 2026-05-12 during verification, reported the same day, fixed upstream by the platform team within two days, and re-verified against production after deploy.

• Set-contradiction commits. Listings whose titles named one set (call it set B) were being committed at high confidence to cards from a sibling set (set A): 106 live rows pre-fix. Post-fix: 0 rows on the original pattern, with the ambiguous population correctly demoted to the review band (444 rows) instead of committed. The residual high-confidence rows were read individually: promotional-era titles and cross-set name twins, correctly matched, a different class.
• Leading set-name parse gap. A leading set-name segment in the title could survive parsing and drag a single-card listing onto the set's sealed booster-box product: 4 live rows pre-fix, 0 post-fix. The one listing still matched to that box on re-verification is a genuine sealed listing correctly matched (only its listing-type classifier tag is wrong), which the report states rather than hides.
• The broader "card matched to a sealed product" class was re-measured at 68 rows and decomposed by reading every row: 54 are a catalog data gap (real cards whose catalog rows carry a null card-type field), 13 are genuine sealed listings correctly matched with only a classifier tag wrong, and exactly 1 is a real single-card-to-sealed mismatch (the set-namesake collision above). The earlier draft had treated the whole class as one bug; the row-by-row read replaced a 53-row scare with a 1-row defect and a catalog-quality follow-up.

The three migrations recommended in M1 were deployed by the platform team between milestones. The M2 report did not take "deployed" on faith; each was independently validated against production from the read-only role:

• The review-band partial index: EXPLAIN on the representative queue query shows the planner actually choosing the index (bitmap-combined with the type index); "exists" is not "used", so the plan was captured, not assumed.
• The two CHECK constraints: both read back as validated in the catalog (so Postgres itself has certified every existing row), and the enum was cross-checked against the live value distribution: all 10 live values inside the constraint, none missing. The pre-flight recon had already widened the constraint from the 5 values the design assumed to the 10 that production actually holds, which is exactly the class of gap pre-validation exists to catch.
• The provenance audit column: present AND being populated (17 non-null rows split across the two bridge strategies at the read). A column that exists but is never written would have been a finding; it was checked, not assumed.

The observability plan hit two managed-platform walls, and the deliverable documents the evolution rather than pretending the first plan worked: the direct stats-role grant is blocked on the managed platform, and the cluster-stats reset is reserved for the platform's internal superuser (so even a definer-owned wrapper cannot fire it). The mechanism that survived: three SECURITY DEFINER read wrappers for the stats views, plus a snapshot-baseline pair for the unused-index observation window (baseline the counters into a table, diff live counters against the baseline later). The report's engineering review of that approach, kept in full: snapshot-baseline beats a stats reset for a windowed observation because it is non-destructive (no collateral wipe of every other counter), self-serve (no platform support ticket), restartable (re-baseline any time), and analytically identical (scans-since-baseline is the same delta a reset would produce with the baseline forced to zero), with one stated caveat (counter monotonicity across the window, void only on crash-recovery or a major-version upgrade).

The draft had claimed the platform's heaviest recurring query (a materialized-view refresh at ~3.5 seconds per call) was spilling its sort to disk and that a memory-setting bump would fix it. An EXPLAIN (ANALYZE, BUFFERS) run for the final report showed the sort is an incremental sort with a 28 kB peak, quicksort, no disk line, all buffers shared-hit. The claim was withdrawn in the shipped deliverable and replaced with the measured picture: the temp traffic attributed to the refresh belongs to the refresh machinery as a whole, the component queries do not individually spill, and the real memory finding is a headroom watch (the refresh's diff-join hash sits at 1.68x headroom under the effective budget and would start batching once the view grows past roughly 92,000 rows; it holds 54,905 today). A wrong mechanism with a confident fix was replaced by a right mechanism with a threshold to watch, and the correction is printed in the deliverable itself.

The platform's strict matcher opens with a warning comment (quoted verbatim in the private report; the client's name is replaced here):

"Wrong matches = wrong arbitrage = [the client] buys the wrong card. Return null rather than guess."

Every M2 decision sits behind that gate: demote ambiguous, abstain on contradiction, never confidently-wrong. The 96.59% overall / 99.40% priority-stratum precision above was reached by a matcher that abstains on a quarter of the sample rather than guess, and the deliverable treats that abstention rate as a feature being verified, not a coverage failure to spin.