THE RECORD
Where I’m wrong
Every model here misses somewhere. These are the misses I know about, with their numbers.
| MISS | MEASURED | AGAINST | WHEN | WHAT I DID | STATUS |
|---|---|---|---|---|---|
| Tourism as a top-three concern | 81% of agents | 1% of residents | Jul 2026 | Wrote up as a fail; rebuilt the archetypes | FAIL |
| Round 2 equivalence | 12 of 24 items missed | Tolerances fixed before the run | 8 Sep 2026 | Reported in full | MISSED |
| Economic self-assessment | 6 of 6 items missed | Survey, 750 respondents | 8 Sep 2026 | Reported in full | MISSED |
| Unemployment | 9.63% synthetic | 2.0% official | 2026 | Recorded as a miss | MISSED |
| Population back-cast (September 2026 update) | 5.5% RMSE | 1.7% (linear trend) | Sep 2026 | Kept, for the policy lever | MISSED |
| Engine wiring (July 2026 audit) | Four scenarios ran one branch; the Monte Carlo band collapsed to one line | The intended design | Jul 2026 | Fixed; pinned with golden-master tests | FIXED |
| Digital twin in the title | A reviewer called it an overclaim | What the system does: scenarios, not a twin | Jun–Jul 2026 | Retitled The Andorra Scenario Engine | FIXED |
| Kendall work-leg mode split | Off by 29.4 points (reference run) | Census | Sep 2026 | Recorded in the project README | MISSED |
| Kendall walking work legs | 9.6% | 28.2% (census) | Sep 2026 | Recorded in the project README | MISSED |
| Kendall tier 3.1 (timetabled transit) | Mode split miss 30.11 points | 29.41 for the reference run | 2026 | Not adopted; kept for demos only | NOT ADOPTED |
| Andorra mobility | Unvalidated | Telecom data: protocol drafted, request not sent | – | Protocol drafted | DESIGNED, NOT YET RUN |
ROUND 2 · 24 ITEMS · 5 PASSED · 12 MISSED (6 NOT EQUIVALENT, 6 OUTSIDE POLICY TOLERANCE) · 7 NOT SCORED (1 INCONCLUSIVE, 4 DESCRIPTIVE ONLY, 2 UNLICENSED)
SOURCES AND STATUS FOR THIS PAGE · 13 CLAIMS
| Claim | Status | Source | Checked |
|---|---|---|---|
| Tourism as a top-three concern: 81% of agents against 1% of residents (July 2026). | DERIVED FROM DATA | Round 1 findings, July 2026 (own repository) | 2026-10-10 |
| Round 2 equivalence: 12 of 24 items missed (8 September 2026). | DERIVED FROM DATA | Round 2 scorecard, 8 September 2026 (own repository) | 2026-10-10 |
| Economic self-assessment: 6 of 6 items missed against a survey of 750 respondents (8 September 2026). | DERIVED FROM DATA | Round 2 findings, ECONOMIA survey block (own repository) | 2026-10-10 |
| Unemployment: 9.63% synthetic against 2.0% official (2026). | DERIVED FROM DATA | Validation schema and unemployment check (own repository) | 2026-10-10 |
| Population back-cast: 5.5% error against 1.7% for a linear trend (September 2026 update). | DERIVED FROM DATA | Scenario engine back-cast, R3c corrected run | 2026-10-10 |
| Engine wiring: four scenarios ran one branch, and p10 equalled p50 equalled p90 (July 2026). | DERIVED FROM DATA | Engine audit, July 2026 (own repository) | 2026-10-10 |
| Digital twin in the title: a reviewer called it an overclaim (June to July 2026). | DERIVED FROM DATA | Peer review of the paper | 2026-10-10 |
| Kendall Square: the reference run misses the work-leg mode split by 29.4 points against the census. | DERIVED FROM DATA | Kendall Square world README and held-out benchmark v2 (tier2, the reference arm), frozen commit e94f175 (private repository) | 2026-10-13 |
| Kendall Square: 9.6% of work legs walked against 28.2% in the census. | DERIVED FROM DATA | Kendall Square world README (tier2, the reference arm), frozen commit e94f175 (private repository) | 2026-10-13 |
| Kendall Square tier 3.1 (timetabled transit): the work-leg mode split miss grew from 29.41 to 30.11 points, so it was not adopted. | DERIVED FROM DATA | Kendall Square world README and docs/TIER3.1_DEMO.md, frozen commit e94f175 (private repository) | 2026-10-13 |
| Andorra mobility: unvalidated, telecom protocol drafted, request not sent. | DERIVED FROM DATA | Draft telecom data request, not sent (own document) | 2026-10-10 |
| Every model here misses somewhere. These are the misses I know about, with their numbers. | OWN WORDING | Round 1, Round 2 and audit findings (own repository) | 2026-10-10 |
| ROUND 2 · 24 ITEMS · 5 PASSED · 12 MISSED (6 NOT EQUIVALENT, 6 OUTSIDE POLICY TOLERANCE) · 7 NOT SCORED (1 INCONCLUSIVE, 4 DESCRIPTIVE ONLY, 2 UNLI… | DERIVED FROM DATA | Round 2 scorecard, 8 September 2026 (own repository) | 2026-10-10 |