THE INDEX · DO SYNTHETIC PEOPLE BEHAVE LIKE REAL ONES?
90,000
- ROUND 2 HASH-LOCKED BEFORE RUN
- IN PREPARATION
PROBLEM
A scenario engine has no people in it, and policies land on people who support, reject or adapt to them.
QUESTION
Does making agents more human make a synthetic population behave more like the real one?
SYSTEM
A synthetic population at national scale: 90,000 agents for a country of about 87,000 people.
A language model writes 123 archetypes ($9.55 in total); deterministic code expands them into 90,000 people (76,321 adults, 13,679 children) in 40,520 households, a four-layer social network of 575,405 ties, and a day of trips routed on the street network (313,527 trips, 312,736 of them routed). Every generated field carries one of four evidence licenses. Two JSON contracts couple the population to the scenario engine.
EVIDENCE
Household structure matches the official register (band TVD 0.0043).
81% of my first synthetic population did. 1% of real residents did. I wrote it up as a fail.
The population was demographically faithful and attitudinally wrong.
The second test was pre-specified and hash-locked in my repository before a single run, an internal pre-registration. I ran it once, on 8 September 2026. Housing tenure matched (TVD 0.003). Economic self-assessment missed on all six items.
Across all items, 12 of 24 missed.
Results moved by less than 0.11 standard deviations. A twin population with Andorra’s names removed reproduced the results, so the fit and the misfit travel with the generated profile, not with the model’s knowledge of Andorra.
STATUS AND LIMITS
Only four constructs in the whole schema can be checked against real surveys at all. The synthetic unemployment rate is 9.63%; the official one is 2.0%.
Mobility is unvalidated: a telecom-based validation protocol is drafted, the data request has not been sent, and the comparison has not been run. It is the subject of a paper in preparation.
The honest answer to the question at the top is still partly no – structure transfers and dispositions do not yet.
PLANNED OUTPUTS
Synthetic Agent Generation Pipeline (working title): the methodology and architecture for generating realistic synthetic populations.
- IN PREPARATION
Mobility Validation Against Telecom Data (working title): validating simulated agent movements against real-world telecommunications data. The comparison has not been run yet.
- IN PREPARATION
- COMPARISON NOT YET RUN
Public Opinion Validation Against Surveys (working title): comparing synthetic agents’ opinions and responses against actual survey data. The Round 1 and Round 2 findings reported on this site are its evidence.
- IN PREPARATION
MY ROLE
I designed the pipeline, the validation harness, the pre-specifications and the coupling contracts.
CREDITS
City Science team (TechRxiv preprint: Mora, Atchade-Adelomou, Alonso-Pastor et al.). My part: the coupling, identification checks and outcome memory.
Survey data: Observatori d’Andorra, aggregates only. Supervision: Kent Larson, Luis Alonso.
SOURCES AND STATUS FOR THIS PAGE · 24 CLAIMS
| Claim | Status | Source | Checked |
|---|---|---|---|
| A scenario engine has no people in it, and policies land on people who support, reject or adapt to them. | OWN WORDING | My Catalan pitch, February 2026 | 2026-10-10 |
| Does making agents more human make a synthetic population behave more like the real one? | OWN RECORD | My Media Lab application progress deck, slide 14 (own document) | 2026-10-10 |
| A synthetic population at national scale: 90,000 agents for a country of about 87,000 people. | DERIVED FROM DATA | Population run P04 against the official register | 2026-10-10 |
| A language model writes 123 archetypes ($9.55 in total); deterministic code expands them into 90,000 people (76,321 adults, 13,679 children) in 40,… | OWN WORDING | Run metadata of populations P04 and P09, frozen September 2026 (own repository) | 2026-10-10 |
| Household structure matches the official register (band TVD 0.0043). | OWN WORDING | Household reconciliation against the official register (own repository) | 2026-10-10 |
| 81% of my first synthetic population did. 1% of real residents did. I wrote it up as a fail. | OWN WORDING | Round 1 findings, July 2026 (own repository) | 2026-10-10 |
| The population was demographically faithful and attitudinally wrong. | OWN RECORD | My independent research proposal, part II (own document) | 2026-10-10 |
| The second test was pre-specified and hash-locked in my repository before a single run, an internal pre-registration. I ran it once, on 8 September… | OWN WORDING | Round 2 findings, 8 September 2026 (own repository) | 2026-10-10 |
| Across all items, 12 of 24 missed. | OWN WORDING | Round 2 scorecard, 8 September 2026 (own repository) | 2026-10-10 |
| Results moved by less than 0.11 standard deviations. | OWN WORDING | Anonymised-country control, Round 2 findings (own repository) | 2026-10-10 |
| A twin population with Andorra's names removed reproduced the results, so the fit and the misfit travel with the generated profile, not with the mo… | OWN WORDING | Anonymised-country control, Round 2 findings (own repository) | 2026-10-10 |
| Only four constructs in the whole schema can be checked against real surveys at all. The synthetic unemployment rate is 9.63%; the official one is … | OWN WORDING | Validation schema and unemployment check, Round 2 (own repository) | 2026-10-10 |
| Mobility is unvalidated: a telecom-based validation protocol is drafted, the data request has not been sent, and the comparison has not been run. I… | OWN RECORD | Draft telecom data request, not sent (own document) | 2026-10-10 |
| The honest answer to the question at the top is still partly no – structure transfers and dispositions do not yet. | OWN RECORD | Speaker notes of my progress deck, slide 14 (own document) | 2026-10-10 |
| Synthetic Agent Generation Pipeline (working title): the methodology and architecture for generating realistic synthetic populations. | OWN RECORD | Paper in preparation, working title (owner direction, 12 October 2026); not submitted | 2026-10-10 |
| IN PREPARATION | OWN RECORD | Paper in preparation, working title (owner direction, 12 October 2026); not submitted | 2026-10-10 |
| Mobility Validation Against Telecom Data (working title): validating simulated agent movements against real-world telecommunications data. The comp… | OWN RECORD | Paper in preparation, working title (owner direction, 12 October 2026); not submitted | 2026-10-10 |
| COMPARISON NOT YET RUN | OWN RECORD | Paper in preparation, working title (owner direction, 12 October 2026) | 2026-10-10 |
| Public Opinion Validation Against Surveys (working title): comparing synthetic agents' opinions and responses against actual survey data. The Round… | OWN RECORD | Paper in preparation, working title (owner direction, 12 October 2026); Round 1 and Round 2 findings (own repository) | 2026-10-10 |
| I designed the pipeline, the validation harness, the pre-specifications and the coupling contracts. | OWN RECORD | Own record; repository history | 2026-10-10 |
| City Science team (TechRxiv preprint: Mora, Atchade-Adelomou, Alonso-Pastor et al.). My part: the coupling, identification checks and outcome memory. | VERIFIED | h-ABM preprint (TechRxiv); my coupling commits, August 2026 | 2026-10-10 |
| Survey data: Observatori d'Andorra, aggregates only. Supervision: Kent Larson, Luis Alonso. | VERIFIED | Survey data agreement; MIT visiting-student letters | 2026-10-10 |
| ROUND 2 HASH-LOCKED BEFORE RUN | OWN RECORD | Round 2 specification and its hash (own repository) | 2026-10-10 |
| 90,000 | DERIVED FROM DATA | Population run P04, frozen September 2026 | 2026-10-10 |