# Japan (JP): migration notes

Internal record of the P5 migration from the old scripts to the producer: the discrepancies found, their causes, and who decided what. This is a dated record, so the numbers are as of the migration and not kept current. It is not shipped in the zip.

- **Issue:** pc-28e.7.29 (closed 2026-10-02)
- **Commit:** `a8c36ac` (worktree commit `b9bb883`, identical diff), landed in batch 2; accuracy wording later changed in `5311ad3`
- **Parity verdict:** `explained` (`row_order` only); signed off by the user
- **Producer:** `custom`, refresh disabled

## Sources

| Input | URL | Notes |
|---|---|---|
| GeoNames postal export | https://download.geonames.org/export/zip/JP.zip | The only input the old builder used |

Japan Post's KEN_ALL.CSV is the stronger official source, but it could not be reached from the original build environment (per the produce.py docstring and `.licensing-notes.md`, 2026-09-03). Not pursued in P5.

## Parity against the live zip (2026-10-02)

Rows: 146,503 live and 146,503 fresh, none only on one side, 0 changed cells. One class, `row_order`. Live is CRLF, fresh is LF. The informational sample check was `format-only`.

**Cause: GeoNames reordered its export.** Rows with the same postal_code and place_name but different wards come out in the other order; the stable sort keeps source order among ties. Examples:
- 950-0101 Eguchi: live has Konan Ku (15104) before Higashi Ku (15102); fresh has the reverse.
- 950-0102 Hosoyama: Kita Ku / Konan Ku swapped the same way.

## Decisions

| Decision | By | Basis |
|---|---|---|
| Accept `row_order` | User, 2026-10-02 ("agree to all", session 2479b1c9) | Batch 2 sign-off in `docs/parity/p5.md` |
| Describe `admin_code3` as GeoNames' 5-digit ward code (the JIS X0402 local-government code, e.g. 01102 Sapporo Kita Ku) instead of "Not populated" | Agent proposed; user approved 2026-10-02 ("agree to all", session 2479b1c9) | Live and fresh both carry these codes. The JIS X0402 label is the agent's identification from the codes; the orchestrator checked 01102 fits the scheme |
| Remove "1181 distinct values" from `admin_name2` | Agent; orchestrator kept it | A count from the data, not a fixed set of admin units. Reported to the user, who replied "agree to all" |
| Remove row, distinct-code, admin3-fill and duplicate counts from README and descriptor; drop the stale `licence_status` sentence; standard accuracy description | Orchestrator instruction (P5 brief) | |
| Accuracy description says GeoNames documents 1/4/6 and other values (e.g. 3) occur upstream | Orchestrator (`5311ad3`, all batch-2 descriptors) | Reported to the user with batch 2; "agree to all" |

## Transforms carried over

From `build_postal_codes_product.py`:
- `admin_code1` → ISO 3166-2:JP by a name crosswalk of the 47 prefectures. GeoNames' own code is alphabetical (01 = Aichi Ken), so prefixing it would be wrong on every row.
- Exact full-row duplicates dropped, first kept (the old builder recorded 380, found byte-identical in 2026-09-03 research).
- Stable sort by (admin_code1, postal_code, place_name), so the file runs north to south.
- `alternative_city_name` empty. Everything else verbatim, including GeoNames' stray spaces ("Sapporo Shi ", " Kita Ku"). `admin_code2` is GeoNames' internal id.
- The old asserts are `ProducerError`s: known prefecture, NNN-NNNN postal code, no literal "nan", no float accuracy, unique (postal_code, place_name, admin_code2, admin_name3), all 47 prefectures present. No row-count assert.
- `build_postal_codes_product.py` and `publish_r2.py` removed.

## Open questions

- `ATTRIBUTION.txt` ("146,503 rows", the 46,265 admin_name3 fill count) and `.licensing-notes.md` still carry counts; left for the end-of-epic sweep (pc-hfz, which subsumes pc-3d2).

## Provenance

Reconstructed 2026-10-05 from the agent transcript `2479b1c9-c4dd-47e5-be82-34ccf8d49a80/subagents/agent-a6ea04a5f5e0b2059.jsonl`, the orchestrator session `2479b1c9-…`, the commit messages (`a8c36ac`, `5311ad3`), the bd close reason and `docs/parity/p5.md`.
