docs: say the pack copies each ZIP byte for byte

Since fed03c3db _add_zip_to_pack copies every archive of the collection as is. architecture.md and tools.md still said MAME and FBNeo romsets were rebuilt deterministically, so a third party rebuilding neogeo.zip to compare would read a sound pack as corrupt.
This commit is contained in:
Abdessamad Derraz committed 2026-11-05 19:53:45 +01:00
1 parent ac98a1fe8a
commit 133afc1df6
3 files changed
+25 -10

No files matched your search

+15
View File
@@ -252,6 +252,21 @@ class TestPackDeterminism(unittest.TestCase):
names = {i.filename.rsplit("/", 1)[-1] for i in zf.infolist()}
self.assertIn("romset.zip", names)
def test_a_romset_is_the_collection_bytes_and_the_wiki_says_so(self):
"""The pack copies each ZIP byte for byte since fed03c3db, and the
wiki still told a third party to rebuild romsets to compare."""
pack = self._build("out1")
with zipfile.ZipFile(pack) as zf:
member = next(n for n in zf.namelist() if n.endswith("romset.zip"))
self.assertEqual(zf.read(member), (self.bios / "romset.zip").read_bytes())
root = Path(__file__).resolve().parent.parent / "wiki"
for page in ("architecture.md", "tools.md"):
with self.subTest(page=page):
text = " ".join((root / page).read_text(encoding="utf-8").split())
self.assertNotIn("rebuilt deterministically", text)
self.assertNotIn("rebuilds MAME", text)
self.assertNotIn("Rebuild MAME BIOS ZIPs", text)
def test_every_member_carries_the_fixed_epoch(self):
"""A member's date must come from the build's inputs, not its clock.
+9 -9
View File
@@ -280,17 +280,16 @@ being intact: see [verifying a release](release-process.md#verifying-a-release).
Three things make that hold. Generated members (`README.txt`, `manifest.json`)
carry a fixed date rather than the wall clock, and the manifest's `generated`
field is read from `database.json` instead of the build time. MAME and FBNeo
romsets are rebuilt deterministically from their ROMs, so a pack does not
inherit whatever metadata the source archive happened to carry. And every
member is written with the same fixed date: `ZipFile.write` copies the source
file's mtime, which is the checkout time for a file from the collection and
the wall clock for one the build just staged in `tmp/`.
field is read from `database.json` instead of the build time. Each ZIP of the
collection, MAME and FBNeo romsets included, is copied byte for byte, so it
hashes to the SHA1 the install list and the site publish, and git fixes those
bytes. And every member is written with the same fixed date: `ZipFile.write`
copies the source file's mtime, which is the checkout time for a file from the
collection and the wall clock for one the build just staged in `tmp/`.
`tests/test_deterministic_zip.py` builds the same fixture twice and compares
the bytes, for both the platform and the emulator pack paths. Its fixture
holds a romset, because without one the comparison never reaches the rebuild
path and passes while the real packs still move.
holds a ZIP, so the comparison covers the archive copy as well as loose files.
## Collection rules
@@ -383,7 +382,8 @@ resolution.
deploy identity. No workflow builds or publishes a release.
- `safe_extract_zip()` prevents zip-slip path traversal attacks
- `deterministic_zip` rebuilds MAME ZIPs so same ROMs always produce the same hash
- a pack copies each ZIP of the collection byte for byte, so every member
hashes to the SHA1 the install list publishes
- `crypto_verify.py` and `sect233r1.py` verify 3DS RSA-2048 signatures and AES-128-CBC integrity
- ZIP inner ROM verification via `checkInsideZip()` replicates Batocera's behavior
- `md5_composite()` replicates Recalbox's composite ZIP hash
+1 -1
View File
@@ -1325,7 +1325,7 @@ before a release (wiki/release-process.md). Its lines stay in
| `validate_site.py` | Validate rendered metadata, heading order, image alternatives, JSON-LD, local resources, links and fragments, and the page size and search index budgets |
| `romset_recipes.py` | Identify which emulator version an arcade archive matches, and rebuild a pinned archive from ROMs already held |
| `scraper/romset_dat_importer.py` | Fetch and import per-set recipes from MAME `-listxml` or FBNeo DATs into `recipes/`. `--fetch` alone takes the newest MAME release, and FBNeo DATs are refreshed by git blob sha. `--fetch <revision>` reads FBNeo at that tag, branch or commit |
| `deterministic_zip.py` | Rebuild MAME BIOS ZIPs deterministically (same ROMs = same hash) |
| `deterministic_zip.py` | The fixed date pack members carry. Its archive rebuild is no longer called by the pack, which copies each ZIP byte for byte |
| `torrentzip.py` | Build TorrentZip archives for MAME/FBNeo ROM sets (archive bytes depend only on contents) |
| `crypto_verify.py` | 3DS RSA signature and AES crypto verification |
| `sect233r1.py` | Pure Python ECDSA verification on sect233r1 curve (3DS OTP cert) |