diff --git a/wiki/architecture.md b/wiki/architecture.md index f26aec29..a0c2ab66 100644 --- a/wiki/architecture.md +++ b/wiki/architecture.md @@ -48,7 +48,7 @@ truth.py generates diff_truth.py export_native.py Pipeline runs all steps in sequence: DB, data dirs, MAME/FBNeo hashes, verify, packs, install manifests, target manifests, consistency check, -README, site. See [tools](tools.md) for the full pipeline reference. +pack integrity, README, site. See [tools](tools.md) for the full pipeline reference. ```mermaid graph LR @@ -59,12 +59,14 @@ graph LR E --> F[install manifests] F --> G[target manifests] G --> H[consistency check] - H --> I[generate_readme] + H --> H2[pack integrity] + H2 --> I[generate_readme] I --> J[generate_site] style A fill:#2d333b,stroke:#adbac7,color:#adbac7 style D fill:#2d333b,stroke:#adbac7,color:#adbac7 style E fill:#2d333b,stroke:#adbac7,color:#adbac7 + style H2 fill:#2d333b,stroke:#adbac7,color:#adbac7 style J fill:#2d333b,stroke:#adbac7,color:#adbac7 ``` @@ -234,14 +236,15 @@ user's platform, filter files by hardware target, and download with SHA1 verific ## Tests -4 test files with synthetic fixtures: +5 test files, 249 tests total: -| File | Coverage | -|------|----------| -| `test_e2e.py` | file resolution, verification, severity, cross-reference, aliases, inheritance, shared groups, data dirs, storage tiers, HLE, launchers, platform grouping, core resolution, target filtering, truth/diff, exporters | -| `test_mame_parser.py` | BIOS root set detection, ROM block parsing, macro expansion | -| `test_fbneo_parser.py` | BIOS set detection, ROM info parsing | -| `test_hash_merge.py` | MAME/FBNeo YAML merge, diff detection | +| File | Tests | Coverage | +|------|-------|----------| +| `test_e2e.py` | 186 | file resolution, verification, severity, cross-reference, aliases, inheritance, shared groups, data dirs, storage tiers, HLE, launchers, platform grouping, core resolution, target filtering, truth/diff, exporters | +| `test_pack_integrity.py` | 8 | extract ZIP packs to disk, verify paths + hashes per platform's native mode | +| `test_mame_parser.py` | 22 | BIOS root set detection, ROM block parsing, macro expansion | +| `test_fbneo_parser.py` | 16 | BIOS set detection, ROM info parsing | +| `test_hash_merge.py` | 17 | MAME/FBNeo YAML merge, diff detection | ```bash python -m unittest tests.test_e2e -v diff --git a/wiki/testing-guide.md b/wiki/testing-guide.md index 6cc59c32..2db7525f 100644 --- a/wiki/testing-guide.md +++ b/wiki/testing-guide.md @@ -11,6 +11,7 @@ Run a single test module: ```bash python -m unittest tests.test_e2e -v +python -m unittest tests.test_pack_integrity -v python -m unittest tests.test_mame_parser -v python -m unittest tests.test_fbneo_parser -v python -m unittest tests.test_hash_merge -v @@ -119,6 +120,30 @@ def test_42_my_new_behavior(self): self.assertEqual(result[0]["status"], Status.OK) ``` +### test_pack_integrity.py + +End-to-end pack verification. Extracts each platform ZIP to `tmp/` (in the +repo, not `/tmp` which is tmpfs on WSL) and verifies that every file declared +in the platform YAML: + +1. Exists at the correct path on disk after extraction +2. Has the correct hash per the platform's native verification mode + +Handles inner ZIP verification for MAME/FBNeo ROM sets (checkInsideZip, +md5_composite, inner ROM MD5) and path collision deduplication. + +8 tests (one per active platform): RetroArch, Batocera, BizHawk, EmuDeck, +Recalbox, RetroBat, RetroDECK, RomM. + +```bash +python -m unittest tests.test_pack_integrity -v +# or via CLI: +python scripts/generate_pack.py --all --verify-packs --output-dir dist/ +``` + +Integrated as pipeline step 6/8 (runs after consistency check, before +README generation). Requires packs in `dist/` — skip with `--skip-packs`. + ## Verification discipline The test suite is one layer of verification. The full quality gate is: @@ -127,6 +152,7 @@ The test suite is one layer of verification. The full quality gate is: 2. The full pipeline completes without error (`python scripts/pipeline.py --offline`) 3. No unexpected CRITICAL entries in the verify output 4. Pack file counts match verification file counts (consistency check) +5. Pack integrity passes (every declared file extractable with correct hash) If a change passes tests but breaks the pipeline, it's worth investigating before merging. Similarly, new CRITICAL entries in the verify output after a change usually indicate something to look into. The pipeline is designed so that all steps agree: if verify reports N files for a platform, the pack should contain exactly N files. diff --git a/wiki/tools.md b/wiki/tools.md index 921a2e18..25d5c9ae 100644 --- a/wiki/tools.md +++ b/wiki/tools.md @@ -7,7 +7,7 @@ All tools are Python scripts in `scripts/`. Single dependency: `pyyaml`. Run everything in sequence: ```bash -python scripts/pipeline.py --offline # DB + verify + packs + manifests + readme + site +python scripts/pipeline.py --offline # DB + verify + packs + manifests + integrity + readme + site python scripts/pipeline.py --offline --skip-packs # DB + verify only python scripts/pipeline.py --offline --skip-docs # skip readme + site generation python scripts/pipeline.py --offline --target switch # filter by hardware target @@ -20,21 +20,22 @@ Pipeline steps: | Step | Description | Skipped by | |------|-------------|------------| -| 1/9 | Generate database | - | -| 2/9 | Refresh data directories | `--offline` | +| 1/8 | Generate database | - | +| 2/8 | Refresh data directories | `--offline` | | 2a | Refresh MAME BIOS hashes | `--offline` | | 2a2 | Refresh FBNeo BIOS hashes | `--offline` | | 2b | Check buildbot staleness | only with `--check-buildbot` | | 2c | Generate truth YAMLs | only with `--with-truth` / `--with-export` | | 2d | Diff truth vs scraped | only with `--with-truth` / `--with-export` | | 2e | Export native formats | only with `--with-export` | -| 3/9 | Verify all platforms | - | -| 4/9 | Generate packs | `--skip-packs` | +| 3/8 | Verify all platforms | - | +| 4/8 | Generate packs | `--skip-packs` | | 4b | Generate install manifests | `--skip-packs` | | 4c | Generate target manifests | `--skip-packs` | -| 5/9 | Consistency check | if verify or pack skipped | -| 8/9 | Generate README | `--skip-docs` | -| 9/9 | Generate site | `--skip-docs` | +| 5/8 | Consistency check | if verify or pack skipped | +| 6/8 | Pack integrity (extract + hash) | `--skip-packs` | +| 7/8 | Generate README | `--skip-docs` | +| 8/8 | Generate site | `--skip-docs` | ## Individual tools @@ -132,6 +133,8 @@ Reports files that cores need beyond what platforms declare. python scripts/cross_reference.py # all python scripts/cross_reference.py --emulator dolphin # single python scripts/cross_reference.py --emulator dolphin --json # JSON output +python scripts/cross_reference.py --platform batocera # single platform +python scripts/cross_reference.py --platform retroarch --target switch ``` ### truth.py, generate_truth.py, diff_truth.py @@ -168,6 +171,8 @@ from upstream repositories into `data/`. ```bash python scripts/refresh_data_dirs.py python scripts/refresh_data_dirs.py --key dolphin-sys --force +python scripts/refresh_data_dirs.py --dry-run # preview without downloading +python scripts/refresh_data_dirs.py --platform batocera # single platform only ``` ### Other tools