docs: sync wiki sources with pipeline changes

Update wiki source files (the single source of truth for the site):
- tools.md: renumber pipeline steps 1-8, add step 6 (pack integrity),
  add missing CLI flags for cross_reference.py and refresh_data_dirs.py
- architecture.md: update mermaid diagram with pack integrity step,
  fix test file count (5 files, 249 tests)
- testing-guide.md: add test_pack_integrity section, add step 5 to
  verification discipline checklist
This commit is contained in:
Abdessamad Derraz
2026-04-01 13:08:19 +02:00
parent 6eca4c416a
commit 0e6db8abdf
3 changed files with 51 additions and 17 deletions

View File

@@ -48,7 +48,7 @@ truth.py generates diff_truth.py export_native.py
Pipeline runs all steps in sequence: DB, data dirs, MAME/FBNeo hashes, Pipeline runs all steps in sequence: DB, data dirs, MAME/FBNeo hashes,
verify, packs, install manifests, target manifests, consistency check, verify, packs, install manifests, target manifests, consistency check,
README, site. See [tools](tools.md) for the full pipeline reference. pack integrity, README, site. See [tools](tools.md) for the full pipeline reference.
```mermaid ```mermaid
graph LR graph LR
@@ -59,12 +59,14 @@ graph LR
E --> F[install manifests] E --> F[install manifests]
F --> G[target manifests] F --> G[target manifests]
G --> H[consistency check] G --> H[consistency check]
H --> I[generate_readme] H --> H2[pack integrity]
H2 --> I[generate_readme]
I --> J[generate_site] I --> J[generate_site]
style A fill:#2d333b,stroke:#adbac7,color:#adbac7 style A fill:#2d333b,stroke:#adbac7,color:#adbac7
style D fill:#2d333b,stroke:#adbac7,color:#adbac7 style D fill:#2d333b,stroke:#adbac7,color:#adbac7
style E fill:#2d333b,stroke:#adbac7,color:#adbac7 style E fill:#2d333b,stroke:#adbac7,color:#adbac7
style H2 fill:#2d333b,stroke:#adbac7,color:#adbac7
style J fill:#2d333b,stroke:#adbac7,color:#adbac7 style J fill:#2d333b,stroke:#adbac7,color:#adbac7
``` ```
@@ -234,14 +236,15 @@ user's platform, filter files by hardware target, and download with SHA1 verific
## Tests ## Tests
4 test files with synthetic fixtures: 5 test files, 249 tests total:
| File | Coverage | | File | Tests | Coverage |
|------|----------| |------|-------|----------|
| `test_e2e.py` | file resolution, verification, severity, cross-reference, aliases, inheritance, shared groups, data dirs, storage tiers, HLE, launchers, platform grouping, core resolution, target filtering, truth/diff, exporters | | `test_e2e.py` | 186 | file resolution, verification, severity, cross-reference, aliases, inheritance, shared groups, data dirs, storage tiers, HLE, launchers, platform grouping, core resolution, target filtering, truth/diff, exporters |
| `test_mame_parser.py` | BIOS root set detection, ROM block parsing, macro expansion | | `test_pack_integrity.py` | 8 | extract ZIP packs to disk, verify paths + hashes per platform's native mode |
| `test_fbneo_parser.py` | BIOS set detection, ROM info parsing | | `test_mame_parser.py` | 22 | BIOS root set detection, ROM block parsing, macro expansion |
| `test_hash_merge.py` | MAME/FBNeo YAML merge, diff detection | | `test_fbneo_parser.py` | 16 | BIOS set detection, ROM info parsing |
| `test_hash_merge.py` | 17 | MAME/FBNeo YAML merge, diff detection |
```bash ```bash
python -m unittest tests.test_e2e -v python -m unittest tests.test_e2e -v

View File

@@ -11,6 +11,7 @@ Run a single test module:
```bash ```bash
python -m unittest tests.test_e2e -v python -m unittest tests.test_e2e -v
python -m unittest tests.test_pack_integrity -v
python -m unittest tests.test_mame_parser -v python -m unittest tests.test_mame_parser -v
python -m unittest tests.test_fbneo_parser -v python -m unittest tests.test_fbneo_parser -v
python -m unittest tests.test_hash_merge -v python -m unittest tests.test_hash_merge -v
@@ -119,6 +120,30 @@ def test_42_my_new_behavior(self):
self.assertEqual(result[0]["status"], Status.OK) self.assertEqual(result[0]["status"], Status.OK)
``` ```
### test_pack_integrity.py
End-to-end pack verification. Extracts each platform ZIP to `tmp/` (in the
repo, not `/tmp` which is tmpfs on WSL) and verifies that every file declared
in the platform YAML:
1. Exists at the correct path on disk after extraction
2. Has the correct hash per the platform's native verification mode
Handles inner ZIP verification for MAME/FBNeo ROM sets (checkInsideZip,
md5_composite, inner ROM MD5) and path collision deduplication.
8 tests (one per active platform): RetroArch, Batocera, BizHawk, EmuDeck,
Recalbox, RetroBat, RetroDECK, RomM.
```bash
python -m unittest tests.test_pack_integrity -v
# or via CLI:
python scripts/generate_pack.py --all --verify-packs --output-dir dist/
```
Integrated as pipeline step 6/8 (runs after consistency check, before
README generation). Requires packs in `dist/` — skip with `--skip-packs`.
## Verification discipline ## Verification discipline
The test suite is one layer of verification. The full quality gate is: The test suite is one layer of verification. The full quality gate is:
@@ -127,6 +152,7 @@ The test suite is one layer of verification. The full quality gate is:
2. The full pipeline completes without error (`python scripts/pipeline.py --offline`) 2. The full pipeline completes without error (`python scripts/pipeline.py --offline`)
3. No unexpected CRITICAL entries in the verify output 3. No unexpected CRITICAL entries in the verify output
4. Pack file counts match verification file counts (consistency check) 4. Pack file counts match verification file counts (consistency check)
5. Pack integrity passes (every declared file extractable with correct hash)
If a change passes tests but breaks the pipeline, it's worth investigating before merging. Similarly, new CRITICAL entries in the verify output after a change usually indicate something to look into. The pipeline is designed so that all steps agree: if verify reports N files for a platform, the pack should contain exactly N files. If a change passes tests but breaks the pipeline, it's worth investigating before merging. Similarly, new CRITICAL entries in the verify output after a change usually indicate something to look into. The pipeline is designed so that all steps agree: if verify reports N files for a platform, the pack should contain exactly N files.

View File

@@ -7,7 +7,7 @@ All tools are Python scripts in `scripts/`. Single dependency: `pyyaml`.
Run everything in sequence: Run everything in sequence:
```bash ```bash
python scripts/pipeline.py --offline # DB + verify + packs + manifests + readme + site python scripts/pipeline.py --offline # DB + verify + packs + manifests + integrity + readme + site
python scripts/pipeline.py --offline --skip-packs # DB + verify only python scripts/pipeline.py --offline --skip-packs # DB + verify only
python scripts/pipeline.py --offline --skip-docs # skip readme + site generation python scripts/pipeline.py --offline --skip-docs # skip readme + site generation
python scripts/pipeline.py --offline --target switch # filter by hardware target python scripts/pipeline.py --offline --target switch # filter by hardware target
@@ -20,21 +20,22 @@ Pipeline steps:
| Step | Description | Skipped by | | Step | Description | Skipped by |
|------|-------------|------------| |------|-------------|------------|
| 1/9 | Generate database | - | | 1/8 | Generate database | - |
| 2/9 | Refresh data directories | `--offline` | | 2/8 | Refresh data directories | `--offline` |
| 2a | Refresh MAME BIOS hashes | `--offline` | | 2a | Refresh MAME BIOS hashes | `--offline` |
| 2a2 | Refresh FBNeo BIOS hashes | `--offline` | | 2a2 | Refresh FBNeo BIOS hashes | `--offline` |
| 2b | Check buildbot staleness | only with `--check-buildbot` | | 2b | Check buildbot staleness | only with `--check-buildbot` |
| 2c | Generate truth YAMLs | only with `--with-truth` / `--with-export` | | 2c | Generate truth YAMLs | only with `--with-truth` / `--with-export` |
| 2d | Diff truth vs scraped | only with `--with-truth` / `--with-export` | | 2d | Diff truth vs scraped | only with `--with-truth` / `--with-export` |
| 2e | Export native formats | only with `--with-export` | | 2e | Export native formats | only with `--with-export` |
| 3/9 | Verify all platforms | - | | 3/8 | Verify all platforms | - |
| 4/9 | Generate packs | `--skip-packs` | | 4/8 | Generate packs | `--skip-packs` |
| 4b | Generate install manifests | `--skip-packs` | | 4b | Generate install manifests | `--skip-packs` |
| 4c | Generate target manifests | `--skip-packs` | | 4c | Generate target manifests | `--skip-packs` |
| 5/9 | Consistency check | if verify or pack skipped | | 5/8 | Consistency check | if verify or pack skipped |
| 8/9 | Generate README | `--skip-docs` | | 6/8 | Pack integrity (extract + hash) | `--skip-packs` |
| 9/9 | Generate site | `--skip-docs` | | 7/8 | Generate README | `--skip-docs` |
| 8/8 | Generate site | `--skip-docs` |
## Individual tools ## Individual tools
@@ -132,6 +133,8 @@ Reports files that cores need beyond what platforms declare.
python scripts/cross_reference.py # all python scripts/cross_reference.py # all
python scripts/cross_reference.py --emulator dolphin # single python scripts/cross_reference.py --emulator dolphin # single
python scripts/cross_reference.py --emulator dolphin --json # JSON output python scripts/cross_reference.py --emulator dolphin --json # JSON output
python scripts/cross_reference.py --platform batocera # single platform
python scripts/cross_reference.py --platform retroarch --target switch
``` ```
### truth.py, generate_truth.py, diff_truth.py ### truth.py, generate_truth.py, diff_truth.py
@@ -168,6 +171,8 @@ from upstream repositories into `data/`.
```bash ```bash
python scripts/refresh_data_dirs.py python scripts/refresh_data_dirs.py
python scripts/refresh_data_dirs.py --key dolphin-sys --force python scripts/refresh_data_dirs.py --key dolphin-sys --force
python scripts/refresh_data_dirs.py --dry-run # preview without downloading
python scripts/refresh_data_dirs.py --platform batocera # single platform only
``` ```
### Other tools ### Other tools