fix: compare pack paths, not only counts

Step 5 compared (ok, total) and the core file count. RetroArch, Lakka
and RetroPie read 526/526 on both sides while verify credited four paths
below SGB1.sfc and SGB2.sfc that the pack never writes: a declared file
blocks them as a directory. verify now places the platform files in the
builder's order with the same path rules, and a user_provided or
external declaration keeps its slot on both sides, so no core file takes
it in the report only.

verify --destinations writes the paths each pack holds by the report,
and step 5 compares them with the archive members (notes and data tree
members set aside). Every emulator profile, in each build it has, is
built in memory (PackRecord, nothing read or written) and compared with
verify --emulator path by path. Cost measured here: about 1 s per
platform archive and 11 s for the 930 emulator packs.
This commit is contained in:
Abdessamad Derraz committed 2026-10-21 05:18:02 +02:00
1 parent fb639647e6
commit 38af19091d
8 files changed
+509 -25

No files matched your search

+2 -2
View File
@@ -494,7 +494,7 @@ This executes in sequence:
2. `refresh_data_dirs.py` - update data directories (skipped with `--offline`)
3. `verify.py --all` - verify all platforms including the new one
4. `generate_pack.py --all` - build ZIP packs + install and target manifests
5. Consistency check - verify counts match between verify and pack
5. Consistency check: verify and the pack agree on counts and on the paths each archive holds
6. Pack integrity - extract ZIPs and verify hashes per platform mode
7. `generate_readme.py` - regenerate README and CONTRIBUTING
8. `generate_site.py` - regenerate documentation site and `mkdocs.yml`
@@ -511,7 +511,7 @@ Check the output for:
- Pack generation succeeds and includes the expected files
- Consistency check passes (verify file counts match pack file counts)
Verification is not optional. A platform that passes `pipeline.py` today may break tomorrow if upstream changes its data format. Run the full pipeline on every change, even if the modification seems trivial. The consistency check (verify counts must match pack counts) catches subtle issues where files resolve during verification but fail during pack generation, or vice versa.
Verification is not optional. A platform that passes `pipeline.py` today may break tomorrow if upstream changes its data format. Run the full pipeline on every change, even if the modification seems trivial. The consistency check (verify and the pack must agree on counts and paths) catches subtle issues where files resolve during verification but fail during pack generation, or vice versa.
## Checklist
+1 -1
View File
@@ -40,7 +40,7 @@ Every module in `tests/`, by the area it guards.
| Core regression suites | synthetic | `test_e2e`, `test_audit_regressions` |
| File resolution and identity | synthetic + collection | `test_adler32_resolution`, `test_agnostic_scan`, `test_agnostic_scan_mode`, `test_agnostic_scope`, `test_agnostic_substitute`, `test_archive_choice`, `test_bios_companions`, `test_bundled_identity`, `test_casefold_cache`, `test_db_aliases`, `test_declared_names`, `test_derivations`, `test_directory_entries`, `test_extra_identity`, `test_foreign_tree`, `test_foreign_tree_names`, `test_frontend_digest`, `test_game_data_homonyms`, `test_hashing`, `test_identity`, `test_md5_pack_context`, `test_name_tiebreak`, `test_nec_filing`, `test_owner_resolution`, `test_path_suffix`, `test_resolution_evidence`, `test_shipped_content`, `test_twin_declarations`, `test_validated_choice`, `test_variant_choice`, `test_variant_identity`, `test_zipped_member` |
| Verification, cross-reference and profile checks | synthetic + profiles | `test_crypto_aes`, `test_crypto_unique_data`, `test_crypto_verify`, `test_case_insensitive_platforms`, `test_checks_have_values`, `test_core_file_fills_declaration`, `test_cross_reference_sources`, `test_file_consumers`, `test_fileless_audit`, `test_gap_settlement`, `test_hidden_extras`, `test_launcher_layer`, `test_native_mode`, `test_no_case_collisions`, `test_path_conflict_status`, `test_path_null_copies`, `test_profile_checks`, `test_romm_hash_rule`, `test_shared_groups`, `test_standalone_mode`, `test_undeclared_dedup`, `test_unplaced_extras`, `test_validation_scope`, `test_verify_all_dir`, `test_verify_emulator_evidence`, `test_verify_listing`, `test_verify_pack_choice` |
| Packs and manifests | synthetic + real packs | `test_artifact_lock`, `test_atomic_artifacts`, `test_batocera_cores`, `test_deterministic_zip`, `test_emulator_pack_counts`, `test_finalize_pack_names`, `test_large_file_cache`, `test_libretro_layout`, `test_manifest_entry`, `test_pack_conformance_dirs`, `test_pack_counts`, `test_pack_failures`, `test_pack_file_names`, `test_pack_grouping`, `test_pack_integrity`, `test_pack_member_identity`, `test_pack_readme`, `test_platform_inheritance`, `test_platform_keys`, `test_required_only_archives`, `test_required_only_twins`, `test_split_extras`, `test_split_names`, `test_split_pack`, `test_system_pack_extras`, `test_system_pack_parity`, `test_torrentzip`, `test_which_pack` |
| Packs and manifests | synthetic + real packs | `test_artifact_lock`, `test_atomic_artifacts`, `test_batocera_cores`, `test_deterministic_zip`, `test_emulator_pack_counts`, `test_finalize_pack_names`, `test_large_file_cache`, `test_libretro_layout`, `test_manifest_entry`, `test_pack_conformance_dirs`, `test_pack_counts`, `test_pack_destinations`, `test_pack_failures`, `test_pack_file_names`, `test_pack_grouping`, `test_pack_integrity`, `test_pack_member_identity`, `test_pack_readme`, `test_platform_inheritance`, `test_platform_keys`, `test_required_only_archives`, `test_required_only_twins`, `test_split_extras`, `test_split_names`, `test_split_pack`, `test_system_pack_extras`, `test_system_pack_parity`, `test_torrentzip`, `test_which_pack` |
| Slots and regions | synthetic + profiles | `test_region`, `test_region_agnostic_scan`, `test_region_groups`, `test_region_held`, `test_region_platform_index`, `test_region_required_only`, `test_slot`, `test_slots`, `test_slots_shipped_file` |
| Hardware targets and systems | synthetic + target files | `test_extract_targets`, `test_lakka_targets`, `test_libretro_slugs`, `test_system_ids`, `test_system_selection`, `test_target_names`, `test_target_scrapers`, `test_target_systems`, `test_target_verify` |
| Installer and downloads | loopback server | `test_check_release`, `test_download`, `test_install`, `test_install_cores`, `test_install_experience`, `test_install_selection` |
+8 -4
View File
@@ -42,7 +42,7 @@ Pipeline steps:
| 4b | Generate install manifests | `--skip-packs` |
| 4c | Generate target manifests | `--skip-packs` |
| 4d | Collected copies only an emulator install provides (`bundled_only.py --strict`, gates the run) | - |
| 5/8 | Consistency check | if verify or pack skipped |
| 5/8 | Consistency check: counts, paths per archive, every emulator pack | if verify or pack skipped |
| 6/8 | Pack integrity (extract + hash) | `--skip-packs` |
| 7/8 | Generate README | `--skip-docs` |
| 8/8 | Generate site | `--skip-docs` |
@@ -78,9 +78,12 @@ alone, run `python scripts/check_buildbot_system.py`.
and manifest names. `--manifest --target switch` once overwrote the
manifest every installer downloads with a narrowed one.
- **A step that did not run says SKIPPED**, never OK.
- **verify.py and generate_pack.py read the same functions.** A count that
differs between the report and the pack is a defect, which is what step 5
checks.
- **verify.py and generate_pack.py read the same functions.** A count or a
path that differs between the report and the pack is a defect, which is
what step 5 checks: the counts, the paths each archive holds against the
ones verify credits (`--destinations`), and every emulator pack, each
profile in each build, built in memory against `verify --emulator`.
Equal counts once hid four RetroArch paths credited and never written.
- **Generators write through `write_if_changed()`.** A new timestamp alone
rewrites nothing, so a diff on a generated file always means content.
- **One run per output directory.** Pack generation takes a lock on
@@ -118,6 +121,7 @@ python scripts/verify.py --emulator handy --strict # exit 1 on a required file
python scripts/verify.py --system atari-lynx # single system
python scripts/verify.py --platform retroarch --target switch # filter by hardware
python scripts/verify.py --platform recalbox --region us # regional priority list
python scripts/verify.py --all --destinations tmp/paths.json # also write the paths each pack holds
python scripts/verify.py --list-emulators # list all emulators
python scripts/verify.py --list-systems # list all systems
python scripts/verify.py --platform retroarch --list-targets # list available targets