Move verification logic to generate_pack.py --verify-packs (single
source of truth). test_pack_integrity.py is now a thin wrapper that
calls the CLI. Pipeline step 6/8 uses the same CLI entry point.
Renumber all pipeline steps 1-8 (was skipping from 5 to 8/9).
Update generate_site.py with pack integrity test documentation.
Extract each platform ZIP to tmp/ (real filesystem, not /tmp tmpfs)
and verify every declared file exists at the correct path with the
correct hash per the platform's native verification mode.
Handles ZIP inner content verification (checkInsideZip, md5_composite,
inner ROM MD5) and path collision deduplication.
Integrated as pipeline step 6/8. Renumber all pipeline steps to be
sequential (was skipping from 5 to 8).
sparse clone upstream repos, parse BIOS root sets from C source,
cache as JSON, merge into emulator profiles with backup.
covers macro expansion, version detection, subset profile protection.
bios_mode: agnostic (profile) and agnostic: true (file) for
emulators that accept any valid BIOS without specific filename.
find_undeclared_files skips agnostic entries, pack extras scan
includes all matching DB files by path prefix + size criteria,
resolve_local_file has agnostic fallback with rename README.
applied to pcsx2, lrps2 (bios_mode), melonds dsi_nand (file).
expand_platform_declared_names resolves platform file MD5s
through the database to recover canonical names and aliases,
eliminating false positive undeclared files when a platform
renames a file (e.g. Batocera ROM1 vs gsplus ROM).
Add many MAME/MESS BIOS entries (TRS-80 family, Bandai RX-78, Sega AI) and update docs/navigation counts (README, mkdocs). Remove empty supplemental file references from database.json and update generated timestamps and totals. Harden and refactor tooling: add MAX_RESPONSE_SIZE limited reader in base_scraper, make target scrapers an abstract base, narrow exception handling in the Batocera targets parser, and switch generate_pack.py and verify.py to use build_target_cores_cache (simplifies target config loading and error handling). verify.py also loads supplemental cross-reference names and accepts them through verify_platform. Update tests to import from updated modules (validation/truth). Misc: small bugfix for case-insensitive path conflict check.
Group archived files by archive unit in find_undeclared_files instead
of reporting individual ROMs. Add path-based fallback for descriptive
names (e.g. "SeaBIOS (128 KB)" resolves via path: bios.bin). Update
_collect_extras to use archive name for pack resolution. Regenerate
database with new bios files. 6 new E2E tests covering archive
in_repo, missing archives, descriptive names, and pack extras.
Loads per-platform target YAML files from platforms/targets/,
resolves aliases and applies add_cores/remove_cores from _overrides.yml.
Includes 7 E2E tests covering alias resolution, overrides, and error paths.
batch re-profiled nekop2 through pokemini. mupen64plus renamed to
mupen64plus_next. new profiles: nes, mupen64plus_next.
validation functions (_build_validation_index, check_file_validation)
consolidated in common.py — single source of truth for verify.py
and generate_pack.py. pipeline 100% consistent on all 6 platforms.
post-generation verification: reopen each ZIP, hash every file,
check against database.json. inject manifest.json inside each pack
(self-documenting: path, sha1, md5, size, status per file).
generate SHA256SUMS.txt alongside packs for download verification.
validation index now uses sets for hashes and sizes to support
multiple valid ROM versions (MT-32 v1.04-v2.07, CM-32L variants).
69 tests pass, pipeline complete.
adler32 hash via zlib.adler32(), min_size/max_size range checks,
signature/crypto tracked as non-reproducible (console-specific keys).
compute_hashes now returns adler32. 69 tests pass including 3 new
tests for adler32, size ranges, and crypto tracking.
