The home page and the stats export counted every file carrying a
provenance record, the provenance page and the README only the system
files. The site published 553 and 566 for the same quantity, one click
apart, and the export paired the wider count with composition.systems as
its denominator. common.count_catalog_matched is now the single source,
scoped to the systems bucket.
The FAQ had drifted from the profiles it describes: MAME pinned at 0.287
against 0.289 in mame.yml, Adler-32 attributed to Dolphin's IPL rather
than the DSP ROMs that carry known_hash_adler32, and the per-emulator
verbose report named as the only content check on an existence platform,
which skips the DISCREPANCY line the platform report raises itself.
Tests read both sides: no generator may count matches inline, and each
FAQ claim is checked against the profile or the script that owns it.
The contributors block is the only part of the README that comes from
the network, and a refused request returned an empty list, which deleted
the section. That happened during a pipeline run and the result was
committed; the freshness check then regenerated the section and failed
on the difference.
An unavailable list now republishes the one already there and says so.
Losing it is a worse answer than a stale one, and it makes an offline
regeneration additive rather than destructive. Reverting the change and
simulating the same outage empties the section again.
The README said 14 files were 'not in the collection yet', which
implies somebody could still put them there. Nine of them cannot be:
WHDLoad.key is a per-user signed registration, Custom.dat and key.txt
are slots the emulator expects the user to fill, gpib.rom and
dragonfly-2.3.rom have never been dumped, CARTS.CRC belongs to a dead
code path. The profiles already record why, so the two are now counted
apart: five still to be found, nine that cannot be.
LICENSE covered the whole repository, so the MIT grant read as
covering 9.9 GB of third-party firmware. It now states its scope and
NOTICE describes the files it does not cover, with a removal channel.
The FAQ cited Connectix and Accolade for redistribution when both
decided intermediate copying during reverse engineering, presented fair
use as settled, read section 1201(f) as a general permission, and
listed abandonware among legal doctrines. Each claim is now stated at
the strength it actually has, and the weakest ground is named.
The metadata behind the verifier, the pack builder and the site is now
served as static files: a versioned JSON API, CSV extracts, a SQLite
snapshot, and a catalog carrying a SHA-256 for each artifact.
The gaps dataset covers both layers behind a layer column. It previously
held one row, the single platform-verification anomaly, while the page
offering it as a download led with the emulator-level count.
source_ref renders as a permalink pinned to the revision the profile
cites. When a profile declares two repositories, a path that belongs to
neither by name is left as plain code: a citation without a link still
names the file and the lines, a link to the wrong repository does not.
The table filter, focus outlines and tap targets are progressive
enhancement; the pages work without them.
Run ruff check --fix: remove unused imports (F401), fix f-strings
without placeholders (F541), remove unused variables (F841), fix
duplicate dict key (F601).
Run isort --profile black: normalize import ordering across all files.
Run ruff format: apply consistent formatting (black-compatible) to
all 58 Python files.
3 intentional E402 remain (imports after require_yaml() must execute
after yaml is available).
9 new wiki pages: getting-started, faq, troubleshooting,
advanced-usage, verification-modes, adding-a-platform,
adding-a-scraper, testing-guide, release-process.
Updated architecture.md with mermaid diagrams, tools.md with
full pipeline and target/exporter sections, profiling.md with
missing fields, index.md with glossary and nav links.
Expanded CONTRIBUTING.md from stub to full contributor guide.
Filter .old.yml from load_emulator_profiles, generate_db alias
collection, and generate_readme counts. Fix BizHawk sha1 mode
in tools.md, fix RetroPie path, fix export_truth.py typos.
write_if_changed in common.py compares content after stripping
timestamps (generated_at, Auto-generated on, Generated on).
applied to generate_db, generate_readme, generate_site.
eliminates timestamp-only diffs in database.json, README.md,
mkdocs.yml, and 423 docs pages.
Group archived files by archive unit in find_undeclared_files instead
of reporting individual ROMs. Add path-based fallback for descriptive
names (e.g. "SeaBIOS (128 KB)" resolves via path: bios.bin). Update
_collect_extras to use archive name for pack resolution. Regenerate
database with new bios files. 6 new E2E tests covering archive
in_repo, missing archives, descriptive names, and pack extras.
- fix KeyError in compute_coverage (generate_readme, generate_site)
- fix comma-separated MD5 handling in generate_pack check_inside_zip
- fix _verify_file_hash to handle multi-MD5 for large files
- fix external downloads not tracked in seen_destinations/file_status
- fix tar path traversal in _is_safe_tar_member (refresh_data_dirs)
- fix predictable tmp path in download.py
- fix _sanitize_path to filter "." components
- remove blanket data_dir suppression in find_undeclared_files
- remove blanket data_dir suppression in cross_reference
- add status_counts to verify_platform return value
- add md5_composite cache for repeated ZIP hashing
Abdessamad Derraz