A CI checkout omits every file over 50 MB, so verify and generate_pack
resolve those database entries against a disk that does not hold them and
report them missing. The generated README then stops matching the
committed one for a reason that has nothing to do with staleness.
restore_large_files.py writes them back from the release cache, matched by
SHA1 rather than by name, and only where the path is gitignored and
absent. The site workflow runs it, and refreshes the data directories, before
generating.
The metadata behind the verifier, the pack builder and the site is now
served as static files: a versioned JSON API, CSV extracts, a SQLite
snapshot, and a catalog carrying a SHA-256 for each artifact.
The gaps dataset covers both layers behind a layer column. It previously
held one row, the single platform-verification anomaly, while the page
offering it as a download led with the emulator-level count.
source_ref renders as a permalink pinned to the revision the profile
cites. When a profile declares two repositories, a path that belongs to
neither by name is left as plain code: a citation without a link still
names the file and the lines, a link to the wrong repository does not.
The table filter, focus outlines and tap targets are progressive
enhancement; the pages work without them.
The manifest is untrusted input. It is now read under a size limit and
validated before use: destinations are contained below the BIOS root,
repo_path stays inside bios/, release_asset is a basename, hashes match
their shape, declared totals match their lists, and every entry has a
download source.
Downloads stream against the declared size, are checked by SHA-256 then
SHA-1, land in a per-process temporary file and are installed with
os.replace. Copies into standalone-emulator directories are opt-in with
--standalone-copies so a detection never writes outside the selected
tree.
Both bootstraps verify install.py against an embedded SHA-256 before
running it, and require the Python version install.py actually needs.
A target that publishes no core list is a target with no filter, not a
broken manifest: rejecting it disabled --target for the whole platform.
A profile's contents: block and a DAT both state, per set, the members
one emulator version expects. Torrentzip makes archive bytes a function
of that list alone, so a recipe plus the roms reproduces the archive
exactly.
MAME ships its -listxml as a release asset and FBNeo keeps its dats in
its repository, so neither needs a browser. The importer streams the
311 MB document with a sliding window, resolves romof parents in two
passes, drops undumped members, and accumulates versions instead of
replacing them. Identical recipes shared across versions are stored once
with dats listing every version that agrees: 22 MAME versions give 23679
entries for 1726 distinct recipes, 18 MB down to 2.2 MB.
Snapshots live in recipes/ because load_provenance_snapshots reads every
provenance/*.json as a dump catalogue, and a recipe is not one.
1113 archives now reproduce byte for byte, against 175 from profiles
alone, and spec128.zip is rebuilt from roms already held.
A declared hash that the local dump contradicts is not one situation. An
existence platform never reads the bytes, so withholding the file lets an
upstream list error remove something the frontend would have loaded; a
hash platform would reject it, so shipping it is pointless.
The mode now decides, at every point that had an opinion: pack building,
core complement, emulator packs, manifests, conformance and
_intentional_hash_exclusion. verify.find_undeclared_files follows, since
verify and generate_pack must agree file for file.
Also here: resolution reports which evidence matched rather than a flat
"exact", a path or filename can no longer override a declared hash, and
safe_extract_zip treats a Windows backslash as the separator it is
instead of refusing the archive.
YAML 1.1 reads an unquoted 01 as the integer 1 and 81 as 81, so the
manager set stopped naming the file its driver opens and the API
published "cores": [2048] as a number where consumers match strings.
Ground truth from MAME 0.289 at the revision the profile cites:
src/mame/vtech/crvision.cpp:957 loads "01" and "23".
The root cause was the scraper: _hash_merge wrote these names through an
f-string, so quoting the profiles alone would be undone on the next
refresh. _yaml_scalar now quotes every name, archive and description it
writes.
JSON Schemas for the database, install and pack manifests, target
manifests, site API envelopes and stats, plus the semantic invariants a
schema cannot express: declared totals matching their lists, no
destination both installed and omitted, database keys matching their
sha1. validate_site.py checks the rendered HTML for metadata, headings,
image alternatives, duplicate ids and unresolved local links.
Pack manifests are read from inside the generated archives, where
generate_pack writes them, rather than from a dist/ glob that matches
nothing.
Emulator and platform schemas gain additionalProperties: false, and
cores[] plus contents[].name must be strings: an unquoted 81 or 01 in
YAML parses as a number and stops matching the upstream name.
Abdessamad Derraz