Commit Graph
733 Commits
Author SHA1 Message Date
Abdessamad Derraz c00314d479 docs: align the wiki with what the code does
The pages described workflows, an installer pinning and a CI permission
model that are not the ones in the tree. Corrected: the release process,
the CI table, the resolution chain and its statuses, the hash-mismatch
policy per verification mode, the archive convention, and the romset
recipe store.

The FAQ keeps the project's own reading of the legal question rather
than a version that reaches no conclusion.
2026-08-10 13:37:20 +02:00
Abdessamad Derraz 892bc34e21 refactor: fold vba_m into the vbam profile
vba_m carried md5 values taken from docs.libretro.com while its entries
declare validation: [size], which is what src/gba/GBA.cpp:3262-3272 and
src/gb/GB.cpp:2172-2192 actually check. Only the source decides a hash,
so they are gone.

The profile becomes an alias of vbam, and the buildbot name vba-m moves
into its cores: list so target filtering still resolves it.
2026-08-10 13:37:20 +02:00
Abdessamad Derraz b4d725b75e chore: keep pr feedback and release path intact
PR validation keeps its four jobs, its comment and its labels. The two
inline schema heredocs become validate_schemas.py --source-only, the
changed-file list is NUL-separated so a path with a space survives, and
the whole suite runs instead of one module.

The offline pipeline, manifest regeneration and mkdocs build stay out of
the pull-request path: they belong to the site workflow, which only runs
on main, and stacking them on every PR does not fit the free-tier
budget.

Deploy Site gains schema validation and rendered-site validation, and
checks that the committed README and CONTRIBUTING match what the
generator produces. write_if_changed compares with the timestamp line
stripped, so that check reports staleness rather than the clock.
2026-08-10 13:37:09 +02:00
Abdessamad Derraz 29ff87a99b feat: publish versioned data exports
The metadata behind the verifier, the pack builder and the site is now
served as static files: a versioned JSON API, CSV extracts, a SQLite
snapshot, and a catalog carrying a SHA-256 for each artifact.

The gaps dataset covers both layers behind a layer column. It previously
held one row, the single platform-verification anomaly, while the page
offering it as a download led with the emulator-level count.

source_ref renders as a permalink pinned to the revision the profile
cites. When a profile declares two repositories, a path that belongs to
neither by name is left as plain code: a citation without a link still
names the file and the lines, a link to the wrong repository does not.

The table filter, focus outlines and tap targets are progressive
enhancement; the pages work without them.
2026-08-10 13:37:01 +02:00
Abdessamad Derraz 9001ebb992 feat: harden the installer boundary
The manifest is untrusted input. It is now read under a size limit and
validated before use: destinations are contained below the BIOS root,
repo_path stays inside bios/, release_asset is a basename, hashes match
their shape, declared totals match their lists, and every entry has a
download source.

Downloads stream against the declared size, are checked by SHA-256 then
SHA-1, land in a per-process temporary file and are installed with
os.replace. Copies into standalone-emulator directories are opt-in with
--standalone-copies so a detection never writes outside the selected
tree.

Both bootstraps verify install.py against an embedded SHA-256 before
running it, and require the Python version install.py actually needs.

A target that publishes no core list is a target with no filter, not a
broken manifest: rejecting it disabled --target for the whole platform.
2026-08-10 13:36:52 +02:00
Abdessamad Derraz bb915ed6aa feat: file triforce ipl and segaboot variants
bios/Sega/Triforce/.variants/segaboot.gcm was never a SegaBoot image:
its header reads "(C) 1999-2001 Nintendo" and it matches Redump's
"Triforce (ARCADE BOOTROM)". crediar's Dolphin fork reads the IPL from
GC/<region>/IPL_TRI.bin and SegaBoot from Triforce/segaboot.gcm, two
separate paths, so it moves to GC/USA/IPL_TRI.bin.

segaboot.gcm.e475c7dc is SegaBoot v3.11, the sha1 emulators/triforce.yml
already declared with no local file.

spec128.zip.1a524bfa is rebuilt from the FinalBurn Neo ZX Spectrum
recipe and roms already held, closing one of the pinned archives listed
as unobtainable.
2026-08-10 13:36:44 +02:00
Abdessamad Derraz 01dc9b2539 feat: add openh264 and lindbergh runtimes
Fourteen files a profiled core loads and the collection lacked.

Eight OpenH264 2.4.1 binaries for ruffle, from ciscobinary.openh264.org.
Cisco serves only the bzip2 artifact; the emulator loads the
decompressed library under the plain name. All eight sha256 match both
emulators/ruffle.yml and ruffle's own table in
video/external/src/decoder/openh264.rs.

Six Lindbergh runtimes: libCg.so and libCgGL.so from NVIDIA's
Cg-3.1_April2012_x86.tgz, libCg2.so from Cg-2.0_Jan2008_x86.tar.gz, and
the three libraries the loader vendors in libs/.

Its logo and font files are not external: hook.c:655-709 serves them
from resources/logo.h and resources/font.h unless built-ins are
disabled. segaboot and lindbergrc have no public source.
2026-08-10 13:36:34 +02:00
Abdessamad Derraz 33a9934a11 feat: identify romsets from version recipes
A profile's contents: block and a DAT both state, per set, the members
one emulator version expects. Torrentzip makes archive bytes a function
of that list alone, so a recipe plus the roms reproduces the archive
exactly.

MAME ships its -listxml as a release asset and FBNeo keeps its dats in
its repository, so neither needs a browser. The importer streams the
311 MB document with a sliding window, resolves romof parents in two
passes, drops undumped members, and accumulates versions instead of
replacing them. Identical recipes shared across versions are stored once
with dats listing every version that agrees: 22 MAME versions give 23679
entries for 1726 distinct recipes, 18 MB down to 2.2 MB.

Snapshots live in recipes/ because load_provenance_snapshots reads every
provenance/*.json as a dump catalogue, and a recipe is not one.

1113 archives now reproduce byte for byte, against 175 from profiles
alone, and spec128.zip is rebuilt from roms already held.
2026-08-10 13:36:12 +02:00
Abdessamad Derraz e8ee8b0954 fix: decide hash mismatch by native mode
A declared hash that the local dump contradicts is not one situation. An
existence platform never reads the bytes, so withholding the file lets an
upstream list error remove something the frontend would have loaded; a
hash platform would reject it, so shipping it is pointless.

The mode now decides, at every point that had an opinion: pack building,
core complement, emulator packs, manifests, conformance and
_intentional_hash_exclusion. verify.find_undeclared_files follows, since
verify and generate_pack must agree file for file.

Also here: resolution reports which evidence matched rather than a flat
"exact", a path or filename can no longer override a declared hash, and
safe_extract_zip treats a Windows backslash as the separator it is
instead of refusing the archive.
2026-08-10 13:36:03 +02:00
Abdessamad Derraz a5f549a1c6 fix: name rom members as their source names them
Four archives carried member names no MAME version uses. Each is decided
against MAME 0.289 at the revision the profiles cite:

  manager.zip   1 -> 01                       crvision.cpp:957
  sys573.zip    700a01(gchgchmp).22g -> ,     ksys573.cpp:3575
  cedmag.zip    Magnet-Master-VID-E03.BIN     cedar_magnet.cpp:1040
  advision.zip  b225__/b8223__ prefixes cut   advision.cpp:456,459

Content is untouched: every member keeps its crc32 and size. The
archives are rebuilt with torrentzip so their bytes depend only on their
contents, which is the repository convention (51 of 60 sampled archives)
and what two of these four already were.

Batocera, RetroBat and RetroDECK pin the older naming, so the upstream
archives stay verbatim under .variants/ and resolve_local_file serves
each layer the copy it expects.
2026-08-10 13:35:53 +02:00
Abdessamad Derraz 597a8ff919 fix: quote rom and core names in yaml
YAML 1.1 reads an unquoted 01 as the integer 1 and 81 as 81, so the
manager set stopped naming the file its driver opens and the API
published "cores": [2048] as a number where consumers match strings.

Ground truth from MAME 0.289 at the revision the profile cites:
src/mame/vtech/crvision.cpp:957 loads "01" and "23".

The root cause was the scraper: _hash_merge wrote these names through an
f-string, so quoting the profiles alone would be undone on the next
refresh. _yaml_scalar now quotes every name, archive and description it
writes.
2026-08-10 13:35:42 +02:00
Abdessamad Derraz 5d417c8229 feat: add data contract validation
JSON Schemas for the database, install and pack manifests, target
manifests, site API envelopes and stats, plus the semantic invariants a
schema cannot express: declared totals matching their lists, no
destination both installed and omitted, database keys matching their
sha1. validate_site.py checks the rendered HTML for metadata, headings,
image alternatives, duplicate ids and unresolved local links.

Pack manifests are read from inside the generated archives, where
generate_pack writes them, rather than from a dist/ glob that matches
nothing.

Emulator and platform schemas gain additionalProperties: false, and
cores[] plus contents[].name must be strings: an unquoted 81 or 01 in
YAML parses as a number and stops matching the upstream name.
2026-08-10 13:35:33 +02:00
Abdessamad Derraz 45f89cc6c0 feat: add region filters and profiles 2026-08-09 14:15:56 +02:00
Abdessamad Derraz 1c537c3a30 fix: keep mame refresh to profiles on mame's tree 2026-08-08 13:52:25 +02:00
Abdessamad Derraz 731096cf12 chore: refresh mame bios refs from 0.289 2026-08-08 13:50:25 +02:00
Abdessamad Derraz 89211003cb fix: refresh mame refs when a driver line moves 2026-08-08 13:48:54 +02:00
Abdessamad Derraz 1232e43e7a chore: recale refs after the parsing fixes 2026-08-08 13:44:47 +02:00
Abdessamad Derraz fa6de11133 feat: parse colon continuations and header stems 2026-08-08 13:44:03 +02:00
Abdessamad Derraz 587aefd282 fix: name the external project on every dosbox ref 2026-08-08 13:41:28 +02:00
Abdessamad Derraz dde5509a39 chore: recale refs followed to their subject 2026-08-08 13:40:37 +02:00
Abdessamad Derraz 17da2ae76f feat: follow refs to a unique declared value 2026-08-08 13:39:54 +02:00
Abdessamad Derraz c8e05013d9 feat: follow a ref when its subject moved with it 2026-08-08 13:38:30 +02:00
Abdessamad Derraz f4a8d8c54b chore: rebuild manifests and docs 2026-08-08 13:36:57 +02:00
Abdessamad Derraz 31c4bee36b fix: point amiberry kickstart refs at candidates 2026-08-08 13:36:51 +02:00
Abdessamad Derraz c42209fa96 fix: point three refs at their real locations 2026-08-08 13:35:19 +02:00
Abdessamad Derraz 82b510de0d fix: relocate gear family bios refs 2026-08-08 13:23:06 +02:00
Abdessamad Derraz 642e9ea9ca fix: mark ecwolf data as standalone only 2026-08-08 13:20:54 +02:00
Abdessamad Derraz 7755367787 docs: document tag-pinned profile handling 2026-08-08 13:17:52 +02:00
Abdessamad Derraz b00ca4d237 docs: state only what the data supports 2026-08-08 13:16:17 +02:00
Abdessamad Derraz 4c5ca17682 feat: judge tag-pinned profiles on their own revision 2026-08-08 13:12:35 +02:00
Abdessamad Derraz 088caef45e docs: use one plain vocabulary across surfaces 2026-08-08 12:59:46 +02:00
Abdessamad Derraz 89fe022196 chore: recale reviewed refs for three cores 2026-08-08 12:54:03 +02:00
Abdessamad Derraz c94771459c chore: rebuild manifests and docs 2026-08-08 12:51:42 +02:00
Abdessamad Derraz 1fda81fb52 feat: record ares laseractive firmware hashes 2026-08-08 12:47:41 +02:00
Abdessamad Derraz 09d586e387 docs: document the ref anchoring strategies 2026-08-08 12:42:29 +02:00
Abdessamad Derraz 38e39af561 docs: say what each platform actually checks 2026-08-08 12:42:06 +02:00
Abdessamad Derraz b65ff255cb chore: recale reviewed refs for two cores 2026-08-08 12:38:47 +02:00
Abdessamad Derraz 7c8958e02b chore: recale refs with their annotations kept 2026-08-08 12:36:00 +02:00
Abdessamad Derraz 763c159a9f feat: rewrite annotated refs keeping their prose 2026-08-08 12:35:02 +02:00
Abdessamad Derraz 16331a1712 feat: report external citations as unverifiable 2026-08-08 12:30:37 +02:00
Abdessamad Derraz ba898063c6 chore: recale refs settled by the new anchors 2026-08-08 12:25:42 +02:00
Abdessamad Derraz dc4b23a883 feat: anchor on declared values when refs miss 2026-08-08 12:24:31 +02:00
Abdessamad Derraz 6b0306b43d feat: settle ambiguity by file line shift 2026-08-08 12:21:13 +02:00
Abdessamad Derraz 9982748e49 fix: realign refs with their pinned revision 2026-08-08 12:17:11 +02:00
Abdessamad Derraz 34876ba610 fix: keep refs and pin on the same revision 2026-08-08 12:16:01 +02:00
Abdessamad Derraz de19473f2c docs: document upstream and exporter base modules 2026-08-08 12:15:48 +02:00
Abdessamad Derraz ace32054f5 chore: recale profile refs onto upstream head 2026-08-08 12:12:29 +02:00
Abdessamad Derraz c585452d46 feat: resolve bare and annotated ref paths 2026-08-08 12:05:59 +02:00
Abdessamad Derraz 344599f7ef docs: name the wanted list for what it is 2026-08-08 11:57:22 +02:00
Abdessamad Derraz 4d34cac45f chore: anchor profile refs to source commits 2026-08-08 11:52:28 +02:00