Commit Graph
747 Commits
Author SHA1 Message Date
Abdessamad Derraz 43cf84d5ec refactor: require evidence before calling a ref stale 2026-08-10 17:17:19 +02:00
Abdessamad Derraz ba699f472c feat: add thomson mo6 machines to clk 2026-08-10 17:15:57 +02:00
Abdessamad Derraz f2670b1ad5 feat: anchor fbneo refs on their rom lines 2026-08-10 17:00:35 +02:00
Abdessamad Derraz f67642095c feat: self-check a profile already on head 2026-08-10 16:55:48 +02:00
Abdessamad Derraz ba84f5701f chore: cover region filter with split and target 2026-08-10 16:54:39 +02:00
Abdessamad Derraz dd4d0c1d35 chore: recale refs after the cap increase 2026-08-10 16:50:50 +02:00
Abdessamad Derraz 77a39b2f4e perf: reuse diffs and raise the file size cap 2026-08-10 16:48:59 +02:00
Abdessamad Derraz 77dff6377e fix: keep fbneo per-rom refs and archive names 2026-08-10 16:47:20 +02:00
Abdessamad Derraz 47182f4cb1 chore: refresh database, manifests and readme
7851 files.
2026-08-10 15:13:15 +02:00
Abdessamad Derraz 76984e8bce feat: close the last anchored gaps
Four files, each with a verifiable origin.

logo.tga and logo_red.tga come from the loader's own resources/logo.h:
hook.c's fread maps FILE_LOGO_TGA to logoLL or logoLLRed depending on
which path was opened, one array per file, so the extraction is exact.
Both decode as valid RLE TGA at their declared lengths.

lgp.zip and timetrav.zip carry crc 0 in Hypseus and in Daphne upstream,
so MAME vouches for them instead. lgp maps onto MAME's set by base name,
confirmed by the five members whose bytes already agreed, and the four
that were missing came from the same set. timetrav is MAME's timetrv:
tt061891.bin, 262144 bytes, crc a3d44219.

The four Sega fonts stay open, and the profile now says why: the loader
embeds one 128x256 atlas and one metrics table answering all four paths,
so it substitutes for the originals rather than reproducing either.
2026-08-10 15:13:15 +02:00
Abdessamad Derraz 8ef73daca3 chore: refresh database, manifests and readme
7847 files. Regenerated after the laserdisc acquisition.
2026-08-10 14:34:34 +02:00
Abdessamad Derraz d588ebbde4 chore: restore large files before ci coverage checks
A CI checkout omits every file over 50 MB, so verify and generate_pack
resolve those database entries against a disk that does not hold them and
report them missing. The generated README then stops matching the
committed one for a reason that has nothing to do with staleness.

restore_large_files.py writes them back from the release cache, matched by
SHA1 rather than by name, and only where the path is gitignored and
absent. The site workflow runs it, and refreshes the data directories, before
generating.
2026-08-10 14:34:27 +02:00
Abdessamad Derraz b76edd2034 feat: add laserdisc romsets for hypseus singe
Sixty-nine of the seventy-one sets Hypseus Singe loads, assembled from
atoms retrieved from public archives and rebuilt with torrentzip under
the member names its source expects.

Every set is anchored: the profile already carried each member with its
CRC32, read from Hypseus' own source, so a candidate is either an exact
match or discarded. Sixty-seven verified that way. For gpworld and
firefoxa, Hypseus declares crc 0 and gives no anchor, so MAME's recipe
for the same game vouched for the bytes instead: 17/17 and 6/6.

lgp and timetrav stay out. Hypseus declares crc 0 for both, MAME covers
only 5 of lgp's 9 members and has no timetrav at all, so nothing can
vouch for them and half-verified is not verified.

The six Lindbergh font and logo files gain has_builtin: true. hook.c
serves them from resources/logo.h and resources/font.h unless built-ins
are disabled, so they are not a gap the collection can close.
2026-08-10 14:34:15 +02:00
Abdessamad Derraz 0d4a4b3355 chore: refresh database, manifests and readme
7779 files. Regenerated after the archive renames, the Triforce filing
and the fourteen acquired runtimes.
2026-08-10 13:37:25 +02:00
Abdessamad Derraz c00314d479 docs: align the wiki with what the code does
The pages described workflows, an installer pinning and a CI permission
model that are not the ones in the tree. Corrected: the release process,
the CI table, the resolution chain and its statuses, the hash-mismatch
policy per verification mode, the archive convention, and the romset
recipe store.

The FAQ keeps the project's own reading of the legal question rather
than a version that reaches no conclusion.
2026-08-10 13:37:20 +02:00
Abdessamad Derraz 892bc34e21 refactor: fold vba_m into the vbam profile
vba_m carried md5 values taken from docs.libretro.com while its entries
declare validation: [size], which is what src/gba/GBA.cpp:3262-3272 and
src/gb/GB.cpp:2172-2192 actually check. Only the source decides a hash,
so they are gone.

The profile becomes an alias of vbam, and the buildbot name vba-m moves
into its cores: list so target filtering still resolves it.
2026-08-10 13:37:20 +02:00
Abdessamad Derraz b4d725b75e chore: keep pr feedback and release path intact
PR validation keeps its four jobs, its comment and its labels. The two
inline schema heredocs become validate_schemas.py --source-only, the
changed-file list is NUL-separated so a path with a space survives, and
the whole suite runs instead of one module.

The offline pipeline, manifest regeneration and mkdocs build stay out of
the pull-request path: they belong to the site workflow, which only runs
on main, and stacking them on every PR does not fit the free-tier
budget.

Deploy Site gains schema validation and rendered-site validation, and
checks that the committed README and CONTRIBUTING match what the
generator produces. write_if_changed compares with the timestamp line
stripped, so that check reports staleness rather than the clock.
2026-08-10 13:37:09 +02:00
Abdessamad Derraz 29ff87a99b feat: publish versioned data exports
The metadata behind the verifier, the pack builder and the site is now
served as static files: a versioned JSON API, CSV extracts, a SQLite
snapshot, and a catalog carrying a SHA-256 for each artifact.

The gaps dataset covers both layers behind a layer column. It previously
held one row, the single platform-verification anomaly, while the page
offering it as a download led with the emulator-level count.

source_ref renders as a permalink pinned to the revision the profile
cites. When a profile declares two repositories, a path that belongs to
neither by name is left as plain code: a citation without a link still
names the file and the lines, a link to the wrong repository does not.

The table filter, focus outlines and tap targets are progressive
enhancement; the pages work without them.
2026-08-10 13:37:01 +02:00
Abdessamad Derraz 9001ebb992 feat: harden the installer boundary
The manifest is untrusted input. It is now read under a size limit and
validated before use: destinations are contained below the BIOS root,
repo_path stays inside bios/, release_asset is a basename, hashes match
their shape, declared totals match their lists, and every entry has a
download source.

Downloads stream against the declared size, are checked by SHA-256 then
SHA-1, land in a per-process temporary file and are installed with
os.replace. Copies into standalone-emulator directories are opt-in with
--standalone-copies so a detection never writes outside the selected
tree.

Both bootstraps verify install.py against an embedded SHA-256 before
running it, and require the Python version install.py actually needs.

A target that publishes no core list is a target with no filter, not a
broken manifest: rejecting it disabled --target for the whole platform.
2026-08-10 13:36:52 +02:00
Abdessamad Derraz bb915ed6aa feat: file triforce ipl and segaboot variants
bios/Sega/Triforce/.variants/segaboot.gcm was never a SegaBoot image:
its header reads "(C) 1999-2001 Nintendo" and it matches Redump's
"Triforce (ARCADE BOOTROM)". crediar's Dolphin fork reads the IPL from
GC/<region>/IPL_TRI.bin and SegaBoot from Triforce/segaboot.gcm, two
separate paths, so it moves to GC/USA/IPL_TRI.bin.

segaboot.gcm.e475c7dc is SegaBoot v3.11, the sha1 emulators/triforce.yml
already declared with no local file.

spec128.zip.1a524bfa is rebuilt from the FinalBurn Neo ZX Spectrum
recipe and roms already held, closing one of the pinned archives listed
as unobtainable.
2026-08-10 13:36:44 +02:00
Abdessamad Derraz 01dc9b2539 feat: add openh264 and lindbergh runtimes
Fourteen files a profiled core loads and the collection lacked.

Eight OpenH264 2.4.1 binaries for ruffle, from ciscobinary.openh264.org.
Cisco serves only the bzip2 artifact; the emulator loads the
decompressed library under the plain name. All eight sha256 match both
emulators/ruffle.yml and ruffle's own table in
video/external/src/decoder/openh264.rs.

Six Lindbergh runtimes: libCg.so and libCgGL.so from NVIDIA's
Cg-3.1_April2012_x86.tgz, libCg2.so from Cg-2.0_Jan2008_x86.tar.gz, and
the three libraries the loader vendors in libs/.

Its logo and font files are not external: hook.c:655-709 serves them
from resources/logo.h and resources/font.h unless built-ins are
disabled. segaboot and lindbergrc have no public source.
2026-08-10 13:36:34 +02:00
Abdessamad Derraz 33a9934a11 feat: identify romsets from version recipes
A profile's contents: block and a DAT both state, per set, the members
one emulator version expects. Torrentzip makes archive bytes a function
of that list alone, so a recipe plus the roms reproduces the archive
exactly.

MAME ships its -listxml as a release asset and FBNeo keeps its dats in
its repository, so neither needs a browser. The importer streams the
311 MB document with a sliding window, resolves romof parents in two
passes, drops undumped members, and accumulates versions instead of
replacing them. Identical recipes shared across versions are stored once
with dats listing every version that agrees: 22 MAME versions give 23679
entries for 1726 distinct recipes, 18 MB down to 2.2 MB.

Snapshots live in recipes/ because load_provenance_snapshots reads every
provenance/*.json as a dump catalogue, and a recipe is not one.

1113 archives now reproduce byte for byte, against 175 from profiles
alone, and spec128.zip is rebuilt from roms already held.
2026-08-10 13:36:12 +02:00
Abdessamad Derraz e8ee8b0954 fix: decide hash mismatch by native mode
A declared hash that the local dump contradicts is not one situation. An
existence platform never reads the bytes, so withholding the file lets an
upstream list error remove something the frontend would have loaded; a
hash platform would reject it, so shipping it is pointless.

The mode now decides, at every point that had an opinion: pack building,
core complement, emulator packs, manifests, conformance and
_intentional_hash_exclusion. verify.find_undeclared_files follows, since
verify and generate_pack must agree file for file.

Also here: resolution reports which evidence matched rather than a flat
"exact", a path or filename can no longer override a declared hash, and
safe_extract_zip treats a Windows backslash as the separator it is
instead of refusing the archive.
2026-08-10 13:36:03 +02:00
Abdessamad Derraz a5f549a1c6 fix: name rom members as their source names them
Four archives carried member names no MAME version uses. Each is decided
against MAME 0.289 at the revision the profiles cite:

  manager.zip   1 -> 01                       crvision.cpp:957
  sys573.zip    700a01(gchgchmp).22g -> ,     ksys573.cpp:3575
  cedmag.zip    Magnet-Master-VID-E03.BIN     cedar_magnet.cpp:1040
  advision.zip  b225__/b8223__ prefixes cut   advision.cpp:456,459

Content is untouched: every member keeps its crc32 and size. The
archives are rebuilt with torrentzip so their bytes depend only on their
contents, which is the repository convention (51 of 60 sampled archives)
and what two of these four already were.

Batocera, RetroBat and RetroDECK pin the older naming, so the upstream
archives stay verbatim under .variants/ and resolve_local_file serves
each layer the copy it expects.
2026-08-10 13:35:53 +02:00
Abdessamad Derraz 597a8ff919 fix: quote rom and core names in yaml
YAML 1.1 reads an unquoted 01 as the integer 1 and 81 as 81, so the
manager set stopped naming the file its driver opens and the API
published "cores": [2048] as a number where consumers match strings.

Ground truth from MAME 0.289 at the revision the profile cites:
src/mame/vtech/crvision.cpp:957 loads "01" and "23".

The root cause was the scraper: _hash_merge wrote these names through an
f-string, so quoting the profiles alone would be undone on the next
refresh. _yaml_scalar now quotes every name, archive and description it
writes.
2026-08-10 13:35:42 +02:00
Abdessamad Derraz 5d417c8229 feat: add data contract validation
JSON Schemas for the database, install and pack manifests, target
manifests, site API envelopes and stats, plus the semantic invariants a
schema cannot express: declared totals matching their lists, no
destination both installed and omitted, database keys matching their
sha1. validate_site.py checks the rendered HTML for metadata, headings,
image alternatives, duplicate ids and unresolved local links.

Pack manifests are read from inside the generated archives, where
generate_pack writes them, rather than from a dist/ glob that matches
nothing.

Emulator and platform schemas gain additionalProperties: false, and
cores[] plus contents[].name must be strings: an unquoted 81 or 01 in
YAML parses as a number and stops matching the upstream name.
2026-08-10 13:35:33 +02:00
Abdessamad Derraz 45f89cc6c0 feat: add region filters and profiles 2026-08-09 14:15:56 +02:00
Abdessamad Derraz 1c537c3a30 fix: keep mame refresh to profiles on mame's tree 2026-08-08 13:52:25 +02:00
Abdessamad Derraz 731096cf12 chore: refresh mame bios refs from 0.289 2026-08-08 13:50:25 +02:00
Abdessamad Derraz 89211003cb fix: refresh mame refs when a driver line moves 2026-08-08 13:48:54 +02:00
Abdessamad Derraz 1232e43e7a chore: recale refs after the parsing fixes 2026-08-08 13:44:47 +02:00
Abdessamad Derraz fa6de11133 feat: parse colon continuations and header stems 2026-08-08 13:44:03 +02:00
Abdessamad Derraz 587aefd282 fix: name the external project on every dosbox ref 2026-08-08 13:41:28 +02:00
Abdessamad Derraz dde5509a39 chore: recale refs followed to their subject 2026-08-08 13:40:37 +02:00
Abdessamad Derraz 17da2ae76f feat: follow refs to a unique declared value 2026-08-08 13:39:54 +02:00
Abdessamad Derraz c8e05013d9 feat: follow a ref when its subject moved with it 2026-08-08 13:38:30 +02:00
Abdessamad Derraz f4a8d8c54b chore: rebuild manifests and docs 2026-08-08 13:36:57 +02:00
Abdessamad Derraz 31c4bee36b fix: point amiberry kickstart refs at candidates 2026-08-08 13:36:51 +02:00
Abdessamad Derraz c42209fa96 fix: point three refs at their real locations 2026-08-08 13:35:19 +02:00
Abdessamad Derraz 82b510de0d fix: relocate gear family bios refs 2026-08-08 13:23:06 +02:00
Abdessamad Derraz 642e9ea9ca fix: mark ecwolf data as standalone only 2026-08-08 13:20:54 +02:00
Abdessamad Derraz 7755367787 docs: document tag-pinned profile handling 2026-08-08 13:17:52 +02:00
Abdessamad Derraz b00ca4d237 docs: state only what the data supports 2026-08-08 13:16:17 +02:00
Abdessamad Derraz 4c5ca17682 feat: judge tag-pinned profiles on their own revision 2026-08-08 13:12:35 +02:00
Abdessamad Derraz 088caef45e docs: use one plain vocabulary across surfaces 2026-08-08 12:59:46 +02:00
Abdessamad Derraz 89fe022196 chore: recale reviewed refs for three cores 2026-08-08 12:54:03 +02:00
Abdessamad Derraz c94771459c chore: rebuild manifests and docs 2026-08-08 12:51:42 +02:00
Abdessamad Derraz 1fda81fb52 feat: record ares laseractive firmware hashes 2026-08-08 12:47:41 +02:00
Abdessamad Derraz 09d586e387 docs: document the ref anchoring strategies 2026-08-08 12:42:29 +02:00
Abdessamad Derraz 38e39af561 docs: say what each platform actually checks 2026-08-08 12:42:06 +02:00