Commit Graph
512 Commits
Author SHA1 Message Date
Abdessamad Derraz 3e6f30c697 fix: send github token to scraper api calls 2026-10-04 15:06:00 +02:00
Abdessamad Derraz 8a2a9c6ffa feat: add freshness tooling and override defaults 2026-10-04 14:54:11 +02:00
Abdessamad Derraz 08ef1e8dcf fix: fail batocera target scraper on empty input 2026-10-04 14:36:32 +02:00
Abdessamad Derraz 08bce47668 fix: keep install manifests and release assets true 2026-09-14 01:17:49 +02:00
Abdessamad Derraz 4f67e3cad3 fix: honour a contradicted hash that is not an md5 2026-09-07 07:48:54 +02:00
Abdessamad Derraz c00a1ecc6c fix: collect a citation to an extensionless file 2026-09-07 07:22:26 +02:00
Abdessamad Derraz 1455e0423c fix: arbitrate on the evidence the builder uses 2026-09-07 07:02:39 +02:00
Abdessamad Derraz c99e292a4d fix: stop a romset member claiming a bios slot 2026-09-07 06:50:56 +02:00
Abdessamad Derraz 1c4eb8adec fix: correct a platform list without replacing it 2026-09-07 06:40:18 +02:00
Abdessamad Derraz 23c4dc2aae fix: name every narrowed artifact from one list 2026-09-07 06:27:35 +02:00
Abdessamad Derraz 29fcfdef34 fix: refuse the flags a mode cannot apply 2026-09-07 06:14:24 +02:00
Abdessamad Derraz aaa1aea452 fix: put every writing mode under the output lock 2026-09-07 05:57:25 +02:00
Abdessamad Derraz 6d97fbe127 fix: stop a check that cannot answer exiting zero 2026-09-07 05:39:50 +02:00
Abdessamad Derraz 1c976390d7 fix: let a refresh failure reach the exit code 2026-09-07 05:30:56 +02:00
Abdessamad Derraz 210ff2b647 fix: keep a member hash out of the archive alias 2026-09-07 04:34:12 +02:00
Abdessamad Derraz 2a4dd59bb2 fix: narrow pack conformance by hardware target 2026-09-06 23:28:27 +02:00
Abdessamad Derraz a61c800462 fix: report a contradicted hash on an archive too 2026-09-06 23:08:04 +02:00
Abdessamad Derraz 8128b47aeb fix: skip full conformance on a targeted pack 2026-09-06 23:06:00 +02:00
Abdessamad Derraz a4455ef6a7 fix: refuse flags verify-packs cannot honour 2026-09-06 23:01:14 +02:00
Abdessamad Derraz fca005f6db fix: narrow the manifest when required-only is set 2026-09-06 22:59:18 +02:00
Abdessamad Derraz 6fab74c1f9 fix: give each download run its own staging dir 2026-09-06 22:53:48 +02:00
Abdessamad Derraz 691ccbfca7 feat: make the native export round-trip a platform file 2026-09-05 17:32:48 +02:00
Abdessamad Derraz 7b93285e1d fix: read the build mode per emulator when arbitrating 2026-09-05 13:25:24 +02:00
Abdessamad Derraz da04df6d6f feat: report a destination a platform fills two ways 2026-09-05 10:51:17 +02:00
Abdessamad Derraz a0b2263bae feat: let a profile say its pin is held on purpose
nethersx2-turnip-classic documents the shim rules of the revision it
pins, and says so in prose: the classic build is that revision. Compared
to HEAD it reported the same two changes every pass, and recaling would
have repointed its refs at rules this build never had.

pin_frozen says the freeze is deliberate, and the profile is then judged
against its own revision, exactly as one pinned to a superseded tag
already was. Recale and bump are refused for it, which is what makes the
declaration safe to trust.
2026-09-05 10:50:53 +02:00
Abdessamad Derraz bc7f131b6f fix: compare a citation without the spacing its author chose
A recale rewrites the located tokens and leaves the sentence alone, so a
prose run keeps `a.c:228, 1439-1443` while the rendered form drops the
space. Comparing the two literally made a run that had just been written
look unwritten, pending_recale counted it, and bump_commit refused for
ever.

mariani was held back by three of them: it recaled twenty-five refs and
could never advance its pin, which is the desync the all-or-nothing rule
forbids. 67 refs, all anchored.
2026-09-05 10:41:58 +02:00
Abdessamad Derraz 02edf40d8f fix: let the fallback mapping reach the largest file cited
openbor.c is 55k lines at the pin and 57k at HEAD, over a ceiling of
40k, so its ref reported CHANGED with nothing to act on: the mapping was
never attempted. Diffing that pair takes nine seconds, and the fallback
only runs once exact anchoring has failed, which is rare.

The ceiling now sits above it and still stops a pathological pair from
stalling a sweep of every profile. savesettings moved from 2675 to 2940
and the range maps cleanly.
2026-09-05 10:17:37 +02:00
Abdessamad Derraz 3c16f6f640 fix: let a deep path answer for the file it names 2026-09-05 09:41:04 +02:00
Abdessamad Derraz 47e267d29c chore: drop files whose bytes are not what they claim 2026-09-05 09:17:42 +02:00
Abdessamad Derraz adcd177e12 feat: settle contested slots in the pack and the report 2026-09-05 09:08:43 +02:00
Abdessamad Derraz c3d9e96d61 feat: arbitrate a contested slot by who the pack serves 2026-09-05 08:53:47 +02:00
Abdessamad Derraz 77929f8c82 feat: report slots where a profile contradicts the pack 2026-09-05 08:41:30 +02:00
Abdessamad Derraz ab170f990d docs: add a first install walkthrough 2026-09-05 07:46:06 +02:00
Abdessamad Derraz 45e21fca51 docs: cover android in the installer pages 2026-09-05 07:02:57 +02:00
Abdessamad Derraz 6582c3237e fix: name the stale pin when the file is only shorter
The diagnostic caught a ref whose file was missing at the pin. nestopia
showed the other shape: the file is there and the line is not yet. Its
palette and database loads were cited at 2041 and 2063, which is where
HEAD carries them, against a pin four hundred lines shorter, and that
reported as a plain GONE with nothing to act on.

A cited line past the end of the pinned file that fits HEAD is the same
finding as before and now says so. nestopia's pin moved to the revision
its refs describe: 8 refs, all anchored.
2026-09-05 01:31:10 +02:00
Abdessamad Derraz c738073f66 fix: move the refs and the pin together or not at all
Recaling refs while the pin stays put produces exactly the state the
all-or-nothing rule exists to prevent: a profile whose refs describe one
revision and whose source_commit names another. The tool manufactured it
on mariani, where three prose runs it could not rewrite kept
bump_commit refusing while eleven refs had already moved.

Asking for both writes is now atomic. The work happens on a copy, which
is promoted only when the pin follows, and a rebase is refused outright
when something visible beforehand will hold the pin: an annotated ref,
one under a mode key, or a prose run whose tokens cannot be located well
enough to rewrite. Where the block only appears after the write, the
copy is discarded and the profile is named on stderr rather than left
half moved.

mariani is back on its pin and stays at four refs to read, which is
honest: three of them have to be rewritten by hand before anything can
advance.
2026-09-04 18:55:15 +02:00
Abdessamad Derraz 6f27a296f7 feat: refuse a citation no repository can hold
kenji-nx cited tmp/es-de/ANDROID.md:470-474, a path from the machine of
whoever profiled it. No revision of any declared repository holds it, so
profile_sync could only report it missing, every pass, forever, and no
amount of reading would ever settle it.

validate_schemas now refuses a scratch directory, an absolute path, a
Windows drive path and one climbing out of the tree, and names the
offending citation rather than the scalar that carries it. Offline, so it
runs on every push and every pull request rather than waiting for a
network pass.

The ES-DE citation reads as external now, which is what it always was.
kenji-nx is at 37 refs, all anchored: the three changed blocks were var
giving way to explicit types.
2026-09-04 18:26:27 +02:00
Abdessamad Derraz d124f6c516 feat: watch the profiles that declare no files
An empty file list is the one assertion here that ages unwatched. Nothing
can go missing and no ref can drift, so nothing notices when a core that
embedded everything grows a path. virtualjaguar said "No external BIOS
files are required or loaded by this core" while its source had grown
eleven filenames read from the system directory, and only a reading
found it.

fileless_audit looks for the request itself, the system directory ask, in
the sources each profile already cites. Over the 151 fileless profiles it
named eleven, of which two were covered by data_directories, six carried
an exclusion_note, and three had nothing written down at all: craft
writes its world database in that directory, dice stores the answer in a
variable no other file in the tree names, lutro hands it to the Lua game.
Each now says so.

The check settles: declared files, a declared directory, or a written
answer all end it, so what it reports is the set nobody has read yet. A
test holds the corpus at zero.
2026-09-04 18:13:53 +02:00
Abdessamad Derraz 1a1f19be2d feat: check a MAME ref against the set it names
profile_sync follows content, so a ref that drifted still anchors where
the cited text went. That is drift detection working, and it cannot
answer the only question a MAME romset ref asks: does this line declare
this set. It flagged five refs in one driver file where nineteen were
stale, the fourteen others having been relocatable somewhere plausible.

mame_ref_audit asks the stronger question and found ninety-two across
the four profiles whose upstream still moves: mame 66, mamearcade 18,
mamemess 6, groovymame 2. Each had exactly one declaration to point at.
The frozen generations, mame2009 through mame2016, come out clean, which
is the check saying it finds drift only where drift can happen.

The set name is argument 1 of the machine macro. Matching it anywhere on
the line matches every clone naming it as parent, which is most of a
driver, and comments are stripped first because a declaration can sit
behind one. A set no machine declares is reported as not judgeable, not
wrong: device archives take their DEFINE_DEVICE_TYPE shortname.
2026-09-04 17:48:24 +02:00
Abdessamad Derraz ece637d52d feat: let a profile state that its upstream is gone
Saying those profiles would stay open no matter what was wrong. A dead
forge is a verifiable fact, and a fact can be recorded: upstream_gone
carries why, and the profile stops being an open problem. The refs
describe the last revision anyone could reach, which is all any reader
can ask of them.

The declaration is guarded rather than trusted, because a forge can come
back and a profile that keeps asserting a death nobody rechecks is worse
than one that fails loudly. Declared and unreachable reports as a
recorded fact; declared and answering reports as the contradiction it
is, and the profile has to be read again.

yuzu and suyu carry GitHub's 451. citron carries the loss of
git.citron-emu.org and the squatter now sitting on the .com. Each names
what was probed and when, so the next reader retraces it rather than
repeating it.
2026-09-04 17:25:23 +02:00
Abdessamad Derraz 1fb717324b fix: name the stale pin instead of a missing file
A ref whose file is absent at the pinned revision reported "pin revision
missing", which reads as code that vanished. When the same file is at
HEAD and the cited range fits it, nothing vanished: the ref was written
against HEAD while source_commit still names an older revision, and the
profile describes two trees at once. The reason now says so, because the
fix is the pin and not a hunt for a move that never happened.

This is the state three profiles were left in during their own
reprofiling, against a warning the repository already carries. A range
that overruns HEAD stays a plain miss, and a file absent from both
revisions is unchanged.
2026-09-04 16:45:51 +02:00
Abdessamad Derraz ca307a4ef1 feat: keep a profile checkable when its forge goes
Four Switch profiles were unverifiable and said so on stderr every pass.
yuzu and suyu answer 451, citron's host stopped resolving, and
git.eden-emu.dev returns 403 to anything that is not a browser. Each
aborted its own report, so nothing could be said about any of them, and
the noise repeated on every run over the whole corpus.

A withdrawn forge is now a fact rather than a failure. GoneError covers
451, 410 and a host that does not resolve; none is retried, since three
attempts with backoff end in the same place. Those profiles land in
their own summary bucket, out of the review backlog where nobody could
act on them anyway. A 403 stays what it was, a refusal, because small
Forgejo instances behind anti-bot filters issue it routinely.

A profile can now name a source_mirror, consulted after source and
upstream so a live primary always decides attribution. Reaching it took
two more changes: a repository that refuses is muted for the rest of the
pass instead of ending it, keyed by host as well as slug because a
mirror carries the same slug on another forge; and a refused miss is not
cached, or the mute would answer for the mirror that was about to be
asked.

eden now reads from its Codeberg copy, which holds the same head and the
pinned commit: 5 refs, all anchored, where the profile could not be
checked at all. yuzu, suyu and citron have no mirror that serves
content, and now say so once instead of failing loudly.
2026-09-04 15:35:00 +02:00
Abdessamad Derraz 6a0fb1e776 fix: resolve a bare cited name at the pin first
A ref citing a bare filename, the way prose does, was matched against the
HEAD tree alone. A file that moved since the pin then resolved to its
HEAD path, which does not exist at the pin, and the ref reported GONE
with "pin revision missing": it failed for the one reason it never
should, its own success at HEAD.

The pin tree is searched first, HEAD stays the fallback, and the rename
search carries the pin path forward as it does for any written path.
This is what the resolver already documents for prefixed and suffixed
paths; the bare-name branch was the one that did not follow it.

linapple cites Memory.cpp, src/Memory.cpp at its pin and
src/apple2/Memory.cpp today. Its refs now name where the code went
instead of reporting it missing.
2026-09-04 14:46:46 +02:00
Abdessamad Derraz 3891d897a6 fix: report the writes a dry run would make
--backfill-commits and --realign-prose have always printed what they
would write. --rebase-refs and --bump-commit took the flag and printed
nothing, so the only way to read a plan was to let it happen, and the
recale and the pin had to be done in two full network passes with
--force in between.

Both now plan. The plan runs the production write path over a throwaway
copy rather than a parallel branch, so it cannot drift from the write,
and the planned bump reads the text the recale would have left: a pin
held back by prose the same pass would move is no longer reported as
blocked. One pass does both, recale before bump on each profile.

bump_commit also stopped announcing a rewrite of the pin to the value it
already held. On the corpus that was 126 of 232 announcements, which
buried the 106 profiles that did move.
2026-09-04 14:41:37 +02:00
Abdessamad Derraz 1a96853aee feat: sign the release checksum list
SHA256SUMS.txt sat beside the artifacts it vouches for, so whoever could
rewrite a release rewrote the list with it. The packs were already
reproducible, which answers corruption and lets a third party rebuild an
archive byte for byte; nothing answered a rewritten release.

The list is now signed with an ed25519 key kept for this alone, and the
public half is allowed_signers at the repository root, so verification
does not go through the release page: ssh-keygen -Y verify against the
committed file, then sha256sum --check. Rehearsed on all three outcomes:
a good signature, a tampered pack caught by the sums, a rewritten list
caught by the signature.

The release steps sign and upload the signature, the README points a
downloader at the procedure, and the reproducibility section says what
each half proves. Rotation keeps retired lines so past releases stay
verifiable. Three tests hold the trust root, the signing step and the
documented principal in agreement.
2026-09-04 14:01:05 +02:00
Abdessamad Derraz fe77535c3b fix: state one catalog ratio, correct the faq
The home page and the stats export counted every file carrying a
provenance record, the provenance page and the README only the system
files. The site published 553 and 566 for the same quantity, one click
apart, and the export paired the wider count with composition.systems as
its denominator. common.count_catalog_matched is now the single source,
scoped to the systems bucket.

The FAQ had drifted from the profiles it describes: MAME pinned at 0.287
against 0.289 in mame.yml, Adler-32 attributed to Dolphin's IPL rather
than the DSP ROMs that carry known_hash_adler32, and the per-emulator
verbose report named as the only content check on an existence platform,
which skips the DISCREPANCY line the platform report raises itself.

Tests read both sides: no generator may count matches inline, and each
FAQ claim is checked against the profile or the script that owns it.
2026-09-04 11:41:17 +02:00
Abdessamad Derraz ee6e7558f9 refactor: assign each module constant once 2026-09-04 10:19:04 +02:00
Abdessamad Derraz 361b958f41 feat: download packs published as split volumes 2026-09-04 10:18:14 +02:00
Abdessamad Derraz d368b37165 docs: surface region filtering in the readme 2026-09-04 08:31:05 +02:00
Abdessamad Derraz b1d1eeab06 docs: give the installer a page of its own 2026-09-04 06:55:23 +02:00