Commit Graph
1064 Commits
Author SHA1 Message Date
Abdessamad Derraz fd6db086d3 fix: keep full lists in check_freshness JSON 2026-10-18 08:40:44 +02:00
Abdessamad Derraz 6dc373fcfd fix: refuse two output forms in diff_truth 2026-10-18 08:40:01 +02:00
Abdessamad Derraz b23f5b04ad fix: refuse a --bios-dir that does not exist 2026-10-18 08:39:27 +02:00
Abdessamad Derraz 4240c98359 fix: name a missing database in one error line 2026-10-18 08:34:09 +02:00
Abdessamad Derraz 9579f6c449 fix: name platforms and core misses in the summary 2026-10-18 08:31:16 +02:00
Abdessamad Derraz e75482777d fix: leave aliases out of the fileless audit 2026-10-18 07:41:54 +02:00
Abdessamad Derraz 7531ccad8f fix: refuse a profile_sync filter matching nothing 2026-10-18 07:33:11 +02:00
Abdessamad Derraz 2362466903 fix: suggest a region for an unknown spelling 2026-10-18 07:32:46 +02:00
Abdessamad Derraz a8c23ad11d docs: number pipeline steps as the run prints them 2026-10-18 07:31:43 +02:00
Abdessamad Derraz 1305516e87 docs: say --offline cuts release asset downloads 2026-10-18 07:30:56 +02:00
Abdessamad Derraz 3116a57617 docs: document every option the scripts accept
Fifty options appeared in no page. fileless_audit also gains --all, so
its example sweeps instead of naming three settled profiles, and it and
download.py --info now fail on an unknown name or an unreachable source.
2026-10-18 07:29:16 +02:00
Abdessamad Derraz ae3dc63bc3 fix: accept RomM files on size and any hash
RomM's verify_file_hashes compares the size, then md5, sha1 or crc. We
compared md5 alone, so a fixture entry with one mistyped hash was left out
of the pack RomM would accept it from. The rule is declared in romm.yml
and read by verify, the builder and the manifest.
2026-10-18 07:24:33 +02:00
Abdessamad Derraz d6d03617db fix: exit 1 when a hash lookup finds nothing
The documented example looked up a test fixture hash, printed NOT FOUND
and exited 0. The docs now use a hash the database holds and say which
line forms a hash file accepts.
2026-10-18 07:19:01 +02:00
Abdessamad Derraz 6d48d9e71e fix: report the emulator cross_reference is asked
A mistyped --emulator exited 0 with no profile found, and a correct one
with no gaps was left out of its own report, totalled across all
emulators. The name is now checked, the emulator always printed, and
--list-emulators added for the suggestion to point at.
2026-10-18 07:16:30 +02:00
Abdessamad Derraz 195fb92036 fix: name the platform a typo meant in every CLI
slots ended a mistyped --platform on a traceback; generate_truth,
refresh_data_dirs and auto_fetch exited 0 having done nothing for it.
All six now stop on require_platform before any work.
2026-10-18 07:14:38 +02:00
Abdessamad Derraz 803f9dce96 fix: hash only files that share a size
identity.py read the whole 18 GB tree on every pipeline. A file whose
size no other file shares cannot have a duplicate.
2026-10-18 07:12:59 +02:00
Abdessamad Derraz 2db83a5e01 feat: confront added profiles in CI
profile_sync --strict exits 1 when a profile has refs to review. The
validation workflow runs it on every profile a pull request or a push to
main adds, so a profile whose refs miss their value no longer lands green.
2026-10-18 06:29:14 +02:00
Abdessamad Derraz 572a2d20de feat: check a release against its record
A count of assets misses a failed upload beside a stray file and a part
re-uploaded from an older build. check_release.py holds each asset's
GitHub sha256 against SHA256SUMS.txt and release.json, and the release
process publishes the draft only on its clean exit.
2026-10-18 06:26:39 +02:00
Abdessamad Derraz 694d1defc0 fix: file truth models by each platform's target
Lakka builds an image named switch while RetroArch files switch as
nintendo-switch. One canonical name taken across the registry put
Lakka's model under RetroArch's target. generate_truth now writes each
platform under its own target name, and diff_truth --target reads it.
2026-10-18 06:24:07 +02:00
Abdessamad Derraz c3162dcd27 feat: scrape Lakka per-device core lists
Lakka builds LIBRETRO_CORES minus the exclusions each project, device
and GLES image adds. Without a list, --target refused Lakka and its RPi1
pack carried firmware for cores the image does not build.
2026-10-18 06:24:01 +02:00
Abdessamad Derraz bac91d0824 fix: stop asking for dumps the collection can make
The Wanted page listed 79 dumps the collection already produces,
such as a Kickstart's byte-swapped words or the halves of a C128
character ROM. The pipeline now records which missing dumps can be
derived, in derivable.json, and the page moves them to a section of
their own that names the held file and the transform.
2026-10-18 06:16:16 +02:00
Abdessamad Derraz fd4c118a38 fix: count a catalogued dump once across its DATs
Several DATs list the same dump, as No-Intro's three FM-7 sets do
for their system ROMs, and the provenance report counted it once per
listing: 2485 missing announced for 2284 distinct dumps, on the site
as well. Dumps are now counted by identity, each keeping the DATs
that list it.
2026-10-18 06:11:12 +02:00
Abdessamad Derraz e1b6aad51b fix: refuse abbreviated flags in every script
argparse completed a prefix into the only flag starting with it:
slots.py --emulator read profiles from a directory named after the
emulator and reported nothing, and pipeline.py --all built every pack
variant. Every parser now refuses an abbreviation.
2026-10-18 05:30:58 +02:00
Abdessamad Derraz 88a3bb8953 fix: let auto_fetch read lists and keep held files
auto_fetch stopped on any profile listing several accepted hashes,
and filed a fetched file in the first folder sharing one word of its
system, overwriting a same-named revision held there. Hash fields are
read as sets, a folder must match the whole system slug, and another
revision goes to .variants/ beside the file already held.
2026-10-18 05:30:10 +02:00
Abdessamad Derraz cc7f9fbdac docs: say what the pack installers really do
The EmuDeck pack README promised standalone keys set up
automatically, which needs --standalone-copies, and named a Citra
folder the installer never uses. The RetroArch README gave Windows
users a shell command only, and RetroPie's sent a PC to a Linux
partition it cannot open. Each guide now matches the installer.
2026-10-18 05:28:31 +02:00
Abdessamad Derraz d3dd045c04 fix: let --write run the report it writes from
romset_recipes --identify --write ran only the identification, so
the reconstructions --write saves were never computed and the run
exited 0 having written nothing. --write now implies --missing.
2026-10-18 05:21:06 +02:00
Abdessamad Derraz 999a77a96a fix: refuse a data key the platform does not use
Under --key the platform restriction was not applied, so dolphin-sys
was downloaded again for RomM, which never reads it. A key whose
platforms exclude the one named is now refused.
2026-10-18 05:20:25 +02:00
Abdessamad Derraz 32d32e5718 fix: refuse validate_pr flags it would drop
Named files beside --changed were replaced by the git diff, and the
run reported no changed files about files it never read; --json
beside --markdown dropped the markdown. Both combinations are now
refused, and an empty JSON run prints an empty list.
2026-10-18 05:20:03 +02:00
Abdessamad Derraz e04646d67a fix: stop the MAME scrape on a clone with no pin
A failed rev-parse in the MAME clone returned an empty commit, which
was cached for a day and let the merge advance the profile's version
and refs while its upstream pin stayed on the previous release. The
scrape now stops instead.
2026-10-18 05:15:52 +02:00
Abdessamad Derraz baf635cf4e fix: refuse to pack an archive that fails its CRC
The deterministic rebuild raises on a member whose bytes fail their
CRC, and the builder caught it and copied the damaged archive into
the pack, expecting verify to flag it, which nothing did. The run now
stops on it and names the file to restore.
2026-10-18 05:15:15 +02:00
Abdessamad Derraz 91b378d641 fix: copy standalone files from their pack path
The registry names one path per standalone copy for every platform,
but each manifest places the file its own way, so the installer
looked where the pack had put nothing and copied nothing: EmuDeck's
aes_keys.txt for Azahar, Batocera's switch/prod.keys. Each manifest
copy now names its source, which the installer validates and reads.
2026-10-18 05:09:05 +02:00
Abdessamad Derraz 648784d875 fix: run the suite so its package setup applies
The documented command, discover tests, makes tests/ the top level,
so tests/__init__.py never ran and the variable it sets to keep tests
offline was missing from every local run. Every documented and CI
run now passes -s tests -t ., and a test requires it.
2026-10-18 05:05:09 +02:00
Abdessamad Derraz e0bf0ee191 fix: make a patched test patch what actually runs
A test isolated the extra collection by patching
resolve_platform_cores on packextras with create=True, while the
module imported it inside the function, so the real resolver ran.
The import is now module level, the test checks its stub was called,
and no test may create the attribute it patches.
2026-10-18 04:15:05 +02:00
Abdessamad Derraz ef0023d78c fix: key the manifest hash cache by its check
The cache of manifest hashes was keyed by the database's id without
holding the database, so a freed database replaced at the same
address got another scan's verified hashes back. The key now carries
the sha1 that database expects for the path.
2026-10-18 04:13:32 +02:00
Abdessamad Derraz 5a4432742c fix: clear an export's old files before writing
An export wrote into its platform directory without removing what an
earlier run left there, so RetroPie scriptmodules and RetroDECK
manifests the current truth no longer corrects stayed beside a
report saying they were untouched. Files the run does not produce
are now removed first.
2026-10-18 04:11:54 +02:00
Abdessamad Derraz 482207264c fix: patch RomM's fixture instead of rebuilding it
The RomM export fetched known_bios_files.json and never read it: the
fixture was rebuilt in our own order, so three corrections came out
as a diff of the whole file and a key we had not transcribed was
deleted. The export now starts from RomM's file, keeps its order and
fields, and only changes the values it corrects.
2026-10-18 04:11:15 +02:00
Abdessamad Derraz b5c4e7340b fix: keep System.dat lines indented with spaces
libretro indents 39 rom lines of System.dat with spaces. The export
read tab-indented lines only, so those left the base and came back
re-rendered at the end of their section, and one our transcription
lacked would have been deleted. Any indentation is read and kept.
2026-10-18 04:10:21 +02:00
Abdessamad Derraz 23d9474f41 fix: serve BizHawk's database as BizHawk wrote it
The export only indexed firmware BizHawk leaves unhashed on purpose,
per-console dumps declared with SHA1Checksum.Dummy, and could only
apply that hash to a same-named call of another system. Every hash
the file states is BizHawk's own, so nothing is rewritten and no
correction is reported.
2026-10-18 04:09:40 +02:00
Abdessamad Derraz 9c0a430f5c fix: settle export sizes, never write None
A truth that accepts several sizes left an export with no size even
where the platform states one of them for the same dump, and RomM's
fixture then received the string None, which RomM cannot parse. A
size list now admits the platform's own size, RomM keeps its value
when nothing settles one, and validation requires a number.
2026-10-18 04:07:57 +02:00
Abdessamad Derraz d5f40a4515 chore: drop the shared groups no platform reads
Six of the eight shared groups injected nothing: three were
included by no platform, three repeated System.dat entry for entry,
so a correction made in them reached no pack. They are removed with
their scraper wiring, and a test requires every remaining group to
contribute an entry somewhere.
2026-10-18 04:03:01 +02:00
Abdessamad Derraz b6f6798c15 fix: place a standalone build's gaps at its path
The gap report took a file's libretro path even where the platform
runs the standalone build, so it named Cemu/keys.txt for Batocera
while verify named keys.txt, and settled the declaration on a slot
the platform never fills. Both now read standalone_path there.
2026-10-18 04:01:02 +02:00
Abdessamad Derraz c047b3828d fix: write a cached original and its URL together
The export cache wrote an original's bytes, then recorded the URL
that produced them under a separate lock, so two runs wanting two
revisions of one file could pair one's bytes with the other's URL.
Both now change under the one index lock.
2026-10-18 03:56:35 +02:00
Abdessamad Derraz 94ca509d3e fix: hold the data trees for the whole pipeline
The pipeline held its output directory for the run but let each step
lock the data directory trees on its own, so another session's
refresh could swap a tree between the packs and the manifests. The
trees are now held shared from right after the pipeline's own
refresh until it exits.
2026-10-18 03:55:51 +02:00
Abdessamad Derraz 383960212d fix: swap a pack in only once it is complete
Packs were opened on their final name, so a rebuild truncated the
previous one and a build that failed half way left a readable ZIP
of part of the files. Each pack is now written beside its name and
replaces it only when the write finished.
2026-10-18 03:54:59 +02:00
Abdessamad Derraz 164bc9e207 fix: ask again for a tag that was not there yet
A tag looked up before it was pushed was cached as absent for good,
so a profile pinned on that release was never recognised as frozen
on it. The absence now expires; a tag found stays cached.
2026-10-18 03:53:36 +02:00
Abdessamad Derraz 2a13e3ee11 fix: recount commits touching a path once stale
The commit list behind the triage drift score was cached for good,
so pushes upstream never moved a profile up the triage. It now
expires like the other moving answers.
2026-10-18 03:53:12 +02:00
Abdessamad Derraz 2dcb49fa4c fix: keep a held file in emulator region packs
Emulator mode reduced region groups without knowing which files the
collection holds, so a requested-region image nobody had collected
withdrew the held one and left the slot empty. Platform mode already
kept it; both emulator readers now pass the database.
2026-10-18 03:52:47 +02:00
Abdessamad Derraz a40a68c1f7 fix: group a system once whatever its spelling
The platform's files and the core extras for the Master System sat
in two region groups, sega-master-system and sega-mastersystem,
reduced apart. A US pack kept the declared Japanese BIOS. A core
extra's system group now takes the platform's spelling, in the pack,
the report and the installer's manifest fields alike.
2026-10-18 03:51:57 +02:00
Abdessamad Derraz e9da4d72ed fix: write one truth model per target, not alias
generate_truth named a narrowed model's directory after the target
as typed, so three aliases of one target wrote three copies, and
the pipeline rebuilt the slug by hand to diff it. One function now
names the directory from the canonical target for both.
2026-10-18 02:55:45 +02:00
Abdessamad Derraz 44d363e739 fix: name a target the way each platform files it
A target was canonicalised once for the run and kept as typed under
--all when platforms file it differently, so the --all build and a
--platform check named the same Batocera pack two ways. Each pack,
group, manifest and expected name now takes the target as its own
platform files it.
2026-10-18 02:53:33 +02:00