Commit Graph
1277 Commits
Author SHA1 Message Date
Abdessamad Derraz 378a9e4da5 chore: merge audit fixes (integrity)
# Conflicts:
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/retropie.json
#	install/rocknix.json
#	install/romm.json
#	scripts/generate_pack.py
#	wiki/tools.md
2026-10-21 06:48:42 +02:00
Abdessamad Derraz 4ba935374e fix: keep pack order outside contested paths
Ranking contested core claims sorted the whole gap report, which
reordered every manifest. Only the claims on one contested path move now,
grouped where the first of them stood, and two spellings of a path are
one group where the filesystem folds case (RetroBat's 64DD_IPL_DEV.bin).

An entry without a destination (EmuDeck's system:md5 whitelist) no
longer counts as a pack path in verify, and step 5 prints one status for
counts and paths together. A full offline run compared all 12 archives
and 930 emulator packs path by path, every one equal.
2026-10-21 06:47:44 +02:00
Abdessamad Derraz 85a44d9bf7 chore: merge amiberry and hatari re-profile
# Conflicts:
#	README.md
#	emulators/amiberry.yml
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/rocknix.json
2026-10-21 06:21:30 +02:00
Abdessamad Derraz 5ce7ab8386 fix: self-check a ref in every repository
A prose citation of the port's own lines was read only in the upstream file that a source_ref attributes the path to, and a range past that file's end came back GONE although the port carries it.
2026-10-21 06:18:00 +02:00
Abdessamad Derraz 58dc7f70b4 chore: merge profile data follow-up 2026-10-21 06:13:25 +02:00
Abdessamad Derraz 6b4010907d feat: list the six BizHawk DSi NAND slots
FirmwareDatabase.cs:405-410 declares NDS+NAND_JPN, EUR, USA, AUS, CHN
and KOR, each with a placeholder hash and 251658240 + 64 bytes. In DSi
mode DecideNAND takes the first one set, limited to the regions a
DSi-enhanced header allows, and stops the load with none
(MelonDS.cs:114, 304, 484, 684-706). DSi mode is off by default, so the
entries are optional, like DSi_Firmware.bin and the DSi BIOS slots.
2026-10-21 06:07:30 +02:00
Abdessamad Derraz 7056dd84ea fix: state why WinUAE needs the CD extended ROMs
A missing extended ROM only raises a notice (memory.cpp:1833-1849), but
the machine then runs on the Kickstart alone. cdstrap, cd.device,
cdtv.device and cdfs.library are resident modules of the CD32 and CDTV
extended ROMs and absent from their Kickstarts, the CD32 and base CDTV
models have no floppy drive (cfgfile.cpp:9375-9379, 9405-9406), so no CD
is read or booted. Both entries stay required. The AROS replacement
stands in only for a missing Kickstart (memory.cpp:3055-3063), so the
CDTV entry loses hle_fallback.
2026-10-21 06:06:35 +02:00
Abdessamad Derraz d678e6acfa fix: declare the FreeIntv overlays on the core
FreeIntv declared no data tree while its alias freeintv_ts_overlay,
the same code, declared freeintv_overlays. The core reads
freeintv_overlays/<rom>.png only with the multi-screen option and draws
its embedded keypad image without one (src/libretro.c:278-280, 294,
321-325), so the tree is optional. A test holds every alias to the trees
of its core.
2026-10-21 06:06:07 +02:00
Abdessamad Derraz 1a41e6a964 chore: merge psx region follow-up 2026-10-21 06:05:04 +02:00
Abdessamad Derraz 47d9e1d629 fix: tag US PlayStation images latin-america too
Each core picks its US image for a disc licensed by Sony Computer
Entertainment America, and DuckStation, SwanStation and Beetle PSX
also for a disc whose region they cannot read. Tagged north-america
alone, a --region brazil,jp pack withdrew those images.
2026-10-21 06:04:42 +02:00
Abdessamad Derraz 35b4cacb5d chore: merge audit fixes (installer)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 06:02:02 +02:00
Abdessamad Derraz 1c4686ab93 feat: place RomM firmware by its own template
RomM 5.4 reads each platform's firmware where filesystem.structure.firmware
in config.yml says, bios/{platform} by default, and suggests {platform}/bios
for a {platform}/roms library (rommapp/romm 5.4.0,
backend/config/config_manager.py:58,101-109,205-225,564-568,1105-1124).
The installer only wrote bios/<platform>/. It now reads the template from
config.yml beside the library, takes --romm-firmware when that file is
elsewhere, refuses a platform-first library with no template, and detects
RomM by /romm/config/config.yml too.
2026-10-21 05:53:23 +02:00
Abdessamad Derraz 734643b8fc chore: merge audit fixes (site)
# Conflicts:
#	wiki/testing-guide.md
2026-10-21 05:53:15 +02:00
Abdessamad Derraz a1a9da59af chore: merge audit fixes (export)
# Conflicts:
#	exports/recalbox.json
#	exports/retropie.json
2026-10-21 05:53:05 +02:00
Abdessamad Derraz ce10323f7c chore: align two tests with the merged lock flow
The lock refusal moved to stderr and pipeline labels lost their /8 in
two branches that met at merge.
2026-10-21 05:53:01 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00
Abdessamad Derraz 53da2a95d2 chore: merge audit fixes (docs)
# Conflicts:
#	tests/test_doc_surfaces.py
#	wiki/installer.md
#	wiki/testing-guide.md
2026-10-21 05:50:25 +02:00
Abdessamad Derraz 43c6df58ee chore: merge audit fixes (region)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 05:48:59 +02:00
Abdessamad Derraz a67265e903 fix: let the site search land on cards
The search splits a page at its headings. Emulator cards were titled in
bold and platform systems folded into blocks under one heading, so a
pasted hash opened the top of a page or the systems overview of a
100-block platform page. Each file card is now a level 4 heading kept
out of the table of contents, and each platform system a heading.

Card lines that list other pages (platforms, packs, source citations,
collection paths) leave the search index: 24.7 MB down to 18.1 MB, the
names, hashes and notes still searchable.
2026-10-21 05:47:31 +02:00
Abdessamad Derraz f4a225488d chore: merge audit fixes (scrapers) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz eb44e3f898 chore: merge audit fixes (profiles) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 73d73e0f2a chore: merge audit fixes (profsync) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz e03fe338ee chore: merge audit fixes (tests) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 89c533e9ba fix: detect Batocera and Knulli by build files
No Batocera build writes /etc/batocera-version: batocera-system.mk:21-22
(batocera.linux 38845725, unchanged since batocera-30) writes
/usr/share/batocera/batocera.version. Knulli (knulli-linux 57afa662) adds
OS_NAME=knulli to os-release, writes /usr/share/knulli/knulli.version and
removes /usr/share/batocera, and no Knulli source writes /etc/knulli-release.
Neither was detected. A Knulli board is named and not narrowed by
Batocera's per-board lists, and an unlisted board is named.
2026-10-21 05:44:27 +02:00
Abdessamad Derraz 5b7f8526a8 refactor: read the gap report through its helper
test_launcher_layer looked for find_undeclared_files in the extras
collector, which now reaches it through undeclared_report.
2026-10-21 05:43:14 +02:00
Abdessamad Derraz aa36ec544d fix: name every gap verify counts
verify --platform printed N optional core files and N game data files
missing and named none, even with --verbose. --verbose now lists them as
MISSING (optional) and MISSING (game data), and without it the count line
says --verbose lists them, as the line about cores with no external file
already did.
2026-10-21 05:40:11 +02:00
Abdessamad Derraz 28435230a7 fix: read each named core's own entry in recalbox
An element's mandatory flag came from the one truth entry its md5 paired
with. saturn/saturn_bios.bin names yabause, yabasanshiro and kronos,
which require it, and was paired with the beetle_saturn entry, which
does not. Each named core's entries for the file, found by name, now
decide, except a namesake elsewhere in the same tree (bluemsx KANJI.rom
under Shared Roms is not the one under MSX2+ - European).
2026-10-21 05:39:17 +02:00
Abdessamad Derraz 82a3e44afd fix: settle a slot in any case where case folds
declaration_settles tested the exact-case name before the slot, so on a
filesystem that folds case a declaration spelled otherwise never settled
the core's file: RetroBat's 32X_G_BIOS.BIN left BlastEm's 32X_G_BIOS.bin
undeclared, and verify counted ten such entries as core files. A slot a
case-insensitive platform fills now settles the entry whatever the
spelling (case_folded_slots), the slot arbitration judging its bytes as
for the exact name. Lakka and RetroPie, which keep both names, are
unchanged. Measured: 10 entries settle on RetroBat, 21 on RetroArch.

generate_site.py builds its gap report through cross_reference() without
the folded slots; it needs folded_declared_slots() passed in the same way
as the command line now does.
2026-10-21 05:38:11 +02:00
Abdessamad Derraz edda080c59 fix: read muOS RetroArch from its own config
muOS runs RetroArch on a config copied from its default, which reads
/run/muos/storage/bios, bound to MUOS/bios on the second card when that
card holds one, and loads cores from /opt/muos/share/core (MustardOS/internal
f41f4a89, script/control/retroarch.sh:8-20, script/device/bind.sh:116-150,
retroarch.default.cfg:2806,3151). The installer wrote to /mnt/mmc/MUOS/bios
and --installed-cores found no core. The config now decides both.
2026-10-21 05:35:45 +02:00
Abdessamad Derraz 184c822e6e fix: name the file each layer reads in slots.py
Each content-checked line ended with a verdict on the platform's list,
printed at pipeline step 3b. The line now says which emulators read the
other file, and the closing count says the rest follow the platform's
list. The module source carries no such verdict, a test reads it.
2026-10-21 05:33:50 +02:00
Abdessamad Derraz bdcc8f0fb6 fix: lower the pipeline floor to Python 3.9
The 3.12 guard rested on generate_readme f-strings that have since been
rewritten. Every script compiles under CPython 3.9.25, 3.10 and 3.11, and
only generate_pack's parenthesized context managers fail under 3.8. Under
3.9 the offline pipeline (--skip-packs) completed with no change to the
committed artifacts, the 15 manifests matched 3.12 byte for byte, and the
RomM pack was identical. CI now compiles every script under a real 3.9.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz ceb8b752e6 fix: count bundled gaps apart in each heading
cross_reference.py --emulator abuse printed "Gaps: 3 missing" above three
[BUNDLED] files, then "0 missing (need to be sourced)" in the total. The
heading and the total now read one count: bundled files appear as
"N bundled", never among the missing.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz c05d29d941 fix: a held name must carry the declared hash
entry_source took a name or path hit before reading the hashes the entry
declares, so the gap report and the site counted as held a file whose
only homonym has other bytes: zquestclassic's ending.mid and gameover.mid
matched Hydra Castle Labyrinth's. A name or path hit now counts only
when its record carries one of the declared hashes, and the declared
content anywhere in bios/ comes before a data/ copy. Measured over every
profile entry: 254 change, 248 bundled ones (report and site only),
three unsourceable SC-55 revisions now read missing, three data hits
read bios by hash.
2026-10-21 05:32:19 +02:00
Abdessamad Derraz 15cb52d551 fix: find EmuDeck Windows where its scripts look
functions/all.ps1:4-14 (emudeck-we 037460ef) reads the APPDATA
settings.ps1, else the one under the profile, and takes C:\Emulation when
neither names a folder. Only the first was read and a forced platform fell
back to %USERPROFILE%\Emulation\bios. A network install names its share
through $VolName (JSONtoPS1.ps1:166-172), and a UTF-16 settings file is
decoded.
2026-10-21 05:31:35 +02:00
Abdessamad Derraz 0dab8f4706 fix: credit file readers through the data trees
FileConsumers resolved profile entries without the data directory
registry the builder uses. Dolphin's hashless codehandler.bin then took
Dolphin-MMJR's copy by name, and the site listed Dolphin and Triforce as
its readers while every pack serves data/dolphin-sys. blueMSX's RS232.ROM
and MSX2P.rom were credited to bios/ copies the same way. The registry
is now read by default, as the builder reads it.

With it, bundled_only.py --strict (pipeline step 4d) names
bios/Other/dolphin-mmjr/codehandler.bin: only dolphin-mmjr's bundled
entry, which already carries its sha1 and size, reaches that copy.
2026-10-21 05:29:42 +02:00
Abdessamad Derraz d2141ac0bb fix: give the MAME clone test a parsed set
The fixture parsed no BIOS root set, which the scraper now refuses.
2026-10-21 05:28:38 +02:00
Abdessamad Derraz cdb2cfa6f0 feat: find RetroBat through the folder it records
RetroBat.exe writes its folder to HKCU\SOFTWARE\RetroBat
LatestKnownInstallPath on every start (retrobat-executable 702da66a,
Program.cs:57 and 99, RegistryTools.cs:15-50). Only C:\RetroBat and the
profile folder were looked at, so an install on D: was missed and a forced
--platform retrobat wrote into C:\RetroBat\bios. The value is read
through winreg, or reg.exe under WSL, before the fixed folders.
2026-10-21 05:28:35 +02:00
Abdessamad Derraz 5e95574143 fix: the strongest proof takes a shared core path
A destination no platform declares went to the first claimant in profile
order whatever the proof: fbalpha2012 asks for hiscore.dat by name and
took the root of six packs over cores that pin it by sha1. Claims on a
contested path are now ranked by what the collection holds for each (a
held declared hash, then a path or every required archive member, then a
name, then a contradicted hash), and profile order only breaks ties.
Measured: 14 slots change file across the platforms, among them
hiscore.dat, kick13.rom on RetroPie, cgb_boot.bin and dmg_boot.bin on
EmuDeck, and ROCKNIX neogeo.zip and neocdz.zip.
2026-10-21 05:28:10 +02:00
Abdessamad Derraz 8f86842117 feat: keep romsets holding a missing catalog dump
The recipe snapshot kept only the archives a profile or platform names,
so provenance_report --missing named no set for dumps a public romset
carries (fm7740sx holds the FM77AV40SX DICROM, EXTSUB and KANJI2 ROMs).
The importer now also keeps the sets holding a catalog entry the
collection lacks. Reimported from mame0289 and the cached FBNeo DATs:
MAME 1726 to 2148 recipes, FBNeo 104 to 111, none dropped.
2026-10-21 05:27:42 +02:00
Abdessamad Derraz 309231afc2 docs: say what --skip-packs still runs
The help said "Only regenerate DB and verify" while steps 1b, 4d, 7 and
8 still ran and rewrote derivable.json, README.md and docs/. It now names
the steps it skips and points to --skip-docs.
2026-10-21 05:27:07 +02:00
Abdessamad Derraz 94a230a063 chore: frame the triage rows in a refusal test 2026-10-21 05:26:53 +02:00
Abdessamad Derraz 738577b5a4 fix: run verify and generate_pack from any dir
Started outside the repository root, both ended on a FileNotFoundError
traceback: the database and the data registry name files from the root.
They now move to the root after reading the paths the caller typed, which
stay relative to where they were typed.
2026-10-21 05:25:29 +02:00
Abdessamad Derraz 4f20b5daaf fix: name the network when every transfer fails
A run whose downloads were all cut ended on "Location that was being
written to" and "once that is resolved", pointing at the disk. A
failure caused by the connection (lookup, reset, timeout, TLS, a cut
transfer, HTTP 5xx or 429) is now marked as such, and the summary says to
check the connection and HTTPS_PROXY.
2026-10-21 05:25:19 +02:00
Abdessamad Derraz 61f0d033b5 chore: split a test docstring sentence 2026-10-21 05:25:10 +02:00
Abdessamad Derraz 76618af460 docs: shorten two script docstrings
_enrich_hashes repeated "ground truth" five times with capitalised
insistence for a three-step lookup, and the MAME ZIP rebuild listed
three guarantees of which two were the same.
2026-10-21 05:24:34 +02:00
Abdessamad Derraz 0457bd8f40 fix: resolve the pipeline output dir once
Every step runs from the repository root, but the lock and the purge of
stale packs read --output-dir from the caller's directory. Started from
scripts/, the run locked and purged scripts/dist/ and wrote to dist/,
leaving the stale packs there. The directory is now made absolute before
the lock, dist/ at the root by default.
2026-10-21 05:24:25 +02:00
Abdessamad Derraz 82d5ff77cf docs: sentence-case and split README paragraphs
Quick Install was the one Title Case heading left in the published
text. The Files sentence was glued to the paragraph above it, two
paragraphs before the table whose column it describes. It now names
the column and sits just above the table.
2026-10-21 05:24:25 +02:00
Abdessamad Derraz 8eda797562 fix: a hashless entry is a path in any mode
Whether the frontend compares content was decided per platform. EmuDeck
(md5) declares neocdz.zip with no hash and checkBIOS.sh never names it,
yet the proven claims of geolith and FBNeo were dropped and the pack
shipped a neocdz.zip without 000-lo.lo. checked_by_path() decides per
entry: no hash of the kind the frontend compares (any of md5, sha1,
crc32 under RomM's rule) and no zipped_file means a path. Measured: the
EmuDeck neocdz.zip slot and three RetroBat DS slots (dsi_bios7.bin,
dsi_bios9.bin, firmware.bin) now take the file the cores prove.
2026-10-21 05:24:19 +02:00
Abdessamad Derraz b464ed9202 fix: count prose that holds the pin as review work
The triage, --changed-only and --strict counted refs only. Prose that
--realign-prose would move or refuse blocks every recale and bump, and
blastem read 0 to review. Each report now carries that reading. The
triage also gets a header, a closing count, and one sampling notice per
run instead of one per profile.
2026-10-21 05:24:17 +02:00
Abdessamad Derraz 905316061e fix: read retropie and emudeck targets at the pin
The RetroPie target scraper read master and the EmuDeck one the default
branch, while both platform files are pinned on a commit: a module flag
changed in between filtered packages the pinned list does not describe.
Both now read the revision pinned_revision returns, which also accepts the
codeload URL the RetroPie scraper records. EmuDeck Windows stays on the
emudeck-we default branch: the platform file pins no revision of it.
2026-10-21 05:24:04 +02:00