Commit Graph
1294 Commits
Author SHA1 Message Date
Abdessamad Derraz 585be38ee3 fix: transform each held content once
Step 1b read 18 924 held files and members for 7 407 distinct contents and ran every transform on each read. A content already transformed is now skipped (the first path still keeps the match), the power-of-two crc shift operators are squared once for the pass instead of once per length, and a group pairs members by the sizes it holds. derivable.json and the step output are byte-identical, 33.4 s to 25.1 s.
2026-10-21 09:56:53 +02:00
Abdessamad Derraz ab16555014 fix: give the bizhawk DSi NAND slots their region
DecideNAND (MelonDS.cs:684-706) keeps, for a DSi title, only the NAND
slots whose bit is set in header byte 0x1B0. The six DSi_Nand entries
now carry that territory, so --region keeps one 240 MB image, not six.
AUS answers oceania, as every other Australian slot does.
2026-10-21 07:00:48 +02:00
Abdessamad Derraz bc2233e725 fix: write truth systems in a stable order
Systems a core covers without a file were added while walking a set,
so the hash seed placed them and two runs wrote dist/truth/<platform>.yml
in two orders. The normalized system map was built from a set too.
2026-10-21 06:57:49 +02:00
Abdessamad Derraz bbd778b390 fix: file nicaiemu and coolbaremu as one system
The buildbot serves nicaiemu_libretro (frozen at 4c9b73f7, last built
2026-10-16 05:15) beside coolbaremu_libretro. They are two builds: each
reads its own orientation list from the system directory, so both
profiles stay, each with its own core name. Both read the same .cbe
archives from the same workspace, which builds a standalone binary at
both revisions: one system id and one build type for the two.
2026-10-21 06:55:42 +02:00
Abdessamad Derraz 2f5b4bd446 chore: resolve the merge markers left in a test 2026-10-21 06:50:36 +02:00
Abdessamad Derraz 6c2204a7a8 chore: merge tooling follow-up
# Conflicts:
#	tests/test_audit_regressions.py
2026-10-21 06:50:20 +02:00
Abdessamad Derraz 768b42e807 fix: fold case in the site gap report too 2026-10-21 06:50:10 +02:00
Abdessamad Derraz 378a9e4da5 chore: merge audit fixes (integrity)
# Conflicts:
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/retropie.json
#	install/rocknix.json
#	install/romm.json
#	scripts/generate_pack.py
#	wiki/tools.md
2026-10-21 06:48:42 +02:00
Abdessamad Derraz 4ba935374e fix: keep pack order outside contested paths
Ranking contested core claims sorted the whole gap report, which
reordered every manifest. Only the claims on one contested path move now,
grouped where the first of them stood, and two spellings of a path are
one group where the filesystem folds case (RetroBat's 64DD_IPL_DEV.bin).

An entry without a destination (EmuDeck's system:md5 whitelist) no
longer counts as a pack path in verify, and step 5 prints one status for
counts and paths together. A full offline run compared all 12 archives
and 930 emulator packs path by path, every one equal.
2026-10-21 06:47:44 +02:00
Abdessamad Derraz 85a44d9bf7 chore: merge amiberry and hatari re-profile
# Conflicts:
#	README.md
#	emulators/amiberry.yml
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/rocknix.json
2026-10-21 06:21:30 +02:00
Abdessamad Derraz 76e44a74b4 refactor: drop comments that restate the code
Thirty one-line labels in verify.py, generate_site.py and
generate_truth.py named the statement under them ("# Load registry",
"# Copy branding assets"). Comments that give a reason stay, as do the
module section headers. A test reads the three files for new ones.
2026-10-21 06:20:50 +02:00
Abdessamad Derraz 3613c0927b docs: open the download page on its content
The which-pack page announced itself ("This page picks the right pack
for a setup") before naming a pack. The sentence goes, the pointer to
Getting started stays.
2026-10-21 06:18:59 +02:00
Abdessamad Derraz 566d36275f fix: count alias readers in a file's requirers
Ymir reads sega_101.bin or saturn_bios.bin, so the sega_101.bin entry
carries saturn_bios.bin as an alias. kronos, yabause and yabasanshiro
open saturn_bios.bin and fall back to sega_101.bin: they were in that
entry's _cores but not its _required_by, while their saturn_bios.bin
entry had them requiring it. A reader that requires one of an entry's
aliases now requires the entry. No reader is added, and same-name
entries kept apart by path are left alone. No export changes.
2026-10-21 06:18:32 +02:00
Abdessamad Derraz 5ce7ab8386 fix: self-check a ref in every repository
A prose citation of the port's own lines was read only in the upstream file that a source_ref attributes the path to, and a range past that file's end came back GONE although the port carries it.
2026-10-21 06:18:00 +02:00
Abdessamad Derraz 58dc7f70b4 chore: merge profile data follow-up 2026-10-21 06:13:25 +02:00
Abdessamad Derraz f19a8f99d3 fix: read composite zip md5s apart in diff_truth
Recalbox hashes a .zip by its members. diff_truth set that composite
against the profile's container md5 and reported neocdz.zip and
neocd.zip as mismatches. _hash_mismatch now drops it through
comparable(), the rule pair_entries and the export already apply.
2026-10-21 06:12:08 +02:00
Abdessamad Derraz b8baf92f1d fix: give atomic writes the umask mode
mkstemp creates its scratch 0600 and os.replace kept that mode, so
database.json, the manifests, the packs, the recipe snapshots and the
large-file cache were readable by their owner only. set_plain_mode
gives the scratch 0o666 less the umask, as open() would, before each
rename. A test walks every function that renames a mkstemp file.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 66937d21d7 docs: install the site packages for contributors
The contributor install line left out mkdocs-material and
pymdown-extensions, so pipeline step 8b (render site) was skipped.
The ranges are the ones deploy-site.yml installs, and a test reads
them from the workflow.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 57ac252482 feat: check the profiles a branch adds or edits
profile_sync gains --changed (added or edited profiles, uncommitted
ones included) and --added, both against the merge base of --base
(origin/main by default). CONTRIBUTING runs --changed --strict instead
of one --emulator call per new profile, and the CI job runs --added
in place of its own diff loop.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 1b3763e58f feat: read the voidpoint.io gitlab
eduke32 and fury declare https://voidpoint.io/terminx/eduke32 and were
skipped as an unsupported host. The instance serves the GitLab v4 API
(/api/v4/projects/terminx%2Feduke32 answers 200) and /-/raw/ paths.
2026-10-21 06:09:33 +02:00
Abdessamad Derraz b784a73d5c fix: name a repository the forge renamed
GitHub, GitLab and Gitea redirect a renamed repository, and every call
follows the redirect: AloysHF/NicaiEmu answers as AloysHF/CoolbarEmu
and the profile read as current. profile_sync now reads the name the
forge answers under and counts a move as review work.
2026-10-21 06:09:31 +02:00
Abdessamad Derraz d6fc66274f fix: let check_freshness read the profile report
check_freshness ran profile_sync with --json and the detection flags,
which profile_sync refuses, so --profiles always reported an unreadable
report. The version check now lives on the report and travels in the
JSON and markdown outputs; check_freshness asks for --json
--check-version only.
2026-10-21 06:09:15 +02:00
Abdessamad Derraz 6b4010907d feat: list the six BizHawk DSi NAND slots
FirmwareDatabase.cs:405-410 declares NDS+NAND_JPN, EUR, USA, AUS, CHN
and KOR, each with a placeholder hash and 251658240 + 64 bytes. In DSi
mode DecideNAND takes the first one set, limited to the regions a
DSi-enhanced header allows, and stops the load with none
(MelonDS.cs:114, 304, 484, 684-706). DSi mode is off by default, so the
entries are optional, like DSi_Firmware.bin and the DSi BIOS slots.
2026-10-21 06:07:30 +02:00
Abdessamad Derraz 7056dd84ea fix: state why WinUAE needs the CD extended ROMs
A missing extended ROM only raises a notice (memory.cpp:1833-1849), but
the machine then runs on the Kickstart alone. cdstrap, cd.device,
cdtv.device and cdfs.library are resident modules of the CD32 and CDTV
extended ROMs and absent from their Kickstarts, the CD32 and base CDTV
models have no floppy drive (cfgfile.cpp:9375-9379, 9405-9406), so no CD
is read or booted. Both entries stay required. The AROS replacement
stands in only for a missing Kickstart (memory.cpp:3055-3063), so the
CDTV entry loses hle_fallback.
2026-10-21 06:06:35 +02:00
Abdessamad Derraz d678e6acfa fix: declare the FreeIntv overlays on the core
FreeIntv declared no data tree while its alias freeintv_ts_overlay,
the same code, declared freeintv_overlays. The core reads
freeintv_overlays/<rom>.png only with the multi-screen option and draws
its embedded keypad image without one (src/libretro.c:278-280, 294,
321-325), so the tree is optional. A test holds every alias to the trees
of its core.
2026-10-21 06:06:07 +02:00
Abdessamad Derraz 1a41e6a964 chore: merge psx region follow-up 2026-10-21 06:05:04 +02:00
Abdessamad Derraz 47d9e1d629 fix: tag US PlayStation images latin-america too
Each core picks its US image for a disc licensed by Sony Computer
Entertainment America, and DuckStation, SwanStation and Beetle PSX
also for a disc whose region they cannot read. Tagged north-america
alone, a --region brazil,jp pack withdrew those images.
2026-10-21 06:04:42 +02:00
Abdessamad Derraz 35b4cacb5d chore: merge audit fixes (installer)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 06:02:02 +02:00
Abdessamad Derraz 1c4686ab93 feat: place RomM firmware by its own template
RomM 5.4 reads each platform's firmware where filesystem.structure.firmware
in config.yml says, bios/{platform} by default, and suggests {platform}/bios
for a {platform}/roms library (rommapp/romm 5.4.0,
backend/config/config_manager.py:58,101-109,205-225,564-568,1105-1124).
The installer only wrote bios/<platform>/. It now reads the template from
config.yml beside the library, takes --romm-firmware when that file is
elsewhere, refuses a platform-first library with no template, and detects
RomM by /romm/config/config.yml too.
2026-10-21 05:53:23 +02:00
Abdessamad Derraz 734643b8fc chore: merge audit fixes (site)
# Conflicts:
#	wiki/testing-guide.md
2026-10-21 05:53:15 +02:00
Abdessamad Derraz a1a9da59af chore: merge audit fixes (export)
# Conflicts:
#	exports/recalbox.json
#	exports/retropie.json
2026-10-21 05:53:05 +02:00
Abdessamad Derraz ce10323f7c chore: align two tests with the merged lock flow
The lock refusal moved to stderr and pipeline labels lost their /8 in
two branches that met at merge.
2026-10-21 05:53:01 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00
Abdessamad Derraz 53da2a95d2 chore: merge audit fixes (docs)
# Conflicts:
#	tests/test_doc_surfaces.py
#	wiki/installer.md
#	wiki/testing-guide.md
2026-10-21 05:50:25 +02:00
Abdessamad Derraz 43c6df58ee chore: merge audit fixes (region)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 05:48:59 +02:00
Abdessamad Derraz a67265e903 fix: let the site search land on cards
The search splits a page at its headings. Emulator cards were titled in
bold and platform systems folded into blocks under one heading, so a
pasted hash opened the top of a page or the systems overview of a
100-block platform page. Each file card is now a level 4 heading kept
out of the table of contents, and each platform system a heading.

Card lines that list other pages (platforms, packs, source citations,
collection paths) leave the search index: 24.7 MB down to 18.1 MB, the
names, hashes and notes still searchable.
2026-10-21 05:47:31 +02:00
Abdessamad Derraz f4a225488d chore: merge audit fixes (scrapers) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz eb44e3f898 chore: merge audit fixes (profiles) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 73d73e0f2a chore: merge audit fixes (profsync) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz e03fe338ee chore: merge audit fixes (tests) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 89c533e9ba fix: detect Batocera and Knulli by build files
No Batocera build writes /etc/batocera-version: batocera-system.mk:21-22
(batocera.linux 38845725, unchanged since batocera-30) writes
/usr/share/batocera/batocera.version. Knulli (knulli-linux 57afa662) adds
OS_NAME=knulli to os-release, writes /usr/share/knulli/knulli.version and
removes /usr/share/batocera, and no Knulli source writes /etc/knulli-release.
Neither was detected. A Knulli board is named and not narrowed by
Batocera's per-board lists, and an unlisted board is named.
2026-10-21 05:44:27 +02:00
Abdessamad Derraz 5b7f8526a8 refactor: read the gap report through its helper
test_launcher_layer looked for find_undeclared_files in the extras
collector, which now reaches it through undeclared_report.
2026-10-21 05:43:14 +02:00
Abdessamad Derraz aa36ec544d fix: name every gap verify counts
verify --platform printed N optional core files and N game data files
missing and named none, even with --verbose. --verbose now lists them as
MISSING (optional) and MISSING (game data), and without it the count line
says --verbose lists them, as the line about cores with no external file
already did.
2026-10-21 05:40:11 +02:00
Abdessamad Derraz 28435230a7 fix: read each named core's own entry in recalbox
An element's mandatory flag came from the one truth entry its md5 paired
with. saturn/saturn_bios.bin names yabause, yabasanshiro and kronos,
which require it, and was paired with the beetle_saturn entry, which
does not. Each named core's entries for the file, found by name, now
decide, except a namesake elsewhere in the same tree (bluemsx KANJI.rom
under Shared Roms is not the one under MSX2+ - European).
2026-10-21 05:39:17 +02:00
Abdessamad Derraz 82a3e44afd fix: settle a slot in any case where case folds
declaration_settles tested the exact-case name before the slot, so on a
filesystem that folds case a declaration spelled otherwise never settled
the core's file: RetroBat's 32X_G_BIOS.BIN left BlastEm's 32X_G_BIOS.bin
undeclared, and verify counted ten such entries as core files. A slot a
case-insensitive platform fills now settles the entry whatever the
spelling (case_folded_slots), the slot arbitration judging its bytes as
for the exact name. Lakka and RetroPie, which keep both names, are
unchanged. Measured: 10 entries settle on RetroBat, 21 on RetroArch.

generate_site.py builds its gap report through cross_reference() without
the folded slots; it needs folded_declared_slots() passed in the same way
as the command line now does.
2026-10-21 05:38:11 +02:00
Abdessamad Derraz edda080c59 fix: read muOS RetroArch from its own config
muOS runs RetroArch on a config copied from its default, which reads
/run/muos/storage/bios, bound to MUOS/bios on the second card when that
card holds one, and loads cores from /opt/muos/share/core (MustardOS/internal
f41f4a89, script/control/retroarch.sh:8-20, script/device/bind.sh:116-150,
retroarch.default.cfg:2806,3151). The installer wrote to /mnt/mmc/MUOS/bios
and --installed-cores found no core. The config now decides both.
2026-10-21 05:35:45 +02:00
Abdessamad Derraz 184c822e6e fix: name the file each layer reads in slots.py
Each content-checked line ended with a verdict on the platform's list,
printed at pipeline step 3b. The line now says which emulators read the
other file, and the closing count says the rest follow the platform's
list. The module source carries no such verdict, a test reads it.
2026-10-21 05:33:50 +02:00
Abdessamad Derraz bdcc8f0fb6 fix: lower the pipeline floor to Python 3.9
The 3.12 guard rested on generate_readme f-strings that have since been
rewritten. Every script compiles under CPython 3.9.25, 3.10 and 3.11, and
only generate_pack's parenthesized context managers fail under 3.8. Under
3.9 the offline pipeline (--skip-packs) completed with no change to the
committed artifacts, the 15 manifests matched 3.12 byte for byte, and the
RomM pack was identical. CI now compiles every script under a real 3.9.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz ceb8b752e6 fix: count bundled gaps apart in each heading
cross_reference.py --emulator abuse printed "Gaps: 3 missing" above three
[BUNDLED] files, then "0 missing (need to be sourced)" in the total. The
heading and the total now read one count: bundled files appear as
"N bundled", never among the missing.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz c05d29d941 fix: a held name must carry the declared hash
entry_source took a name or path hit before reading the hashes the entry
declares, so the gap report and the site counted as held a file whose
only homonym has other bytes: zquestclassic's ending.mid and gameover.mid
matched Hydra Castle Labyrinth's. A name or path hit now counts only
when its record carries one of the declared hashes, and the declared
content anywhere in bios/ comes before a data/ copy. Measured over every
profile entry: 254 change, 248 bundled ones (report and site only),
three unsourceable SC-55 revisions now read missing, three data hits
read bios by hash.
2026-10-21 05:32:19 +02:00