Commit Graph
2546 Commits
Author SHA1 Message Date
Abdessamad Derraz 648db6bcbf fix: merge mame devices when the pin moves
Step 2a advanced mame.yml's core_version and upstream_commit to a new
release and left the device ROM sets on the previous one. A plain run
that moves the pin now fetches that release's listxml into .cache/dats
and runs the device merge, which checks version and commit itself.
2026-10-21 10:06:13 +02:00
Abdessamad Derraz a74a56abc4 feat: name the profile a missing dump would fill
The acquisition report never read a profile, so XM7's required DICROM.ROM
sat among 2 287 targets at the rank of dumps no core reads. A missing
entry that a profile reads and the collection lacks is now listed first
with the profile, matched by hash or, for a hashless entry with a size,
by name and size. 29 entries on the current snapshots.
2026-10-21 10:05:10 +02:00
Abdessamad Derraz d16f8fd17a fix: accept downloads by cryptographic hash only
A crc32 and a size can be met by crafted bytes, so a remote download
matched on them could land in bios/. verify_content now requires every
sha256, sha1 and md5 the entry declares to match, and an entry declaring
none is reported as not verifiable from the network and nothing is
written. crc32 and size still find a file the database already holds.
2026-10-21 10:04:20 +02:00
Abdessamad Derraz 6c076884ed fix: let auto_fetch prove sha256 and crc32 entries
verify_content read sha1 then md5 only, so an entry pinned by sha256 or
crc32 could be searched and never accepted, and one without any hash was
announced as searched. The strongest declared hash now decides, crc32
counts with its size, the database lookup reads sha256 and crc32, and an
entry with nothing to prove a download by is reported, not searched.
2026-10-21 10:02:05 +02:00
Abdessamad Derraz f3586001ab fix: fail freshness on unknown, name the refresher
check_freshness exited 0 when no upstream answered. An unknown row now
holds the exit code, and a profile skipped by nature is reported skipped
so it does not hold it forever. --only names the valid areas, a stale
native row names its platform, and the summary names refresh_stale.py.
2026-10-21 10:00:36 +02:00
Abdessamad Derraz cc57d44de5 fix: order inherited rescrapes, end on verify
refresh_stale ran a parent's rescrape and its children's truth and
export jobs side by side, so a child read the parent mid-rewrite. A job
now starts once the rescrapes of its ancestors have ended, and a child
no row names gets its export rewritten after them. A rescrape chain ends
on verify.py --platform, whose summary closes the report.
2026-10-21 09:58:58 +02:00
Abdessamad Derraz 9bb1a73f91 fix: hold every data tree, present or not
export_native and cross_reference read data/ without the shared hold, so
a refresh could swap a tree mid-walk. hold_data_dirs skipped a cache not
yet on disk, so a refresh could install it between verify and the packs
of one pipeline run. Both now hold every registered tree.
2026-10-21 09:56:49 +02:00
Abdessamad Derraz 563ce1ebf2 fix: read fbneo dats at one checked commit
The FBNeo import recorded the listing's blob sha for bytes fetched from a
branch URL a CDN caches, and --fetch <revision> still read the tip. The
revision is now resolved to a commit, listed and downloaded there, each
DAT is held against its blob sha, and a DAT read at a named revision is
labelled with it without replacing the tip's recorded blobs.
2026-10-21 09:55:50 +02:00
Abdessamad Derraz 12cc71b634 fix: hold the pipeline's fixed outputs for a run
The pipeline locked only --output-dir while it also writes database.json,
install/, README.md and docs/. A run on a scratch output directory could
start beside the run on dist/ and rewrite them. Every run now holds the
repository's .lock first.
2026-10-21 09:53:15 +02:00
Abdessamad Derraz 43337c6719 fix: match release packs by exact name
release_record rebuilt nothing from the name builder: a substring match
took RetroArch_v1.22.2_AndroidArm64V8A_BIOS_Pack.zip for the RetroArch
pack. The record now refuses any pack that is not a registered platform's
full pack and stores the platforms each pack serves, which the readers
use instead of the pack name.
2026-10-21 09:51:55 +02:00
Abdessamad Derraz 0242dc77ee chore: regenerate manifests, readme and site nav 2026-10-21 09:01:49 +02:00
Abdessamad Derraz 2f327fbd37 chore: drop two hiscore.dat copies cores install
groovymame, mame4droid-2024 and raine install their hiscore.dat with the
emulator; each profile records the sha1 of its copy.
2026-10-21 07:32:34 +02:00
Abdessamad Derraz ab16555014 fix: give the bizhawk DSi NAND slots their region
DecideNAND (MelonDS.cs:684-706) keeps, for a DSi title, only the NAND
slots whose bit is set in header byte 0x1B0. The six DSi_Nand entries
now carry that territory, so --region keeps one 240 MB image, not six.
AUS answers oceania, as every other Australian slot does.
2026-10-21 07:00:48 +02:00
Abdessamad Derraz bc2233e725 fix: write truth systems in a stable order
Systems a core covers without a file were added while walking a set,
so the hash seed placed them and two runs wrote dist/truth/<platform>.yml
in two orders. The normalized system map was built from a set too.
2026-10-21 06:57:49 +02:00
Abdessamad Derraz bbd778b390 fix: file nicaiemu and coolbaremu as one system
The buildbot serves nicaiemu_libretro (frozen at 4c9b73f7, last built
2026-10-16 05:15) beside coolbaremu_libretro. They are two builds: each
reads its own orientation list from the system directory, so both
profiles stay, each with its own core name. Both read the same .cbe
archives from the same workspace, which builds a standalone binary at
both revisions: one system id and one build type for the two.
2026-10-21 06:55:42 +02:00
Abdessamad Derraz 2f5b4bd446 chore: resolve the merge markers left in a test 2026-10-21 06:50:36 +02:00
Abdessamad Derraz 6c2204a7a8 chore: merge tooling follow-up
# Conflicts:
#	tests/test_audit_regressions.py
2026-10-21 06:50:20 +02:00
Abdessamad Derraz 768b42e807 fix: fold case in the site gap report too 2026-10-21 06:50:10 +02:00
Abdessamad Derraz 91a9d2279e chore: drop the dolphin-mmjr codehandler copy
Only dolphin-mmjr's bundled entry reaches it, and that entry records its
sha1 and size: the APK installs the file with the emulator.
2026-10-21 06:50:10 +02:00
Abdessamad Derraz 378a9e4da5 chore: merge audit fixes (integrity)
# Conflicts:
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/retropie.json
#	install/rocknix.json
#	install/romm.json
#	scripts/generate_pack.py
#	wiki/tools.md
2026-10-21 06:48:42 +02:00
Abdessamad Derraz 88f9c40a78 chore: regenerate the install manifests
The slot and claim decisions above change which copy some paths take
(EmuDeck neocdz.zip, RetroBat DS firmware, hiscore.dat on six platforms)
and settle case variants on folding filesystems.
2026-10-21 06:47:50 +02:00
Abdessamad Derraz 4ba935374e fix: keep pack order outside contested paths
Ranking contested core claims sorted the whole gap report, which
reordered every manifest. Only the claims on one contested path move now,
grouped where the first of them stood, and two spellings of a path are
one group where the filesystem folds case (RetroBat's 64DD_IPL_DEV.bin).

An entry without a destination (EmuDeck's system:md5 whitelist) no
longer counts as a pack path in verify, and step 5 prints one status for
counts and paths together. A full offline run compared all 12 archives
and 930 emulator packs path by path, every one equal.
2026-10-21 06:47:44 +02:00
Abdessamad Derraz 85a44d9bf7 chore: merge amiberry and hatari re-profile
# Conflicts:
#	README.md
#	emulators/amiberry.yml
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/rocknix.json
2026-10-21 06:21:30 +02:00
Abdessamad Derraz 76e44a74b4 refactor: drop comments that restate the code
Thirty one-line labels in verify.py, generate_site.py and
generate_truth.py named the statement under them ("# Load registry",
"# Copy branding assets"). Comments that give a reason stay, as do the
module section headers. A test reads the three files for new ones.
2026-10-21 06:20:50 +02:00
Abdessamad Derraz 3613c0927b docs: open the download page on its content
The which-pack page announced itself ("This page picks the right pack
for a setup") before naming a pack. The sentence goes, the pointer to
Getting started stays.
2026-10-21 06:18:59 +02:00
Abdessamad Derraz 566d36275f fix: count alias readers in a file's requirers
Ymir reads sega_101.bin or saturn_bios.bin, so the sega_101.bin entry
carries saturn_bios.bin as an alias. kronos, yabause and yabasanshiro
open saturn_bios.bin and fall back to sega_101.bin: they were in that
entry's _cores but not its _required_by, while their saturn_bios.bin
entry had them requiring it. A reader that requires one of an entry's
aliases now requires the entry. No reader is added, and same-name
entries kept apart by path are left alone. No export changes.
2026-10-21 06:18:32 +02:00
Abdessamad Derraz 4648ed0de3 docs: regenerate readme coverage 2026-10-21 06:18:13 +02:00
Abdessamad Derraz e1a4f19737 feat: re-profile amiberry at 8.4.2
Pin moves to 2de3bf8 (v8.4.2). roms/aros-ext.bin, roms/aros-rom.bin and whdboot/game-data/whdload_db.json changed upstream and are collected at that revision. The WHDLoad database is matched on the archive SHA1 first and refreshed from the bundled copy when newer. kick02019.AROS and its .ext are names the Recalbox launcher writes, so the profile now documents the kickstart_rom_file settings instead.
2026-10-21 06:18:10 +02:00
Abdessamad Derraz 27ab0c3872 feat: re-profile hatari at libretro 5092994
Pins move to libretro/hatari 5092994 and hatari/hatari 90949cf. The port now reads floppy.raw from the system directory (floppy_sound.c:146-186), documented as an optional user sample. Upstream decodes IPF without capsimage, the icon block moved by four lines, and the launcher file names leave the tos.img note for the platform layer.
2026-10-21 06:18:04 +02:00
Abdessamad Derraz 5ce7ab8386 fix: self-check a ref in every repository
A prose citation of the port's own lines was read only in the upstream file that a source_ref attributes the path to, and a range past that file's end came back GONE although the port carries it.
2026-10-21 06:18:00 +02:00
Abdessamad Derraz 58dc7f70b4 chore: merge profile data follow-up 2026-10-21 06:13:25 +02:00
Abdessamad Derraz 67f1f05ff2 docs: state what three cores do without data trees
The Dolphin Sys, PPSSPP assets and blueMSX trees stay required, and the
profiles now cite why. blueMSX finds no machine without Machines/, the
C-BIOS ones included, and the load fails (Src/Board/Machine.c:637-656,
libretro.c:1447-1449). Its tree reference cited libretro.c:1119-1120,
which at the pinned revision is the Sunrise IDE option, and now cites
1196-1197. PPSSPP has no internal font without flash0/font, so
sceFontOpen fails for games using the system fonts
(Core/HLE/sceFont.cpp:805-864, 984-986). Dolphin loads no GameCube font
without Sys/GC/ or an IPL dump and warns that games may crash
(Source/Core/Core/HW/EXI/EXI_DeviceIPL.cpp:328-340).
2026-10-21 06:12:41 +02:00
Abdessamad Derraz f19a8f99d3 fix: read composite zip md5s apart in diff_truth
Recalbox hashes a .zip by its members. diff_truth set that composite
against the profile's container md5 and reported neocdz.zip and
neocd.zip as mismatches. _hash_mismatch now drops it through
comparable(), the rule pair_entries and the export already apply.
2026-10-21 06:12:08 +02:00
Abdessamad Derraz b8baf92f1d fix: give atomic writes the umask mode
mkstemp creates its scratch 0600 and os.replace kept that mode, so
database.json, the manifests, the packs, the recipe snapshots and the
large-file cache were readable by their owner only. set_plain_mode
gives the scratch 0o666 less the umask, as open() would, before each
rename. A test walks every function that renames a mkstemp file.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 66937d21d7 docs: install the site packages for contributors
The contributor install line left out mkdocs-material and
pymdown-extensions, so pipeline step 8b (render site) was skipped.
The ranges are the ones deploy-site.yml installs, and a test reads
them from the workflow.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 57ac252482 feat: check the profiles a branch adds or edits
profile_sync gains --changed (added or edited profiles, uncommitted
ones included) and --added, both against the merge base of --base
(origin/main by default). CONTRIBUTING runs --changed --strict instead
of one --emulator call per new profile, and the CI job runs --added
in place of its own diff loop.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 1b3763e58f feat: read the voidpoint.io gitlab
eduke32 and fury declare https://voidpoint.io/terminx/eduke32 and were
skipped as an unsupported host. The instance serves the GitLab v4 API
(/api/v4/projects/terminx%2Feduke32 answers 200) and /-/raw/ paths.
2026-10-21 06:09:33 +02:00
Abdessamad Derraz b784a73d5c fix: name a repository the forge renamed
GitHub, GitLab and Gitea redirect a renamed repository, and every call
follows the redirect: AloysHF/NicaiEmu answers as AloysHF/CoolbarEmu
and the profile read as current. profile_sync now reads the name the
forge answers under and counts a move as review work.
2026-10-21 06:09:31 +02:00
Abdessamad Derraz d6fc66274f fix: let check_freshness read the profile report
check_freshness ran profile_sync with --json and the detection flags,
which profile_sync refuses, so --profiles always reported an unreadable
report. The version check now lives on the report and travels in the
JSON and markdown outputs; check_freshness asks for --json
--check-version only.
2026-10-21 06:09:15 +02:00
Abdessamad Derraz 6b4010907d feat: list the six BizHawk DSi NAND slots
FirmwareDatabase.cs:405-410 declares NDS+NAND_JPN, EUR, USA, AUS, CHN
and KOR, each with a placeholder hash and 251658240 + 64 bytes. In DSi
mode DecideNAND takes the first one set, limited to the regions a
DSi-enhanced header allows, and stops the load with none
(MelonDS.cs:114, 304, 484, 684-706). DSi mode is off by default, so the
entries are optional, like DSi_Firmware.bin and the DSi BIOS slots.
2026-10-21 06:07:30 +02:00
Abdessamad Derraz 7056dd84ea fix: state why WinUAE needs the CD extended ROMs
A missing extended ROM only raises a notice (memory.cpp:1833-1849), but
the machine then runs on the Kickstart alone. cdstrap, cd.device,
cdtv.device and cdfs.library are resident modules of the CD32 and CDTV
extended ROMs and absent from their Kickstarts, the CD32 and base CDTV
models have no floppy drive (cfgfile.cpp:9375-9379, 9405-9406), so no CD
is read or booted. Both entries stay required. The AROS replacement
stands in only for a missing Kickstart (memory.cpp:3055-3063), so the
CDTV entry loses hle_fallback.
2026-10-21 06:06:35 +02:00
Abdessamad Derraz d678e6acfa fix: declare the FreeIntv overlays on the core
FreeIntv declared no data tree while its alias freeintv_ts_overlay,
the same code, declared freeintv_overlays. The core reads
freeintv_overlays/<rom>.png only with the multi-screen option and draws
its embedded keypad image without one (src/libretro.c:278-280, 294,
321-325), so the tree is optional. A test holds every alias to the trees
of its core.
2026-10-21 06:06:07 +02:00
Abdessamad Derraz 1a41e6a964 chore: merge psx region follow-up 2026-10-21 06:05:04 +02:00
Abdessamad Derraz 47d9e1d629 fix: tag US PlayStation images latin-america too
Each core picks its US image for a disc licensed by Sony Computer
Entertainment America, and DuckStation, SwanStation and Beetle PSX
also for a disc whose region they cannot read. Tagged north-america
alone, a --region brazil,jp pack withdrew those images.
2026-10-21 06:04:42 +02:00
Abdessamad Derraz 35b4cacb5d chore: merge audit fixes (installer)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 06:02:02 +02:00
Abdessamad Derraz 1c4686ab93 feat: place RomM firmware by its own template
RomM 5.4 reads each platform's firmware where filesystem.structure.firmware
in config.yml says, bios/{platform} by default, and suggests {platform}/bios
for a {platform}/roms library (rommapp/romm 5.4.0,
backend/config/config_manager.py:58,101-109,205-225,564-568,1105-1124).
The installer only wrote bios/<platform>/. It now reads the template from
config.yml beside the library, takes --romm-firmware when that file is
elsewhere, refuses a platform-first library with no template, and detects
RomM by /romm/config/config.yml too.
2026-10-21 05:53:23 +02:00
Abdessamad Derraz 734643b8fc chore: merge audit fixes (site)
# Conflicts:
#	wiki/testing-guide.md
2026-10-21 05:53:15 +02:00
Abdessamad Derraz a1a9da59af chore: merge audit fixes (export)
# Conflicts:
#	exports/recalbox.json
#	exports/retropie.json
2026-10-21 05:53:05 +02:00
Abdessamad Derraz ce10323f7c chore: align two tests with the merged lock flow
The lock refusal moved to stderr and pipeline labels lost their /8 in
two branches that met at merge.
2026-10-21 05:53:01 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00