Commit Graph
2818 Commits
Author SHA1 Message Date
Abdessamad Derraz 287a16f41c fix: name --fetch when an original is not cached
The export failed with 'absent and fetching is off'. It now names the cache directory it looked in and the flag that downloads the file.
2026-10-21 09:57:44 +02:00
Abdessamad Derraz 83a4b21d30 fix: keep platform hashes cores disagree on
When two cores declare different values for one file, the truth keeps the first and marks the entry hash_conflict. Its hashes pair an entry and never replace the platform's: armsx2's GameIndex.yaml no longer overwrites System.dat's line for LRPS2.
2026-10-21 09:57:20 +02:00
Abdessamad Derraz f997bdc81a fix: check a recalbox zip by its composite only
Bios::Scan takes Zip::Md5Composite for every .zip (Bios.cpp:64-68). The resolver already honoured zip_digest: composite, but digest_verdict accepted the container's own md5 first, so the pack check passed bytes Recalbox marks red. The page described that order.
2026-10-21 09:57:06 +02:00
Abdessamad Derraz 585be38ee3 fix: transform each held content once
Step 1b read 18 924 held files and members for 7 407 distinct contents and ran every transform on each read. A content already transformed is now skipped (the first path still keeps the match), the power-of-two crc shift operators are squared once for the pass instead of once per length, and a group pairs members by the sizes it holds. derivable.json and the step output are byte-identical, 33.4 s to 25.1 s.
2026-10-21 09:56:53 +02:00
Abdessamad Derraz 9bb1a73f91 fix: hold every data tree, present or not
export_native and cross_reference read data/ without the shared hold, so
a refresh could swap a tree mid-walk. hold_data_dirs skipped a cache not
yet on disk, so a refresh could install it between verify and the packs
of one pipeline run. Both now hold every registered tree.
2026-10-21 09:56:49 +02:00
Abdessamad Derraz 4fec09a014 fix: name the selection mode in --from-md5 packs
BizHawk is both a platform and a profile display name: --from-md5 with --platform bizhawk and with --emulator bizhawk wrote one file in two layouts. The emulator mode now carries _Emulator in the name.
2026-10-21 09:56:45 +02:00
Abdessamad Derraz c3fdfba412 fix: profile flycast from flyinghead/flycast
libretro/flycast now answers as libretro/parallel-dc, a separate core.
The buildbot builds flycast from flyinghead/flycast master
(libretro-super recipes, cores-linux-x64-generic), so the profile is
read again at 34ad8c82 and pinned there.

At that revision the Dreamcast boot ROM is dc_boot.bin or dc_bios.bin,
the flash is dc_nvmem.bin, arcade platforms read no flat boot ROM, the
system font is compiled in, and NAOMI 2 and System SP load naomi2.zip
and segasp.zip. Archive members carry the CRC the loader looks up.
2026-10-21 09:56:25 +02:00
Abdessamad Derraz 4ceca6acc5 fix: credit platforms with the file the pack ships
FileConsumers resolved platform declarations with their destination as
a hint, which the builder, the manifest and verify never pass. RetroBat's
63 mame/hash lists and the curated gc-ntsc-12.bin credited another copy
than the one the manifests install.
2026-10-21 09:56:08 +02:00
Abdessamad Derraz 2af9509ef4 fix: resolve declarations through one function
verify resolved platform entries with common.resolve_local_file and put
the slot override first, and the region pass added a destination hint.
packresolve.resolve_declaration now serves the builder, the manifest,
verify and the region pass: release assets cached for a file absent from
bios/ count as shipped in all four, and a release asset keeps its answer
over a slot override.
2026-10-21 09:56:07 +02:00
Abdessamad Derraz a3e625cd6a fix: refuse a default value typed out as a flag
The refusal tables compared each flag with its default value, so --source full and --group-by system passed beside the modes that refuse every other value. Both now default to None and take their effective value after the checks, in generate_pack and pipeline.
2026-10-21 09:55:51 +02:00
Abdessamad Derraz ed610c7bf3 fix: hold the consumer guards on fixtures
The Dolphin-MMJR codehandler.bin and armsx2's game_controller_db.txt left
the collection, and the two guards that read them skipped on every run.
Both now build the trees they need, and the guards still reading the
repository fail instead of skipping when their data is gone.
2026-10-21 09:55:50 +02:00
Abdessamad Derraz 563ce1ebf2 fix: read fbneo dats at one checked commit
The FBNeo import recorded the listing's blob sha for bytes fetched from a
branch URL a CDN caches, and --fetch <revision> still read the tip. The
revision is now resolved to a commit, listed and downloaded there, each
DAT is held against its blob sha, and a DAT read at a named revision is
labelled with it without replacing the tip's recorded blobs.
2026-10-21 09:55:50 +02:00
Abdessamad Derraz d8e9ad23d3 docs: describe short md5 handling as the code does
No platform YAML holds an md5 shorter than 32 characters: the Batocera scraper completes one when a single collection md5 starts with it. The pages said some entries carry 29 characters and that scrapers store them as-is.
2026-10-21 09:54:57 +02:00
Abdessamad Derraz 45db77cfd0 fix: link a system to its maker's folder
The database keeps one path per content, so a system placed by the
copy deduplication kept linked to another maker: nintendo-nes and
nintendo-fds to Arcade > FBNeo, sega-mastersystem and
sinclair-zx-spectrum to Other > zesarux. Every folder holding a copy now
votes, an id that spells one console folder links to it, and a maker's
folder outranks the folders that file copies by emulator.
2026-10-21 09:54:56 +02:00
Abdessamad Derraz 10c94369df fix: no sudo advice for a read-only mount
EROFS was advised like a permission refusal, and sudo fails the same way on a read-only mount. The advice now reads the mount flag and says to check and remount the card or drive.
2026-10-21 09:54:55 +02:00
Abdessamad Derraz 78bf456bb2 fix: run the DuckStation US slot guard again
Every US image now lists latin-america beside north-america, so the
equality filter matched nothing and the test skipped. It filters by
membership and fails instead of skipping when the profile stops ranking
US candidates.
2026-10-21 09:54:44 +02:00
Abdessamad Derraz 443a4e56c9 fix: let busybox wget fetch the installer
BusyBox wget has no --https-only and refused the whole command, so a curl-less Alpine system (the RomM image) could never bootstrap. The flag is passed only when wget lists it.
2026-10-21 09:54:11 +02:00
Abdessamad Derraz 730f900ff5 fix: detect emudeck on macos
EmuDeck's macOS build reads ~/.config/EmuDeck/settings.sh like the Linux one (functions/all.sh:3-4,24 and vars.sh:2 at dddb3022). The settings and ~/Emulation/bios rules now run on darwin too.
2026-10-21 09:53:37 +02:00
Abdessamad Derraz 8721dafd2a fix: load every script under the 3.9 floor
A module-level alias written with a PEP 604 union compiled under 3.9 and
raised TypeError when check_release_assets.py, and check_release.py
through it, were loaded. The floor test now imports every module under
scripts/ with a real 3.9 interpreter, which CI gives pyyaml and
jsonschema.
2026-10-21 09:53:34 +02:00
Abdessamad Derraz 12cc71b634 fix: hold the pipeline's fixed outputs for a run
The pipeline locked only --output-dir while it also writes database.json,
install/, README.md and docs/. A run on a scratch output directory could
start beside the run on dist/ and rewrite them. Every run now holds the
repository's .lock first.
2026-10-21 09:53:15 +02:00
Abdessamad Derraz a6d37e40ae docs: show the handy output verify prints 2026-10-21 09:52:57 +02:00
Abdessamad Derraz f7b758a41d fix: patch batocera entries past commented items
_patched_entry read '#{ ... }' lines as live items, found no match in the model and gave up, so atarist was written again whole (52 lines changed for 3 additions). A commented item is now left as written.
2026-10-21 09:52:34 +02:00
Abdessamad Derraz 3d2a521f8a fix: match profile_sync --system as verify does
--system compared the raw id, so the normalized spelling printed by verify.py --list-systems selected 1 Saturn profile of 11. The hint now counts the spellings that listing prints.
2026-10-21 09:52:13 +02:00
Abdessamad Derraz 43337c6719 fix: match release packs by exact name
release_record rebuilt nothing from the name builder: a substring match
took RetroArch_v1.22.2_AndroidArm64V8A_BIOS_Pack.zip for the RetroArch
pack. The record now refuses any pack that is not a registered platform's
full pack and stores the platforms each pack serves, which the readers
use instead of the pack name.
2026-10-21 09:51:55 +02:00
Abdessamad Derraz f2628c1170 docs: drop hand-kept --derive figures from tools 2026-10-21 09:51:51 +02:00
Abdessamad Derraz 9bb3dfbd1b fix: keep release checks importable on python 3.9
check_release_assets.py built a tuple[..., int | None] alias at import, a TypeError on the 3.9 floor tools.md states, and check_release.py imports it. A test reads every script for unions and match statements evaluated at runtime.
2026-10-21 09:51:33 +02:00
Abdessamad Derraz 5ec1208a83 fix: write romm keys with a bare file name
RomM lists the platform firmware folder without recursing and looks each file up as slug:file_name (models/firmware.py, base_handler.list_files at 5.4.0). An addition named with a directory (fuse/48.rom) is not written.
2026-10-21 09:51:32 +02:00
Abdessamad Derraz 6877512af7 feat: carry new upstream names and hashes in json
--detect-new-files and --watch-hashes were printed only by the text report and refused with --json, so check_freshness --profiles never saw a name or hash the cited code gained. Both now live on the report, and check_freshness passes them and marks such a profile stale.
2026-10-21 09:51:14 +02:00
Abdessamad Derraz 4205bc2233 fix: count one retrobat addition per path
The render writes one biosFiles item per path, as checkBios reads one md5 per item. A second addition at a held path was dropped from the file and still counted added (78 announced, 75 written).
2026-10-21 09:50:46 +02:00
Abdessamad Derraz 9409ec12b3 fix: keep the two mt32emu.h citations together
Each MT-32 entry cites the SHA-1 line and the ROMInfo line of one
mt32emu.h. dosbox_pure wrote them under two repository words, so the
ROMInfo line was self-checked alone and read MOVED onto the SHA-1 line.
Both now sit in the port that carries the file, and in the upstream for
dosbox-pure-unleashed, whose tree has no src/gui.

A recale that lands on a line a sibling part already cites goes to
review, and a part settled by its anchor keeps that anchor's repository.
2026-10-21 09:50:40 +02:00
Abdessamad Derraz 3545c1b333 fix: scope gap report entries by platform
cross_reference --platform read bundled from the profile alone and never
read unread_on. It now calls validation.installed_with_emulator and
unread_on with the platform key, as verify does, and a file installed
with its emulator is reported as bundled whether held or not.
2026-10-21 09:49:49 +02:00
Abdessamad Derraz fd9a9b14f8 fix: map rows edited in place one to one
A replace opcode of equal size is a line-by-line edit. Collapsing it onto its first line sent every ref citing a table row to the first row.
2026-10-21 09:48:33 +02:00
Abdessamad Derraz 0242dc77ee chore: regenerate manifests, readme and site nav 2026-10-21 09:01:49 +02:00
Abdessamad Derraz 2f327fbd37 chore: drop two hiscore.dat copies cores install
groovymame, mame4droid-2024 and raine install their hiscore.dat with the
emulator; each profile records the sha1 of its copy.
2026-10-21 07:32:34 +02:00
Abdessamad Derraz ab16555014 fix: give the bizhawk DSi NAND slots their region
DecideNAND (MelonDS.cs:684-706) keeps, for a DSi title, only the NAND
slots whose bit is set in header byte 0x1B0. The six DSi_Nand entries
now carry that territory, so --region keeps one 240 MB image, not six.
AUS answers oceania, as every other Australian slot does.
2026-10-21 07:00:48 +02:00
Abdessamad Derraz bc2233e725 fix: write truth systems in a stable order
Systems a core covers without a file were added while walking a set,
so the hash seed placed them and two runs wrote dist/truth/<platform>.yml
in two orders. The normalized system map was built from a set too.
2026-10-21 06:57:49 +02:00
Abdessamad Derraz bbd778b390 fix: file nicaiemu and coolbaremu as one system
The buildbot serves nicaiemu_libretro (frozen at 4c9b73f7, last built
2026-10-16 05:15) beside coolbaremu_libretro. They are two builds: each
reads its own orientation list from the system directory, so both
profiles stay, each with its own core name. Both read the same .cbe
archives from the same workspace, which builds a standalone binary at
both revisions: one system id and one build type for the two.
2026-10-21 06:55:42 +02:00
Abdessamad Derraz 2f5b4bd446 chore: resolve the merge markers left in a test 2026-10-21 06:50:36 +02:00
Abdessamad Derraz 6c2204a7a8 chore: merge tooling follow-up
# Conflicts:
#	tests/test_audit_regressions.py
2026-10-21 06:50:20 +02:00
Abdessamad Derraz 768b42e807 fix: fold case in the site gap report too 2026-10-21 06:50:10 +02:00
Abdessamad Derraz 91a9d2279e chore: drop the dolphin-mmjr codehandler copy
Only dolphin-mmjr's bundled entry reaches it, and that entry records its
sha1 and size: the APK installs the file with the emulator.
2026-10-21 06:50:10 +02:00
Abdessamad Derraz 378a9e4da5 chore: merge audit fixes (integrity)
# Conflicts:
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/retropie.json
#	install/rocknix.json
#	install/romm.json
#	scripts/generate_pack.py
#	wiki/tools.md
2026-10-21 06:48:42 +02:00
Abdessamad Derraz 88f9c40a78 chore: regenerate the install manifests
The slot and claim decisions above change which copy some paths take
(EmuDeck neocdz.zip, RetroBat DS firmware, hiscore.dat on six platforms)
and settle case variants on folding filesystems.
2026-10-21 06:47:50 +02:00
Abdessamad Derraz 4ba935374e fix: keep pack order outside contested paths
Ranking contested core claims sorted the whole gap report, which
reordered every manifest. Only the claims on one contested path move now,
grouped where the first of them stood, and two spellings of a path are
one group where the filesystem folds case (RetroBat's 64DD_IPL_DEV.bin).

An entry without a destination (EmuDeck's system:md5 whitelist) no
longer counts as a pack path in verify, and step 5 prints one status for
counts and paths together. A full offline run compared all 12 archives
and 930 emulator packs path by path, every one equal.
2026-10-21 06:47:44 +02:00
Abdessamad Derraz 85a44d9bf7 chore: merge amiberry and hatari re-profile
# Conflicts:
#	README.md
#	emulators/amiberry.yml
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/rocknix.json
2026-10-21 06:21:30 +02:00
Abdessamad Derraz 76e44a74b4 refactor: drop comments that restate the code
Thirty one-line labels in verify.py, generate_site.py and
generate_truth.py named the statement under them ("# Load registry",
"# Copy branding assets"). Comments that give a reason stay, as do the
module section headers. A test reads the three files for new ones.
2026-10-21 06:20:50 +02:00
Abdessamad Derraz 3613c0927b docs: open the download page on its content
The which-pack page announced itself ("This page picks the right pack
for a setup") before naming a pack. The sentence goes, the pointer to
Getting started stays.
2026-10-21 06:18:59 +02:00
Abdessamad Derraz 566d36275f fix: count alias readers in a file's requirers
Ymir reads sega_101.bin or saturn_bios.bin, so the sega_101.bin entry
carries saturn_bios.bin as an alias. kronos, yabause and yabasanshiro
open saturn_bios.bin and fall back to sega_101.bin: they were in that
entry's _cores but not its _required_by, while their saturn_bios.bin
entry had them requiring it. A reader that requires one of an entry's
aliases now requires the entry. No reader is added, and same-name
entries kept apart by path are left alone. No export changes.
2026-10-21 06:18:32 +02:00
Abdessamad Derraz 4648ed0de3 docs: regenerate readme coverage 2026-10-21 06:18:13 +02:00
Abdessamad Derraz e1a4f19737 feat: re-profile amiberry at 8.4.2
Pin moves to 2de3bf8 (v8.4.2). roms/aros-ext.bin, roms/aros-rom.bin and whdboot/game-data/whdload_db.json changed upstream and are collected at that revision. The WHDLoad database is matched on the archive SHA1 first and refreshed from the bundled copy when newer. kick02019.AROS and its .ext are names the Recalbox launcher writes, so the profile now documents the kickstart_rom_file settings instead.
2026-10-21 06:18:10 +02:00