Commit Graph
1662 Commits
Author SHA1 Message Date
Abdessamad Derraz 648db6bcbf fix: merge mame devices when the pin moves
Step 2a advanced mame.yml's core_version and upstream_commit to a new
release and left the device ROM sets on the previous one. A plain run
that moves the pin now fetches that release's listxml into .cache/dats
and runs the device merge, which checks version and commit itself.
2026-10-21 10:06:13 +02:00
Abdessamad Derraz 8c7c705169 fix: ask the release only for registered names
asset_candidates fell back to the bare name when no .gitignore line carried it: every online build asked large-files for aes.zip and five others and printed an HTTP 404 per name, pointing at an upload that never existed. The tests that fetch now register their names.
2026-10-21 10:06:11 +02:00
Abdessamad Derraz 377e53f4c0 feat: show load_from, built_by and pin fields
load_from, found_by, on_mismatch, built_by, pin_frozen and
version_source were in the profiles and on no page: Dolphin's
gba_bios.bin showed a path under system/ although the libretro core reads
it from its save directory, and mame0278 did not say only Recalbox ships
it.
2026-10-21 10:05:18 +02:00
Abdessamad Derraz a74a56abc4 feat: name the profile a missing dump would fill
The acquisition report never read a profile, so XM7's required DICROM.ROM
sat among 2 287 targets at the rank of dumps no core reads. A missing
entry that a profile reads and the collection lacks is now listed first
with the profile, matched by hash or, for a hashless entry with a size,
by name and size. 29 entries on the current snapshots.
2026-10-21 10:05:10 +02:00
Abdessamad Derraz 4e41fad9b6 fix: give bios_U.sms every export territory
sms_cart_region_detect returns REGION_USA for any cartridge outside its
CRC list whose header is not Japanese, so Genesis Plus GX and its two
forks open bios_U.sms for Australian and Brazilian cartridges as well.
Tagged north-america and europe only, --region australia,japan dropped
it from the pack. The tag now adds latin-america and oceania.
2026-10-21 10:04:58 +02:00
Abdessamad Derraz c29c456206 fix: name the python install command per system
Without Python the bootstrap told every system to run sudo apt install python3. It now reads Termux, macOS and /etc/os-release to name pkg, xcode-select, apt, dnf, pacman, zypper or apk, and in Termux offers to run pkg install python, which needs no root.
2026-10-21 10:04:54 +02:00
Abdessamad Derraz d16f8fd17a fix: accept downloads by cryptographic hash only
A crc32 and a size can be met by crafted bytes, so a remote download
matched on them could land in bios/. verify_content now requires every
sha256, sha1 and md5 the entry declares to match, and an entry declaring
none is reported as not verifiable from the network and nothing is
written. crc32 and size still find a file the database already holds.
2026-10-21 10:04:20 +02:00
Abdessamad Derraz 7cebbfe2ec feat: fill standalone emulator folders
With no platform detected, a machine running standalone emulators (ES-DE with DuckStation and PCSX2) was offered RetroArch's folder. The installer now fills the emulator folders of the standalone copies list that exist here, from the EmuDeck then RetroArch file lists, asking first at a terminal.
2026-10-21 10:03:58 +02:00
Abdessamad Derraz d904116a79 fix: verify each 3DS file once per content
check_crypto_validation keeps its answer keyed by the file and keys file as compute_hashes keys them (path, inode, size, mtime, ctime). verify --all ran the verifiers 35 times for 5 files (0.88 s), now 5 times (0.13 s). verify --all --json output is byte-identical.
2026-10-21 10:03:27 +02:00
Abdessamad Derraz e65ece743d fix: name where a held profile entry sits
164 entries wore the in repo badge with no copy named: they are held as
a ZIP member or in a data directory, which the database does not index.
held_copy() finds that copy and the card names it. storage: large_file
no longer counts as held by itself: kenji-nx's user-picked firmware
package read as held although nothing collects it.
2026-10-21 10:02:50 +02:00
Abdessamad Derraz b057684b94 fix: hold the private-names guard to itself
The guard spelled out every name it forbids and skipped its own file.
The names are now assembled from parts, the file is checked like any
other, and the generated-page filter reads the site's own list of
local note directories.
2026-10-21 10:02:48 +02:00
Abdessamad Derraz 99c9993c17 fix: profile stella from stella-emu/stella
The buildbot builds the stella core from stella-emu/stella master
(src/os/libretro), not from stella2014 code, and stella2023 claimed the
stella name. stella.yml now documents that tree at 2c3d7023: the KidVid
tapes are read from a stella folder under the save directory, loose or
in KidVidAudio.zip, and played. stella2023 keeps its own name only.
2026-10-21 10:02:32 +02:00
Abdessamad Derraz e255348f52 fix: read mame hiscore.dat where the cores open it
mame2000 opens hiscore.dat, history.dat and mameinfo.dat from IMAMEBASEPATH, the content directory (src/libretro/fileio.c:1002-1014, libretro.c:1044-1052). mame2014 and mame2015 open hiscore.dat through emu_file with no search path, the working directory (src/emu/hiscore.c:330-337, fileio.c:141-143). With no load_from the entries claimed system/ and the packs shipped files neither core opens there. ANoteOutsideTheSystemDirCarriesLoadFrom holds libretro notes and load_from together.
2026-10-21 10:02:30 +02:00
Abdessamad Derraz 62a4f19ce3 fix: keep ares labels off other dumps' names
ares finds firmware by sha256 (desktop-ui/settings/firmware.cpp:116-136) and names no file. Its FDS entry (RP2C33-01, crc32 1c7ae5d5) was labelled disksys.rom, the name nestopia, fceumm and four more cores read for RP2C33A-01A: ROCKNIX and EmuDeck got ares's revision at disksys.rom. It now carries MAME's member name rp2c33-01.bin.

The LaserActive v1.05 entry, the PAC-S10, PAC-N10, PAC-N1, PCE-LP1 and Neo Geo Pocket aliases are names RetroBat's launcher passes (Ares.Generator.cs:435-555): mega-ld.cpp:16-17 holds only the v1.04 US and v1.02 Japan hashes. RetroBat declares each name in its own list.

AHashScanLabelNamesNoOtherDump compares every found_by: content label with other profiles' entries of that name through the hash indexes.
2026-10-21 10:02:30 +02:00
Abdessamad Derraz f4010bfd18 docs: state what each .info declares, not a grade
Ten profile notes graded a .info (correct, accurate, still, missing, omits). Each now says what the file declares and what the code reads. test_no_grade_on_another_projects_file reads every profile line that names a .info, core-info or System.dat.
2026-10-21 10:02:16 +02:00
Abdessamad Derraz 0b0480a121 fix: name standalone folders instead of skipped
Every install ended on 'Standalone copies skipped', on machines with no standalone emulator as on one whose DuckStation lacked its BIOS. The line now lists the emulator folders that exist here, and nothing is said when there are none.
2026-10-21 10:02:15 +02:00
Abdessamad Derraz 5d5c431cce fix: store each s8k_cpu.zip member once
ROM_START(s8k_cpu) in src/devices/bus/zbi/s8k_cpu.cpp (mamedev f34f0250) names cpu_34-0715-03a.u76, cpu_34-0717-03a.u75 and cpu_34-0718-03a.u77 under two system BIOSes with the same CRC. The archive held them twice. Rebuilt with build_torrentzip from the 13 distinct members, each checked against the mame0289 recipe sha1.
2026-10-21 10:02:12 +02:00
Abdessamad Derraz 6c076884ed fix: let auto_fetch prove sha256 and crc32 entries
verify_content read sha1 then md5 only, so an entry pinned by sha256 or
crc32 could be searched and never accepted, and one without any hash was
announced as searched. The strongest declared hash now decides, crc32
counts with its size, the database lookup reads sha256 and crc32, and an
entry with nothing to prove a download by is reported, not searched.
2026-10-21 10:02:05 +02:00
Abdessamad Derraz e9f945857c docs: describe the native export without verdicts 2026-10-21 10:01:46 +02:00
Abdessamad Derraz ff8a390ce6 fix: say when --dest does not exist
A mistyped --dest was created with every missing parent and reported done. The run now names the missing folder and the nearest one that exists, asks at a terminal before creating more than one level, and prints the folder it created. The free-space check now runs before the folder is made and ends with 2, nothing installed.
2026-10-21 10:01:34 +02:00
Abdessamad Derraz fe2fcf6034 fix: create tmp/tests when the test package loads
Most modules write under tmp/, which a fresh clone lacks, and only three
created it on import. A module run alone failed with FileNotFoundError on
a temporary directory.
2026-10-21 10:01:23 +02:00
Abdessamad Derraz 3b60facd34 fix: count core copies under their own category
The second pass, the archive-prefix pass and the agnostic scan built
extras without source_name or source_path, so verify found no gap report
entry for them and counted game data such as hiscore.dat as core files.
Every extra now carries its entry's category and key, and verify reads
the category from the extra the builder places.
2026-10-21 10:01:08 +02:00
Abdessamad Derraz 0b772fe255 docs: compare scraper output in the repo tmp/ 2026-10-21 10:01:07 +02:00
Abdessamad Derraz 11c836cdee fix: hash only same-size files in dedup scan
Equal files have equal sizes, and the scan only uses the sha1. Files alone at their size are no longer read and the rest get one digest instead of five. Same 278 groups (645 paths), same dry-run output, 36.7 s to 1.9 s.
2026-10-21 10:00:53 +02:00
Abdessamad Derraz 7b9c8ae663 fix: refuse a mistyped --target in cross_reference
load_target_config raised out of cross_reference.py as a traceback. The hint now names the full verify.py command, since cross_reference and slots have no --list-targets.
2026-10-21 10:00:47 +02:00
Abdessamad Derraz 932410e00d fix: send GITHUB_TOKEN from download.sh
download.py authenticated its release lookup and download.sh did not, so
past the anonymous quota the shell twin failed with 403. The header goes
to api.github.com only and is read from stdin, out of the process list.
2026-10-21 10:00:44 +02:00
Abdessamad Derraz f3586001ab fix: fail freshness on unknown, name the refresher
check_freshness exited 0 when no upstream answered. An unknown row now
holds the exit code, and a profile skipped by nature is reported skipped
so it does not hold it forever. --only names the valid areas, a stale
native row names its platform, and the summary names refresh_stale.py.
2026-10-21 10:00:36 +02:00
Abdessamad Derraz e257504e20 docs: point the CI table to each workflow's paths 2026-10-21 10:00:21 +02:00
Abdessamad Derraz 6c4f056f08 fix: keep the listing in the no-platform hint
--list-systems, --list-cores, --list-emulators and --list-targets with nothing detected were answered with an install command that dropped the listing and added --dest. The hint now names the platform and keeps the listing.
2026-10-21 10:00:07 +02:00
Abdessamad Derraz 7e926ac909 docs: say which entries skip the name steps 2026-10-21 09:59:53 +02:00
Abdessamad Derraz 0c65d93248 fix: state only what lands in the export summary
The summary in exports/<platform>.json read '82 hashes corrected' and '854 additions left out'. It now counts what the written file carries, in the exports page's terms (hashes updated to the emulator source, required flags aligned with the code). Files the format cannot carry, withheld updates and content-found labels are printed on a line of their own and stay in the report's counts.
2026-10-21 09:59:49 +02:00
Abdessamad Derraz 5e4d9199c2 feat: accept platform system names in --system
The manifest now carries system_names, each system's native_system names from the platform's own file, and --system, its suggestions and --list-systems read them: --platform batocera --system psx installs sony-playstation.
2026-10-21 09:59:25 +02:00
Abdessamad Derraz 7a120d7a51 fix: generate the site before CI runs the suite
docs/ is gitignored and was built after the tests, so the three guards
that read rendered pages skipped on every CI run. The site is generated
first, the suite runs whatever that step returns, and the guards fail
in CI when the pages are absent.
2026-10-21 09:59:22 +02:00
Abdessamad Derraz 6bca75b23e docs: name the registry keys the tools read 2026-10-21 09:59:22 +02:00
Abdessamad Derraz cc57d44de5 fix: order inherited rescrapes, end on verify
refresh_stale ran a parent's rescrape and its children's truth and
export jobs side by side, so a child read the parent mid-rewrite. A job
now starts once the rescrapes of its ancestors have ended, and a child
no row names gets its export rewritten after them. A rescrape chain ends
on verify.py --platform, whose summary closes the report.
2026-10-21 09:58:58 +02:00
Abdessamad Derraz 8ed1a737a8 fix: keep external downloads under the run policy
download_external stayed in generate_pack, outside the offline switch: it fetched under --offline, wrote to the system /tmp and held the body in memory. It now lives in packresolve, honours the switch, streams into the repository tmp/ and hashes on the way.
2026-10-21 09:58:21 +02:00
Abdessamad Derraz ffd3a66839 docs: keep the derivable count in its file alone 2026-10-21 09:58:20 +02:00
Abdessamad Derraz 084dff84f4 fix: list what the emulator pack writes
verify --emulator listed only OK destinations, while the emulator pack
also writes a copy whose hash the profile contradicts, loose or archive.
pack_destinations now records the paths the replay places, and both
branches resolve through packresolve.resolve_file as the pack does.
2026-10-21 09:58:09 +02:00
Abdessamad Derraz 287a16f41c fix: name --fetch when an original is not cached
The export failed with 'absent and fetching is off'. It now names the cache directory it looked in and the flag that downloads the file.
2026-10-21 09:57:44 +02:00
Abdessamad Derraz 83a4b21d30 fix: keep platform hashes cores disagree on
When two cores declare different values for one file, the truth keeps the first and marks the entry hash_conflict. Its hashes pair an entry and never replace the platform's: armsx2's GameIndex.yaml no longer overwrites System.dat's line for LRPS2.
2026-10-21 09:57:20 +02:00
Abdessamad Derraz f997bdc81a fix: check a recalbox zip by its composite only
Bios::Scan takes Zip::Md5Composite for every .zip (Bios.cpp:64-68). The resolver already honoured zip_digest: composite, but digest_verdict accepted the container's own md5 first, so the pack check passed bytes Recalbox marks red. The page described that order.
2026-10-21 09:57:06 +02:00
Abdessamad Derraz 585be38ee3 fix: transform each held content once
Step 1b read 18 924 held files and members for 7 407 distinct contents and ran every transform on each read. A content already transformed is now skipped (the first path still keeps the match), the power-of-two crc shift operators are squared once for the pass instead of once per length, and a group pairs members by the sizes it holds. derivable.json and the step output are byte-identical, 33.4 s to 25.1 s.
2026-10-21 09:56:53 +02:00
Abdessamad Derraz 9bb1a73f91 fix: hold every data tree, present or not
export_native and cross_reference read data/ without the shared hold, so
a refresh could swap a tree mid-walk. hold_data_dirs skipped a cache not
yet on disk, so a refresh could install it between verify and the packs
of one pipeline run. Both now hold every registered tree.
2026-10-21 09:56:49 +02:00
Abdessamad Derraz 4fec09a014 fix: name the selection mode in --from-md5 packs
BizHawk is both a platform and a profile display name: --from-md5 with --platform bizhawk and with --emulator bizhawk wrote one file in two layouts. The emulator mode now carries _Emulator in the name.
2026-10-21 09:56:45 +02:00
Abdessamad Derraz 4ceca6acc5 fix: credit platforms with the file the pack ships
FileConsumers resolved platform declarations with their destination as
a hint, which the builder, the manifest and verify never pass. RetroBat's
63 mame/hash lists and the curated gc-ntsc-12.bin credited another copy
than the one the manifests install.
2026-10-21 09:56:08 +02:00
Abdessamad Derraz 2af9509ef4 fix: resolve declarations through one function
verify resolved platform entries with common.resolve_local_file and put
the slot override first, and the region pass added a destination hint.
packresolve.resolve_declaration now serves the builder, the manifest,
verify and the region pass: release assets cached for a file absent from
bios/ count as shipped in all four, and a release asset keeps its answer
over a slot override.
2026-10-21 09:56:07 +02:00
Abdessamad Derraz a3e625cd6a fix: refuse a default value typed out as a flag
The refusal tables compared each flag with its default value, so --source full and --group-by system passed beside the modes that refuse every other value. Both now default to None and take their effective value after the checks, in generate_pack and pipeline.
2026-10-21 09:55:51 +02:00
Abdessamad Derraz ed610c7bf3 fix: hold the consumer guards on fixtures
The Dolphin-MMJR codehandler.bin and armsx2's game_controller_db.txt left
the collection, and the two guards that read them skipped on every run.
Both now build the trees they need, and the guards still reading the
repository fail instead of skipping when their data is gone.
2026-10-21 09:55:50 +02:00
Abdessamad Derraz 563ce1ebf2 fix: read fbneo dats at one checked commit
The FBNeo import recorded the listing's blob sha for bytes fetched from a
branch URL a CDN caches, and --fetch <revision> still read the tip. The
revision is now resolved to a commit, listed and downloaded there, each
DAT is held against its blob sha, and a DAT read at a named revision is
labelled with it without replacing the tip's recorded blobs.
2026-10-21 09:55:50 +02:00
Abdessamad Derraz d8e9ad23d3 docs: describe short md5 handling as the code does
No platform YAML holds an md5 shorter than 32 characters: the Batocera scraper completes one when a single collection md5 starts with it. The pages said some entries carry 29 characters and that scrapers store them as-is.
2026-10-21 09:54:57 +02:00