Commit Graph
242 Commits
Author SHA1 Message Date
Abdessamad Derraz 54eac3b2df refactor: verify keeps its lock stack name
The --destinations branch of _main rebound held, the ExitStack that
holds the data directory locks, to the dict it writes. It is now
named destinations, and a test refuses a rebinding of the stack.
2026-11-06 08:19:00 +01:00
Abdessamad Derraz df801fd311 chore: merge resolver and export fixes 2026-11-06 05:54:29 +01:00
Abdessamad Derraz 7f6ba6952c feat: verify reports the required-only pack 2026-11-06 03:45:07 +01:00
Abdessamad Derraz 8f7dbb1b31 fix: gap report reads the launcher layer
cross_reference --platform listed core files the platform's launcher already settles, as Batocera's DuckStation scph1002a.bin while the PAL slot holds scph5502.bin. It now hands the launcher layer the same entries verify does, and counts what it settles as set aside.
2026-11-06 03:31:27 +01:00
Abdessamad Derraz d265453b32 feat: compare narrowed packs with verify
Step 5 builds the --required-only, --no-game-data, --region, --target
and --system packs of every platform in memory and compares their paths
with verify run under the same narrowing. verify_platform gains
required_only, generate_pack a record mode. A run samples three regions,
targets and systems per platform (155 packs, about 3 minutes);
--full-narrowings compares all 1,150 (25 minutes). A test fails when
_narrowings() gains a dimension that is neither compared nor exempted.
2026-11-06 00:07:40 +01:00
Abdessamad Derraz c1f0e8c2ca fix: compare system packs in pipeline step 5
Step 5 built emulator packs one profile at a time, where two profiles never share a path. It now also builds each system pack that gathers several profiles and compares it with verify. The first run found five standalone MSX and ColecoVision packs where verify skipped the data tree of a profile the system filter left without files, and credited msx-emu's copies the pack never wrote. verify now places those trees too: 541 system packs, all equal.
2026-11-05 19:45:39 +01:00
Abdessamad Derraz d8cfd283b4 fix: show packed core files with an empty gap list
verify --platform recalbox --system nec-pc-engine printed no Core files line while the pack held six syscard copies for geargrafx: the section returned on an empty gap report before reading packed_extras. It now prints the pack's counts whenever they are not zero.
2026-11-05 19:32:05 +01:00
Abdessamad Derraz dde6229775 fix: count a core's release asset as packed
verify replayed the core extras offline: a release asset an online
build fetched and packed took no path in its report (finding 2).
2026-11-05 11:55:53 +01:00
Abdessamad Derraz 71020c3974 fix: list a pending release asset as packed
verify counted it OK but kept its path out of pack_destinations, so an
online step 5 reported the fetched file as pack only (finding 1).
2026-11-05 11:54:42 +01:00
Abdessamad Derraz f2e25aeea2 feat: list platforms with --list-platforms
install.py took --list-platforms, generate_pack.py --list and verify.py
nothing. Both tools now take --list-platforms and print the same list
from common.py. generate_pack.py keeps --list (finding 80).
2026-11-05 11:52:26 +01:00
Abdessamad Derraz c7df790744 fix: one system scope for verify and the packs
A system the target leaves no core for was verified 0/0 with exit 0,
and generate_pack blamed the platform for lacking it. Both read
platform_scope_systems and system_scope_error now (findings 13, 14).
2026-11-05 11:47:03 +01:00
Abdessamad Derraz f3fe6f58b4 fix: name the path of a file read at two places
MyMinUI printed the same MISSING line twice for Bios/FBN/ and Bios/NG/
sfiii2a.zip (finding 78).
2026-11-05 11:41:46 +01:00
Abdessamad Derraz 666883ad53 fix: print missing core files under their count
They came after the held-but-not-packed line and read as files
installed with their emulator (finding 79).
2026-11-05 11:41:02 +01:00
Abdessamad Derraz 5e4becf876 fix: count each reason a held file is not packed
The first entry's reason stood for the whole count: 16 Lakka files
under one cause that 2 of them carried (findings 23 and 76).
2026-11-05 11:40:40 +01:00
Abdessamad Derraz 6e77da4b82 fix: write every platform count in the summary
A platform with one entry had no number, so the named counts did not
add up to the total (finding 81).
2026-11-05 11:39:25 +01:00
Abdessamad Derraz 7c6af30a0c fix: name the file existence mode finds
verify_entry_existence returned no path, so RetroArch, Lakka and
RetroPie pages showed no Served by line. It now returns the resolved
path as the hash modes do. verify.py --all prints the same bytes and
its JSON is unchanged.
2026-10-23 07:16:05 +02:00
Abdessamad Derraz 2d26385efe feat: place save-dir files in each system folder
Batocera, KNULLI, Recalbox, RetroBat and RetroDECK hand a libretro core <saves>/<system> at each launch. Each copy lands below every system that runs the core. EmuDeck keeps one unsorted saves/retroarch/saves. The unsupported platforms record why.
2026-10-22 21:16:59 +02:00
Abdessamad Derraz 1cd6c9ee4d feat: place save-dir files in packs and lists
load_from: save_dir entries of a libretro build were left out of every
pack. They now go below the platform's save folder (registry
save_directory: RetroArch saves/, Lakka savefiles/, by core), the
manifest marks them root: save, and verify replays the same placement.
Platforms keeping saves per system or per game list them as omitted.
2026-10-22 19:58:26 +02:00
Abdessamad Derraz cd7d993cac feat: settle entries by their system's cores
A system that lists the cores it runs is settled by those cores alone, so an unprofiled core of another system no longer leaves its entries open. Readers whose core runs the entry's system are named first, and cores that only share the file name are listed as such.
2026-10-22 06:30:56 +02:00
Abdessamad Derraz a0b27d612d fix: align tests with the ten-value pack result
Two test modules still unpacked nine values from verify_pack_against_platform. The category evidence joined readers with semicolons, which the prose rule refuses on generated pages, and three comments used clause semicolons.
2026-10-22 06:23:15 +02:00
Abdessamad Derraz 8ba65de1ca chore: merge the game data filter 2026-10-22 06:19:55 +02:00
Abdessamad Derraz 8b952968d6 feat: add a --no-game-data pack narrowing
The install manifest marks each game data file "category": "game_data":
a platform entry only game data profile entries read, or a core file its
profile declares game data. Firmware carries no category. Every manifest
says "categorized": true so the installer can refuse an older list.

generate_pack.py --no-game-data cuts those files from the platform list
and the core profiles before the build, names each one with the cores that
read it as game data, and tags the pack _NoGameData through _narrowings.
verify.py and the pack check take the same cut, so --verify-packs and the
report describe the pack the builder wrote. Modes without a platform
(--emulator, --system alone, --from-md5) and --all-variants refuse it.
2026-10-22 06:19:05 +02:00
Abdessamad Derraz be9036f2b2 feat: document the category of every entry
Held entries carry their category and readers too, not only missing ones, so each platform page says which cores read every file it lists and where the code does it. Severity still applies to firmware gaps only.
2026-10-22 05:46:09 +02:00
Abdessamad Derraz a75ea05b86 feat: document what settles each category
Each missing platform entry now carries its evidence: the profile entries that read it with their source reference, the cores of its system examined without finding a reader, or why the question stays open. verify --verbose prints it, --json carries it and the platform pages show it.
2026-10-22 05:43:06 +02:00
Abdessamad Derraz f66adfdcbc feat: categorize missing platform entries
A platform list can name a game's data files or a file no core it runs opens. Verify gives each missing entry a category read from the profiles of the platform's cores: firmware, game data or unread. Only firmware gaps carry a severity and fail --strict; the others are counted on their own line, and the pack check no longer counts them as pack errors. Missing firmware stays a conformance error.
2026-10-22 05:38:57 +02:00
Abdessamad Derraz d0802c5360 feat: place core files per handed system dir
A platform can hand each libretro core a system directory of its own
(core_system_dirs). platform_layouts then gives one view per directory,
so the gap report, the builder, the manifest, slots and cross_reference
place a core's files below each folder it is handed, and a core handed
none gets no file. Resolution strips the folder again and searches the
collection by the core's own path.
2026-10-22 04:04:31 +02:00
Abdessamad Derraz d6a0639330 chore: merge audit fixes (packrules)
# Conflicts:
#	scripts/packextras.py
2026-10-21 11:52:28 +02:00
Abdessamad Derraz cd589b2742 chore: merge audit fixes (mame) 2026-10-21 11:48:30 +02:00
Abdessamad Derraz 7030ef3350 chore: merge audit fixes (sysnames) 2026-10-21 11:48:30 +02:00
Abdessamad Derraz 8643d4a4ea chore: merge audit fixes (site)
# Conflicts:
#	emulators/flycast.yml
#	emulators/primehack.yml
2026-10-21 11:48:30 +02:00
Abdessamad Derraz 56e8cbe046 feat: file mame device sets by their machines
The -listxml names, for every machine, its parent (romof), the devices of its default configuration (device_ref) and the cards its slots offer (slotoption). A machine takes the system of the first set on its romof chain a written entry lists, and passes it to each device it can instantiate. A device set reached from one system gets system:, from several the new per-entry systems:, from none neither, and keeps serving every system of the profile. The gap report, the core extras and slots read the entry's own systems, so generate_pack --system and the installer pick by them.
2026-10-21 11:33:39 +02:00
Abdessamad Derraz 3fe88332e7 feat: accept platform system names in --system
generate_pack and verify read --system beside --platform through common.resolve_system_arg, built on platform_system_names, the mapping the manifest's system_names comes from. verify --platform --system now reports on the narrowed pack. Unknown names are refused with suggestions before the database is read.
2026-10-21 10:52:40 +02:00
Abdessamad Derraz 9a0501d9d3 fix: count an uncached release asset as fetched
verify resolved offline, so a release asset absent from bios/ and the
cache read as missing while an online build fetched and packed it. verify
now reads the database record of the registered path and counts such a
declaration present, listed as fetched at build time. --offline keeps the
old count, and the offline pipeline passes it to its verify step.
2026-10-21 10:50:03 +02:00
Abdessamad Derraz b4e5e42c97 fix: leave out core files the core refuses by size
A core extra resolved by name shipped on existence platforms even when
the core's own size check turns the copy away. The RetroPie pack carried
the 128 KB Apple IIGS ROM as Basilisk II's ROM (min 512 KB). The builder,
the manifest and verify now read validation.core_size_refusal, which
accepts a copy any version of the same profile slot takes.
2026-10-21 10:46:52 +02:00
Abdessamad Derraz 052f837e69 chore: merge round 2 fixes (integrity) 2026-10-21 10:31:04 +02:00
Abdessamad Derraz 6409757c39 chore: merge round 2 fixes (docs)
# Conflicts:
#	CONTRIBUTING.md
#	scripts/check_release_assets.py
#	scripts/generate_readme.py
2026-10-21 10:31:04 +02:00
Abdessamad Derraz 3ccee1b216 fix: drop the gba_bios.bin alias of nanoboyadvance
NanoBoyAdvance opens general.bios_path, default bios.bin (src/platform/core/src/config.cpp:36 at 3bb6f478). gba_bios.bin is the path ROCKNIX's config.toml sets, recorded under launchers: in rocknix.yml.

Without the alias the entry met only PCem's and QEMU's bios.bin, 64 and 128 KB, which BIOSLoader refuses (bios.cpp:17-48). verify counted it held by name, so the launcher layer left it alone and the ROCKNIX pack gained PCem's image. A hashless entry whose every same-named copy fails its size is now not held, and the gba_bios.bin ROCKNIX declares settles it.
2026-10-21 10:14:15 +02:00
Abdessamad Derraz e65ece743d fix: name where a held profile entry sits
164 entries wore the in repo badge with no copy named: they are held as
a ZIP member or in a data directory, which the database does not index.
held_copy() finds that copy and the card names it. storage: large_file
no longer counts as held by itself: kenji-nx's user-picked firmware
package read as held although nothing collects it.
2026-10-21 10:02:50 +02:00
Abdessamad Derraz 3b60facd34 fix: count core copies under their own category
The second pass, the archive-prefix pass and the agnostic scan built
extras without source_name or source_path, so verify found no gap report
entry for them and counted game data such as hiscore.dat as core files.
Every extra now carries its entry's category and key, and verify reads
the category from the extra the builder places.
2026-10-21 10:01:08 +02:00
Abdessamad Derraz 084dff84f4 fix: list what the emulator pack writes
verify --emulator listed only OK destinations, while the emulator pack
also writes a copy whose hash the profile contradicts, loose or archive.
pack_destinations now records the paths the replay places, and both
branches resolve through packresolve.resolve_file as the pack does.
2026-10-21 09:58:09 +02:00
Abdessamad Derraz 2af9509ef4 fix: resolve declarations through one function
verify resolved platform entries with common.resolve_local_file and put
the slot override first, and the region pass added a destination hint.
packresolve.resolve_declaration now serves the builder, the manifest,
verify and the region pass: release assets cached for a file absent from
bios/ count as shipped in all four, and a release asset keeps its answer
over a slot override.
2026-10-21 09:56:07 +02:00
Abdessamad Derraz a6d37e40ae docs: show the handy output verify prints 2026-10-21 09:52:57 +02:00
Abdessamad Derraz 6c2204a7a8 chore: merge tooling follow-up
# Conflicts:
#	tests/test_audit_regressions.py
2026-10-21 06:50:20 +02:00
Abdessamad Derraz 378a9e4da5 chore: merge audit fixes (integrity)
# Conflicts:
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/retropie.json
#	install/rocknix.json
#	install/romm.json
#	scripts/generate_pack.py
#	wiki/tools.md
2026-10-21 06:48:42 +02:00
Abdessamad Derraz 4ba935374e fix: keep pack order outside contested paths
Ranking contested core claims sorted the whole gap report, which
reordered every manifest. Only the claims on one contested path move now,
grouped where the first of them stood, and two spellings of a path are
one group where the filesystem folds case (RetroBat's 64DD_IPL_DEV.bin).

An entry without a destination (EmuDeck's system:md5 whitelist) no
longer counts as a pack path in verify, and step 5 prints one status for
counts and paths together. A full offline run compared all 12 archives
and 930 emulator packs path by path, every one equal.
2026-10-21 06:47:44 +02:00
Abdessamad Derraz 76e44a74b4 refactor: drop comments that restate the code
Thirty one-line labels in verify.py, generate_site.py and
generate_truth.py named the statement under them ("# Load registry",
"# Copy branding assets"). Comments that give a reason stay, as do the
module section headers. A test reads the three files for new ones.
2026-10-21 06:20:50 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00
Abdessamad Derraz aa36ec544d fix: name every gap verify counts
verify --platform printed N optional core files and N game data files
missing and named none, even with --verbose. --verbose now lists them as
MISSING (optional) and MISSING (game data), and without it the count line
says --verbose lists them, as the line about cores with no external file
already did.
2026-10-21 05:40:11 +02:00
Abdessamad Derraz 82a3e44afd fix: settle a slot in any case where case folds
declaration_settles tested the exact-case name before the slot, so on a
filesystem that folds case a declaration spelled otherwise never settled
the core's file: RetroBat's 32X_G_BIOS.BIN left BlastEm's 32X_G_BIOS.bin
undeclared, and verify counted ten such entries as core files. A slot a
case-insensitive platform fills now settles the entry whatever the
spelling (case_folded_slots), the slot arbitration judging its bytes as
for the exact name. Lakka and RetroPie, which keep both names, are
unchanged. Measured: 10 entries settle on RetroBat, 21 on RetroArch.

generate_site.py builds its gap report through cross_reference() without
the folded slots; it needs folded_declared_slots() passed in the same way
as the command line now does.
2026-10-21 05:38:11 +02:00
Abdessamad Derraz 738577b5a4 fix: run verify and generate_pack from any dir
Started outside the repository root, both ended on a FileNotFoundError
traceback: the database and the data registry name files from the root.
They now move to the root after reading the paths the caller typed, which
stay relative to where they were typed.
2026-10-21 05:25:29 +02:00