Commit Graph
136 Commits
Author SHA1 Message Date
Abdessamad Derraz 84cffe3f8d chore: merge origin/main 2026-11-06 16:42:16 +01:00
Abdessamad Derraz 057fd2d0f6 fix: isolate every test module however it runs
tests/__init__.py keeps a run offline, its scratch under tmp/ and its
installer cache its own, but only discover -s tests -t . imported it.
discover tests ran with RETROBIOS_OFFLINE unset, scratch on the /tmp
tmpfs and the user's real installer cache, the setting in which e2e
fixtures had written into the shared .cache/large. Every test module now
imports the package first, and a test fails on a module that does not.
test_verify_listing.py took its line with the previous commit.
2026-11-06 15:14:05 +01:00
Abdessamad Derraz 52b7ee0f46 fix: succeed when an emulator needs no file
--emulator handy --required-only removed the empty ZIP, printed the
path it had just removed and exited 1, while verify reported 0/0 OK and
exited 0. The builder now says no file is needed and succeeds.
2026-11-06 15:06:28 +01:00
Abdessamad Derraz 6bf62d5746 fix: refuse profile narrowings without cores
MiSTer and BizHawk declare cores: []. --region, --one-per-slot and
--no-game-data read the profiles of the cores a platform runs, so their
packs and manifests withdrew nothing under a name and README that said
they had. generate_pack and verify now refuse the flags for a named
platform and leave it out under --all.
2026-11-06 15:01:58 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz fb9b5592a5 feat: let --emulator take an alias name
--emulator mednafen_psx refused with 'use --emulator beetle_psx' although the profile names its target. The alias now selects that profile and says so on stderr, in verify, generate_pack and cross_reference alike. Launchers are still refused, and pipeline step 5 skips aliases since they compare under the profile they name.
2026-11-05 19:26:36 +01:00
Abdessamad Derraz cfc2515909 fix: check the zipped_file member before composite
A one-member archive whose member carries the zipped_file md5 under
another name has that md5 as its composite. The name step returned it as
md5_composite_exact, which slots and the site take as content proof, while
Batocera looks the member up by name and finds nothing.
2026-11-05 11:45:58 +01:00
Abdessamad Derraz 0d510eb4bd chore: merge core versions read from code 2026-10-27 19:42:04 +01:00
Abdessamad Derraz 25c356669b fix: record each platform's own list version
install/<platform>.json wrote the constant "1.0" as the version of the
list it was built from. The manifest now carries the version the platform
YAML records, and generate_pack refuses a YAML without one.

inherits: no longer passes a parent's version to a child: Lakka was
published as RetroArch v1.22.2 in its manifest, pack name and site page.
lakka.yml records v6.1, the Lakka-LibreELEC tag its target list is read
at. The MiSTer scraper refuses a BiosDB without a timestamp instead of
writing an empty version.

A pack's manifest.json records list_version, so an install from the pack
alone knows it too. Manifests regenerated twice and compared: only
version and generated changed.
2026-10-27 19:10:39 +01:00
Abdessamad Derraz 07c82c885f refactor: read the fbneo version from version.h
FBNeo bumps src/burn/version.h without tagging: the scraper wrote the
last tag, v1.0.0.02, while the source said 1.0.0.03. It now reads the
header the way the libretro core prints it. The merge no longer writes
core_version: the hashes come from finalburnneo/FBNeo's tip, which the
libretro/FBNeo pin does not name, so the version stays the one
version_basis reads at that pin. A MAME release profile is written with
its version_basis.
2026-10-27 18:48:16 +01:00
Abdessamad Derraz fa43053362 chore: pin packed zips to the collection bytes 2026-10-23 08:27:16 +02:00
Abdessamad Derraz a0b27d612d fix: align tests with the ten-value pack result
Two test modules still unpacked nine values from verify_pack_against_platform. The category evidence joined readers with semicolons, which the prose rule refuses on generated pages, and three comments used clause semicolons.
2026-10-22 06:23:15 +02:00
Abdessamad Derraz 8b952968d6 feat: add a --no-game-data pack narrowing
The install manifest marks each game data file "category": "game_data":
a platform entry only game data profile entries read, or a core file its
profile declares game data. Firmware carries no category. Every manifest
says "categorized": true so the installer can refuse an older list.

generate_pack.py --no-game-data cuts those files from the platform list
and the core profiles before the build, names each one with the cores that
read it as game data, and tags the pack _NoGameData through _narrowings.
verify.py and the pack check take the same cut, so --verify-packs and the
report describe the pack the builder wrote. Modes without a platform
(--emulator, --system alone, --from-md5) and --all-variants refuse it.
2026-10-22 06:19:05 +02:00
Abdessamad Derraz f66adfdcbc feat: categorize missing platform entries
A platform list can name a game's data files or a file no core it runs opens. Verify gives each missing entry a category read from the profiles of the platform's cores: firmware, game data or unread. Only firmware gaps carry a severity and fail --strict; the others are counted on their own line, and the pack check no longer counts them as pack errors. Missing firmware stays a conformance error.
2026-10-22 05:38:57 +02:00
Abdessamad Derraz b7b407698f fix: count files the collection lacks apart
The pack check failed a pack for a declared file nothing in the collection resolves to, though the builder cannot ship it and verify already reports it missing. Such destinations are now counted as not in the collection; a held file absent from the pack stays an error.
2026-10-22 04:28:43 +02:00
Abdessamad Derraz 055627f39c chore: merge round 2 fixes (tooling) 2026-10-21 10:31:04 +02:00
Abdessamad Derraz 4fec09a014 fix: name the selection mode in --from-md5 packs
BizHawk is both a platform and a profile display name: --from-md5 with --platform bizhawk and with --emulator bizhawk wrote one file in two layouts. The emulator mode now carries _Emulator in the name.
2026-10-21 09:56:45 +02:00
Abdessamad Derraz a3e625cd6a fix: refuse a default value typed out as a flag
The refusal tables compared each flag with its default value, so --source full and --group-by system passed beside the modes that refuse every other value. Both now default to None and take their effective value after the checks, in generate_pack and pipeline.
2026-10-21 09:55:51 +02:00
Abdessamad Derraz d8e9ad23d3 docs: describe short md5 handling as the code does
No platform YAML holds an md5 shorter than 32 characters: the Batocera scraper completes one when a single collection md5 starts with it. The pages said some entries carry 29 characters and that scrapers store them as-is.
2026-10-21 09:54:57 +02:00
Abdessamad Derraz f957b5f893 fix: open no System.dat section libretro lacks
libretro titles each section with the system as it spells it,
"Commodore - Amiga". An addition for a system it has no section for
opened one under our slug: 21 sections came out as commodore-c64,
quake, laseractive or xrick, which no libretro DAT carries. An addition
now lands only in a section the original holds and is otherwise counted
as left out, so the export keeps the original's 66 sections. A platform
entry is always written, under its own section.
2026-10-20 17:21:16 +02:00
Abdessamad Derraz 71b59eb2a3 docs: rewrite half-dash asides in comments
Dropping the double-dash asides left 72 comments and docstrings with
half a dash glued to the next word ("shape -a size or a size range -is
all"). They now read as a colon, a comma or parentheses, and a prose
style test names any that comes back.
2026-10-20 09:17:10 +02:00
Abdessamad Derraz 16f6ef8cbf feat: record the commit a branch scrape read
Six platforms are read from a branch: System.dat, batocera-systems.json, checkBIOS.sh, the MiSTer BiosDB, the RetroPie scriptmodules and the RetroDECK component manifests. Their YAML named the branch, so the native export patched whatever the branch had become since the scrape. Each scraper now resolves the branch head once, reads every file at that commit and writes it into source. The export follows a recorded tarball and a recorded tree as it already followed a raw file URL.
2026-10-19 22:50:45 +02:00
Abdessamad Derraz 4231902a6c docs: write prose without dashes or semicolons
Profiles, the wiki, comments, docstrings, command output and schema descriptions carried em-dashes, clause-joining semicolons, unicode arrows, box-drawing banners and check-mark glyphs. Each sentence was rewritten rather than having its character swapped. Hashes, paths, source_ref values and every cited file:line are unchanged, which a comparison of each parsed profile against the previous revision confirms.

validate_pr comments now say PASS and FAIL in words, and source_ref values in a profile's structured blocks render as code. test_prose_style fails if any of these comes back, in tracked files or in the generated site.
2026-10-19 06:47:31 +02:00
Abdessamad Derraz d6d03617db fix: exit 1 when a hash lookup finds nothing
The documented example looked up a test fixture hash, printed NOT FOUND
and exited 0. The docs now use a hash the database holds and say which
line forms a hash file accepts.
2026-10-18 07:19:01 +02:00
Abdessamad Derraz a40a68c1f7 fix: group a system once whatever its spelling
The platform's files and the core extras for the Master System sat
in two region groups, sega-master-system and sega-mastersystem,
reduced apart. A US pack kept the declared Japanese BIOS. A core
extra's system group now takes the platform's spelling, in the pack,
the report and the installer's manifest fields alike.
2026-10-18 03:51:57 +02:00
Abdessamad Derraz 78902ccc6a fix: name the flag one-per-slot conflicts with
--platform --system --one-per-slot was refused with a message
asking for --platform, which the command carried. The refusal now
names --system or --emulator, and every refusal in the mode matrix
is checked never to ask for a flag it was given.
2026-10-18 02:52:02 +02:00
Abdessamad Derraz d7f382339c chore: name the platform core in export fixtures 2026-10-17 19:38:08 +02:00
Abdessamad Derraz af4b43d64c fix: tell a pack note by its name, not its path 2026-10-17 10:38:18 +02:00
Abdessamad Derraz 921de6a1a6 fix: refuse a truth pack a platform cannot fill 2026-10-17 10:31:02 +02:00
Abdessamad Derraz e09a37d21e feat: keep platform-built cores out of all_libretro
Recalbox compiles libretro/mame at 0.258 and 0.278 itself; typed
libretro, both profiles were resolved for every all_libretro platform
although the buildbot RetroArch installs from has neither. A profile
now names the platforms that build it (built_by), and is reached only
when a platform lists it by name.
2026-10-17 04:44:25 +02:00
Abdessamad Derraz 42950a553c feat: refresh data trees once per online pipeline
Step 2 refreshed the data directories, then the pack build and the
manifest build refreshed them again, so a tree tracking a moving
branch could differ between the pack and the manifest. generate_pack
gains --no-refresh-data, which packs the cached trees and still
fetches release assets; the online pipeline passes it to both builds.
2026-10-17 01:59:20 +02:00
Abdessamad Derraz 9966029cf7 fix: keep the ci test job off the network 2026-10-16 10:18:53 +02:00
Abdessamad Derraz fd8db894d8 fix: test extras grouping through the real chain 2026-10-10 12:16:11 +02:00
Abdessamad Derraz 6a2f9bf2e7 chore: read the three copy counts 2026-10-09 23:18:36 +02:00
Abdessamad Derraz 82bdcfa1b1 fix: read the clone map beside the scanned tree 2026-10-09 22:45:57 +02:00
Abdessamad Derraz 0c39fc479d fix: drop the --include-extras flag nothing read 2026-10-06 12:27:21 +02:00
Abdessamad Derraz c39588807f fix: refuse --include-archived without --all 2026-10-06 12:26:47 +02:00
Abdessamad Derraz 3183ad982c fix: read gaps by normalized system, sha256, file names 2026-10-06 09:32:32 +02:00
Abdessamad Derraz cdecb096a1 fix: refuse --refresh-data where nothing refreshes 2026-10-06 09:24:24 +02:00
Abdessamad Derraz 7b62f7f702 fix: refuse every flag a quick mode does not read 2026-10-06 04:39:08 +02:00
Abdessamad Derraz dbc8d63318 fix: one agnostic substitute rule for pack and verify 2026-10-06 04:19:40 +02:00
Abdessamad Derraz d6afb09663 refactor: one upstream name index for core lookups 2026-10-06 01:47:16 +02:00
Abdessamad Derraz 64f84e7ca6 fix: verify the pack the flags name, by exact name 2026-10-06 00:13:38 +02:00
Abdessamad Derraz 556e632efb fix: one held release fills a slot in emulator verify 2026-10-04 17:41:06 +02:00
Abdessamad Derraz 59ccb5b7d5 feat: scope a profile data directory to a build mode 2026-10-04 17:09:14 +02:00
Abdessamad Derraz 97f741bf0f fix: publish target lists as the builder resolves them 2026-10-04 16:55:30 +02:00
Abdessamad Derraz 887fc34bf6 feat: lay out standalone files by build mode 2026-10-04 16:35:59 +02:00
Abdessamad Derraz 23c4dc2aae fix: name every narrowed artifact from one list 2026-09-07 06:27:35 +02:00
Abdessamad Derraz 29fcfdef34 fix: refuse the flags a mode cannot apply 2026-09-07 06:14:24 +02:00
Abdessamad Derraz 2a4dd59bb2 fix: narrow pack conformance by hardware target 2026-09-06 23:28:27 +02:00