Commit Graph
385 Commits
Author SHA1 Message Date
Abdessamad Derraz 072380c268 feat: install the fewest files for a card's games
--for-games --required-only on a dArkOS card from a computer kept every
file of the list: --dest skipped the card's detection, so its rom
folders were never found. The games part beside the BIOS folder now
names them. The manifest marks a data tree every core runs without
(optional), and --required-only leaves it out: one GBA and one SNES game
took FBNeo's cheats and samples, 3,472 files no game opened.
2026-11-06 19:58:41 +01:00
Abdessamad Derraz de8f20f409 fix: count listed systems in the --system hint
A mistyped --system said '--list-systems lists all 579' and the listing
prints 532: the hint counted every spelling across profiles, and
system_index merges the ones that differ in case or separator before
printing. Both verify.py and generate_pack.py now pass that count to
suggest_names, whose listed parameter exists for this case.
2026-11-06 14:55:40 +01:00
Abdessamad Derraz c4ae8d9c6e fix: type pack helpers as ZipFile or PackRecord
Six helpers that write a pack member declared zipfile.ZipFile while every
call site also passes the PackRecord the pipeline builds emulator and
narrowed packs with. mypy reported the 19 call sites as arg-type errors.
With the union in the signature a helper that calls writestr or write
without its isinstance branch is a union-attr error in CI, instead of an
AttributeError in the pipeline's record pass.
2026-11-06 14:49:19 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz d0c65dca22 chore: merge pack and verify fixes 2026-11-06 10:28:27 +01:00
Abdessamad Derraz 462b61a4bc refactor: compute slot choices once per manifest
generate_manifest called slot_choices, and pack_overrides called it again with the same five arguments. pack_overrides takes the choices when the caller holds them. Measured on generate_pack.py --all --manifest --offline: 48 calls and 7.0 s in slot_choices before, 24 calls and 4.3 s after (wall 45.7 s and 42.4 s, one run each on a loaded host). The manifests written are identical.
2026-11-06 09:11:37 +01:00
Abdessamad Derraz 55e2c8328e feat: pack readme names spellings left out
A pack folded for Windows and macOS keeps one spelling of each case twin. Extracted on a Steam Deck or Linux, the core asking for the other spelling finds nothing, and only the installer added it. Where the platform also runs on a filesystem that keeps case, the README now lists each spelling left out beside the one it holds: 12 for EmuDeck and 10 for RetroArch, the manifest's case_variants.
2026-11-06 09:02:51 +01:00
Abdessamad Derraz 4efcb3da64 fix: group core files under their pack path
On a platform that writes bios/ in front of its destinations (RetroDECK, AmberELEC, EmuELEC), a core extra was grouped, dropped and held under its bare destination while the platform's declaration of the same file used bios/<name>. Beetle PSX's scph5501.bin was kept by the --region japan pack while the declaration was dropped and the installer dropped the path. extra_region_key keys every extra with the platform's declared root, in the builder, the manifest selection, verify's replay and the omissions. The same key gives the declared entries the systems of the core files at their path: every --system case of RetroDECK differed between pack and installer before, none does now.
2026-11-06 09:00:06 +01:00
Abdessamad Derraz 725325f0a1 fix: announce a data tree at its archive size
A tarball tree was announced at its extracted size: yaps2 at 6.5 MB for a 21.2 MB download, Cemu's game profiles at 9.9 KB for 21.4 MB. refresh_data_dirs records the size of each archive it downloads (a tree current from before is measured, not refetched), the install lists carry it as archive_size, and the plan, the menu and the data event count it.
2026-11-06 08:59:10 +01:00
Abdessamad Derraz 3e2a5279a1 fix: split parts hold what the full pack holds
Each part was built alone: the Other_Cores part never saw the platform's destinations and each part had its own case fold. RetroDECK's parts wrote bios/SGB1.sfc/ members beside the file bios/SGB1.sfc, RetroBat's wrote 21 openmsx/ twins of openMSX/ files, and a path two systems declare got each system's file. The full pack is now built in memory first and a part writes a declared file or core extra only where the full pack writes the same bytes.
2026-11-06 08:21:52 +01:00
Abdessamad Derraz c634a282d8 fix: a shipped twin path is untested, not excluded
A path declared twice, once unpinned and once with a pin the copy misses: the builder wrote excluded over the file it shipped, in either order of the two systems. It now counts the path untested, as verify does, and lists no exclusion.
2026-11-06 08:11:28 +01:00
Abdessamad Derraz 19898cb6ea chore: merge site and docs fixes, third pass 2026-11-06 06:38:47 +01:00
Abdessamad Derraz 83076818b0 chore: merge origin/main 2026-11-06 05:18:32 +01:00
Abdessamad Derraz 7a98d73d79 chore: merge engine fixes, third pass 2026-11-06 05:06:59 +01:00
Abdessamad Derraz 7f6ba6952c feat: verify reports the required-only pack 2026-11-06 03:45:07 +01:00
Abdessamad Derraz 1ed605969d fix: standalone writers take the repository lock 2026-11-06 03:42:09 +01:00
Abdessamad Derraz a2e8809945 fix: map a card's image name to its board
Batocera and KNULLI write the image name, not the board, to the card's board file (post-image-script.sh). The target scraper now records each board's images from Config.in, the target lists carry them, and the installer maps a card's image to its board. KNULLI's boot/knulli.board is read too.
2026-11-06 03:40:34 +01:00
Abdessamad Derraz 9b08925dd0 fix: finalize only the packs this run built 2026-11-06 03:30:47 +01:00
Abdessamad Derraz f96e7c10f4 fix: name a missing --from-md5-file 2026-11-06 03:27:29 +01:00
Abdessamad Derraz 7d33bd6e8e fix: count core data trees apart from baseline 2026-11-06 03:26:50 +01:00
Abdessamad Derraz 7b4c03afc2 docs: say what the archived status changes
The README, the setup pages and adding-a-platform tied archived to a
scraping schedule and a CI matrix that do not exist. Archived keeps a
platform out of the runs that cover every platform unless
--include-archived is passed.
2026-11-06 03:20:01 +01:00
Abdessamad Derraz 1c3462e7fd fix: name the python floor in generate_pack 2026-11-06 03:13:13 +01:00
Abdessamad Derraz 6acd48a344 chore: merge narrowed pack comparison 2026-11-06 01:37:34 +01:00
Abdessamad Derraz d265453b32 feat: compare narrowed packs with verify
Step 5 builds the --required-only, --no-game-data, --region, --target
and --system packs of every platform in memory and compares their paths
with verify run under the same narrowing. verify_platform gains
required_only, generate_pack a record mode. A run samples three regions,
targets and systems per platform (155 packs, about 3 minutes);
--full-narrowings compares all 1,150 (25 minutes). A test fails when
_narrowings() gains a dimension that is neither compared nor exempted.
2026-11-06 00:07:40 +01:00
Abdessamad Derraz 54a3876a8b feat: carry the data trees a platform's cores read
A platform pack carried only the trees its own systems declared, so for_platforms reached RetroArch, Lakka and RetroPie alone and every other platform running blueMSX, FBNeo or Dolphin got no tree. The pack, the install list and the pipeline's tree members now add the trees of the libretro cores the platform runs, below the system directory and build folder the core reads, unless the platform installs or never opens them. A platform entry the manifest resolved into a carried tree becomes that tree's member, which the installer can fetch: AmberELEC and EmuELEC no longer hold 192 blueMSX machine configs back. stonesoup reads its tree beside the content and pcee2 compiles its resources in, so neither tree reaches a BIOS pack.
2026-11-05 23:25:29 +01:00
Abdessamad Derraz 5749c66e32 fix: keep --system packs among the verified packs
The pack suffix test of b20d65d12 missed the platform --system packs, which carry their system after it (_BIOS_Pack_Playstation.zip): their manifest was no longer injected and install.py refused them. A pack name holds _BIOS_Pack, then an optional tag, then .zip.
2026-11-05 20:29:53 +01:00
Abdessamad Derraz b20d65d129 fix: verify only pack archives in the output dir
verify_and_finalize_packs took every ZIP of the directory, so the launcher archives build_port.py writes to the same dist/ were judged as packs, failed, and were rewritten with a manifest after build_port had printed their hashes. Every name the builder gives ends in _BIOS_Pack.zip, and only those are taken.
2026-11-05 19:38:21 +01:00
Abdessamad Derraz 0d21566183 fix: take an unchecked version as a match in packs
Raze lists three sized BLOOD.RFF versions and a fourth it takes without a check, the shareware RFF the collection holds. The Batocera pack printed two DISCREPANCY lines for it that verify never showed. A slot with an unchecked sibling now accepts any copy for that core, in the validation rules and in _matches_an_alternative. When every version refuses a copy, the message names all accepted values instead of the first.
2026-11-05 19:37:33 +01:00
Abdessamad Derraz 9ae3919d34 fix: spell cited commands so a paste runs them
Messages in check_freshness, generate_pack, common, profile_sync and fileless_audit named commands as export_native.py --platform x, which fails from the repository root. They now read python scripts/<name>.py like the rest of the report.
2026-11-05 19:19:04 +01:00
Abdessamad Derraz 46302de5bb chore: merge installer batch 2026-11-05 15:21:48 +01:00
Abdessamad Derraz ddafc2ff42 feat: carry the board list in the pack 2026-11-05 12:30:06 +01:00
Abdessamad Derraz f2e25aeea2 feat: list platforms with --list-platforms
install.py took --list-platforms, generate_pack.py --list and verify.py
nothing. Both tools now take --list-platforms and print the same list
from common.py. generate_pack.py keeps --list (finding 80).
2026-11-05 11:52:26 +01:00
Abdessamad Derraz 000877689f fix: name the split directory by the pack stem
The prefix was assembled by hand and kept a '/' or ':' the full pack
name makes safe (finding 17).
2026-11-05 11:50:44 +01:00
Abdessamad Derraz 1718813728 fix: skip a platform without the --system asked
--all --system built every pack asked for and exited 1 on a platform
that has no such system. The scope is now decided before the lock:
skipped under --all, refused under --platform (finding 15).
2026-11-05 11:49:41 +01:00
Abdessamad Derraz 7f31cf37cf fix: split only the systems the target runs
--split --target asked for a part per declared system, and each one
the target drops ended on a refusal naming it unknown (finding 14).
2026-11-05 11:47:37 +01:00
Abdessamad Derraz c7df790744 fix: one system scope for verify and the packs
A system the target leaves no core for was verified 0/0 with exit 0,
and generate_pack blamed the platform for lacking it. Both read
platform_scope_systems and system_scope_error now (findings 13, 14).
2026-11-05 11:47:03 +01:00
Abdessamad Derraz 9ae0ede21d fix: check --target before the output lock
A mistyped target waited on the dist lock and the data directory refresh
before it was refused (finding 62).
2026-11-05 11:44:22 +01:00
Abdessamad Derraz 8ef4eb47fc fix: refuse a target --verify-packs cannot honour
An unknown target, or one the platform does not publish, was dropped and
the full pack verified under it with rc 0 (finding 34).
2026-11-05 11:43:53 +01:00
Abdessamad Derraz 25c356669b fix: record each platform's own list version
install/<platform>.json wrote the constant "1.0" as the version of the
list it was built from. The manifest now carries the version the platform
YAML records, and generate_pack refuses a YAML without one.

inherits: no longer passes a parent's version to a child: Lakka was
published as RetroArch v1.22.2 in its manifest, pack name and site page.
lakka.yml records v6.1, the Lakka-LibreELEC tag its target list is read
at. The MiSTer scraper refuses a BiosDB without a timestamp instead of
writing an empty version.

A pack's manifest.json records list_version, so an install from the pack
alone knows it too. Manifests regenerated twice and compared: only
version and generated changed.
2026-10-27 19:10:39 +01:00
Abdessamad Derraz 0fa4552dae fix: refuse only the axis a list lacks
The MinUI family's lists record cores and regions and no system: every
narrowing was refused with a remedy that fetched the same list. Each
option is now refused by the axis missing, with what works, and a pack
names its platform's list only when that list records the axis. Line
mode names what c chooses and skips a question with one answer, and
Options leaves out parallel downloads when nothing is downloaded.
2026-10-27 15:37:24 +01:00
Abdessamad Derraz 31c574aab0 fix: line mode and screen share one decision
Answers code_audit_17 and tui_audit_10: the folder found after o, SHARE
drive roots, one same-choice command, a counted confirm step in line
mode, counts from the narrowed list, read-only advice without sudo,
systems recorded in packs, Ctrl-Z stopping the whole job, and the
wording, layout and docs items.
2026-10-27 12:24:26 +01:00
Abdessamad Derraz 1998580772 fix: start each pack run with no stale notices
_MISSING_DATA_DIRS is module state: a test that called main() after one
whose data directories were not cached exited 1 with the earlier run's
notice. main() now clears it first.
2026-10-24 02:34:47 +02:00
Abdessamad Derraz eeef4fc6fb feat: packs for the most downloaded setups 2026-10-23 17:18:00 +02:00
Abdessamad Derraz 32c648b67d feat: release packs for marked platforms only
release_pack: true in the registry is the list a release builds: RetroArch,
EmuDeck, RetroBat, Batocera, RomM and MiSTer, 83% of the downloads of
v2026.10.10. generate_pack --all, --verify-packs, the pipeline consistency
step, release_record and check_release read it. Manifests are still written
for every platform.
2026-10-23 17:06:46 +02:00
Abdessamad Derraz 4a1d875f43 feat: name systems in the install manifests
Each manifest carries system_titles, the name the site prints for every system its files list (Sony PlayStation, not sony-playstation or psx), so the installer can show it. Older installers ignore the field.
2026-10-23 15:55:46 +02:00
Abdessamad Derraz fed03c3db7 fix: pack each zip with the collection's bytes 2026-10-23 07:47:35 +02:00
Abdessamad Derraz bf22f1da47 fix: keep Batocera's save files apart from bios/
batocera-systems lists bios/saves/dolphin-emu/Triforce/segaboot.gcm, so the pack folder for the save directory is userdata-saves/. validate_schemas refuses a pack folder the BIOS tree already uses.
2026-10-22 21:44:14 +02:00
Abdessamad Derraz 2d26385efe feat: place save-dir files in each system folder
Batocera, KNULLI, Recalbox, RetroBat and RetroDECK hand a libretro core <saves>/<system> at each launch. Each copy lands below every system that runs the core. EmuDeck keeps one unsorted saves/retroarch/saves. The unsupported platforms record why.
2026-10-22 21:16:59 +02:00
Abdessamad Derraz 1cd6c9ee4d feat: place save-dir files in packs and lists
load_from: save_dir entries of a libretro build were left out of every
pack. They now go below the platform's save folder (registry
save_directory: RetroArch saves/, Lakka savefiles/, by core), the
manifest marks them root: save, and verify replays the same placement.
Platforms keeping saves per system or per game list them as omitted.
2026-10-22 19:58:26 +02:00
Abdessamad Derraz 8b952968d6 feat: add a --no-game-data pack narrowing
The install manifest marks each game data file "category": "game_data":
a platform entry only game data profile entries read, or a core file its
profile declares game data. Firmware carries no category. Every manifest
says "categorized": true so the installer can refuse an older list.

generate_pack.py --no-game-data cuts those files from the platform list
and the core profiles before the build, names each one with the cores that
read it as game data, and tags the pack _NoGameData through _narrowings.
verify.py and the pack check take the same cut, so --verify-packs and the
report describe the pack the builder wrote. Modes without a platform
(--emulator, --system alone, --from-md5) and --all-variants refuse it.
2026-10-22 06:19:05 +02:00