Commit Graph
257 Commits
Author SHA1 Message Date
Abdessamad Derraz b270a24d7b feat: take a platform's own libretro builds
A list naming retroarch now takes the libretro profiles whose built_by names the platform: ROCKNIX ships yabadabadoo with its RetroArch and named it nowhere, so its files never reached the ROCKNIX pack. Only change across the registry: rocknix + yabadabadoo. Also quotes the installer's current unproven-drive message.
2026-11-09 17:18:44 +01:00
Abdessamad Derraz 6c14468862 feat: say the installer and setup pages plainly
- The installer speaks without "you": the first screen asks "Where
  to install?", rows read "Copy the BIOS files from an old card or a
  folder", a close match reads "Close matches: ...". The handheld
  launchers and their port text follow, and the pronoun guard no
  longer lists any installer message.
- A setup that runs as an app opens on its one-line command, the
  folder table after it; a setup that runs on its own device keeps
  the network folder first, which its ZIP route needs.
- The home steps say what to do: "Its page gives one line to copy",
  "Paste it, press Enter".
2026-11-09 16:33:56 +01:00
Abdessamad Derraz 6cc646acd6 fix: type errors the merged batches brought
The name step reused one variable for paths and for (path, md5, size)
tuples, the standalone copies reused targets inside its own loop, and
the WSL state read as object. Ten baseline entries no longer occur.
2026-11-07 00:55:46 +01:00
Abdessamad Derraz 84cffe3f8d chore: merge origin/main 2026-11-06 16:42:16 +01:00
Abdessamad Derraz 537f814d4a refactor: one refusal for profile-blind narrowings
generate_pack and verify each held a copy of the refusal added for MiSTer
and BizHawk. without_profile_blind in common.py is the one both read.
2026-11-06 15:23:13 +01:00
Abdessamad Derraz ed6643022d fix: write the cut md5 rule once
Six places read an md5 shorter than 32 digits, each with its own bound.
digest_verdict accepted any prefix down to one digit, while Batocera
compares the whole string (batocera-systems:1088). A cut value now names
a dump only when one held md5 starts with it (cut_md5_owner, read by the
scraper, the resolver and --from-md5), and every verdict compares whole
digests.
2026-11-06 15:18:38 +01:00
Abdessamad Derraz 1b4b31fb32 fix: run the read-only tools from the repo root
Only verify.py and generate_pack.py moved to the repository root. From
any other folder slots, region_audit, provenance_report and bundled_only
said database.json was missing and to rebuild it, identity and
list_platforms named a missing folder, and game_rules ended on a
FileNotFoundError traceback. work_from_repo_root moves to common.py and
the eight tools call it: defaults are read from the root, a path the
user typed from where it was typed.
2026-11-06 15:09:35 +01:00
Abdessamad Derraz 1fe37d16ac fix: keep stderr after the report it closes
verify.py --emulator fbneo --strict > log 2>&1 put the ERROR verdict on
the first line of the log: redirected, stdout is block-buffered and
stderr is not, and run_cli flushed stdout only after main returned.
run_cli now sets stdout to line buffering, so every tool it runs keeps
the two streams in the order they were written.
2026-11-06 15:06:26 +01:00
Abdessamad Derraz 1834d20dc9 fix: name the file a YAML syntax error is in
Every profile is loaded whatever the command, so one misindented key in
an unrelated profile stopped verify.py --emulator handy and generate_pack
on a 37-line yaml traceback. The sources common.py reads (profiles,
platform files, registry, shared groups, targets) now raise
SourceUnreadable, which run_cli prints on one line with the file, the
line and the validate_schemas command. It is not a FileNotFoundError:
ten callers skip a platform whose file is absent, and a damaged one must
not be skipped the same way.
2026-11-06 15:02:01 +01:00
Abdessamad Derraz 6bf62d5746 fix: refuse profile narrowings without cores
MiSTer and BizHawk declare cores: []. --region, --one-per-slot and
--no-game-data read the profiles of the cores a platform runs, so their
packs and manifests withdrew nothing under a name and README that said
they had. generate_pack and verify now refuse the flags for a named
platform and leave it out under --all.
2026-11-06 15:01:58 +01:00
Abdessamad Derraz 13a10da993 fix: serve the dump a name's hashes prove
dsi_bios7.bin and dsi_bios9.bin hold the DSi Panda devkit boot ROMs, eMMC
key-Y zeroed. Every md5 and crc32 declared under those names designates
a retail dump in the same directory, yet hashless entries (RetroBat,
dsperate) got the devkit ROMs by name. by_proven_name maps such a name to
the proven primaries beside it. kick13.rom, dsi_firmware.bin and
spectrum.zip follow the same rule.
2026-11-06 15:00:28 +01:00
Abdessamad Derraz 8b55ab45a5 fix: suggest the database beside a mistyped --db
--db databse.json, with database.json in the same folder, was told to run
generate_db --force --output databse.json: a rescan of the whole
collection into a second file under the typo. load_database now names the
closest .json in that folder first, and gives the rebuild command only
when none is close.
2026-11-06 14:57:48 +01:00
Abdessamad Derraz 82bccd9dae fix: refuse Python 3.8 in every build script
Only generate_pack.py and pipeline.py checked the interpreter. verify.py,
slots.py, validate_pr.py, cross_reference.py and the probe tools started
under 3.8 and stopped halfway on a str.removeprefix AttributeError. The
check now lives in common.py, which every build script imports, and in
core_probe.py and region_check.py, which do not. generate_site.py imports
common before siterender, which raised a TypeError at load under 3.8.
2026-11-06 14:52:50 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz d0c65dca22 chore: merge pack and verify fixes 2026-11-06 10:28:27 +01:00
Abdessamad Derraz 9633ce3dd4 refactor: normalize each system id once
SelectionIndex.build and system_group called _norm_system_id 1.7 million times per manifest run for about a thousand ids, each through the manufacturer prefix loop. The function reads only its argument and two constant tables, so its answer is cached. Measured on the call volume of one manifest run (630 ids, 1,686,618 calls): 1.53 s uncached, 0.07 s cached. The end-to-end manifest run moves inside its own noise on this host (45.9 to 50.6 s across four runs).
2026-11-06 09:08:09 +01:00
Abdessamad Derraz a1ddee8e5e fix: a lock reader refuses a missing directory
artifact_lock created its directory even for a shared reader, so a
mistyped path given to --verify-packs or release_record was left as a
directory holding a .lock, and split_pack created the parent before
printing a bare errno. A reader now raises ArtifactDirectoryMissing,
which run_cli prints on one line, and split_pack checks its target
before taking the lock.
2026-11-06 08:54:06 +01:00
Abdessamad Derraz df801fd311 chore: merge resolver and export fixes 2026-11-06 05:54:29 +01:00
Abdessamad Derraz 7f6ba6952c feat: verify reports the required-only pack 2026-11-06 03:45:07 +01:00
Abdessamad Derraz 1ed605969d fix: standalone writers take the repository lock 2026-11-06 03:42:09 +01:00
Abdessamad Derraz af3c90e448 fix: unwind a terminated run to drop its scratch 2026-11-06 03:31:28 +01:00
Abdessamad Derraz 27991c84ba fix: refuse a missing platforms directory 2026-11-06 03:25:20 +01:00
Abdessamad Derraz 6445e5d7d6 fix: refuse a missing emulator profile directory 2026-11-06 03:21:53 +01:00
Abdessamad Derraz 402e688b01 fix: platform entries skip other emulators' trees
A platform entry names no profile, so bios/Other/<emulator>/ was never foreign to it. EmuELEC's ports/VVVVVV/data.zip resolved to xrick's data.zip, then to xrick's data directory copy. A system that lists its cores now takes no name match only another emulator's tree answers.
2026-11-06 03:21:07 +01:00
Abdessamad Derraz b274e2303d fix: annotate the five affinity sort keys 2026-11-06 03:12:15 +01:00
Abdessamad Derraz 172639d0dd chore: merge demul profile 2026-11-06 01:01:37 +01:00
Abdessamad Derraz 755259517f feat: assemble the naomi.zip demul reads
Demul 0.7 reads 44 NAOMI BIOS rows from naomi.zip, the HOTD2, F355 and Airline Pilots BIOS among them. No copy held them all. The set is built with build_torrentzip from members already in the collection, each checked against the crc32 and sha1 of demul.exe's table, and kept in bios/Other/demul/. Batocera, KNULLI and RetroBat place it at bios/naomi.zip, the file MAME cores there also open, so it carries the nine MAME members the Demul table does not list.

The resolver left another emulator's own copy out of the member and superset ranking, so Flycast and the MAME cores keep the copies they were served.
2026-11-05 23:37:06 +01:00
Abdessamad Derraz 55f7ed24c2 fix: rank archives by name on a shared crc32
segadimm.zip holds fpr23905.ic36 and fpr23905c.ic36 under the same crc32, and segasp.zip four fpr-*.ic72 images. Ranked by crc32 alone, a copy missing some of them tied with the full set and the index order chose it. A crc32 that two declared members share now also requires the member name.
2026-11-05 23:25:21 +01:00
Abdessamad Derraz d1ebd411ab fix: narrow hashless name matches to their maker
AmberELEC's nec-pc-88 declares DISK.ROM with an empty md5. QUASI88 reads
that name as n88sub.rom, the 2 KB PC-88 disk ROM (libretro.c:1085), but
the name step served the 16 KB MSX disk ROM, the only primary of that
name. The quasi88 alias never reached by_name, which refuses an alias a
primary already carries.

load_platform_config now writes each entry's system as platform_system.
A platform entry with no hash and no owning emulator, whose system id
names a maker, takes no name candidate filed under another maker, in
the name step and in the hash_mismatch step after it. When nothing is
left, by_shadowed_alias, the profile aliases by_name refuses, answers
within the system's maker folder. names_manufacturer and BUCKET_FOLDERS
move to common.py, and generate_site imports them.

Across every registered platform the only resolution that moves is
AmberELEC nec-pc-88 DISK.ROM, from bios/Microsoft/MSX/DISK.ROM to
bios/NEC/PC-88/N88SUB.ROM. Manifests, truth and exports are otherwise
byte-identical.
2026-11-05 23:17:04 +01:00
Abdessamad Derraz afd062d111 chore: merge resolver and profiles batch 2026-11-05 21:47:10 +01:00
Abdessamad Derraz da19bb5755 fix: keep MinUI core dirs out of the tree view
Without --platform, cross_reference and the site's gap report merged every
platform's standalone set, MinUI, NextUI and MyMinUI coredir: markers
included. A marker says which cores one platform hands a folder, so in the
union every core those three do not list got no layout: px68k showed 0 of
6 files and the global report lost 238 profiles and 1201 files.
merged_standalone_set drops the markers, and both readers use it.
2026-11-05 19:39:07 +01:00
Abdessamad Derraz fb9b5592a5 feat: let --emulator take an alias name
--emulator mednafen_psx refused with 'use --emulator beetle_psx' although the profile names its target. The alias now selects that profile and says so on stderr, in verify, generate_pack and cross_reference alike. Launchers are still refused, and pipeline step 5 skips aliases since they compare under the profile they name.
2026-11-05 19:26:36 +01:00
Abdessamad Derraz 3b0112001e fix: serve a data dir file at its platform path
AmberELEC and EmuELEC declare blueMSX's Machines/<model>/config.ini and
Shared Roms/RS232.ROM with no hash. The name step served msx-emu's
ColecoVision config.ini in 192 machine folders and the 16 KB RS232.ROM,
while data/bluemsx held blueMSX's own file at that very path. A platform
declaration with no hash now reads its destination's tail in the data
directories before the name step. Profile entries keep the foreign-tree
redirect, so Dolphin's dsp_coef.bin dump still outranks the free copy.

Manifests: amberelec and emuelec omit 192 config.ini entries (held in
data/bluemsx only, so the installer has no URL) and serve colecodb.xml,
MSX2+/config.ini, MSXturboR/config.ini and RS232.ROM from blueMSX's bytes.
2026-11-05 19:21:26 +01:00
Abdessamad Derraz 4b54b82966 fix: suggest the target when no platform has it
verify --all --target swich ended on 'no platform has it' after seventeen INFO lines, two of which held the suggestion. The refusal now names the closest targets across every platform with the platforms that have them, and the per-platform lines print only when some platform keeps the target.
2026-11-05 19:21:23 +01:00
Abdessamad Derraz 9ae3919d34 fix: spell cited commands so a paste runs them
Messages in check_freshness, generate_pack, common, profile_sync and fileless_audit named commands as export_native.py --platform x, which fails from the repository root. They now read python scripts/<name>.py like the rest of the report.
2026-11-05 19:19:04 +01:00
Abdessamad Derraz e50eeb52fd chore: merge audit batch, resolver and profiles 2026-11-05 13:16:03 +01:00
Abdessamad Derraz f2e25aeea2 feat: list platforms with --list-platforms
install.py took --list-platforms, generate_pack.py --list and verify.py
nothing. Both tools now take --list-platforms and print the same list
from common.py. generate_pack.py keeps --list (finding 80).
2026-11-05 11:52:26 +01:00
Abdessamad Derraz c7df790744 fix: one system scope for verify and the packs
A system the target leaves no core for was verified 0/0 with exit 0,
and generate_pack blamed the platform for lacking it. Both read
platform_scope_systems and system_scope_error now (findings 13, 14).
2026-11-05 11:47:03 +01:00
Abdessamad Derraz cfc2515909 fix: check the zipped_file member before composite
A one-member archive whose member carries the zipped_file md5 under
another name has that md5 as its composite. The name step returned it as
md5_composite_exact, which slots and the site take as content proof, while
Batocera looks the member up by name and finds nothing.
2026-11-05 11:45:58 +01:00
Abdessamad Derraz 609710e2d7 fix: say a platform has no hardware targets
The error named platforms/targets/<name>.yml, which read as a file
missing from the repository (finding 63).
2026-11-05 11:42:46 +01:00
Abdessamad Derraz 08cf4dd914 fix: skip other casings in a foreign tree
reminiscence's Flashback module reunion resolved to skyline's
zoneinfo/Indian/Reunion through the case-insensitive name step, and the
emulator pack shipped the time-zone file. Another casing found only in
another emulator's tree says nothing about the entry.
2026-11-05 11:42:29 +01:00
Abdessamad Derraz c2ad67e4fa feat: publish the games index with install lists 2026-11-03 17:45:03 +01:00
Abdessamad Derraz 1aa36258b2 feat: scrape the rom folders of each setup 2026-11-03 17:37:33 +01:00
Abdessamad Derraz a0c560479b fix: join platform system names to profiles
The rom folder survey joined 463 folders to their default core's whole
profile, no system of it matching. SYSTEM_ALIASES, the one table
_norm_system_id reads, now folds ten more spellings of one machine:
apple2gs, astrocade, astrocde, creativision, gamepocketcomputer,
nintendo-super-game-boy, cps1, cps2, cps3 and pgm. The profiles that
used them take the canonical id (mame, groovymame, mamemess, mame0258,
mame0278, bsnes, mesen-s, mesence, jollycv, fbalpha2012 and its three
CPS cores, fbneo_cps12), as test_system_ids requires.

With bsnes and bsnes-hd now filing SGB1.sfc under one system, their
claims fold onto one report entry. _names_content read only the first
claim, hashless, and nSide's SGB1.sfc/ tree took the path before the
file. A hash on any folded claim now counts: every pack keeps the
destinations it had.

The site drops the eight display names the canonical ids replace and
names tvgames. README counts 533 systems instead of 541.
2026-11-03 15:23:30 +01:00
Abdessamad Derraz 25c356669b fix: record each platform's own list version
install/<platform>.json wrote the constant "1.0" as the version of the
list it was built from. The manifest now carries the version the platform
YAML records, and generate_pack refuses a YAML without one.

inherits: no longer passes a parent's version to a child: Lakka was
published as RetroArch v1.22.2 in its manifest, pack name and site page.
lakka.yml records v6.1, the Lakka-LibreELEC tag its target list is read
at. The MiSTer scraper refuses a BiosDB without a timestamp instead of
writing an empty version.

A pack's manifest.json records list_version, so an install from the pack
alone knows it too. Manifests regenerated twice and compared: only
version and generated changed.
2026-10-27 19:10:39 +01:00
Abdessamad Derraz 32c648b67d feat: release packs for marked platforms only
release_pack: true in the registry is the list a release builds: RetroArch,
EmuDeck, RetroBat, Batocera, RomM and MiSTer, 83% of the downloads of
v2026.10.10. generate_pack --all, --verify-packs, the pipeline consistency
step, release_record and check_release read it. Manifests are still written
for every platform.
2026-10-23 17:06:46 +02:00
Abdessamad Derraz aeb68fe753 fix: credit readers the file their pack resolves
The name step tried the destination's basename alongside the entry's
own names. With dsi_nand.bin absent from a checkout, nand.bin from
dsi/nand.bin let the Wii NAND outrank the entry's own DSi variant, and
the site named melonDS Nightly a reader of it. The basename is now tried
only when no own name answers.

FileConsumers also resolved profile entries without the archive members
the emulator pack passes, so gngeopi and neo-emu were credited with the
MAME neogeo.zip instead of the archive their pack carries.
2026-10-23 13:03:39 +02:00
Abdessamad Derraz cea5a3d0b2 fix: count the Marathon data trees as game data
bios/Bungie holds the Marathon scenarios Aleph One reads, entries its profile files as category game_data. Listed with the other publishers' trees, they leave the system files count and sit under Game engines and PC games on the site.
2026-10-22 11:54:03 +02:00
Abdessamad Derraz 841017c887 chore: merge the minui family platforms 2026-10-22 04:09:17 +02:00
Abdessamad Derraz d0802c5360 feat: place core files per handed system dir
A platform can hand each libretro core a system directory of its own
(core_system_dirs). platform_layouts then gives one view per directory,
so the gap report, the builder, the manifest, slots and cross_reference
place a core's files below each folder it is handed, and a core handed
none gets no file. Resolution strips the folder again and searches the
collection by the core's own path.
2026-10-22 04:04:31 +02:00