Commit Graph
122 Commits
Author SHA1 Message Date
Abdessamad Derraz ae3dc63bc3 fix: accept RomM files on size and any hash
RomM's verify_file_hashes compares the size, then md5, sha1 or crc. We
compared md5 alone, so a fixture entry with one mistyped hash was left out
of the pack RomM would accept it from. The rule is declared in romm.yml
and read by verify, the builder and the manifest.
2026-10-18 07:24:33 +02:00
Abdessamad Derraz c3162dcd27 feat: scrape Lakka per-device core lists
Lakka builds LIBRETRO_CORES minus the exclusions each project, device
and GLES image adds. Without a list, --target refused Lakka and its RPi1
pack carried firmware for cores the image does not build.
2026-10-18 06:24:01 +02:00
Abdessamad Derraz e1b6aad51b fix: refuse abbreviated flags in every script
argparse completed a prefix into the only flag starting with it:
slots.py --emulator read profiles from a directory named after the
emulator and reported nothing, and pipeline.py --all built every pack
variant. Every parser now refuses an abbreviation.
2026-10-18 05:30:58 +02:00
Abdessamad Derraz e04646d67a fix: stop the MAME scrape on a clone with no pin
A failed rev-parse in the MAME clone returned an empty commit, which
was cached for a day and let the merge advance the profile's version
and refs while its upstream pin stayed on the previous release. The
scrape now stops instead.
2026-10-18 05:15:52 +02:00
Abdessamad Derraz d5f40a4515 chore: drop the shared groups no platform reads
Six of the eight shared groups injected nothing: three were
included by no platform, three repeated System.dat entry for entry,
so a correction made in them reached no pack. They are removed with
their scraper wiring, and a test requires every remaining group to
contribute an entry somewhere.
2026-10-18 04:03:01 +02:00
Abdessamad Derraz e4960dd4f8 fix: declare EmuDeck's emulators as standalone 2026-10-18 01:19:52 +02:00
Abdessamad Derraz faa8af0800 refactor: annotate every remaining signature 2026-10-18 00:44:20 +02:00
Abdessamad Derraz 27d119cb7f fix: scrape targets at the pinned platform release 2026-10-18 00:01:36 +02:00
Abdessamad Derraz a8bfadd4c1 feat: list recalbox cores per board
Recalbox builds seven board images with different emulators. The new
target scraper reads each board's defconfig against the package every
system.ini core names. The names that reached no profile are mapped to
the projects Recalbox builds or profiled from their source, and the
mupen64plus video plugins are set aside.
2026-10-17 15:51:39 +02:00
Abdessamad Derraz d9c74888e8 feat: list rocknix cores per device
ROCKNIX builds a different emulator set per SoC, and without a target
list an H700 handheld received the PS2, PS3 and GameCube firmware of
emulators its image lacks. The new target scraper reads the emulators
metapackage per device; seven cores ROCKNIX ships get profiles and the
Super Famicom Box ROMs one of them needs are collected.
2026-10-17 15:38:07 +02:00
Abdessamad Derraz 2de8e11489 feat: list retroarch's linux arm64 and x86 targets 2026-10-17 13:41:10 +02:00
Abdessamad Derraz dbcd26d3ab fix: file every platform system under a known id
Thirty platform system ids, such as sufami, pokemini, vsmile, and
Recalbox's comma-joined msx1,msx2, matched no system any profile
names, so no core file was attributed to them and --system missed them.
The scraper maps now carry those names, and a test holds every platform
system to a profile system apart from two named folds.
2026-10-17 12:47:22 +02:00
Abdessamad Derraz 0894b67d3f feat: read recalbox system files in batches 2026-10-17 12:46:59 +02:00
Abdessamad Derraz 4544006aff feat: wait out a forge rate limit while scraping 2026-10-17 12:34:12 +02:00
Abdessamad Derraz ae0a918f74 fix: evaluate batocera's kconfig per board
The target scraper treated every select in Config.in as unconditional
and never set the symbols Buildroot sets, so nearly every board listed
the same 171 cores and xroar, which needs OpenGL, reached none. Selects
now fire from their own config, to a fixed point, from the flags each
board file sets, with Buildroot's arch and OpenGL provider rules read at
the commit Batocera pins. A board that builds no image is no target.
2026-10-17 12:33:44 +02:00
Abdessamad Derraz c2ea021699 fix: file retrobat systems with batocera's slugs 2026-10-17 12:03:54 +02:00
Abdessamad Derraz e540dac996 fix: give one native system id one slug
RetroDECK filed coco as tandy-coco and Dragon as dragon-32-64, MiSTer
filed COCO3 as tandy-coco and Recalbox kept its raw ids, so none of
them met xroar's coco, coco3 and dragon64 and its ROMs reached no
system. The scraper maps now agree and a test holds every shared
native id to one system.
2026-10-17 11:59:38 +02:00
Abdessamad Derraz 83d8f62f68 fix: map bizhawk's ares to its n64 core only
BizHawk embeds ares as Ares64, its N64 core, whose 64DD IPL its own
FirmwareDatabase declares. Mapped to the whole ares profile, it pulled
thirty systems into the BizHawk pack and made slots accuse
FirmwareDatabase.cs of a wrong MSX.rom it never declared for ares.
2026-10-17 10:39:27 +02:00
Abdessamad Derraz 3395236966 feat: pack the standalone emulators rocknix ships
ROCKNIX was declared libretro-only although it packages duckstation,
dolphin, ppsspp and 33 other emulators standalone, and its BIOS list
was read on the next branch under the latest release's version. The
scraper now reads the release tag and lists RetroArch plus every
standalone package.
2026-10-17 04:45:13 +02:00
Abdessamad Derraz e09a37d21e feat: keep platform-built cores out of all_libretro
Recalbox compiles libretro/mame at 0.258 and 0.278 itself; typed
libretro, both profiles were resolved for every all_libretro platform
although the buildbot RetroArch installs from has neither. A profile
now names the platforms that build it (built_by), and is reached only
when a platform lists it by name.
2026-10-17 04:44:25 +02:00
Abdessamad Derraz d9ef6ad0ad feat: read retrobat's cores from its es_systems
RetroBat's platform file carried no core list, and a registry list
copied from Batocera stood in with ports RetroBat never runs and none
of its standalone emulators marked as such. The scraper now reads
es_systems.cfg at the release tag: libretro cores, and every other
emulator as standalone.
2026-10-17 03:40:09 +02:00
Abdessamad Derraz 6280f4c370 feat: read recalbox's cores from its system files
es_bios.xml names only the cores that need a BIOS, so a hand-kept
registry list stood in, with mame and mamemess that Recalbox does not
ship and without vice, ymir, vircon32 and others it does. The scraper
now reads each system.ini of recalbox-romfs2 at the scraped tag, and
the registry list is gone.
2026-10-17 03:31:11 +02:00
Abdessamad Derraz a4da6d205f feat: profile the MAME releases recalbox builds
Recalbox ships libretro/mame built at 0.258 and at 0.278, and no
profile described either release. mame_hash_scraper gains --release,
which reads the BIOS root sets at one mamedev tag and writes a profile
pinned to it; mame0258 and mame0278 are generated from it, every
source_ref anchored at its tag.
2026-10-17 03:31:11 +02:00
Abdessamad Derraz decd70b19e feat: pack every core batocera offers in es
Batocera's cores were read from configgen-defaults.yml, one per system,
and a hand-kept registry list overrode even that; DOSBox Staging,
melonDS DS and nine other cores a user can pick in ES were outside the
pack, while mamemess, which Batocera does not ship, was in. The
scraper now reads es_systems.yml, the registry list is gone, and the
target scraper names standalone emulators (amiberry, fsuae, vice)
instead of only the machines listed under them.
2026-10-17 02:26:14 +02:00
Abdessamad Derraz d4e2e5ac8a fix: skip hash merge backups in every loader 2026-10-16 18:09:45 +02:00
Abdessamad Derraz 4dc6af5a89 fix: send the token from the tag helper 2026-10-16 16:58:26 +02:00
Abdessamad Derraz 0198445ee4 fix: stop the core-info scrape on a failed fetch 2026-10-16 15:56:50 +02:00
Abdessamad Derraz 6a39264fa6 fix: send the github token on every api call 2026-10-10 12:17:21 +02:00
Abdessamad Derraz 88d7ca4814 fix: stop a retrodeck scrape on a failed fetch 2026-10-10 12:08:52 +02:00
Abdessamad Derraz e26ee44229 fix: export only what the platform declares 2026-10-10 11:15:25 +02:00
Abdessamad Derraz a00f57ad37 fix: write scraped files whole or not at all 2026-10-10 04:45:10 +02:00
Abdessamad Derraz a116743471 fix: scrape the mame release that labels the data 2026-10-10 04:40:27 +02:00
Abdessamad Derraz d8ad89902b refactor: split helpers out of grown functions 2026-10-06 13:59:50 +02:00
Abdessamad Derraz 8bf8fdba9e fix: refuse a libretro scrape without core info 2026-10-06 12:23:48 +02:00
Abdessamad Derraz 1e61b749cf fix: refuse an emudeck scrape with no cores 2026-10-06 12:23:48 +02:00
Abdessamad Derraz 0eecb23724 fix: refuse batocera targets with unread boards 2026-10-06 12:22:51 +02:00
Abdessamad Derraz 548ea5935b fix: stop a scrape whose release tag is unread 2026-10-06 12:22:51 +02:00
Abdessamad Derraz f3d01aeeeb refactor: split this round's fixes into small helpers 2026-10-06 11:16:06 +02:00
Abdessamad Derraz a70ff264d2 fix: refuse an unknown mame or fbneo version 2026-10-06 09:42:33 +02:00
Abdessamad Derraz 6b57377111 fix: name each retropie package once per target 2026-10-06 05:33:18 +02:00
Abdessamad Derraz 240a30f1d7 fix: read retropie targets the way retropie does 2026-10-06 05:22:34 +02:00
Abdessamad Derraz 29a285d37e fix: stop target scrapes on a failed request 2026-10-06 05:19:05 +02:00
Abdessamad Derraz 077351bab3 feat: list retrodeck standalone emulators 2026-10-06 01:43:30 +02:00
Abdessamad Derraz 6d6cd7d668 fix: map pc-8801, dsi and cd-i system slugs 2026-10-06 01:22:56 +02:00
Abdessamad Derraz 6453a96d47 feat: read retropie version from its setup script 2026-10-04 17:47:51 +02:00
Abdessamad Derraz 1a993edc0d feat: record upstream dat blobs in recipe snapshots 2026-10-04 16:35:59 +02:00
Abdessamad Derraz 887fc34bf6 feat: lay out standalone files by build mode 2026-10-04 16:35:59 +02:00
Abdessamad Derraz 3e6f30c697 fix: send github token to scraper api calls 2026-10-04 15:06:00 +02:00
Abdessamad Derraz 08ef1e8dcf fix: fail batocera target scraper on empty input 2026-10-04 14:36:32 +02:00
Abdessamad Derraz 691ccbfca7 feat: make the native export round-trip a platform file 2026-09-05 17:32:48 +02:00