Commit Graph
215 Commits
Author SHA1 Message Date
Abdessamad Derraz ed6643022d fix: write the cut md5 rule once
Six places read an md5 shorter than 32 digits, each with its own bound.
digest_verdict accepted any prefix down to one digit, while Batocera
compares the whole string (batocera-systems:1088). A cut value now names
a dump only when one held md5 starts with it (cut_md5_owner, read by the
scraper, the resolver and --from-md5), and every verdict compares whole
digests.
2026-11-06 15:18:38 +01:00
Abdessamad Derraz 8100d671f9 chore: merge origin/main 2026-11-06 13:02:20 +01:00
Abdessamad Derraz f55e473183 chore: one clause per sentence in a comment 2026-11-06 12:55:13 +01:00
Abdessamad Derraz 5e90a455e4 feat: add EmuELEC hardware targets
The v4.8 release publishes four images (Amlogic-ng, GameForce,
OdroidGoAdvance, OdroidM1, read from its asset names). The metapackage
(packages/sx05re/emuelec/package.mk:8-77) builds the S922X cores, dolphinSA
and mame only on Amlogic-ng, Amlogic-no, RK356x and OdroidM1, and removes
duckstation on the handhelds with a sed that replaces the first match only:
pcsx_rearmed, written twice, keeps its 64-bit build. Each package is mapped
to its cores by the .so or program its install step copies, PKG_ARCH is
applied as scripts/build:28-30 does, and es_systems.cfg is edited per
device as emuelec-emulationstation/package.mk:114-143 does. Amlogic-ng and
OdroidM1 list 124 cores, the handhelds 117.
2026-11-06 12:09:05 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz c0e4bd8379 chore: merge resolver fixes, fourth pass 2026-11-06 10:28:27 +01:00
Abdessamad Derraz 43cb0f84ab feat: add AmberELEC hardware targets
AmberELEC's metapackage adds LIBRETRO_CORES_EXTRA (bsnes, Mesen, melonDS, geolith) only on RG552, RG353 and RG503, and deletes those cores from es_systems.cfg on every other device (packages/amberelec/package.mk:23-31, 45-58 at a21975bf). The new target scraper reads the Makefile's device images, that test and es_systems.cfg the way the platform scraper does: RG351P, RG351V and RG351MP list 125 cores, the other three 136. The installer reads the device from COREELEC_DEVICE in os-release (scripts/image:168).
2026-11-06 09:04:51 +01:00
Abdessamad Derraz 57ccca640a refactor: importers write through write_if_changed
game_dat_importer and arcade_parent_importer each kept an exact-text
copy of write_if_changed, so a redump refresh rewrote every catalog for
its fetched date alone. Both now use artifacts.write_if_changed, which
treats fetched as a timestamp. A No-Intro rebuild still rewrites its
catalogs, since asset_updated_at moves with it, so check_freshness keeps
reading a fetched date no older than the pack.
2026-11-06 08:13:32 +01:00
Abdessamad Derraz 5c93c6dc6b refactor: one emudeck script-to-profile rule
The target scraper kept its own copy of the EmuScripts table and took
.ps1 files the platform scraper never read, so emudeck-we's template
skeleton was published as a core and removed afterwards by an override.
Both scrapers now read emulator_of_script, and the platform scraper
lists the installers at the pinned revision, as launchers: and the
SteamOS target already did. Targets rescraped, override dropped.
2026-11-06 08:12:19 +01:00
Abdessamad Derraz df801fd311 chore: merge resolver and export fixes 2026-11-06 05:54:29 +01:00
Abdessamad Derraz 7a98d73d79 chore: merge engine fixes, third pass 2026-11-06 05:06:59 +01:00
Abdessamad Derraz 47ece52410 docs: drop a semicolon in the RetroPie scraper 2026-11-06 04:07:14 +01:00
Abdessamad Derraz 3cb1d9c06a fix: keep the -sa name in ROCKNIX targets
The target scraper stripped -sa from every standalone package while the platform scraper keeps both spellings. vice-standalone, scummvm-standalone and supermodel3 answer to vice-sa, scummvm-sa and supermodel-sa only, so every targeted ROCKNIX build left them out. Both now read standalone_emulators(), and the targets are rescraped at tag 20261001.
2026-11-06 04:05:10 +01:00
Abdessamad Derraz 87f6211983 fix: one core name per RetroPie package
The platform scraper kept lr-dosbox-pure as dosbox-pure, which matched the standalone DOSBox Pure Unleashed profile, while the target scraper wrote dosbox_pure. Both now read core_name(), and platforms/retropie.yml is rescraped at its pinned revision 46128cd2: only the libretro core names change, and six Unleashed files leave the RetroPie manifest.
2026-11-06 04:03:36 +01:00
Abdessamad Derraz a2e8809945 fix: map a card's image name to its board
Batocera and KNULLI write the image name, not the board, to the card's board file (post-image-script.sh). The target scraper now records each board's images from Config.in, the target lists carry them, and the installer maps a card's image to its board. KNULLI's boot/knulli.board is read too.
2026-11-06 03:40:34 +01:00
Abdessamad Derraz 6445e5d7d6 fix: refuse a missing emulator profile directory 2026-11-06 03:21:53 +01:00
Abdessamad Derraz a9ecfea1a6 feat: add knulli target scraper
KNULLI builds per board on Batocera's Kconfig model from its own tree (knulli-cfw/knulli-linux at the tag platforms/knulli.yml pins): configs/knulli-*.board, package/system/knulli-system/Config.in, knulli-es-system/es_systems.yml and the knulli-cfw/buildroot submodule. Ten boards, the armhf library builds excluded.
2026-11-05 12:42:23 +01:00
Abdessamad Derraz 313693e58b fix: settle batocera kconfig before mapping cores
Continued if-conditions were cut at their first line, so GLES3 held on every board, and selects guarded by a negation stayed on once fired. Derived flags are read line by line, each pass rebuilds the symbol set from the board, and the architecture comes from the board's BR2_<arch> choice.
2026-11-05 12:42:19 +01:00
Abdessamad Derraz d38cd58036 feat: scrape recalbox rom folders per board
GitLab refused the hundred raw reads per IP. A GitLab directory is now
read from one archive request, else a shallow sparse git fetch, and the
cache keeps its file list so offline runs know what is absent.

Each board builds system.ini against its defconfig: a core whose package
the board does not enable is dropped (pisnes was the SNES default
everywhere), and a system with no core is not built. The row names the
default most boards share, board_defaults the others. A files: list
names whole files and is carried as files through the index and the
installer.

A folder no name resolves, run by a core of several machines, now holds
those machines: thomson, multivision and pcv2 had no fallback, so their
games got no file. Arcade-indexed cores keep naming files per set.
2026-11-04 16:48:34 +01:00
Abdessamad Derraz a05fc599d9 fix: read recalbox board sections as romfs does
board_cores walked only the plain [core.N] sections and replaced a board
section whole. romfs2 systemholder.py reads a slot present as
[core.N.<board>] alone (gb and gbc bsnes on x86_64) and each key from the
board section, then <key>.<board>, then the plain key. The committed
target lists are unchanged on all seven boards.
2026-11-04 16:48:34 +01:00
Abdessamad Derraz fd7dd0efe5 fix: games tools write whole files, name options
The games importers wrote through fixed .part names, the new options had
no help text, and romfolders stopped on the first setup the offline cache
lacked. RetroArch's rom folder map is regenerated from its source spec
(path null, as the schema says). The installer's Python 3.8 job imports
the games tests without pyyaml, so the region rules load lazily.
2026-11-03 22:46:17 +01:00
Abdessamad Derraz 8ba0a8c3f6 chore: merge main 2026-11-03 19:12:38 +01:00
Abdessamad Derraz ff1f94e473 feat: read 0.78 era arcade parents from code
mame2000, mame2003 and mame2003-plus carry a DAT their code moved past. Their parents and BIOS sets now come from src/driver.c and src/drivers/ at the pin, named by the src tree sha. The DAT is still read to report gaps. FBNeo DATs are read one namespace each, console sets keyed by driver. A BIOS with no dumped ROM is no longer listed.
2026-11-03 18:46:45 +01:00
Abdessamad Derraz b69b33178b feat: publish board sets and sets with no bios
The games index now names, per arcade core, the game sets that load no
BIOS set and every BIOS set, so a set on a card the DAT does not know is
told apart from one that needs nothing. Each board default core of a rom
folder gets its own fallback set, and index.json maps every folder name
a setup uses to its systems, for a games folder on a computer.
2026-11-03 18:17:03 +01:00
Abdessamad Derraz 1271b338f4 fix: keep listxml machines larger than a read 2026-11-03 18:07:08 +01:00
Abdessamad Derraz 1aa36258b2 feat: scrape the rom folders of each setup 2026-11-03 17:37:33 +01:00
Abdessamad Derraz 249b189ec9 feat: index arcade sets by bios parent 2026-11-03 16:36:26 +01:00
Abdessamad Derraz 8d8689c2e4 feat: import game catalogs and header rules 2026-11-03 16:36:22 +01:00
Abdessamad Derraz ae8aae8469 fix: file mame sets under their systems
934 of the 1,563 mame.yml entries named no system and so served every
system of the profile: each MAME-run system's set was the whole profile.

The 65 BIOS root sets written by hand take the system groovymame.yml
gives the same 0.289 archive (sega-systemsp, konami-gx, taito-gnet and
41 more ids, added to systems:). The 19 XaviX sets take tvgames, the
directory MAME keeps xavix.cpp in. They seed the device walk.

mame_hash_scraper --devices files a device set under arcade when a
machine no entry places loads it: MAME starts such a machine from its
set name, as an arcade folder passes it. 1,106 of the 1,254 device sets
now carry arcade, alone or beside the systems of their placed machines.
qsound, ym2413 and ym2608, written by hand, take the same answer.

Left without a system: 41 device sets no machine of the listxml
instantiates (a software list slot selects them), and the hash/,
plugins/ and floppy/ trees every machine reads.
2026-11-03 15:23:03 +01:00
Abdessamad Derraz aa0158c65a fix: read redump bios dats from /bios/<code>
redump.info moved its BIOS DAT links from /static/bios/<name>.dat to one
route per system (/bios/PSX, /bios/PS2, /bios/GC, /bios/XBOX), served as
an attachment. The scraper matched only the old shape and found nothing.
Both shapes are read now.

Refreshed snapshot: the PS2 DAT moved from 2026-06-16 to 2026-10-10,
191 entries become 225 (34 mechacon, DVD and font ROMs added, the
ps2-dvd-302c.bin sha1 corrected, five descriptions completed). Only
the provenance fields of database.json change.
2026-11-03 14:37:09 +01:00
Abdessamad Derraz 0d510eb4bd chore: merge core versions read from code 2026-10-27 19:42:04 +01:00
Abdessamad Derraz 25c356669b fix: record each platform's own list version
install/<platform>.json wrote the constant "1.0" as the version of the
list it was built from. The manifest now carries the version the platform
YAML records, and generate_pack refuses a YAML without one.

inherits: no longer passes a parent's version to a child: Lakka was
published as RetroArch v1.22.2 in its manifest, pack name and site page.
lakka.yml records v6.1, the Lakka-LibreELEC tag its target list is read
at. The MiSTer scraper refuses a BiosDB without a timestamp instead of
writing an empty version.

A pack's manifest.json records list_version, so an install from the pack
alone knows it too. Manifests regenerated twice and compared: only
version and generated changed.
2026-10-27 19:10:39 +01:00
Abdessamad Derraz 07c82c885f refactor: read the fbneo version from version.h
FBNeo bumps src/burn/version.h without tagging: the scraper wrote the
last tag, v1.0.0.02, while the source said 1.0.0.03. It now reads the
header the way the libretro core prints it. The merge no longer writes
core_version: the hashes come from finalburnneo/FBNeo's tip, which the
libretro/FBNeo pin does not name, so the version stays the one
version_basis reads at that pin. A MAME release profile is written with
its version_basis.
2026-10-27 18:48:16 +01:00
Abdessamad Derraz c826e0fc05 fix: keep release profiles on the build they pin
mame_hash_scraper --release rewrote the whole profile and dropped the
pin_frozen that mame0258 and mame0278 carry: libretro/mame HEAD is a
later release, so without it profile_sync compares their refs with code
the Recalbox build never had. A release profile now states it, from
built_by, and keeps the reason a reader wrote.
2026-10-23 16:41:58 +02:00
Abdessamad Derraz 0b4504992b feat: read batocera md5 written as a list
batocera.linux 6b08b873 lets a biosFiles item declare its md5 as a
list of strings, and checkBios accepts a file matching any of them.
The scrapers join the values with commas, as for Recalbox, and record
md5_list so the export writes the list back as a list, byte for byte.
2026-10-23 07:39:25 +02:00
Abdessamad Derraz be09d517d9 feat: read retrodeck dual-mode cores from es-de
RetroDECK listed every component standalone only. Azahar, Cemu, Dolphin,
MAME, melonDS, PCSX2 and PPSSPP also run through RetroArch: RetroDECK/ES-DE
es_systems.xml at 15e1b236 loads <name>_libretro.so beside each standalone
command, and the cores ship whole from the buildbot nightly
RetroArch_cores.7z (retroarch/component_recipe.json:250-262). Their
libretro layouts, Azahar's save-dir files among them, now reach the pack.
2026-10-22 22:59:47 +02:00
Abdessamad Derraz 9a1bf4baa6 feat: record the systems each libretro core runs 2026-10-22 21:16:47 +02:00
Abdessamad Derraz d60fe814ed feat: read required from romload for MAME sets
A set whose every ROM is NO_DUMP or ROM_OPTIONAL only warns in
handle_missing_file, so the machine runs without it. The parser now
keeps undumped and optional ROMs for that decision, the merge and the
release profiles write required false for such sets, and the text
patcher rewrites the required line it used to leave. A curated false
over a set with dumped ROMs is kept, except over no contents, which is
what an undumped set leaves once a later release dumps it.

pSxMAME builds mamedev/mame itself at 0.133 and left the current
refresh, which would have moved it onto 0.289 refs. Release profiles
keep their citations double-quoted on regeneration.
2026-10-22 14:06:18 +02:00
Abdessamad Derraz 9648b9fc3b feat: read elec ports and per-system cores
EmuELEC's ports system has one emulator, `ports`, the dispatch to a
script ports.yaml generates. The scraper now reads each port's engine
(the libretro core handed to emuelecRunEmu.sh with -C, else port_exe),
so devilutionx, sonic2013, soniccd, sonic-mania, reminiscence and
rigelengine are examined. Standalone emulators keep the spelling the
launch code dispatches on instead of invented lowercase forms, and
media players and runtimes go to the overrides' remove_cores.

Each system of the AmberELEC, EmuELEC and KNULLI lists now carries the
cores its ES list offers it, in menu order, under `cores:`.
2026-10-22 06:28:24 +02:00
Abdessamad Derraz 841017c887 chore: merge the minui family platforms 2026-10-22 04:09:17 +02:00
Abdessamad Derraz 43bdf83973 feat: add minui, nextui and myminui platforms
minarch hands every core SDCARD_PATH/Bios/<TAG> as its system directory
and opens no file in it. The scrapers read each pak's launch.sh (core,
tag, devices), each device's core makefile (the project a library is
built from: MyMinUI's dosbox_libretro.so is dosbox-pure) and the minarch
statement, at a pinned commit. The installer recognises the card from
.system/version.txt and refuses a folder inside it.
2026-10-22 04:04:36 +02:00
Abdessamad Derraz 2bdac466a9 feat: add knulli, amberelec and emuelec platforms
Each OS publishes Batocera's systems dict (knulli-systems, and
batocera-systems at AmberELEC's newest prerelease commit and EmuELEC v4.8),
read with the shared parser. Cores come from each OS's own ES list.
AmberELEC and EmuELEC declare neogeo/ and ports/ beside bios/ from
/storage/roms, so their packs extract there and core_destination puts
core files under bios/.
2026-10-22 03:42:20 +02:00
Abdessamad Derraz 2fc62bd249 refactor: share the batocera-systems parser
extract_systems_dict, requirements_from_systems, platform_systems and
configgen_cores move to module level so the scrapers of lists with the
same shape read them with the same code.
2026-10-22 03:42:11 +02:00
Abdessamad Derraz f2c0d13ab2 chore: merge round 4 fixes (links)
Both branches extracted RetroBat's Flycast BiosPath: value_folders
already produces the record, so the dedicated extractor is dropped and
its test reads the generic one.
2026-10-21 13:19:44 +02:00
Abdessamad Derraz 43fe5a7b15 chore: merge round 4 fixes (profiles) 2026-10-21 13:17:10 +02:00
Abdessamad Derraz 3e5deed8fd chore: merge round 4 fixes (retrobat) 2026-10-21 13:17:10 +02:00
Abdessamad Derraz d9809efa36 fix: pack emudeck libretro builds as dual mode
RetroArch_installCores (emuDeckRetroArch.sh) downloads flycast, mame,
melonds, mgba, mupen64plus_next, ppsspp and scummvm as libretro cores
next to their standalone apps. Listed as standalone only, they got one
layout: once Flycast's standalone files moved to flycast/bios/, the
libretro core lost dc/airlbios.zip, dc/naomi2.zip and dc/segasp.zip.
The scraper now writes them as dual_mode_cores.
2026-10-21 12:45:36 +02:00
Abdessamad Derraz 8036d36fe8 fix: drop names DraStic never opens
DraStic opens nds_bios_arm9.bin, nds_bios_arm7.bin and nds_firmware.bin and checks their size. Its nine aliases are in none of its binaries, and neither platform launcher passes them: ROCKNIX links the two BIOS names from its bios folder, and RetroPie runs DraStic from $md_conf_root/nds/drastic (drastic.sh:48), now a passes_path record. The collection answers DraStic's names with its DS dumps in generate_db, where repository names belong.
2026-10-21 12:44:33 +02:00
Abdessamad Derraz bd6ab38fe1 feat: extract the remaining retrobat generators
The launcher layer now reads the Saturn emulators whose option replaces
the default path, DeSmuME's pair and firmware, the melonDS DSi files and
NAND region, jzIntv arguments, BlastEm's SetBiosPath, Dolphin, Raine,
GSplus, the DOSBox and capsimg copies, Tsugaru's folder chain, RPCS3,
EKA2L1, BizHawk's GB switches and the folders that MAME, FBNeo, Snes9x,
Flycast, ScummVM, XRoar, XM6 Pro, ArcadeDuck, Demul and B-Syndrome get.
Every line of these generators that names the bios folder must be read
by a record or a stated exception, or the scrape stops.
2026-10-21 12:39:15 +02:00
Abdessamad Derraz d5a25190a3 fix: place standalone files in the linked folder
standalone_path is relative to the directories a standalone build
resolves it against, now declared per profile (standalone_directories).
A launcher record that links or passes a whole folder of the BIOS tree
under one of them gives that folder, and platform_layouts rebases the
standalone view once for the builder, the manifest, verify and slots
(launchers.standalone_root, common.standalone_destination).

Flycast on RetroPie and ROCKNIX reads bios/dc, on EmuDeck
bios/flycast/bios, on Batocera (FLYCAST_BIOS_PATH) and RetroBat
(Dreamcast.BiosPath) bios/dc. The ROCKNIX, Batocera and RetroBat
scrapers now record the patched data directory, the environment
variable and the setting that say so.
2026-10-21 12:39:03 +02:00