Commit Graph
100 Commits
Author SHA1 Message Date
Abdessamad Derraz 9f229753b2 feat: platform-specific notes in zip readme 2026-03-28 19:27:49 +01:00
Abdessamad Derraz ddf2937f41 fix: eliminate cross-reference false positives
Skip placeholder names (<bios>.bin), resolve by MD5/SHA1 hash
match for alias files, fix directory basename extraction for
trailing slash entries, index bios/ directory names for
directory-type file entries. 1011 -> 113 true missing.
149/149 tests pass.
2026-03-28 19:24:16 +01:00
Abdessamad Derraz 1825fff893 fix: repo_path fallback by name for normalized zips 2026-03-28 19:11:36 +01:00
Abdessamad Derraz a418f7ef68 feat: add Dragon CoCo ROMs and mameinfo.dat
cdos20.rom (Cumana DOS 2.0), yados.rom (YA-DOS 0.5a),
dragonfly-1.3.rom (Ikon Ultra Drive v1.3) from World of
Dragon Archive and CCHDD GitHub. delta-premier.rom variant.
mameinfo.dat for MAME 2000/2003/2003-Plus game info.
2026-03-28 19:09:42 +01:00
Abdessamad Derraz 0419c6a780 fix: generate manifest aliases for grouped platforms 2026-03-28 18:58:24 +01:00
Abdessamad Derraz 63f2450943 feat: add Xbox BIOS variants and PC-98 sasi.rom
8 Xbox BIOS variants (mcpx-1.1, xbox-3944 through 5838)
from Myrient MAME bios-devices. PC-98 SASI controller ROM
for NP2kai/nekop2. Cross-reference: 146 true missing.
2026-03-28 18:50:59 +01:00
Abdessamad Derraz 6243271ab3 fix: regenerate manifests without base_dest prefix 2026-03-28 18:47:38 +01:00
Abdessamad Derraz 0a721fbef1 fix: url-encode repo paths with spaces 2026-03-28 18:46:36 +01:00
Abdessamad Derraz 2118571ca2 fix: emudeck detection path and base url override 2026-03-28 18:45:36 +01:00
Abdessamad Derraz 080cc8f19d fix: manifest dest paths relative to bios dir 2026-03-28 18:42:19 +01:00
Abdessamad Derraz 8777706a01 chore: generate initial install manifests 2026-03-28 18:29:01 +01:00
Abdessamad Derraz 2ff5be8420 feat: complete DuckStation PS1/PS2 BIOS coverage
Add 9 PS2 BIOS from Myrient (pre-shutdown raid), fix
incorrect scph5000.bin, add DuckStation-named symlinks
for 80 PS2 entries. Add Saturn cdb_rom, Fuse peripheral
ROMs, Amiberry MT-32 aliases, DOSBox soundfont. PS1
coverage: 25/25. PS2 coverage: 80/80 via MD5 matching.
2026-03-28 18:26:57 +01:00
Abdessamad Derraz 536c0c242e fix: resolve descriptive names via dest_hint basename fallback
resolve_local_file now tries the path: field basename when name:
lookup fails. Fixes 139 PCem false positives where descriptive
names ("MDA font ROM") didn't match actual filenames (mda.rom).
Also add 3 QEMU firmware (MacROM.bin, bios_loongson3, pmon_2e).
Cross-reference path: fallback already added. 149/149 tests pass.
2026-03-28 18:24:53 +01:00
Abdessamad Derraz edd44d4137 feat: add manifest generation to pipeline 2026-03-28 18:13:05 +01:00
Abdessamad Derraz f453a7c750 feat: include readme.txt in pack zips 2026-03-28 18:11:37 +01:00
Abdessamad Derraz 6ba2519c4e feat: add quick install section to readme 2026-03-28 18:09:40 +01:00
Abdessamad Derraz 8fe32103e6 feat: add install.ps1 powershell installer 2026-03-28 18:09:13 +01:00
Abdessamad Derraz 46426aa1e4 feat: add install.py universal installer engine 2026-03-28 18:07:28 +01:00
Abdessamad Derraz 42f2cc5617 fix: cross-reference resolves by path: field as fallback
Many emulator profiles use descriptive names (e.g., "SeaBIOS
(128 KB)") while files exist under their path: field basename
(e.g., "bios.bin"). Try path: when name: fails. Eliminates
206 false positives. True missing: 448 -> 242.
2026-03-28 18:06:04 +01:00
Abdessamad Derraz 7f46996b4d feat: add --manifest-targets for installer 2026-03-28 18:04:27 +01:00
Abdessamad Derraz 1a0eef563b feat: add install.sh posix bootstrap 2026-03-28 18:04:08 +01:00
Abdessamad Derraz 5763ce731d feat: add --manifest mode to generate_pack 2026-03-28 18:02:54 +01:00
Abdessamad Derraz 76fe7dd76f fix: cross-reference checks inside ZIP archives
_build_supplemental_index scans both data/ directories and
contents of bios/ ZIP files. Eliminates 197 false positives
where files existed inside archive ZIPs (neogeo.zip, pgm.zip,
stvbios.zip, etc.) but were counted as missing. True missing
drops from 645 to 448.
2026-03-28 18:00:11 +01:00
Abdessamad Derraz c6a24446ba feat: add install metadata to platform registry 2026-03-28 17:58:05 +01:00
Abdessamad Derraz 68824bf3d4 chore: regenerate database after wave 2 collection
7545 files indexed. Cross-reference: 645 true missing (down
from 1011). All platform baselines at 100% or pre-existing
untested. Only required missing: 3 CLK Enterprise 128 ROMs.
2026-03-28 17:53:39 +01:00
Abdessamad Derraz 31ec407c91 feat: add FBNeo st010/st011/sfiii3na, PCem rivatnt, misc
sfiii3na.zip (CPS3 Asia No-CD), st010.zip and st011.zip
(SNES coprocessors) complete FBNeo required coverage.
rivatnt.bin completes PCem at 139/139. Also add QEMU
firmware (4), FM Towns Marty ROMs (4), Fuse mf1.rom,
Hatari BOOT.ST, PS1 extracts (2), ZC210 data (2).
2026-03-28 17:42:53 +01:00
Abdessamad Derraz 23dd7657aa fix: restore all emulator profiles to ground truth
Previous commit failed to fully revert fuse.yml (agent-added
MD5/SHA1 still present). Also revert qemu.yml and higan_sfc.yml.
All profiles now match their pre-collection state (2666ebd).
Emulator profiles are source-verified ground truth — agents
must never modify them without code verification.
2026-03-28 17:39:31 +01:00
Abdessamad Derraz fea0338062 fix: revert agent modifications to fuse.yml and qemu.yml
Agents added MD5/SHA1 hashes to fuse.yml computed from downloaded
files, not from source code (ground truth violation: validation
is [size] only). Agents added firmware entries to qemu.yml without
source code verification. Profiles must reflect code, not files.
2026-03-28 17:36:39 +01:00
Abdessamad Derraz 3092d73122 fix: cross-reference checks data/ directories for false positives
_find_in_repo and _name_in_index now scan data/ in addition to
bios/ via database.json. Eliminates 129 false positives from
game data migrated to data/ (OpenTyrian, ScummVM, SDLPAL, Cave
Story, Syobon Action). True missing: 782 -> 653.
2026-03-28 17:31:22 +01:00
Abdessamad Derraz 3d5d5b601c feat: mass BIOS collection (+1020 files, 7520 total)
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.

New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz 7dc8428ac1 refactor: fix cross-reference archive grouping and path resolution
Group archived files by archive unit in find_undeclared_files instead
of reporting individual ROMs. Add path-based fallback for descriptive
names (e.g. "SeaBIOS (128 KB)" resolves via path: bios.bin). Update
_collect_extras to use archive name for pack resolution. Regenerate
database with new bios files. 6 new E2E tests covering archive
in_repo, missing archives, descriptive names, and pack extras.
2026-03-28 14:00:08 +01:00
Abdessamad Derraz 1ee8623b2e feat: add 47 missing bios files and fix cross-reference false positives
Add BIOS files for CLK (Apple II, Mac, Atari ST, Enterprise, MSX,
Commodore, Thomson, PC, Acorn), FBNeo (CPS3 redearthn/sfiii2h),
and QEMU variants. Sources: Asimov mirror, MAME chip extraction,
86Box, Theodore, XRoar upstream, official QEMU repo.

Fix cross-reference in verify.py to group archived files by archive
unit instead of reporting individual ROMs as missing. Add path-based
fallback for descriptive names. Update generate_pack.py extras to
use archive name for resolution. 6 new E2E tests.

MISSING required: 321 -> 6 (zero false positives).
2026-03-28 13:59:48 +01:00
Abdessamad Derraz 543cb504bb fix: skip file/directory path conflicts in pack generation 2026-03-28 12:57:30 +01:00
Abdessamad Derraz 645dc013a4 feat: add qemu bios hash variants 2026-03-28 12:33:03 +01:00
Abdessamad Derraz 5b5b304288 feat: add dragon and coco clone bios variants 2026-03-28 12:33:02 +01:00
Abdessamad Derraz e07842ac4b chore: add pyproject.toml for uv/pip compatibility 2026-03-28 12:33:00 +01:00
Abdessamad Derraz 366695b764 Update README.md 2026-03-28 12:12:05 +01:00
Abdessamad Derraz 460c49dbcb fix: bizhawk scraper lowercase sha1 + complete bios 2026-03-28 11:49:52 +01:00
Abdessamad Derraz a5285a9fc0 feat: add ti83 v1.10 and doom2.wad, zero platform missing
TI-83 v1.10 ROM from TOSEC archive (sha1 verified).
DOOM II WAD from archive.org for prboom libretro core.
All platforms now at 0 missing files.
2026-03-28 11:40:06 +01:00
Abdessamad Derraz 84d344450f feat: add vice, qemu, pcem, freedoom, x16, iigs, dsi bios
VICE: basic, chargen, kernal for x64sdl libretro core.
QEMU: SeaBIOS 1.16.3 + VGA BIOS from system package.
PCem: MDA font ROM from PCem v17 ROM set.
Freedoom: freedoom1.wad + freedoom2.wad for prboom.
Commander X16: rom.bin R49 from x16-rom releases.
Apple IIGS: ROM01 from asimov mirrors.
DSi: NDS_Bios7i, NDS_Bios9i, DSi_Firmware, SGB_(JU).
2026-03-28 11:32:58 +01:00
Abdessamad Derraz dd689bbd2a feat: add name aliases for bizhawk platform resolution
Copy existing BIOS files under BizHawk-expected names:
MCD_eu_100.bin, MCD_jp_100p.bin, PSX_DTLH2000.bin,
MSX_4000bios.rom, MSX_8020-20bios.rom.
2026-03-28 10:29:13 +01:00
Abdessamad Derraz 1f65662fd2 feat: add ~110 missing bios files from archive.org and bizhawk
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.

Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz b854f71b44 feat: add bizhawk platform config 2026-03-28 09:45:12 +01:00
Abdessamad Derraz 2738a4d326 feat: add bizhawk scraper for FirmwareDatabase.cs 2026-03-28 09:41:52 +01:00
Abdessamad Derraz b75f2b2a43 feat: add sha1 verification mode for bizhawk 2026-03-28 09:35:13 +01:00
Abdessamad Derraz f1855641c5 docs: add granular pack options to tools reference 2026-03-28 09:29:31 +01:00
Abdessamad Derraz 2666ebd9b7 chore: regenerate readme and database timestamps 2026-03-28 09:24:01 +01:00
Abdessamad Derraz 22a1e7caf4 fix: sync NstDatabase.xml hashes in platform configs 2026-03-28 09:19:37 +01:00
Abdessamad Derraz 67186448a2 fix: verify and add manifests to split packs 2026-03-28 08:43:41 +01:00
Abdessamad Derraz 70891314d3 fix: include core extras in split packs 2026-03-28 08:29:37 +01:00
Abdessamad Derraz 97b9900f62 chore: update NstDatabase.xml from upstream nestopia
Closes #40
2026-03-28 08:24:13 +01:00
Abdessamad Derraz a1aa97a70e fix: include core extras in split packs 2026-03-28 08:06:22 +01:00
Abdessamad Derraz 97b1c2c08a feat: add --from-md5 lookup and pack builder 2026-03-28 01:02:25 +01:00
Abdessamad Derraz 0624e9d87e test: add validation tests for pack arg combos 2026-03-28 00:59:02 +01:00
Abdessamad Derraz 3ded72f72b feat: add --group-by manufacturer for split packs 2026-03-28 00:45:12 +01:00
Abdessamad Derraz 94a28f5459 feat: add --split flag for per-system packs 2026-03-28 00:43:20 +01:00
Abdessamad Derraz 837ac80cca refactor: deduplicate manufacturer prefix list 2026-03-28 00:39:28 +01:00
Abdessamad Derraz 43cb7a9884 feat: allow --platform + --system combination 2026-03-28 00:36:51 +01:00
Abdessamad Derraz 020ff148c2 feat: add --required-only flag to generate_pack 2026-03-28 00:32:58 +01:00
Abdessamad Derraz 5cbd461a97 docs: update readme and database for ground truth feature 2026-03-27 23:55:30 +01:00
Abdessamad Derraz 6cbdd4c40c test: add integration tests for ground truth full chain 2026-03-27 23:41:06 +01:00
Abdessamad Derraz 37acc8d0fc feat: add --verbose flag and ground truth rendering 2026-03-27 23:38:43 +01:00
Abdessamad Derraz 2cf1398786 feat: attach ground truth to emulator verification results 2026-03-27 23:33:53 +01:00
Abdessamad Derraz 6b14b5e2b1 feat: attach ground truth to platform verification results 2026-03-27 23:30:49 +01:00
Abdessamad Derraz 6d959ff2b0 feat: add per-emulator ground truth to validation index 2026-03-27 23:25:42 +01:00
Abdessamad Derraz 3672912de7 chore: clean up sc3000 variant swap, update database 2026-03-27 22:50:54 +01:00
Abdessamad Derraz 569781c104 fix: rename misleading exclusion label in verify report 2026-03-27 22:44:05 +01:00
Abdessamad Derraz 7f265b3cb2 refactor: split pack check into baseline and cores with clear counts 2026-03-27 20:37:48 +01:00
Abdessamad Derraz 3ea1e09cb0 feat: verify core extras presence in pack alongside baseline 2026-03-27 19:41:47 +01:00
Abdessamad Derraz 89d6dd2eee feat: add platform conformance check to pack verification 2026-03-27 19:21:29 +01:00
Abdessamad Derraz acd2daf7c1 fix: filter pattern placeholders, skip standalone exclusions for standalone platforms 2026-03-27 18:30:18 +01:00
Abdessamad Derraz 0ad8324d46 refactor: clearer verify report for core files coverage 2026-03-27 18:11:26 +01:00
Abdessamad Derraz 29749898f8 fix: correct sc3000, plus3 and plus3e hashes in platform configs 2026-03-27 17:47:39 +01:00
Abdessamad Derraz 59d7582c0e fix: swap sc3000.zip primary to batocera-compatible dump 2026-03-27 16:50:10 +01:00
Abdessamad Derraz e94ce6b194 feat: source 10 missing bios variants for romm and retrobat coverage 2026-03-27 16:00:10 +01:00
Abdessamad Derraz 181248b6db fix: case-sensitive packs for linux platforms, remove empty bios placeholder 2026-03-27 12:58:08 +01:00
Abdessamad Derraz a117b13b49 fix: data directory path construction avoids double slash duplicates 2026-03-27 12:42:21 +01:00
Abdessamad Derraz 8f1c7e47de fix: emudeck scraper outputs cores field, base_scraper uses generate_platform_yaml 2026-03-27 12:27:51 +01:00
Abdessamad Derraz 106130ca52 refactor: extract _norm_system_id, apply to resolve_platform_cores fallback 2026-03-27 12:23:18 +01:00
Abdessamad Derraz 79f2d82072 feat: add batocera standalone overrides for wine-dependent emulators 2026-03-27 12:18:08 +01:00
Abdessamad Derraz c8cba9d519 fix: add beetle_supergrafx, caprice32, mess2016 aliases 2026-03-27 12:00:27 +01:00
Abdessamad Derraz 6b87c75e11 fix: add upstream buildbot aliases to 18 emulator profiles 2026-03-27 11:57:10 +01:00
Abdessamad Derraz 21fa98c82f fix: add missing buildbot aliases to avoid false target skips 2026-03-27 11:43:29 +01:00
Abdessamad Derraz 1bde69f173 fix: add mednafen_wswan alias to beetle_wswan profile 2026-03-27 11:26:47 +01:00
Abdessamad Derraz 5d6760d120 fix: populate romm target cores, normalize system IDs in filter 2026-03-27 11:15:41 +01:00
Abdessamad Derraz 17631f741c fix: normalize system IDs for target filtering cross-platform matching 2026-03-27 11:14:24 +01:00
Abdessamad Derraz cdc71525d0 feat: add emulator profiles and update docs 2026-03-27 10:29:38 +01:00
Abdessamad Derraz 74f48b8881 Add emulator profiles and update docs
Add new emulator profiles (ares, BigPEmu, Eden, Model2, PrimeHack, Supermodel, Suyu, Xenia Canary, Yuzu) under emulators/*.yml with metadata, notes, file lists and exclusion/hle details. Update README counts (emulators/systems/files summary and auto-generated timestamp) and bump generated_at in database.json. Adjust mkdocs.yml navigation counts and add Yuzu entry to the Emulators nav.
2026-03-26 13:22:11 +01:00
Abdessamad Derraz 0a1880f606 fix: filter baseline by platform-scoped cores, include retroarch cores in emudeck targets 2026-03-26 10:20:43 +01:00
Abdessamad Derraz 6402b77374 fix: filter baseline systems by target-available cores 2026-03-26 09:54:28 +01:00
Abdessamad Derraz a3de47dd88 fix: normalize core names in emudeck and retropie scrapers 2026-03-26 09:44:11 +01:00
Abdessamad Derraz 7e8491fdf7 fix: resolve upstream core names to profile keys via reverse index 2026-03-26 09:42:08 +01:00
Abdessamad Derraz bccea60f8d fix: add vita via recipe, rewrite retropie scraper via github api 2026-03-26 09:32:22 +01:00
Abdessamad Derraz 16b4dcc270 feat: rewrite retropie scraper using scriptmodules 2026-03-26 09:31:37 +01:00
Abdessamad Derraz 15e8c0eccb feat: add Vita target via libretro-super recipes 2026-03-26 09:31:34 +01:00
Abdessamad Derraz 14eaff73f7 feat: add retroarch target aliases to overrides 2026-03-26 09:25:39 +01:00
Abdessamad Derraz dfb7d9a25a fix: correct scraper paths and patterns, populate target files 2026-03-26 09:18:39 +01:00
Abdessamad Derraz 03a9fa3276 fix: batocera targets scraper es_systems path and condition parser
- Fix es_systems.yml URL (batocera-emulationstation -> batocera-es-system)
- Replace if/endif block parser with select-if condition parser that
  matches the actual Config.in structure (select PACKAGE if CONDITION)
- Add Kconfig boolean condition checker (handles &&, ||, !, parentheses)
- Add meta-flag expansion (X86_64_ANY, GLES3, ROCKCHIP_GLES3, etc.)
  iterated to fixpoint for chained derivations
- Fix es_systems.yml parser for the actual dict format with requireAnyOf
2026-03-26 09:16:06 +01:00
Abdessamad Derraz 2980100fba feat: add static target files for retrobat, retrodeck, romm 2026-03-26 08:56:00 +01:00
Abdessamad Derraz 6b86c543af feat: add emudeck and retropie target scrapers 2026-03-26 08:55:58 +01:00