Abdessamad Derraz
9e331687a6
feat: generate defaults and extract compiled-in ROMs
...
WiiSD.raw/sd.raw (128MB FAT32), eeprom.dat (PCSX2 DEV9),
CMOS.ROM (fMSX RTC), key.txt (NP2kai), wildmidi.cfg
(EasyRPG), NXEngine sprites.sif+tilekey.dat extracted from
source headers, Frodo C64 ROMs extracted from compiled-in
arrays, Custom.dat (ZC210 empty default).
2026-03-28 21:30:30 +01:00
Abdessamad Derraz
7b0527bb0c
feat: add FM Towns FMT_ALL.ROM combined container
...
Built from individual ROMs (FMT_SYS, FMT_DOS, FMT_FNT,
FMT_F20, FMT_DIC, MYTOWNS, MAR_EX0-3) using the tsugaru
structured container format (12-byte name + length + data).
2026-03-28 21:04:12 +01:00
Abdessamad Derraz
3eb6e0e530
feat: add Dolphin Wii certs, fMSX GameMaster ROMs
...
Wii SSL certs (clientca/clientcakey/rootca.pem) from archived
Nintendo HTTPS certs. SYSCONF and setting.txt generated with
correct format. Konami GameMaster 1+2 cartridge ROMs for fMSX.
2026-03-28 20:57:41 +01:00
Abdessamad Derraz
6c4f18056b
feat: add Rustation PS1 aliases, FBNeo arcade BIOS, misc
...
8 Rustation PS1 BIOS (SHA256-verified copies of existing files
under Rustation-expected names). 4 FBNeo arcade BIOS extracted
from MAME game ROMs (236-bios.sp1, bios.u42, svg_bios.u49).
DuckStation ps2_scph18000.bin resolved from existing ps2-50j.
2026-03-28 20:53:23 +01:00
Abdessamad Derraz
ddf2937f41
fix: eliminate cross-reference false positives
...
Skip placeholder names (<bios>.bin), resolve by MD5/SHA1 hash
match for alias files, fix directory basename extraction for
trailing slash entries, index bios/ directory names for
directory-type file entries. 1011 -> 113 true missing.
149/149 tests pass.
2026-03-28 19:24:16 +01:00
Abdessamad Derraz
a418f7ef68
feat: add Dragon CoCo ROMs and mameinfo.dat
...
cdos20.rom (Cumana DOS 2.0), yados.rom (YA-DOS 0.5a),
dragonfly-1.3.rom (Ikon Ultra Drive v1.3) from World of
Dragon Archive and CCHDD GitHub. delta-premier.rom variant.
mameinfo.dat for MAME 2000/2003/2003-Plus game info.
2026-03-28 19:09:42 +01:00
Abdessamad Derraz
63f2450943
feat: add Xbox BIOS variants and PC-98 sasi.rom
...
8 Xbox BIOS variants (mcpx-1.1, xbox-3944 through 5838)
from Myrient MAME bios-devices. PC-98 SASI controller ROM
for NP2kai/nekop2. Cross-reference: 146 true missing.
2026-03-28 18:50:59 +01:00
Abdessamad Derraz
2ff5be8420
feat: complete DuckStation PS1/PS2 BIOS coverage
...
Add 9 PS2 BIOS from Myrient (pre-shutdown raid), fix
incorrect scph5000.bin, add DuckStation-named symlinks
for 80 PS2 entries. Add Saturn cdb_rom, Fuse peripheral
ROMs, Amiberry MT-32 aliases, DOSBox soundfont. PS1
coverage: 25/25. PS2 coverage: 80/80 via MD5 matching.
2026-03-28 18:26:57 +01:00
Abdessamad Derraz
536c0c242e
fix: resolve descriptive names via dest_hint basename fallback
...
resolve_local_file now tries the path: field basename when name:
lookup fails. Fixes 139 PCem false positives where descriptive
names ("MDA font ROM") didn't match actual filenames (mda.rom).
Also add 3 QEMU firmware (MacROM.bin, bios_loongson3, pmon_2e).
Cross-reference path: fallback already added. 149/149 tests pass.
2026-03-28 18:24:53 +01:00
Abdessamad Derraz
76fe7dd76f
fix: cross-reference checks inside ZIP archives
...
_build_supplemental_index scans both data/ directories and
contents of bios/ ZIP files. Eliminates 197 false positives
where files existed inside archive ZIPs (neogeo.zip, pgm.zip,
stvbios.zip, etc.) but were counted as missing. True missing
drops from 645 to 448.
2026-03-28 18:00:11 +01:00
Abdessamad Derraz
68824bf3d4
chore: regenerate database after wave 2 collection
...
7545 files indexed. Cross-reference: 645 true missing (down
from 1011). All platform baselines at 100% or pre-existing
untested. Only required missing: 3 CLK Enterprise 128 ROMs.
2026-03-28 17:53:39 +01:00
Abdessamad Derraz
31ec407c91
feat: add FBNeo st010/st011/sfiii3na, PCem rivatnt, misc
...
sfiii3na.zip (CPS3 Asia No-CD), st010.zip and st011.zip
(SNES coprocessors) complete FBNeo required coverage.
rivatnt.bin completes PCem at 139/139. Also add QEMU
firmware (4), FM Towns Marty ROMs (4), Fuse mf1.rom,
Hatari BOOT.ST, PS1 extracts (2), ZC210 data (2).
2026-03-28 17:42:53 +01:00
Abdessamad Derraz
fea0338062
fix: revert agent modifications to fuse.yml and qemu.yml
...
Agents added MD5/SHA1 hashes to fuse.yml computed from downloaded
files, not from source code (ground truth violation: validation
is [size] only). Agents added firmware entries to qemu.yml without
source code verification. Profiles must reflect code, not files.
2026-03-28 17:36:39 +01:00
Abdessamad Derraz
3092d73122
fix: cross-reference checks data/ directories for false positives
...
_find_in_repo and _name_in_index now scan data/ in addition to
bios/ via database.json. Eliminates 129 false positives from
game data migrated to data/ (OpenTyrian, ScummVM, SDLPAL, Cave
Story, Syobon Action). True missing: 782 -> 653.
2026-03-28 17:31:22 +01:00
Abdessamad Derraz
3d5d5b601c
feat: mass BIOS collection (+1020 files, 7520 total)
...
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.
New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz
7dc8428ac1
refactor: fix cross-reference archive grouping and path resolution
...
Group archived files by archive unit in find_undeclared_files instead
of reporting individual ROMs. Add path-based fallback for descriptive
names (e.g. "SeaBIOS (128 KB)" resolves via path: bios.bin). Update
_collect_extras to use archive name for pack resolution. Regenerate
database with new bios files. 6 new E2E tests covering archive
in_repo, missing archives, descriptive names, and pack extras.
2026-03-28 14:00:08 +01:00
Abdessamad Derraz
460c49dbcb
fix: bizhawk scraper lowercase sha1 + complete bios
2026-03-28 11:49:52 +01:00
Abdessamad Derraz
a5285a9fc0
feat: add ti83 v1.10 and doom2.wad, zero platform missing
...
TI-83 v1.10 ROM from TOSEC archive (sha1 verified).
DOOM II WAD from archive.org for prboom libretro core.
All platforms now at 0 missing files.
2026-03-28 11:40:06 +01:00
Abdessamad Derraz
84d344450f
feat: add vice, qemu, pcem, freedoom, x16, iigs, dsi bios
...
VICE: basic, chargen, kernal for x64sdl libretro core.
QEMU: SeaBIOS 1.16.3 + VGA BIOS from system package.
PCem: MDA font ROM from PCem v17 ROM set.
Freedoom: freedoom1.wad + freedoom2.wad for prboom.
Commander X16: rom.bin R49 from x16-rom releases.
Apple IIGS: ROM01 from asimov mirrors.
DSi: NDS_Bios7i, NDS_Bios9i, DSi_Firmware, SGB_(JU).
2026-03-28 11:32:58 +01:00
Abdessamad Derraz
dd689bbd2a
feat: add name aliases for bizhawk platform resolution
...
Copy existing BIOS files under BizHawk-expected names:
MCD_eu_100.bin, MCD_jp_100p.bin, PSX_DTLH2000.bin,
MSX_4000bios.rom, MSX_8020-20bios.rom.
2026-03-28 10:29:13 +01:00
Abdessamad Derraz
1f65662fd2
feat: add ~110 missing bios files from archive.org and bizhawk
...
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.
Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz
2666ebd9b7
chore: regenerate readme and database timestamps
2026-03-28 09:24:01 +01:00
Abdessamad Derraz
97b9900f62
chore: update NstDatabase.xml from upstream nestopia
...
Closes #40
2026-03-28 08:24:13 +01:00
Abdessamad Derraz
5cbd461a97
docs: update readme and database for ground truth feature
2026-03-27 23:55:30 +01:00
Abdessamad Derraz
3672912de7
chore: clean up sc3000 variant swap, update database
2026-03-27 22:50:54 +01:00
Abdessamad Derraz
59d7582c0e
fix: swap sc3000.zip primary to batocera-compatible dump
2026-03-27 16:50:10 +01:00
Abdessamad Derraz
e94ce6b194
feat: source 10 missing bios variants for romm and retrobat coverage
2026-03-27 16:00:10 +01:00
Abdessamad Derraz
181248b6db
fix: case-sensitive packs for linux platforms, remove empty bios placeholder
2026-03-27 12:58:08 +01:00
Abdessamad Derraz
74f48b8881
Add emulator profiles and update docs
...
Add new emulator profiles (ares, BigPEmu, Eden, Model2, PrimeHack, Supermodel, Suyu, Xenia Canary, Yuzu) under emulators/*.yml with metadata, notes, file lists and exclusion/hle details. Update README counts (emulators/systems/files summary and auto-generated timestamp) and bump generated_at in database.json. Adjust mkdocs.yml navigation counts and add Yuzu entry to the Emulators nav.
2026-03-26 13:22:11 +01:00
Abdessamad Derraz
8ac64d6143
docs: add build guide and documentation site section to readme
...
Quick start commands for generating packs by platform, emulator, or
system. Documentation site section explains what users can find there.
2026-03-26 07:15:01 +01:00
Abdessamad Derraz
bbe4936cf2
feat: add 12 batocera standalone profiles, update 3 existing
...
New profiles: citron (Switch), clk (multi-system), demul (Hikaru),
eka2l1 (Symbian/N-Gage), gsplus (Apple IIGS), lexaloffle (PICO-8),
openmsx (MSX), ryujinx (Switch), shadps4 (PS4), tsugaru (FM Towns),
x16emu (Commander X16), xroar (Dragon/CoCo), zc210 (Zelda Classic).
Updated: amiberry, fsuae (standalone Amiga), ymir (Saturn standalone).
2026-03-26 07:01:16 +01:00
Abdessamad Derraz
3f676b75e8
feat: standalone emulator support for batocera and multi-platform name mapping
...
resolve_platform_cores() builds reverse index from profile cores: field,
fixing 17 name mismatches across Batocera, RetroBat, and Recalbox
(genesisplusgx, pce_fast, pcfx, vb, mame078plus, vice cores, etc.).
standalone_path field on file entries + standalone_cores on platform
YAMLs enable mode-aware pack generation. find_undeclared_files() uses
standalone_path for cores the platform runs standalone, filters by
mode: libretro/standalone per file.
batocera.yml gains standalone_cores (92 entries from configgen-defaults).
generate_readme.py dynamically lists platforms from registry.
3 profiles updated for standalone type/path (mame, hatari, mupen64plus_next).
78 E2E tests pass, pipeline verified.
2026-03-26 00:44:21 +01:00
Abdessamad Derraz
d2cc9b8f29
feat: add doom engine wad files, emulatorjs base config
2026-03-25 23:12:53 +01:00
Abdessamad Derraz
38059e0fdf
chore: regenerate database, readme, mkdocs nav
2026-03-25 16:49:49 +01:00
Abdessamad Derraz
6e237e3b6d
chore: regenerate database, readme, mkdocs nav
2026-03-25 15:54:17 +01:00
Abdessamad Derraz
5529f8474a
chore: regenerate database and readme
2026-03-25 15:41:46 +01:00
Abdessamad Derraz
dbc26b11c1
refactor: move fetch_large_file to common, auto-download on db rebuild
2026-03-25 13:19:12 +01:00
Abdessamad Derraz
910428c6f1
fix: resolve large files from cache in database paths
2026-03-25 12:52:20 +01:00
Abdessamad Derraz
47e6174ed4
fix: pack naming, large file preservation, discrepancy reporting
2026-03-25 12:23:40 +01:00
Abdessamad Derraz
ebb55a445b
feat: re-profile 40 emulators, harden CI workflows
...
profile emulators pd777 through tic80, add frozen snapshots
(puae2021, snes9x2002/2005/2010, stella2014/2023).
CI: replace github-script with gh CLI, add test execution,
job-level permissions, propagate changed output, pin jsonschema.
2026-03-25 07:00:17 +01:00
Abdessamad Derraz
94000bdaef
fix: align verify and pack validation, pipeline 100% consistent
...
generate_pack.py now applies emulator-level validation (crc32, sha1,
adler32) matching verify.py behavior. existence mode: validation is
informational (file present = OK). md5 mode: validation downgrades
to UNTESTED. clone resolution moved to common.py resolve_local_file.
all 6 platforms pass consistency check.
2026-03-24 22:21:47 +01:00
Abdessamad Derraz
ae4846550f
fix: clone resolution in common.py, move clone map to root
...
moved _mame_clones.json out of bios/ (was indexed by generate_db.py
as BIOS file). clone resolution now in common.py resolve_local_file
so all tools (verify, pack, cross_reference) resolve clones
transparently. removed duplicate clone code from generate_pack.py.
added error handling on os.remove in dedup.py. consistency check
now passes for Batocera/EmuDeck/Lakka/RetroArch (4/6 platforms).
2026-03-24 21:57:49 +01:00
Abdessamad Derraz
fb1007496d
chore: deduplicate bios/ — remove 427 files, save 227 MB
...
true duplicates (same file in multiple dirs): removed copies, kept
canonical. MAME device clones (different names, identical content):
removed copies, created _mame_clones.json mapping for pack-time
assembly via deterministic ZIP rebuild. generate_pack.py resolves
clones transparently. 95 canonical ZIPs serve 392 clone names.
2026-03-24 21:35:50 +01:00
Abdessamad Derraz
69987cc0bb
feat: re-profile 30 emulators, beetle to parallel_n64
...
source-verified: mesen FdsBios.bin→alias, MesenDB.txt phantom (embedded).
mupen64plus 64DD_IPL.bin→IPL.n64 (code path fix). neocd 000-lo.lo/ng-lo.rom
phantoms (algorithmically generated). new profiles: mesen-s, mupen64plus
next develop/gles2/gles3. regenerate database.
2026-03-24 14:06:20 +01:00
Abdessamad Derraz
ef25f8cebf
feat: re-profile 40 emulators from fbalpha2012 to holani
...
batch re-profiled beetle, holani, kronos, mame variants, mednafen
cores with upstream verification. beetle_pce, mame, mame2010 excluded
(regressions detected, need manual review).
2026-03-24 11:36:58 +01:00
Abdessamad Derraz
4c1caea621
feat: re-profile 40 emulators from fbalpha2012 to holani
2026-03-24 09:33:06 +01:00
Abdessamad Derraz
65b6ab9354
feat: re-profile 14 emulators, add dosbox and ep128emu_core
...
batch re-profiled dosbox_pure through fbalpha2012_cps1 with upstream
verification. new profiles: dosbox, ep128emu_core. regenerate database.
2026-03-23 15:04:43 +01:00
Abdessamad Derraz
438fb6e9af
feat: re-profile 12 emulators, fix dosbox_core regression
...
batch re-profiled clownmdemu through dosbox_core with upstream
verification. restored dosbox_core mt-32/cm-32l rom halves removed
by batch (14 files), added core_classification. regenerate database.
2026-03-23 12:37:43 +01:00
Abdessamad Derraz
78fa5c3d0f
feat: re-profile citra, regenerate database
2026-03-23 11:02:53 +01:00
Abdessamad Derraz
06f5157c35
feat: profile mame, mame2010, mame2015, mame2016, fix aliases
...
source-verified BIOS root sets for all MAME cores:
- mame (0.286): 99 sets, official_port, new profile
- mame2010 (0.139): 51 sets, remove cvs.zip phantom
- mame2015 (0.160): 69 sets, replace false mame2016 alias
- mame2016 (0.174): 69 sets, fix 3 phantoms, add 2 missing
- mamearcade/mamemess: fix alias_of mame2016 to mame
2026-03-22 22:30:11 +01:00