Commit Graph

67 Commits

Author SHA1 Message Date
Abdessamad Derraz
5413c6a5ac feat: add Cemu Wii U sound RPLs and Lexaloffle Voxatron
snd_user.rpl + snduser2.rpl from Cafe SDK 2.12.13 archive.org.
Voxatron v0.3.5b binaries (vox, vox_dyn, vox.dat) from
archive.org pico-8-tron-and-voxatron. libHoloPlayCore.so from
Looking Glass SDK GitHub. All 6 "impossible" files found.
2026-03-29 00:21:55 +01:00
Abdessamad Derraz
99c8fd62f0 feat: derive Amiga rom.key via XOR cryptanalysis
Cloanto decryption key (2069 bytes) derived by XOR-ing
encrypted and plain Kickstart ROM pairs from archive.org
commodore-amiga-firmware collection. Unlocks 5 emulators
(Amiberry, PUAE, PUAE2021, FS-UAE, uae4arm).
2026-03-29 00:20:00 +01:00
Abdessamad Derraz
e91caa6444 feat: add Saturn AR4MP firmware and QEMU Virtex DTB
satar4mp.bin (256KB) extracted from existing ar_bios.zip
MAME ROM. virtex-ml507.dtb compiled from Linux 4.19 DTS
with dtc. Both resolve long-standing optional missing files.
2026-03-29 00:07:40 +01:00
Abdessamad Derraz
054cb940b0 feat: resolve last 3 required files + 26 optional
CLK os.rom (Acorn Electron MOS), exos10.bin and basic10.bin
(Enterprise EXOS/BASIC 1.0) from TOSEC/Myrient — zero
required missing across all platforms. Also add Enterprise
ROMs, ZX Spectrum 128/+2/+3, MSX CLK variants, FBA2012
samples from progettosnaps.
2026-03-28 22:18:19 +01:00
Abdessamad Derraz
c00f4bca80 feat: add PC-98 IDE/SCSI ROMs, OpenTyrian Tyrian 2000 data
PC-98 ide.rom and scsi.rom from archive.org np2kai_bios pack.
Tyrian 2000 Episode 5 data + shapes + sprites moved to data/
opentyrian/ (53 files from archive.org msdos_Tyrian_2000).
2026-03-28 22:00:35 +01:00
Abdessamad Derraz
9e331687a6 feat: generate defaults and extract compiled-in ROMs
WiiSD.raw/sd.raw (128MB FAT32), eeprom.dat (PCSX2 DEV9),
CMOS.ROM (fMSX RTC), key.txt (NP2kai), wildmidi.cfg
(EasyRPG), NXEngine sprites.sif+tilekey.dat extracted from
source headers, Frodo C64 ROMs extracted from compiled-in
arrays, Custom.dat (ZC210 empty default).
2026-03-28 21:30:30 +01:00
Abdessamad Derraz
7b0527bb0c feat: add FM Towns FMT_ALL.ROM combined container
Built from individual ROMs (FMT_SYS, FMT_DOS, FMT_FNT,
FMT_F20, FMT_DIC, MYTOWNS, MAR_EX0-3) using the tsugaru
structured container format (12-byte name + length + data).
2026-03-28 21:04:12 +01:00
Abdessamad Derraz
3eb6e0e530 feat: add Dolphin Wii certs, fMSX GameMaster ROMs
Wii SSL certs (clientca/clientcakey/rootca.pem) from archived
Nintendo HTTPS certs. SYSCONF and setting.txt generated with
correct format. Konami GameMaster 1+2 cartridge ROMs for fMSX.
2026-03-28 20:57:41 +01:00
Abdessamad Derraz
6c4f18056b feat: add Rustation PS1 aliases, FBNeo arcade BIOS, misc
8 Rustation PS1 BIOS (SHA256-verified copies of existing files
under Rustation-expected names). 4 FBNeo arcade BIOS extracted
from MAME game ROMs (236-bios.sp1, bios.u42, svg_bios.u49).
DuckStation ps2_scph18000.bin resolved from existing ps2-50j.
2026-03-28 20:53:23 +01:00
Abdessamad Derraz
a418f7ef68 feat: add Dragon CoCo ROMs and mameinfo.dat
cdos20.rom (Cumana DOS 2.0), yados.rom (YA-DOS 0.5a),
dragonfly-1.3.rom (Ikon Ultra Drive v1.3) from World of
Dragon Archive and CCHDD GitHub. delta-premier.rom variant.
mameinfo.dat for MAME 2000/2003/2003-Plus game info.
2026-03-28 19:09:42 +01:00
Abdessamad Derraz
63f2450943 feat: add Xbox BIOS variants and PC-98 sasi.rom
8 Xbox BIOS variants (mcpx-1.1, xbox-3944 through 5838)
from Myrient MAME bios-devices. PC-98 SASI controller ROM
for NP2kai/nekop2. Cross-reference: 146 true missing.
2026-03-28 18:50:59 +01:00
Abdessamad Derraz
2ff5be8420 feat: complete DuckStation PS1/PS2 BIOS coverage
Add 9 PS2 BIOS from Myrient (pre-shutdown raid), fix
incorrect scph5000.bin, add DuckStation-named symlinks
for 80 PS2 entries. Add Saturn cdb_rom, Fuse peripheral
ROMs, Amiberry MT-32 aliases, DOSBox soundfont. PS1
coverage: 25/25. PS2 coverage: 80/80 via MD5 matching.
2026-03-28 18:26:57 +01:00
Abdessamad Derraz
536c0c242e fix: resolve descriptive names via dest_hint basename fallback
resolve_local_file now tries the path: field basename when name:
lookup fails. Fixes 139 PCem false positives where descriptive
names ("MDA font ROM") didn't match actual filenames (mda.rom).
Also add 3 QEMU firmware (MacROM.bin, bios_loongson3, pmon_2e).
Cross-reference path: fallback already added. 149/149 tests pass.
2026-03-28 18:24:53 +01:00
Abdessamad Derraz
31ec407c91 feat: add FBNeo st010/st011/sfiii3na, PCem rivatnt, misc
sfiii3na.zip (CPS3 Asia No-CD), st010.zip and st011.zip
(SNES coprocessors) complete FBNeo required coverage.
rivatnt.bin completes PCem at 139/139. Also add QEMU
firmware (4), FM Towns Marty ROMs (4), Fuse mf1.rom,
Hatari BOOT.ST, PS1 extracts (2), ZC210 data (2).
2026-03-28 17:42:53 +01:00
Abdessamad Derraz
3d5d5b601c feat: mass BIOS collection (+1020 files, 7520 total)
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.

New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz
1ee8623b2e feat: add 47 missing bios files and fix cross-reference false positives
Add BIOS files for CLK (Apple II, Mac, Atari ST, Enterprise, MSX,
Commodore, Thomson, PC, Acorn), FBNeo (CPS3 redearthn/sfiii2h),
and QEMU variants. Sources: Asimov mirror, MAME chip extraction,
86Box, Theodore, XRoar upstream, official QEMU repo.

Fix cross-reference in verify.py to group archived files by archive
unit instead of reporting individual ROMs as missing. Add path-based
fallback for descriptive names. Update generate_pack.py extras to
use archive name for resolution. 6 new E2E tests.

MISSING required: 321 -> 6 (zero false positives).
2026-03-28 13:59:48 +01:00
Abdessamad Derraz
645dc013a4 feat: add qemu bios hash variants 2026-03-28 12:33:03 +01:00
Abdessamad Derraz
5b5b304288 feat: add dragon and coco clone bios variants 2026-03-28 12:33:02 +01:00
Abdessamad Derraz
460c49dbcb fix: bizhawk scraper lowercase sha1 + complete bios 2026-03-28 11:49:52 +01:00
Abdessamad Derraz
a5285a9fc0 feat: add ti83 v1.10 and doom2.wad, zero platform missing
TI-83 v1.10 ROM from TOSEC archive (sha1 verified).
DOOM II WAD from archive.org for prboom libretro core.
All platforms now at 0 missing files.
2026-03-28 11:40:06 +01:00
Abdessamad Derraz
84d344450f feat: add vice, qemu, pcem, freedoom, x16, iigs, dsi bios
VICE: basic, chargen, kernal for x64sdl libretro core.
QEMU: SeaBIOS 1.16.3 + VGA BIOS from system package.
PCem: MDA font ROM from PCem v17 ROM set.
Freedoom: freedoom1.wad + freedoom2.wad for prboom.
Commander X16: rom.bin R49 from x16-rom releases.
Apple IIGS: ROM01 from asimov mirrors.
DSi: NDS_Bios7i, NDS_Bios9i, DSi_Firmware, SGB_(JU).
2026-03-28 11:32:58 +01:00
Abdessamad Derraz
dd689bbd2a feat: add name aliases for bizhawk platform resolution
Copy existing BIOS files under BizHawk-expected names:
MCD_eu_100.bin, MCD_jp_100p.bin, PSX_DTLH2000.bin,
MSX_4000bios.rom, MSX_8020-20bios.rom.
2026-03-28 10:29:13 +01:00
Abdessamad Derraz
1f65662fd2 feat: add ~110 missing bios files from archive.org and bizhawk
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.

Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz
97b9900f62 chore: update NstDatabase.xml from upstream nestopia
Closes #40
2026-03-28 08:24:13 +01:00
Abdessamad Derraz
3672912de7 chore: clean up sc3000 variant swap, update database 2026-03-27 22:50:54 +01:00
Abdessamad Derraz
59d7582c0e fix: swap sc3000.zip primary to batocera-compatible dump 2026-03-27 16:50:10 +01:00
Abdessamad Derraz
e94ce6b194 feat: source 10 missing bios variants for romm and retrobat coverage 2026-03-27 16:00:10 +01:00
Abdessamad Derraz
181248b6db fix: case-sensitive packs for linux platforms, remove empty bios placeholder 2026-03-27 12:58:08 +01:00
Abdessamad Derraz
d2cc9b8f29 feat: add doom engine wad files, emulatorjs base config 2026-03-25 23:12:53 +01:00
Abdessamad Derraz
94000bdaef fix: align verify and pack validation, pipeline 100% consistent
generate_pack.py now applies emulator-level validation (crc32, sha1,
adler32) matching verify.py behavior. existence mode: validation is
informational (file present = OK). md5 mode: validation downgrades
to UNTESTED. clone resolution moved to common.py resolve_local_file.
all 6 platforms pass consistency check.
2026-03-24 22:21:47 +01:00
Abdessamad Derraz
ae4846550f fix: clone resolution in common.py, move clone map to root
moved _mame_clones.json out of bios/ (was indexed by generate_db.py
as BIOS file). clone resolution now in common.py resolve_local_file
so all tools (verify, pack, cross_reference) resolve clones
transparently. removed duplicate clone code from generate_pack.py.
added error handling on os.remove in dedup.py. consistency check
now passes for Batocera/EmuDeck/Lakka/RetroArch (4/6 platforms).
2026-03-24 21:57:49 +01:00
Abdessamad Derraz
fb1007496d chore: deduplicate bios/ — remove 427 files, save 227 MB
true duplicates (same file in multiple dirs): removed copies, kept
canonical. MAME device clones (different names, identical content):
removed copies, created _mame_clones.json mapping for pack-time
assembly via deterministic ZIP rebuild. generate_pack.py resolves
clones transparently. 95 canonical ZIPs serve 392 clone names.
2026-03-24 21:35:50 +01:00
Abdessamad Derraz
d4849681a7 feat: add 3DS signature/crypto verification to verify.py
pure python RSA-2048 PKCS1v15 SHA256 for SecureInfo_A,
LocalFriendCodeSeed_B, movable.sed. AES-128-CBC + SHA256 for otp.bin.
keys extracted from azahar default_keys.h, added RSA/ECC sections
to aes_keys.txt. sect233r1 ECC not reproducible (binary field curve).
2026-03-24 11:36:29 +01:00
Abdessamad Derraz
1d350f0578 feat: add emulator/system pack generation, validation checks, path resolution
add --emulator, --system, --standalone, --list-emulators, --list-systems
to verify.py and generate_pack.py. packs are RTU with data directories,
regional BIOS variants, and archive support.

validation: field per file (size, crc32, md5, sha1) with conflict
detection. by_path_suffix index in database.json for regional variant
resolution via dest_hint. restructure GameCube IPL to regional subdirs.

66 E2E tests, full pipeline verified.
2026-03-22 14:02:20 +01:00
Abdessamad Derraz
74f17694c2 feat: add category field to emulator profiles, source missing BIOS
Add category: game_data to sdlpal, nxengine, opentyrian, easyrpg,
mkxp_z profiles. verify.py separates game_data from bios in core
gap metrics for cleaner coverage numbers.

New BIOS files: Cemu fonts (4), QEMU bios-256k + vgabios-stdvga,
GAM4980 ROMs (2), SC-3000 Export variant.
2026-03-21 07:37:22 +01:00
Abdessamad Derraz
27df5c8fb5 fix: resolve case collisions on case-insensitive filesystems
Move 17 case-colliding BIOS variants to .variants/ so Windows
and macOS clones no longer lose files. Existence-based platforms
keep the primary, MD5-based platforms resolve from .variants/.

Also fix resolve_local_file zipped_file resolution: when multiple
ZIPs share a name, verify the inner ROM exists before accepting
a candidate. Fixes adam_fdc.zip resolving to the wrong archive.
2026-03-20 20:02:42 +01:00
Abdessamad Derraz
59ee9088fa chore: add Atari 1450XLD OS Rev 3 (1984-03-27) 2026-03-20 12:52:34 +01:00
Abdessamad Derraz
1c9261d304 chore: add Atari 1450XLD, Arabic 65XE, compiled OS 2026-03-20 10:04:06 +01:00
Abdessamad Derraz
4a76c5ca15 chore: add Atari OS-A NTSC, XEGS, doc format ares 2026-03-20 09:50:59 +01:00
Abdessamad Derraz
7b712ca089 chore: add Game Master, MSX, Atari BIOS files 2026-03-20 07:06:11 +01:00
Abdessamad Derraz
789ff0a32b chore: add MAME and RetroDECK ROM sets 2026-03-20 06:52:57 +01:00
Abdessamad Derraz
6ee162f8fb chore: add MAME and RetroDECK ROM sets 2026-03-19 23:26:49 +01:00
Abdessamad Derraz
fbb2079f9b feat: complete 81 profile, archive 27 standalone ROMs 2026-03-18 17:52:12 +01:00
Abdessamad Derraz
846640dd7c feat: emulator mode field, archive ZX81 standalone ROMs
emulator profiles support mode: standalone | libretro | both.
cross_reference.py skips standalone-only files for libretro platforms.
81.yml: type standalone + libretro, upstream ref added, files listed
with mode: standalone and source_refs to both codebases.
bios/Sinclair/ZX 81/: zx81.rom (8K) and dkchr.rom (4K) archived.
2026-03-18 17:37:01 +01:00
Abdessamad Derraz
0451ff6b67 fix: restore dolphin DSP/font firmware to bios/, integration test pass
dsp_coef.bin, dsp_rom.bin, font_western.bin, font_japanese.bin copied
back to bios/Nintendo/GameCube/ (needed by verify.py, were moved to
data/ with the Sys folder). Pack includes 2537 files with Dolphin Sys.
2026-03-18 16:13:00 +01:00
Abdessamad Derraz
e07d85fc63 refactor: move dolphin Sys to data/, reduce DB by 2500 entries 2026-03-18 15:59:02 +01:00
Abdessamad Derraz
bb307aa250 feat: archive full dolphin-emu/Sys, add DSP/font/IPL paths to pack
dolphin-emu/Sys/ folder (2562 files) from libretro/dolphin Data/Sys.
retroarch.yml: DSP firmware (dsp_coef.bin, dsp_rom.bin), fonts
(font_western.bin, font_japanese.bin) at dolphin-emu/Sys/GC/ paths.
ref: DolphinLibretro/Boot.cpp:72-73, HW/DSPLLE/DSPHost.cpp,
HW/EXI/EXI_DeviceIPL.cpp. pack: 452 files, 0 missing.
2026-03-18 15:16:20 +01:00
Abdessamad Derraz
d920231c50 feat: add bioscv.rom, GameIndex.yaml, enrich mu.yml with MD5s 2026-03-18 14:22:19 +01:00
Abdessamad Derraz
c71378a71c feat: shared groups, scraper subdir prefixes, arcade + emulator profiles
shared groups in _shared.yml: np2kai, keropi, quasi88, kronos, ep128emu
with source references for each subdirectory requirement.

libretro_scraper: CORE_SUBDIR_MAP applies subdirectory prefixes at
generation time (np2kai/, keropi/, dc/). EXTRA_SYSTEMS adds QUASI88.
SYSTEM_SHARED_GROUPS injects includes for kronos/np2kai/keropi.

new BIOS: CPS3 (23 ZIPs), Cannonball OutRun (40 ROMs), PCem PC BIOS
(73 files), VICE Commodore ROMs, Spectrum ZIPs, dc_bios.bin, X1 fonts.

new emulator profiles: redream, melonds_ds, lrps2 with doc vs source
notes. platforms/README.md documents shared groups architecture.
2026-03-18 13:49:59 +01:00
Abdessamad Derraz
7653d5d108 feat: add 19 BIOS files, fix cross_reference resolution
New files: OpenTyrian data (11), Cave Story (2), SeaBIOS,
VGA BIOS, OpenSBI, Cromwell, xbox_hdd, Sega CD Model 2 (3),
NGP Color BIOS, Pentagon 128p-1.rom, X1 font, BK TERAK.
cross_reference.py: basename + case-insensitive lookup.
2026-03-18 12:50:55 +01:00