Commit Graph
37 Commits
Author SHA1 Message Date
Abdessamad Derraz d2e1cbf8f0 feat: collect the firmware BizHawk 2.11.1 names
40 files BizHawk 2.11.1's FirmwareDatabase.cs names that the collection lacked, each verified against the sha1 it states: Mega CD and Master System revisions, TI-83 and TI-83 Plus ROMs, Game Boy and Game Boy Color boot ROMs, Super Game Boy, NDS firmware, 64DD IPL, 3DO and LaserActive PAC-S10 v1.02 (No-Intro [BIOS] LaserActive (USA) (v1.02)). Sources: archive.org bizhawk-firmware-files_20250516 and pioneer-laseractive-bios-pack.
2026-10-20 19:21:42 +02:00
Abdessamad Derraz ef8b372810 chore: drop copies only an install provides
A file that arrives with the emulator is already on the user's machine
and the packs leave it out. 4558 collected copies (1.4 GB) were reached
only by such entries, with no platform, other profile entry or install
manifest needing them. They are removed, and two release-asset copies
leave the ignore list. bundled_only.py now lists every copy of a listed
file, since the database keeps one path per content, and pipeline step
4d fails while one remains.
2026-10-20 10:17:08 +02:00
Abdessamad Derraz 63ded27336 feat: extract the DSi font tables from held NANDs
DSperate reads TWLFontTable.dat for DSiWare run without a NAND. The file sits in nand:/sys/ of the DSi NAND dumps the collection holds. Decrypted with the console ID and CID in each dump's nocash footer, as DSperate's own NAND reader does, the four western and Japanese consoles give one table, the Chinese and Korean ones their own: 863296, 581664 and 162688 bytes, the sizes and sha1 already known for them.
2026-10-20 03:17:57 +02:00
Abdessamad Derraz 5fa3f2aebe fix: drop variants whose bytes a file holds
A copy under .variants/ exists to hold a content its primary does
not. 29 held bytes another file already carried, under a second or
third identity: msx.rom and MSX.ROM variants equal to
MSX_8020-20bios.rom, naomi2.zip.da79eca4 whose suffix named the
primary's sha1 rather than its own. dedup now removes such a copy,
keeping one when only variants hold the content. The LaserActive
PAC-S10 BIOS lived only under .variants/ while RetroBat names it
without a hash; it moves beside its siblings.
2026-10-18 02:31:13 +02:00
Abdessamad Derraz e506cbeb2f fix: identify the files daedalus and touchhle ship 2026-10-17 16:58:37 +02:00
Abdessamad Derraz d9c74888e8 feat: list rocknix cores per device
ROCKNIX builds a different emulator set per SoC, and without a target
list an H700 handheld received the PS2, PS3 and GameCube firmware of
emulators its image lacks. The new target scraper reads the emulators
metapackage per device; seven cores ROCKNIX ships get profiles and the
Super Famicom Box ROMs one of them needs are collected.
2026-10-17 15:38:07 +02:00
Abdessamad Derraz 776e50f3c2 fix: file the older switch keys as variants 2026-10-06 08:50:38 +02:00
Abdessamad Derraz 3f65defbf0 feat: collect ap6, shareddata, introset, a2 scsi 2026-10-05 02:51:23 +02:00
Abdessamad Derraz 6148ab2e55 feat: collect files for the september profiles 2026-10-04 16:35:48 +02:00
Abdessamad Derraz 08ef1e8dcf fix: fail batocera target scraper on empty input 2026-10-04 14:36:32 +02:00
Abdessamad Derraz 1873b9fa37 chore: refile two ds firmware dumps under nintendo 2026-09-07 04:34:12 +02:00
Abdessamad Derraz 50b22d34fc feat: build the 3ds client certificate from a console otp 2026-08-12 05:38:37 +02:00
Abdessamad Derraz 3a58167823 chore: keep one copy per unique content 2026-08-11 05:22:18 +02:00
Abdessamad Derraz 8663ec17b7 feat: add misterfpga platform support 2026-08-07 13:28:15 +02:00
Abdessamad Derraz f88aeee6ff feat: add bios from verified upstream sources 2026-08-06 10:40:18 +02:00
神龟 e72a5e80b9 feat: add gba j debug bios 2026-08-06 14:59:16 +08:00
Abdessamad Derraz 51c25ea4ce feat: extend 3ds aes key list from retail set 2026-08-06 05:00:50 +02:00
Abdessamad Derraz d685ad111d feat: add 63 missing BIOS files from multiple sources
MAME/MESS ROM sets (23 ZIPs from archive.org MESS 0.151 + MAME 0.285 split),
Dolphin BT firmware (15 files from linux-firmware), Commodore VIC-20/Plus4
regional ROMs (8 from zimmers.net), IBM AT 5170 BIOS + BASIC (3 from
minuszerodegrees.net + PCem), Apple DiskII boot/state-machine ROMs (4 from
Asimov mirror), Commodore 1540/1541 drive ROMs (2 reconstructed from MAME
chips), Oric Microdisc ROM, SMS Japanese BIOS, Videoton TVC64 + BK-0010
enriched ZIPs. All platforms at 0 required missing except Batocera bk0010.zip
(2/11 ROM chips unavailable).
2026-03-31 11:00:31 +02:00
Abdessamad Derraz 44145d41cd chore: remove 399 bios/ files now sourced from buildbot
Files verified by MD5 to be identical to their buildbot-fetched
copies in data/. resolve_local_file data directory fallback ensures
they remain resolvable for verify and pack generation.
2026-03-29 11:13:08 +02:00
Abdessamad Derraz 718ea2192d feat: add FDS original BIOS and GameCube DSP hardware dumps
disksys.rom non-Rev1 variant (SHA256 fdc1a76e, ares-compatible)
from Myrient No-Intro. GameCube dsp_rom.bin + dsp_coef.bin real
hardware dumps (adler32 match Dolphin) from Redump collection.
All placed as .variants/ — primaries unchanged.
2026-03-29 09:35:48 +02:00
Abdessamad Derraz 48598d61b0 feat: add Wii keys.bin from archive.org NAND dump
1024-byte OTP/SEEPROM dump for Dolphin/PrimeHack/Ishiiruka
NAND import. From archive.org nand_20240722 collection.
2026-03-29 01:01:47 +01:00
Abdessamad Derraz 5413c6a5ac feat: add Cemu Wii U sound RPLs and Lexaloffle Voxatron
snd_user.rpl + snduser2.rpl from Cafe SDK 2.12.13 archive.org.
Voxatron v0.3.5b binaries (vox, vox_dyn, vox.dat) from
archive.org pico-8-tron-and-voxatron. libHoloPlayCore.so from
Looking Glass SDK GitHub. All 6 "impossible" files found.
2026-03-29 00:21:55 +01:00
Abdessamad Derraz 3eb6e0e530 feat: add Dolphin Wii certs, fMSX GameMaster ROMs
Wii SSL certs (clientca/clientcakey/rootca.pem) from archived
Nintendo HTTPS certs. SYSCONF and setting.txt generated with
correct format. Konami GameMaster 1+2 cartridge ROMs for fMSX.
2026-03-28 20:57:41 +01:00
Abdessamad Derraz 3d5d5b601c feat: mass BIOS collection (+1020 files, 7520 total)
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.

New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz 84d344450f feat: add vice, qemu, pcem, freedoom, x16, iigs, dsi bios
VICE: basic, chargen, kernal for x64sdl libretro core.
QEMU: SeaBIOS 1.16.3 + VGA BIOS from system package.
PCem: MDA font ROM from PCem v17 ROM set.
Freedoom: freedoom1.wad + freedoom2.wad for prboom.
Commander X16: rom.bin R49 from x16-rom releases.
Apple IIGS: ROM01 from asimov mirrors.
DSi: NDS_Bios7i, NDS_Bios9i, DSi_Firmware, SGB_(JU).
2026-03-28 11:32:58 +01:00
Abdessamad Derraz 1f65662fd2 feat: add ~110 missing bios files from archive.org and bizhawk
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.

Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz 97b9900f62 chore: update NstDatabase.xml from upstream nestopia
Closes #40
2026-03-28 08:24:13 +01:00
Abdessamad Derraz d4849681a7 feat: add 3DS signature/crypto verification to verify.py
pure python RSA-2048 PKCS1v15 SHA256 for SecureInfo_A,
LocalFriendCodeSeed_B, movable.sed. AES-128-CBC + SHA256 for otp.bin.
keys extracted from azahar default_keys.h, added RSA/ECC sections
to aes_keys.txt. sect233r1 ECC not reproducible (binary field curve).
2026-03-24 11:36:29 +01:00
Abdessamad Derraz 1d350f0578 feat: add emulator/system pack generation, validation checks, path resolution
add --emulator, --system, --standalone, --list-emulators, --list-systems
to verify.py and generate_pack.py. packs are RTU with data directories,
regional BIOS variants, and archive support.

validation: field per file (size, crc32, md5, sha1) with conflict
detection. by_path_suffix index in database.json for regional variant
resolution via dest_hint. restructure GameCube IPL to regional subdirs.

66 E2E tests, full pipeline verified.
2026-03-22 14:02:20 +01:00
Abdessamad Derraz 74f17694c2 feat: add category field to emulator profiles, source missing BIOS
Add category: game_data to sdlpal, nxengine, opentyrian, easyrpg,
mkxp_z profiles. verify.py separates game_data from bios in core
gap metrics for cleaner coverage numbers.

New BIOS files: Cemu fonts (4), QEMU bios-256k + vgabios-stdvga,
GAM4980 ROMs (2), SC-3000 Export variant.
2026-03-21 07:37:22 +01:00
Abdessamad Derraz 0451ff6b67 fix: restore dolphin DSP/font firmware to bios/, integration test pass
dsp_coef.bin, dsp_rom.bin, font_western.bin, font_japanese.bin copied
back to bios/Nintendo/GameCube/ (needed by verify.py, were moved to
data/ with the Sys folder). Pack includes 2537 files with Dolphin Sys.
2026-03-18 16:13:00 +01:00
Abdessamad Derraz e07d85fc63 refactor: move dolphin Sys to data/, reduce DB by 2500 entries 2026-03-18 15:59:02 +01:00
Abdessamad Derraz bb307aa250 feat: archive full dolphin-emu/Sys, add DSP/font/IPL paths to pack
dolphin-emu/Sys/ folder (2562 files) from libretro/dolphin Data/Sys.
retroarch.yml: DSP firmware (dsp_coef.bin, dsp_rom.bin), fonts
(font_western.bin, font_japanese.bin) at dolphin-emu/Sys/GC/ paths.
ref: DolphinLibretro/Boot.cpp:72-73, HW/DSPLLE/DSPHost.cpp,
HW/EXI/EXI_DeviceIPL.cpp. pack: 452 files, 0 missing.
2026-03-18 15:16:20 +01:00
Abdessamad Derraz 617f5a3654 feat: expand bios collection, emudeck at 97.6%
Add 36 new files from multiple sources:
- PSX rare variants (DTL-H2000, SCPH-5903 VCD, SCPH-7000/7000W, DTL-H1100)
- Sega Mega CD 9 region variants (EU/JP/US firmware revisions)
- Saturn HiSaturn GameNavi v1.03
- N64DD IPL (US/JP/DEV), PS2 SCPH-70004, Xbox Complex BIOS
- PPSSPP system pack, 3DS/Switch/Wii U keys, Nokia N-Gage
- Apple Macintosh mac755.chd

Platform coverage:
  RetroArch  398/398 (100%)
  Lakka      398/398 (100%)
  RetroPie   398/398 (100%)
  Batocera   589/680 (86.6%, 91 untested MAME versions)
  Recalbox   336/346 (97.1%, 10 untested)
  RetroBat   341/343 (99.4%, 2 untested)
  EmuDeck    160/164 (97.6%, 4 missing rare dumps)
2026-03-17 14:15:33 +01:00
Abdessamad Derraz 4d98f17308 feat: add dolphin dsp, mame hash xmls, scummvm fonts from retrobat-bios 2026-03-17 12:42:55 +01:00
Abdessamad Derraz e6ea0484a8 expand bios collection, retrobat at 93% coverage 2026-03-17 11:53:42 +01:00
Abdessamad Derraz 13c561888d v2: automated BIOS platform with full pipeline
Reorganized 6 branches into bios/Manufacturer/Console/.
Scrapers for RetroArch, Batocera, Recalbox, and libretro core-info.
Platform-aware verification replicating native logic per platform.
Pack generation with dedup, alias resolution, variant support.
CI/CD: weekly auto-scrape, auto-release, PR validation.
Large files (>50MB) stored as GitHub Release assets, auto-fetched at build time.
2026-03-17 10:54:39 +01:00