Commit Graph
41 Commits
Author SHA1 Message Date
Abdessamad Derraz 15568b37aa feat: reprofile beetle_saturn, collect mpeg cards 2026-10-05 02:51:22 +02:00
Abdessamad Derraz 620e1244e6 feat: collect firmware and flagged backlog files 2026-10-04 23:47:55 +02:00
Abdessamad Derraz 6148ab2e55 feat: collect files for the september profiles 2026-10-04 16:35:48 +02:00
Abdessamad Derraz 47e267d29c chore: drop files whose bytes are not what they claim 2026-09-05 09:17:42 +02:00
Abdessamad Derraz cbb81fb176 feat: collect desktop emulator bundle files 2026-09-03 22:05:27 +02:00
Abdessamad Derraz 9bd1cfec9e fix: serve the real c128 swedish kernal
bios/Commodore/C128/vice/C128/kernal-318034-01.bin held 32768 bytes
where VICE reads 16384. It was two ROMs end to end: kernal64-325182-01
followed by the Swedish kernal the name promises. The RetroArch, Lakka
and RetroPie manifests shipped it, so the file reaching users was one
the emulator cannot load.

The second half is the genuine dump, and it matches every value
vice/src/c128/c128rom.h:104 declares: 16384 bytes, sha1 925177c9,
md5 78529222, crc32 1cf7f729. The same bytes are in the VICE tree at
the revision the profile pins. The assembly is kept in .variants/ under
its own hash, though both halves were already held separately.

Supermodel's Games.xml is collected in the same pass, from the commit
the profile pins. It is the ROM database the loader parses at startup,
so without it no set resolves and the launch returns before the
emulator is built.
2026-08-23 08:10:17 +02:00
Abdessamad Derraz 3a58167823 chore: keep one copy per unique content 2026-08-11 05:22:18 +02:00
Abdessamad Derraz 76984e8bce feat: close the last anchored gaps
Four files, each with a verifiable origin.

logo.tga and logo_red.tga come from the loader's own resources/logo.h:
hook.c's fread maps FILE_LOGO_TGA to logoLL or logoLLRed depending on
which path was opened, one array per file, so the extraction is exact.
Both decode as valid RLE TGA at their declared lengths.

lgp.zip and timetrav.zip carry crc 0 in Hypseus and in Daphne upstream,
so MAME vouches for them instead. lgp maps onto MAME's set by base name,
confirmed by the five members whose bytes already agreed, and the four
that were missing came from the same set. timetrav is MAME's timetrv:
tt061891.bin, 262144 bytes, crc a3d44219.

The four Sega fonts stay open, and the profile now says why: the loader
embeds one 128x256 atlas and one metrics table answering all four paths,
so it substitutes for the originals rather than reproducing either.
2026-08-10 15:13:15 +02:00
Abdessamad Derraz bb915ed6aa feat: file triforce ipl and segaboot variants
bios/Sega/Triforce/.variants/segaboot.gcm was never a SegaBoot image:
its header reads "(C) 1999-2001 Nintendo" and it matches Redump's
"Triforce (ARCADE BOOTROM)". crediar's Dolphin fork reads the IPL from
GC/<region>/IPL_TRI.bin and SegaBoot from Triforce/segaboot.gcm, two
separate paths, so it moves to GC/USA/IPL_TRI.bin.

segaboot.gcm.e475c7dc is SegaBoot v3.11, the sha1 emulators/triforce.yml
already declared with no local file.

spec128.zip.1a524bfa is rebuilt from the FinalBurn Neo ZX Spectrum
recipe and roms already held, closing one of the pinned archives listed
as unobtainable.
2026-08-10 13:36:44 +02:00
Abdessamad Derraz 01dc9b2539 feat: add openh264 and lindbergh runtimes
Fourteen files a profiled core loads and the collection lacked.

Eight OpenH264 2.4.1 binaries for ruffle, from ciscobinary.openh264.org.
Cisco serves only the bzip2 artifact; the emulator loads the
decompressed library under the plain name. All eight sha256 match both
emulators/ruffle.yml and ruffle's own table in
video/external/src/decoder/openh264.rs.

Six Lindbergh runtimes: libCg.so and libCgGL.so from NVIDIA's
Cg-3.1_April2012_x86.tgz, libCg2.so from Cg-2.0_Jan2008_x86.tar.gz, and
the three libraries the loader vendors in libs/.

Its logo and font files are not external: hook.c:655-709 serves them
from resources/logo.h and resources/font.h unless built-ins are
disabled. segaboot and lindbergrc have no public source.
2026-08-10 13:36:34 +02:00
Abdessamad Derraz e749ac382b feat: add bios variants from tosec sets 2026-08-08 08:11:09 +02:00
Abdessamad Derraz 8663ec17b7 feat: add misterfpga platform support 2026-08-07 13:28:15 +02:00
Abdessamad Derraz 65062dd2be feat: add bios harvested from platform packs 2026-08-06 19:10:14 +02:00
Abdessamad Derraz 1a915e67f1 feat: add bios recovered from verified sources 2026-08-06 16:41:55 +02:00
Abdessamad Derraz f88aeee6ff feat: add bios from verified upstream sources 2026-08-06 10:40:18 +02:00
Abdessamad Derraz d6afe0c1ac fix: align mega cd bios contents with filenames 2026-08-06 05:00:50 +02:00
Abdessamad Derraz d685ad111d feat: add 63 missing BIOS files from multiple sources
MAME/MESS ROM sets (23 ZIPs from archive.org MESS 0.151 + MAME 0.285 split),
Dolphin BT firmware (15 files from linux-firmware), Commodore VIC-20/Plus4
regional ROMs (8 from zimmers.net), IBM AT 5170 BIOS + BASIC (3 from
minuszerodegrees.net + PCem), Apple DiskII boot/state-machine ROMs (4 from
Asimov mirror), Commodore 1540/1541 drive ROMs (2 reconstructed from MAME
chips), Oric Microdisc ROM, SMS Japanese BIOS, Videoton TVC64 + BK-0010
enriched ZIPs. All platforms at 0 required missing except Batocera bk0010.zip
(2/11 ROM chips unavailable).
2026-03-31 11:00:31 +02:00
Abdessamad Derraz 44145d41cd chore: remove 399 bios/ files now sourced from buildbot
Files verified by MD5 to be identical to their buildbot-fetched
copies in data/. resolve_local_file data directory fallback ensures
they remain resolvable for verify and pack generation.
2026-03-29 11:13:08 +02:00
Abdessamad Derraz ce6c218942 fix: restore original ZIPs to match platform MD5 hashes
Agents enriched neogeo/pgm/stvbios/neocdz/nmk004/channelf
ZIPs with extra ROMs, breaking platform MD5 verification.
Restore originals from git history. RetroBat: 0 untested.
RetroDECK 2 untested remain (upstream scraper bug).
2026-03-29 09:28:52 +02:00
Abdessamad Derraz e91caa6444 feat: add Saturn AR4MP firmware and QEMU Virtex DTB
satar4mp.bin (256KB) extracted from existing ar_bios.zip
MAME ROM. virtex-ml507.dtb compiled from Linux 4.19 DTS
with dtc. Both resolve long-standing optional missing files.
2026-03-29 00:07:40 +01:00
Abdessamad Derraz 2ff5be8420 feat: complete DuckStation PS1/PS2 BIOS coverage
Add 9 PS2 BIOS from Myrient (pre-shutdown raid), fix
incorrect scph5000.bin, add DuckStation-named symlinks
for 80 PS2 entries. Add Saturn cdb_rom, Fuse peripheral
ROMs, Amiberry MT-32 aliases, DOSBox soundfont. PS1
coverage: 25/25. PS2 coverage: 80/80 via MD5 matching.
2026-03-28 18:26:57 +01:00
Abdessamad Derraz 31ec407c91 feat: add FBNeo st010/st011/sfiii3na, PCem rivatnt, misc
sfiii3na.zip (CPS3 Asia No-CD), st010.zip and st011.zip
(SNES coprocessors) complete FBNeo required coverage.
rivatnt.bin completes PCem at 139/139. Also add QEMU
firmware (4), FM Towns Marty ROMs (4), Fuse mf1.rom,
Hatari BOOT.ST, PS1 extracts (2), ZC210 data (2).
2026-03-28 17:42:53 +01:00
Abdessamad Derraz 3d5d5b601c feat: mass BIOS collection (+1020 files, 7520 total)
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.

New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz dd689bbd2a feat: add name aliases for bizhawk platform resolution
Copy existing BIOS files under BizHawk-expected names:
MCD_eu_100.bin, MCD_jp_100p.bin, PSX_DTLH2000.bin,
MSX_4000bios.rom, MSX_8020-20bios.rom.
2026-03-28 10:29:13 +01:00
Abdessamad Derraz 1f65662fd2 feat: add ~110 missing bios files from archive.org and bizhawk
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.

Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz 3672912de7 chore: clean up sc3000 variant swap, update database 2026-03-27 22:50:54 +01:00
Abdessamad Derraz 59d7582c0e fix: swap sc3000.zip primary to batocera-compatible dump 2026-03-27 16:50:10 +01:00
Abdessamad Derraz fb1007496d chore: deduplicate bios/ — remove 427 files, save 227 MB
true duplicates (same file in multiple dirs): removed copies, kept
canonical. MAME device clones (different names, identical content):
removed copies, created _mame_clones.json mapping for pack-time
assembly via deterministic ZIP rebuild. generate_pack.py resolves
clones transparently. 95 canonical ZIPs serve 392 clone names.
2026-03-24 21:35:50 +01:00
Abdessamad Derraz 74f17694c2 feat: add category field to emulator profiles, source missing BIOS
Add category: game_data to sdlpal, nxengine, opentyrian, easyrpg,
mkxp_z profiles. verify.py separates game_data from bios in core
gap metrics for cleaner coverage numbers.

New BIOS files: Cemu fonts (4), QEMU bios-256k + vgabios-stdvga,
GAM4980 ROMs (2), SC-3000 Export variant.
2026-03-21 07:37:22 +01:00
Abdessamad Derraz c71378a71c feat: shared groups, scraper subdir prefixes, arcade + emulator profiles
shared groups in _shared.yml: np2kai, keropi, quasi88, kronos, ep128emu
with source references for each subdirectory requirement.

libretro_scraper: CORE_SUBDIR_MAP applies subdirectory prefixes at
generation time (np2kai/, keropi/, dc/). EXTRA_SYSTEMS adds QUASI88.
SYSTEM_SHARED_GROUPS injects includes for kronos/np2kai/keropi.

new BIOS: CPS3 (23 ZIPs), Cannonball OutRun (40 ROMs), PCem PC BIOS
(73 files), VICE Commodore ROMs, Spectrum ZIPs, dc_bios.bin, X1 fonts.

new emulator profiles: redream, melonds_ds, lrps2 with doc vs source
notes. platforms/README.md documents shared groups architecture.
2026-03-18 13:49:59 +01:00
Abdessamad Derraz 7653d5d108 feat: add 19 BIOS files, fix cross_reference resolution
New files: OpenTyrian data (11), Cave Story (2), SeaBIOS,
VGA BIOS, OpenSBI, Cromwell, xbox_hdd, Sega CD Model 2 (3),
NGP Color BIOS, Pentagon 128p-1.rom, X1 font, BK TERAK.
cross_reference.py: basename + case-insensitive lookup.
2026-03-18 12:50:55 +01:00
Abdessamad Derraz e82a498124 feat: update sc3000 to basic iii v1.1, regenerate db and readme
Replace sc3000.rom with BASIC Level III v1.1 (Japan) from MAME 0.270+
software list (SHA1 737a06b3, closer to Batocera target).

Batocera 679/680, Recalbox 346/346, all others 100%.
2026-03-17 17:51:11 +01:00
Abdessamad Derraz 1257653c9b feat: batocera 679/680, fix variant indexing, add hikaru + segaboot
Fix variant name indexing: files in .variants/ now indexed under
canonical name (naomi2.zip instead of naomi2.zip.da79eca4).
Fix .zip detection for variant paths in verify.py.
Add composite MD5 matching in resolver for ZIP variants.

Add hikaru.zip (MAME 0.285, 6 ROMs) and segaboot.gcm (Triforce)
from archive.org. Both match Batocera expected MD5s.

Batocera 679/680 (1 untested: sc3000 private dump)
Recalbox 346/346 (100%)
2026-03-17 17:06:02 +01:00
Abdessamad Derraz 36a17b315a feat: batocera 678/680, add 6 naomi2 roms from mame 0.278
Add missing NAOMI 2 ROMs (315-6146, 315-6215, sp5001/a/b, sp5002-a)
from MAME 0.278 non-merged set. Clean up naomi2 variant files.

Recalbox composite naomi2.zip kept as variant (.da79eca4).
Batocera 678/680 (2 untested: sc3000, triforce)
Recalbox 345/346 (1 untested: naomi2 composite mismatch)
2026-03-17 16:19:21 +01:00
Abdessamad Derraz 4bf62b8126 feat: fix enterprise zips, restore deleted files, add sms aliases
- Create ep64.zip/ep128.zip MAME sets from existing ROMs
- Restore Complex_4627v1.03.bin, N64DD IPL, PS2 SCPH-70004 files
  (accidentally deleted by background agent)
- Add Xbox Complex_4627.bin alias for Batocera
- Add SMS BIOS aliases for RetroBat naming convention

All platforms at 0 missing:
  Batocera 677/680, Recalbox 337/346, all others 100%
2026-03-17 15:44:12 +01:00
Abdessamad Derraz 5ab82a7898 refactor: security hardening + mame arcade bios updates
Security fixes:
- Zip-slip protection in _extract_zip_to_archive (sanitize paths)
- Hash verification for large file downloads (cache + post-download)
- Sanitize YAML destination fields against path traversal
- Size limit on ZIP entry reads (512MB cap, prevents zip bombs)
- Download size limits in auto_fetch (100MB cap)
- Reject hashless external downloads
- Sanitize filenames in place_file with basename()

MAME arcade updates from Batocera v38 pack:
- Updated naomi, naomi2, naomigd, awbios, airlbios, hod2bios, hikaru
- Old versions preserved in .variants/ for RetroBat compatibility

Batocera 675/680 (+9), all other platforms unchanged at 0 missing
2026-03-17 15:32:14 +01:00
Abdessamad Derraz 936cec07cd feat: emudeck at 163/164, add mega cd + ps1_rom variants
Add 3 rare BIOS variants from TOSEC/archive.org:
- Mega CD v1.00s JP (a3ddcc8483) from Myrient TOSEC
- Sega CD Model 2 v2.00 US (310a9081d2) from Myrient TOSEC
- ps1_rom.bin 4MB PS3-extracted (fbb5f59ec3) from archive.org

EmuDeck: 163/164 (99.4%), 1 unknown hash remains (476d68a9)
All other platforms: 0 missing
2026-03-17 14:37:58 +01:00
Abdessamad Derraz 617f5a3654 feat: expand bios collection, emudeck at 97.6%
Add 36 new files from multiple sources:
- PSX rare variants (DTL-H2000, SCPH-5903 VCD, SCPH-7000/7000W, DTL-H1100)
- Sega Mega CD 9 region variants (EU/JP/US firmware revisions)
- Saturn HiSaturn GameNavi v1.03
- N64DD IPL (US/JP/DEV), PS2 SCPH-70004, Xbox Complex BIOS
- PPSSPP system pack, 3DS/Switch/Wii U keys, Nokia N-Gage
- Apple Macintosh mac755.chd

Platform coverage:
  RetroArch  398/398 (100%)
  Lakka      398/398 (100%)
  RetroPie   398/398 (100%)
  Batocera   589/680 (86.6%, 91 untested MAME versions)
  Recalbox   336/346 (97.1%, 10 untested)
  RetroBat   341/343 (99.4%, 2 untested)
  EmuDeck    160/164 (97.6%, 4 missing rare dumps)
2026-03-17 14:15:33 +01:00
Abdessamad Derraz 0ffb8cbd0d feat: complete retrobat coverage, fix large file resolution 2026-03-17 13:03:57 +01:00
Abdessamad Derraz e6ea0484a8 expand bios collection, retrobat at 93% coverage 2026-03-17 11:53:42 +01:00
Abdessamad Derraz 13c561888d v2: automated BIOS platform with full pipeline
Reorganized 6 branches into bios/Manufacturer/Console/.
Scrapers for RetroArch, Batocera, Recalbox, and libretro core-info.
Platform-aware verification replicating native logic per platform.
Pack generation with dedup, alias resolution, variant support.
CI/CD: weekly auto-scrape, auto-release, PR validation.
Large files (>50MB) stored as GitHub Release assets, auto-fetched at build time.
2026-03-17 10:54:39 +01:00