Commit Graph

65 Commits

Author SHA1 Message Date
Abdessamad Derraz
d929424988 feat: add profiled_date to all YAMLs, create 56 alias profiles
- Added profiled_date field to all 204 existing profiles for update tracking
- Created 56 alias profiles for cores that share BIOS with a parent
  (e.g., mednafen_psx -> beetle_psx, fbalpha2012 -> fbneo)

260 total profiles covering all 294 libretro cores (204 unique + 56 alias).
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
62262db6ac feat: complete 294/294 libretro core profiling
Final batch: stonesoup, vitaquake3+vitavoyager, uw8, uxn, wasm4,
tempgba, retro8, moonlight, ffmpeg, imageviewer, mpv, remotejoy,
puzzlescript, lutro, mojozork, simcp (SAM Coupe, ROM built-in),
romcleaner, redbook, onsyuri, mcsoftserve, x64sdl (VICE alias)

204 total emulator/core profiles covering ALL 294 libretro cores.
Every core analyzed from source code. Complete museum documentation.
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
b43d71e113 feat: add emulator profiles (119-series batch 6 partial)
dolphin_launcher (stub launcher), oberon (RISC, bootloader embedded),
17 test cores (00_example through vidtest), 17 game/utility cores
(2048 through vaporspec), bennugd, boytacean, emux_chip8, fake08,
jaxe, lowresnx, opentyrian, openlara, squirreljme, vitaquake2, xrick

182 total profiles. 6 agents failed (API 500), will retry.
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
a5553def94 feat: add 10 emulator profiles (119-series batch 5)
boytacean (GB Rust, 6 boot ROMs embedded), emux_chip8 (no BIOS),
fake08 (PICO-8, Lua runtime compiled in), jaxe (CHIP-8, fonts hardcoded),
lowresnx (fantasy console), opentyrian (Tyrian freeware data),
openlara (Tomb Raider engine, game data), squirreljme (J2ME, 8 JARs),
vitaquake2 (Quake 2, 4 variants), xrick (data.zip sound pack)

145 total profiles.
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
31501211de feat: add 10 emulator profiles (119-series batch 4)
bennugd (game engine, no BIOS), cruzes (puzzle game, no BIOS),
dice (discrete arcade, no BIOS), ishiiruka (Dolphin fork, same BIOS),
mkxp_z (RPG Maker, 3 RTP dirs), onscripter (VN engine, no BIOS),
qemu (55 firmware files! SeaBIOS/VGA/iPXE/OpenBIOS/OpenSBI),
rustation (PS1 Rust, 22 BIOS by SHA-256), rvvm (RISC-V, OpenSBI),
reminiscence (Flashback engine, game data only)

135 total profiles.
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
6a778a09a3 feat: add 10 emulator profiles (119-series batch 3)
arduous (Arduboy AVR sim, no BIOS), boom3 (Doom 3, game data only),
directxbox (Xbox, mcpx + cromwell), doukutsu_rs (Cave Story, no BIOS),
meteor (GBA, full HLE), nxengine (Cave Story, Doukutsu.exe + data/),
tgbdual (Game Boy link, HLE), tic80 (fantasy console),
virtualxt (IBM XT, open BIOS embedded), sdlpal (Sword & Fairy, 13 .mkf)

125 total profiles.
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
ebc8e4413f feat: add 10 emulator profiles (119-series batch 2)
freeintv_ts_overlay (same BIOS as FreeIntv), bnes (NES, no BIOS),
clownmdemu (Mega Drive, open-source MCD boot ROM embedded),
m2000 (Philips P2000T, all embedded), mesen (NES, FDS + StudyBox BIOS),
race (NGP, HLE with disabled BIOS loader), vemulator (VMU, HLE only),
gw (Game & Watch, Lua simulator), pd777 (Cassette Vision, no BIOS),
play (PS2, full HLE, LoadBIOS commented out)

115 total profiles.
2026-03-18 05:20:05 +01:00
Abdessamad Derraz
a77477cbe8 feat: add 10 emulator profiles (119-series batch 1)
81 (ZX81, embedded), DoubleCherryGB (GB dual, no BIOS),
amiarcadia (Arcadia 2001, no BIOS), applewin (Apple II, 26 embedded),
ardens (Arduboy, 6 bootloaders embedded), daphne (LaserDisc, no BIOS),
pcem (IBM PC, 75+ machine/video/sound ROMs), panda3ds (3DS, optional keys),
easyrpg (RPG Maker, soundfont + RTP), frodo (C64, 4 ROMs embedded)

105 total profiles.
2026-03-18 05:20:05 +01:00
github-actions[bot]
9a69b49eac regenerate database and docs 2026-03-17 19:10:52 +00:00
Abdessamad Derraz
ff82f64cb6 feat: complete all emulator profiles, 95 engines total
Final batch: scummvm (55 files, MT-32/engine data/themes/shaders),
gearcoleco (colecovision.rom), geargrafx (4 syscard variants),
gearlynx (lynxboot.img), dosbox_core (MT-32 + BASS libs),
geolith (Neo Geo FPGA, aes.zip/neogeo.zip), retrodream (DC boot+flash),
noods (NDS, full HLE), freej2me (JAR), jollycv (coleco+creativision),
fixgb (gbc_bios), numero (TI-83), smsplus (SMS/GG/CV), vircon32

95 profiles covering every libretro core with firmware requirements.
Cross-reference: 803 undeclared files, 249 in repo, 554 to source.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
60d9d38d69 feat: complete emulator profiling, 81 engines (final batch)
mame2003_plus (16 BIOS ZIPs, MAME 0.78 naming), mame2010 (51 ZIPs),
mame2016 (37 ZIPs incl Lindbergh), hbmame (MAME 0.220, no custom BIOS),
tyrquake (no BIOS), cannonball (29 OutRun ROMs with CRC32),
uzem (open source, no BIOS), vecx (ROM embedded), emuscv (UPD7801G),
b2 (BBC Micro standalone, ROMs bundled)

81 total emulator profiles covering all libretro cores with firmware.
Cross-reference: 731 undeclared files, 186 already in repo.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
99581d8aba feat: add 10 emulator profiles (batch 6)
gpsp (GBA, open BIOS fallback), ecwolf (ecwolf.pk3),
prboom (prboom.wad), x1 (Sharp X1, 5 files in xmil/),
minivmac (MacII.ROM/MacIIx.ROM), theodore (Thomson, all embedded),
crocods (CPC 6128, all embedded), vba_next (GBA, HLE fallback),
same_cdi (CD-i, 3 MAME ZIPs + 2 undeclared), beetle_lynx (lynxboot.img)

71 total profiles. Cross-reference: 690 undeclared, 186 in repo.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
4bc7085c1c feat: add 10 emulator profiles (batch 5)
snes9x (BS-X + STBIOS, HLE coprocessors), mupen64plus (64DD IPL),
pcsx_rearmed (7 BIOS + dynamic scan fallback), stella (no BIOS),
fmsx (12 MSX ROMs), virtualjaguar (6 BIOS embedded),
sameboy (8 boot ROM types, all built-in), beetle_vb (no BIOS),
handy (lynxboot.img 512B), pokemini (bios.min + FreeBIOS fallback)

61 total profiles. Cross-reference: 667 undeclared, 182 in repo.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
9eede1c957 feat: add 10 emulator profiles (batch 4)
blastem (MD, no BIOS needed), dosbox_pure (14 MT-32 ROMs + SF2),
desmume (NDS, 3 files all HLE), cap32 (CPC, all embedded),
mu (Palm OS, 5 ROMs), bk (Elektronika BK, 9 Soviet ROMs),
gearsystem (SMS/GG, 2 optional boot ROMs),
beetle_ngp (Neo Geo Pocket, pure HLE), beetle_wswan (WonderSwan, pure HLE),
galaksija (Yugoslav computer, all embedded)

51 total profiles. Cross-reference: 658 undeclared, 177 in repo.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
8768aed7aa feat: add 10 emulator profiles (batch 3)
freechaf (Channel F, 3 ROMs), freeintv (Intellivision, exec+grom),
ep128emu (Enterprise/TVC/CPC/ZX, 22 ROMs all built-in),
kronos (Saturn + 14 ST-V blobs), gambatte (GB/GBC boot ROMs),
mgba (GBA/GB/GBC/SGB, 10 CRC32 revisions), nestopia (FDS + NstDatabase),
px68k (X68000, 6 ROMs in keropi/), fuse (ZX Spectrum, 17 machines),
o2em (Odyssey2/Videopac, 4 BIOS by CRC32)

41 total profiles. Cross-reference: 644 undeclared, 167 in repo.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
639ff69d6c feat: add 10 emulator profiles (batch 2)
a5200, prosystem (Atari 7800), atari800 (400/800/XL/XE),
vice (C64/C128/VIC-20/Plus4/PET - 10 cores, all ROMs embedded),
neocd (Neo Geo CD, 12 BIOS variants), beetle_pce (PC Engine, 6 files),
beetle_pcfx (PC-FX), np2kai (PC-9801, 14 files + YM2608 samples),
quasi88 (PC-8801, 13 ROMs + 3 hidden fonts), opera (3DO, 10 BIOS + 3 kanji)

31 total profiles. Cross-reference: 572 undeclared, 148 in repo.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
f6a44c9409 feat: add 10 emulator profiles (batch 1 P1)
genesis_plus_gx (12 files), picodrive (18 files), fbneo (19 systems),
hatari (TOS ROMs), bluemsx (60+ MSX/ColecoVision/SG-1000),
fceumm (FDS/Game Genie), bsnes (20 coprocessor ROMs),
beetle_psx (5 BIOS + aliases), beetle_saturn (5 files),
puae (14 Kickstart ROMs, 3 naming conventions)

21 total emulator profiles. Cross-reference: 436 undeclared files,
120 already in repo, 316 to source.
2026-03-17 20:08:27 +01:00
Abdessamad Derraz
9052a6b750 feat: add emulator profiles and cross-reference engine (tier 2)
New two-tier architecture:
- Tier 1: Platform configs (what the UI checks) - unchanged
- Tier 2: Emulator profiles (what the code actually loads)

11 emulator profiles from source code analysis:
  cemu, citra, dolphin, duckstation, flycast,
  melonds, pcsx2, ppsspp, rpcs3, vita3k, xemu

Each profile documents every file the emulator loads with
source code references (file:line), hashes, and notes.

New scripts/cross_reference.py computes gaps between what
platforms declare and what emulators need.

Current gap: 200 undeclared files, 24 already in repo.
DuckStation alone recognizes 105 PS1/PS2 BIOS variants.

generate_pack.py gains --include-extras flag (future use).
_registry.yml maps platforms to their emulators.
2026-03-17 20:08:27 +01:00
github-actions[bot]
a38c86cf1e regenerate database and docs 2026-03-17 16:53:26 +00:00
Abdessamad Derraz
e82a498124 feat: update sc3000 to basic iii v1.1, regenerate db and readme
Replace sc3000.rom with BASIC Level III v1.1 (Japan) from MAME 0.270+
software list (SHA1 737a06b3, closer to Batocera target).

Batocera 679/680, Recalbox 346/346, all others 100%.
2026-03-17 17:51:11 +01:00
github-actions[bot]
29d4c6e5b9 regenerate database and docs v2026.03.17.2 2026-03-17 16:25:41 +00:00
Abdessamad Derraz
159bb427df chore: update gitignore, add log and node_modules exclusions 2026-03-17 17:06:33 +01:00
Abdessamad Derraz
1257653c9b feat: batocera 679/680, fix variant indexing, add hikaru + segaboot
Fix variant name indexing: files in .variants/ now indexed under
canonical name (naomi2.zip instead of naomi2.zip.da79eca4).
Fix .zip detection for variant paths in verify.py.
Add composite MD5 matching in resolver for ZIP variants.

Add hikaru.zip (MAME 0.285, 6 ROMs) and segaboot.gcm (Triforce)
from archive.org. Both match Batocera expected MD5s.

Batocera 679/680 (1 untested: sc3000 private dump)
Recalbox 346/346 (100%)
2026-03-17 17:06:02 +01:00
Abdessamad Derraz
36a17b315a feat: batocera 678/680, add 6 naomi2 roms from mame 0.278
Add missing NAOMI 2 ROMs (315-6146, 315-6215, sp5001/a/b, sp5002-a)
from MAME 0.278 non-merged set. Clean up naomi2 variant files.

Recalbox composite naomi2.zip kept as variant (.da79eca4).
Batocera 678/680 (2 untested: sc3000, triforce)
Recalbox 345/346 (1 untested: naomi2 composite mismatch)
2026-03-17 16:19:21 +01:00
Abdessamad Derraz
bb1855d3f7 feat: recalbox 346/346 via md5_composite, add mame variants
Add md5_composite() to verify.py replicating Recalbox Zip::Md5Composite
(sorted filenames, sequential content hash). Independent of ZIP
compression level, resolves all 9 MAME arcade untested entries.

Add Recalbox-specific MAME ZIP variants from Recalbox 10 pack.
Batocera 671/680 (9 untested MAME-specific), all others 100%.
2026-03-17 16:08:39 +01:00
Abdessamad Derraz
4bf62b8126 feat: fix enterprise zips, restore deleted files, add sms aliases
- Create ep64.zip/ep128.zip MAME sets from existing ROMs
- Restore Complex_4627v1.03.bin, N64DD IPL, PS2 SCPH-70004 files
  (accidentally deleted by background agent)
- Add Xbox Complex_4627.bin alias for Batocera
- Add SMS BIOS aliases for RetroBat naming convention

All platforms at 0 missing:
  Batocera 677/680, Recalbox 337/346, all others 100%
2026-03-17 15:44:12 +01:00
Abdessamad Derraz
8d81aee235 refactor: quality audit fixes, honest verification reporting
- batocera_scraper: fix OrderedDict parsing for ast.literal_eval
- auto_fetch: fix TypeError when sha1/md5 is None
- verify: filter non-ZIP files for zipped_file entries (F2)
- verify: distinguish ZIP read errors from hash mismatches (F5)
- generate_pack: track seen_destinations with source hash (F7)

Batocera ep64/ep128.zip now correctly reported as MISSING
instead of false UNTESTED (resolved to .rom instead of .zip)
2026-03-17 15:35:30 +01:00
Abdessamad Derraz
5ab82a7898 refactor: security hardening + mame arcade bios updates
Security fixes:
- Zip-slip protection in _extract_zip_to_archive (sanitize paths)
- Hash verification for large file downloads (cache + post-download)
- Sanitize YAML destination fields against path traversal
- Size limit on ZIP entry reads (512MB cap, prevents zip bombs)
- Download size limits in auto_fetch (100MB cap)
- Reject hashless external downloads
- Sanitize filenames in place_file with basename()

MAME arcade updates from Batocera v38 pack:
- Updated naomi, naomi2, naomigd, awbios, airlbios, hod2bios, hikaru
- Old versions preserved in .variants/ for RetroBat compatibility

Batocera 675/680 (+9), all other platforms unchanged at 0 missing
2026-03-17 15:32:14 +01:00
Abdessamad Derraz
af74fffa14 refactor: fix code review findings across all scripts
Critical: stream large file downloads (OOM fix), fix basename match
in auto_fetch, include hashes in pack grouping fingerprint, handle
not_in_zip status in verify, fix escaped quotes in batocera parser.

Important: deduplicate shared group includes, catch coreinfo network
errors, fix NODEDUP path component match, fix CI word splitting on
spaces, replace bare except Exception in 3 files.

Minor: argparse in list_platforms, specific exceptions in download.py.
2026-03-17 15:16:51 +01:00
Abdessamad Derraz
ff38eebde7 feat: fix retro computer bios, retrobat ecwolf/freej2me variants
Add missing ROMs to Dragon 32/64, TRS-80, TRS-80 M3/M4 ZIPs.
Add RetroBat-specific ecwolf.pk3 and freej2me-lr.jar variants.

Batocera 666/680 (+5), RetroBat 343/343 (100%), EmuDeck 164/164
2026-03-17 15:13:32 +01:00
Abdessamad Derraz
9104ec68e3 fix: verify.py resolve by md5 when multiple name candidates
When a file exists under multiple SHA1s (e.g. awbios.zip in both
Arcade/ and Sega/Dreamcast/), prefer the candidate whose MD5
matches the expected hash instead of always picking the first.

Batocera: 589 -> 661 verified (+72), RetroBat: 341 -> 343 (100%)
2026-03-17 15:08:01 +01:00
Abdessamad Derraz
7ea649da7c feat: emudeck 164/164, complete coverage all platforms
Add SCPH-1000R (PlayStation Classic JP) romJP.bin from archive.org
NAND dump. Last missing file across all 7 platforms.

All platforms at 0 missing:
  RetroArch  398/398, Lakka 398/398, RetroPie 398/398
  RetroBat   341/343 (2 untested)
  Recalbox   336/346 (10 untested)
  Batocera   589/680 (91 untested, MAME version diffs)
  EmuDeck    164/164
2026-03-17 14:44:10 +01:00
Abdessamad Derraz
936cec07cd feat: emudeck at 163/164, add mega cd + ps1_rom variants
Add 3 rare BIOS variants from TOSEC/archive.org:
- Mega CD v1.00s JP (a3ddcc8483) from Myrient TOSEC
- Sega CD Model 2 v2.00 US (310a9081d2) from Myrient TOSEC
- ps1_rom.bin 4MB PS3-extracted (fbb5f59ec3) from archive.org

EmuDeck: 163/164 (99.4%), 1 unknown hash remains (476d68a9)
All other platforms: 0 missing
2026-03-17 14:37:58 +01:00
Abdessamad Derraz
617f5a3654 feat: expand bios collection, emudeck at 97.6%
Add 36 new files from multiple sources:
- PSX rare variants (DTL-H2000, SCPH-5903 VCD, SCPH-7000/7000W, DTL-H1100)
- Sega Mega CD 9 region variants (EU/JP/US firmware revisions)
- Saturn HiSaturn GameNavi v1.03
- N64DD IPL (US/JP/DEV), PS2 SCPH-70004, Xbox Complex BIOS
- PPSSPP system pack, 3DS/Switch/Wii U keys, Nokia N-Gage
- Apple Macintosh mac755.chd

Platform coverage:
  RetroArch  398/398 (100%)
  Lakka      398/398 (100%)
  RetroPie   398/398 (100%)
  Batocera   589/680 (86.6%, 91 untested MAME versions)
  Recalbox   336/346 (97.1%, 10 untested)
  RetroBat   341/343 (99.4%, 2 untested)
  EmuDeck    160/164 (97.6%, 4 missing rare dumps)
2026-03-17 14:15:33 +01:00
Abdessamad Derraz
29d475b8b7 feat: add emudeck platform support, 126/164 verified 2026-03-17 13:33:07 +01:00
Abdessamad Derraz
c13d3d13bf feat: complete retrobat coverage, fix large file resolution, fix readme variants 2026-03-17 13:12:51 +01:00
github-actions[bot]
d834688593 regenerate database and docs 2026-03-17 12:06:43 +00:00
Abdessamad Derraz
0ffb8cbd0d feat: complete retrobat coverage, fix large file resolution 2026-03-17 13:03:57 +01:00
github-actions[bot]
a443ba74e8 regenerate database and docs 2026-03-17 11:44:55 +00:00
Abdessamad Derraz
4d98f17308 feat: add dolphin dsp, mame hash xmls, scummvm fonts from retrobat-bios 2026-03-17 12:42:55 +01:00
github-actions[bot]
1218bc0bb8 regenerate database and docs 2026-03-17 11:35:56 +00:00
Abdessamad Derraz
3453f89d9d refactor: consolidate CI pipeline, remove third-party deps 2026-03-17 12:33:10 +01:00
Abdessamad Derraz
e6ea0484a8 expand bios collection, retrobat at 93% coverage 2026-03-17 11:53:42 +01:00
Abdessamad Derraz
851a14e49a add retrobat platform support (scraper, config, verify) 2026-03-17 11:38:52 +01:00
Abdessamad Derraz
1129aebfc4 update all references from retroarch_system to retrobios 2026-03-17 11:17:50 +01:00
Abdessamad Derraz
c23c565c6d update repo references after rename to retrobios 2026-03-17 11:16:37 +01:00
Abdessamad Derraz
13c561888d v2: automated BIOS platform with full pipeline
Reorganized 6 branches into bios/Manufacturer/Console/.
Scrapers for RetroArch, Batocera, Recalbox, and libretro core-info.
Platform-aware verification replicating native logic per platform.
Pack generation with dedup, alias resolution, variant support.
CI/CD: weekly auto-scrape, auto-release, PR validation.
Large files (>50MB) stored as GitHub Release assets, auto-fetched at build time.
2026-03-17 10:54:39 +01:00
Abdessamad DERRAZ
5f96368f6d Readme : Rename coleco.rom to colecovision.rom large-files 2022-03-08 13:28:17 +01:00
Abdessamad DERRAZ
df70aa6818 Add MacII.ROM 2022-03-08 13:24:59 +01:00
Abdessamad DERRAZ
0ebbfebba5 Rename coleco.rom to colecovision.rom 2022-03-08 13:14:42 +01:00