Commit Graph

94 Commits

Author SHA1 Message Date
Abdessamad Derraz
f9a612db4a refactor: simplify readme contributor section to avatars only 2026-04-02 15:39:34 +02:00
Abdessamad Derraz
812775f6b4 Add platform contribution steps and nav
Add an "Add a new platform" section to CONTRIBUTING.md (instructions to write a scraper in scripts/scraper/, create platform YAML in platforms/, register in platforms/_registry.yml, and submit a PR) and note contributor crediting. Update README: bump verified files from 7,296 to 7,302, add RomM and RetroDECK contributor credits with PR links, and refresh the auto-generated timestamp. Add sdlpal to the mkdocs.yml navigation.
2026-04-02 15:36:27 +02:00
Abdessamad Derraz
f4626ce3bd Add MSX2J ROM and update manifests
Add MSX2J.rom (sha1: 0081ea0d25bc5cd8d70b60ad8cfdc7307812c0fd, size: 32768) to multiple install manifests and the RetroDECK bios list. Update generated timestamps and adjust total_files/total_size counts in batocera, lakka, recalbox, retroarch, retrobat, and retrodeck manifests. Also bump README verified file count and regenerate the auto-generated timestamp to reflect the new entry.
2026-04-02 10:44:25 +02:00
Abdessamad Derraz
5eeaf87a3a fix: resolve all untested and missing bios across platforms
Batocera: fix sc3000.rom md5 (no dump matches upstream hash),
remove erroneous bk0010.zip mame entries (upstream confirmed
mame needs no bios for bk), add PSP2UPDAT.PUP correct version.
Recalbox: add MSX2R2.ROM from blueMSX v2.82.
RetroDECK: fix stale peribox_ev/gen.zip md5 hashes.

Regenerate database, manifests, readme.
2026-04-01 01:42:39 +02:00
Abdessamad Derraz
ab3255b0c7 chore: regenerate readme, manifests after md5 resolution fix 2026-03-31 14:37:52 +02:00
Abdessamad Derraz
03002515fe fix: readme and site coverage use data_dir and supplemental indexes 2026-03-31 13:58:52 +02:00
Abdessamad Derraz
b56f8dd05f feat: add archive_prefix for core-specific BIOS subdirectories
Closes #43

FBNeo and Kronos expect BIOS archives in core-specific subdirectories
(system/fbneo/, system/kronos/). RetroArch firmware check uses .info
paths which include these prefixes, so files at root show as Missing.

Add archive_prefix field to emulator profiles. The pack code now places
archive copies in the prefixed subdirectory while keeping root copies
for cores that expect them there (e.g. Geolith for neogeo.zip).
2026-03-31 09:17:54 +02:00
Abdessamad Derraz
d0dd05ddf6 docs: add wiki pages for all audiences, fix .old.yml leak
9 new wiki pages: getting-started, faq, troubleshooting,
advanced-usage, verification-modes, adding-a-platform,
adding-a-scraper, testing-guide, release-process.

Updated architecture.md with mermaid diagrams, tools.md with
full pipeline and target/exporter sections, profiling.md with
missing fields, index.md with glossary and nav links.

Expanded CONTRIBUTING.md from stub to full contributor guide.

Filter .old.yml from load_emulator_profiles, generate_db alias
collection, and generate_readme counts. Fix BizHawk sha1 mode
in tools.md, fix RetroPie path, fix export_truth.py typos.
2026-03-30 23:58:12 +02:00
Abdessamad Derraz
2aca4927c0 chore: regenerate database, readme, manifests, site 2026-03-30 14:19:00 +02:00
Abdessamad Derraz
d3a2224dd2 chore: regenerate database, readme, manifests, site 2026-03-30 09:42:39 +02:00
Abdessamad Derraz
0c5cde83e1 Add TRS-80, RX-78, Sega AI entries; refactor tools
Add many MAME/MESS BIOS entries (TRS-80 family, Bandai RX-78, Sega AI) and update docs/navigation counts (README, mkdocs). Remove empty supplemental file references from database.json and update generated timestamps and totals. Harden and refactor tooling: add MAX_RESPONSE_SIZE limited reader in base_scraper, make target scrapers an abstract base, narrow exception handling in the Batocera targets parser, and switch generate_pack.py and verify.py to use build_target_cores_cache (simplifies target config loading and error handling). verify.py also loads supplemental cross-reference names and accepts them through verify_platform. Update tests to import from updated modules (validation/truth). Misc: small bugfix for case-insensitive path conflict check.
2026-03-29 23:04:30 +02:00
Abdessamad Derraz
463fca7e7d Regenerate database and update emulator profiles
Regenerate database.json and update README counts/timestamps; add and normalize numerous BIOS entries and hashes. Key changes: update generated_at timestamp and system count (355→357) in README; add OpenBIOS / HLE fallback and additional aliases to beetle_psx, include beetle_psx core name and profiled_date update; add laseractive to ares systems; adjust atari800 systems and source_ref line numbers; mark dinothawr as a system and expand its note; update gsplus upstream/profile date, add apple-iie system and tweak source_refs; add pcsx2 core to lrps2; refresh mame profiled_date and add multiple systems and BIOS root sets. Miscellaneous script changes and other JSON normalization to reflect newly discovered/merged BIOS files.
2026-03-29 22:41:01 +02:00
Abdessamad Derraz
95e16c9e7a Add Adam/Advision/APF systems and update metadata
Add support for Coleco Adam, Entex Adventure Vision and APF M-1000 BIOS/ROM sets in MAME and MESS metadata (multiple Adam device MCU ROMs and optional FDC/SPI variants, Advision and APF BIOS entries). Update generated metadata across the repo: README coverage numbers and per-platform coverage rows, database generated timestamp and totals (total_files 7245), and various install manifests (notably batocera.json) with new timestamps, adjusted file counts/sizes, SHA1s, repo_path fixes and an added adam_fdc_320kb.zip entry. Also update notes to reflect the new system ROM sets in the emulators entries.
2026-03-29 16:07:15 +02:00
Abdessamad Derraz
36f6ad1379 fix: correct stale scummvm.zip and cpc464.rom hashes
RetroDECK had stale buildbot hash a17e0e01 for scummvm.zip
(from old 9.5MB build, current is 79MB) copied to cpc464.rom.
RomM had same stale hash. Updated to current verified values.
All 8 platforms now 100% OK, 0 untested.
2026-03-29 09:52:32 +02:00
Abdessamad Derraz
ce6c218942 fix: restore original ZIPs to match platform MD5 hashes
Agents enriched neogeo/pgm/stvbios/neocdz/nmk004/channelf
ZIPs with extra ROMs, breaking platform MD5 verification.
Restore originals from git history. RetroBat: 0 untested.
RetroDECK 2 untested remain (upstream scraper bug).
2026-03-29 09:28:52 +02:00
Abdessamad Derraz
536300984d feat: add howard.o Lynx development bootloader
513-byte BS93-format bootloader reconstructed from Handy
emulator source code analysis. Header verified: BRA +8,
load_addr 0x0212, magic BS93, SEI at code entry. CRam
reset simulation passed.
2026-03-29 09:18:30 +02:00
Abdessamad Derraz
a37bf11524 fix: add path fields for correct pack subdirectory placement
Cores that load files from system_dir subdirectories (same_cdi/bios/,
neocd/, cannonball/, Citra/sysdata/, mame2003/, etc.) need path: on
each file entry so cross-reference and pack generation place files at
the correct destination. Also fixes neocd.yml using non-standard dest:
field instead of path:.

Closes #41
2026-03-29 00:59:57 +01:00
Abdessamad Derraz
87d3c4c645 chore: regenerate database and revert agent profile violations
16 emulator profiles reverted to ground truth (agents modified
cannonball, citra, fbalpha2012, mame variants, neocd, nxengine,
same_cdi without source code verification). OpenTyrian residuals
moved from bios/ to data/. Database regenerated: 7616 files.
2026-03-29 00:54:44 +01:00
Abdessamad Derraz
c835336999 fix: correct OpenTyrian shapes profiling error
Remove 30 phantom shapes[2-v].dat entries. shapeFile[34] in
lvlmast.c maps to newsh?.shp (enemy sprites), NOT shapes?.dat
(level tilesets). Only 5 shapes exist: ), W, X, Y, Z — the
characters actually referenced by level data. Verified against
trapexit/libretro-opentyrian source. Cross-ref: 54 -> 24.
2026-03-29 00:30:06 +01:00
Abdessamad Derraz
054cb940b0 feat: resolve last 3 required files + 26 optional
CLK os.rom (Acorn Electron MOS), exos10.bin and basic10.bin
(Enterprise EXOS/BASIC 1.0) from TOSEC/Myrient — zero
required missing across all platforms. Also add Enterprise
ROMs, ZX Spectrum 128/+2/+3, MSX CLK variants, FBA2012
samples from progettosnaps.
2026-03-28 22:18:19 +01:00
Abdessamad Derraz
3eb6e0e530 feat: add Dolphin Wii certs, fMSX GameMaster ROMs
Wii SSL certs (clientca/clientcakey/rootca.pem) from archived
Nintendo HTTPS certs. SYSCONF and setting.txt generated with
correct format. Konami GameMaster 1+2 cartridge ROMs for fMSX.
2026-03-28 20:57:41 +01:00
Abdessamad Derraz
ddf2937f41 fix: eliminate cross-reference false positives
Skip placeholder names (<bios>.bin), resolve by MD5/SHA1 hash
match for alias files, fix directory basename extraction for
trailing slash entries, index bios/ directory names for
directory-type file entries. 1011 -> 113 true missing.
149/149 tests pass.
2026-03-28 19:24:16 +01:00
Abdessamad Derraz
a418f7ef68 feat: add Dragon CoCo ROMs and mameinfo.dat
cdos20.rom (Cumana DOS 2.0), yados.rom (YA-DOS 0.5a),
dragonfly-1.3.rom (Ikon Ultra Drive v1.3) from World of
Dragon Archive and CCHDD GitHub. delta-premier.rom variant.
mameinfo.dat for MAME 2000/2003/2003-Plus game info.
2026-03-28 19:09:42 +01:00
Abdessamad Derraz
63f2450943 feat: add Xbox BIOS variants and PC-98 sasi.rom
8 Xbox BIOS variants (mcpx-1.1, xbox-3944 through 5838)
from Myrient MAME bios-devices. PC-98 SASI controller ROM
for NP2kai/nekop2. Cross-reference: 146 true missing.
2026-03-28 18:50:59 +01:00
Abdessamad Derraz
2ff5be8420 feat: complete DuckStation PS1/PS2 BIOS coverage
Add 9 PS2 BIOS from Myrient (pre-shutdown raid), fix
incorrect scph5000.bin, add DuckStation-named symlinks
for 80 PS2 entries. Add Saturn cdb_rom, Fuse peripheral
ROMs, Amiberry MT-32 aliases, DOSBox soundfont. PS1
coverage: 25/25. PS2 coverage: 80/80 via MD5 matching.
2026-03-28 18:26:57 +01:00
Abdessamad Derraz
76fe7dd76f fix: cross-reference checks inside ZIP archives
_build_supplemental_index scans both data/ directories and
contents of bios/ ZIP files. Eliminates 197 false positives
where files existed inside archive ZIPs (neogeo.zip, pgm.zip,
stvbios.zip, etc.) but were counted as missing. True missing
drops from 645 to 448.
2026-03-28 18:00:11 +01:00
Abdessamad Derraz
3092d73122 fix: cross-reference checks data/ directories for false positives
_find_in_repo and _name_in_index now scan data/ in addition to
bios/ via database.json. Eliminates 129 false positives from
game data migrated to data/ (OpenTyrian, ScummVM, SDLPAL, Cave
Story, Syobon Action). True missing: 782 -> 653.
2026-03-28 17:31:22 +01:00
Abdessamad Derraz
3d5d5b601c feat: mass BIOS collection (+1020 files, 7520 total)
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.

New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz
7dc8428ac1 refactor: fix cross-reference archive grouping and path resolution
Group archived files by archive unit in find_undeclared_files instead
of reporting individual ROMs. Add path-based fallback for descriptive
names (e.g. "SeaBIOS (128 KB)" resolves via path: bios.bin). Update
_collect_extras to use archive name for pack resolution. Regenerate
database with new bios files. 6 new E2E tests covering archive
in_repo, missing archives, descriptive names, and pack extras.
2026-03-28 14:00:08 +01:00
Abdessamad Derraz
366695b764 Update README.md 2026-03-28 12:12:05 +01:00
Abdessamad Derraz
a5285a9fc0 feat: add ti83 v1.10 and doom2.wad, zero platform missing
TI-83 v1.10 ROM from TOSEC archive (sha1 verified).
DOOM II WAD from archive.org for prboom libretro core.
All platforms now at 0 missing files.
2026-03-28 11:40:06 +01:00
Abdessamad Derraz
1f65662fd2 feat: add ~110 missing bios files from archive.org and bizhawk
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.

Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz
2666ebd9b7 chore: regenerate readme and database timestamps 2026-03-28 09:24:01 +01:00
Abdessamad Derraz
5cbd461a97 docs: update readme and database for ground truth feature 2026-03-27 23:55:30 +01:00
Abdessamad Derraz
74f48b8881 Add emulator profiles and update docs
Add new emulator profiles (ares, BigPEmu, Eden, Model2, PrimeHack, Supermodel, Suyu, Xenia Canary, Yuzu) under emulators/*.yml with metadata, notes, file lists and exclusion/hle details. Update README counts (emulators/systems/files summary and auto-generated timestamp) and bump generated_at in database.json. Adjust mkdocs.yml navigation counts and add Yuzu entry to the Emulators nav.
2026-03-26 13:22:11 +01:00
Abdessamad Derraz
8ac64d6143 docs: add build guide and documentation site section to readme
Quick start commands for generating packs by platform, emulator, or
system. Documentation site section explains what users can find there.
2026-03-26 07:15:01 +01:00
Abdessamad Derraz
bbe4936cf2 feat: add 12 batocera standalone profiles, update 3 existing
New profiles: citron (Switch), clk (multi-system), demul (Hikaru),
eka2l1 (Symbian/N-Gage), gsplus (Apple IIGS), lexaloffle (PICO-8),
openmsx (MSX), ryujinx (Switch), shadps4 (PS4), tsugaru (FM Towns),
x16emu (Commander X16), xroar (Dragon/CoCo), zc210 (Zelda Classic).
Updated: amiberry, fsuae (standalone Amiga), ymir (Saturn standalone).
2026-03-26 07:01:16 +01:00
Abdessamad Derraz
3f676b75e8 feat: standalone emulator support for batocera and multi-platform name mapping
resolve_platform_cores() builds reverse index from profile cores: field,
fixing 17 name mismatches across Batocera, RetroBat, and Recalbox
(genesisplusgx, pce_fast, pcfx, vb, mame078plus, vice cores, etc.).

standalone_path field on file entries + standalone_cores on platform
YAMLs enable mode-aware pack generation. find_undeclared_files() uses
standalone_path for cores the platform runs standalone, filters by
mode: libretro/standalone per file.

batocera.yml gains standalone_cores (92 entries from configgen-defaults).
generate_readme.py dynamically lists platforms from registry.
3 profiles updated for standalone type/path (mame, hatari, mupen64plus_next).
78 E2E tests pass, pipeline verified.
2026-03-26 00:44:21 +01:00
Abdessamad Derraz
44dc946217 docs: add romm to platform references 2026-03-25 23:19:52 +01:00
Abdessamad Derraz
d2cc9b8f29 feat: add doom engine wad files, emulatorjs base config 2026-03-25 23:12:53 +01:00
Abdessamad Derraz
c61d40d8ef docs: rewrite readme for clarity and discoverability 2026-03-25 17:18:43 +01:00
Abdessamad Derraz
38059e0fdf chore: regenerate database, readme, mkdocs nav 2026-03-25 16:49:49 +01:00
Abdessamad Derraz
6e237e3b6d chore: regenerate database, readme, mkdocs nav 2026-03-25 15:54:17 +01:00
Abdessamad Derraz
5529f8474a chore: regenerate database and readme 2026-03-25 15:41:46 +01:00
Abdessamad Derraz
313637663a docs: soften tone, explain methodology without dismissing other sources 2026-03-25 14:51:57 +01:00
Abdessamad Derraz
593466b655 feat: add methodology and ground truth narrative to readme and site 2026-03-25 14:50:09 +01:00
github-actions[bot]
1996135cd1 regenerate database and docs 2026-03-20 19:12:35 +00:00
github-actions[bot]
3a7050a8cb regenerate database and docs 2026-03-20 12:10:56 +00:00
github-actions[bot]
16fd815099 regenerate database and docs 2026-03-19 22:32:31 +00:00
Abdessamad Derraz
6ee162f8fb chore: add MAME and RetroDECK ROM sets 2026-03-19 23:26:49 +01:00