Commit Graph

340 Commits

Author SHA1 Message Date
Abdessamad Derraz
6ba2519c4e feat: add quick install section to readme 2026-03-28 18:09:40 +01:00
Abdessamad Derraz
8fe32103e6 feat: add install.ps1 powershell installer 2026-03-28 18:09:13 +01:00
Abdessamad Derraz
46426aa1e4 feat: add install.py universal installer engine 2026-03-28 18:07:28 +01:00
Abdessamad Derraz
42f2cc5617 fix: cross-reference resolves by path: field as fallback
Many emulator profiles use descriptive names (e.g., "SeaBIOS
(128 KB)") while files exist under their path: field basename
(e.g., "bios.bin"). Try path: when name: fails. Eliminates
206 false positives. True missing: 448 -> 242.
2026-03-28 18:06:04 +01:00
Abdessamad Derraz
7f46996b4d feat: add --manifest-targets for installer 2026-03-28 18:04:27 +01:00
Abdessamad Derraz
1a0eef563b feat: add install.sh posix bootstrap 2026-03-28 18:04:08 +01:00
Abdessamad Derraz
5763ce731d feat: add --manifest mode to generate_pack 2026-03-28 18:02:54 +01:00
Abdessamad Derraz
76fe7dd76f fix: cross-reference checks inside ZIP archives
_build_supplemental_index scans both data/ directories and
contents of bios/ ZIP files. Eliminates 197 false positives
where files existed inside archive ZIPs (neogeo.zip, pgm.zip,
stvbios.zip, etc.) but were counted as missing. True missing
drops from 645 to 448.
2026-03-28 18:00:11 +01:00
Abdessamad Derraz
c6a24446ba feat: add install metadata to platform registry 2026-03-28 17:58:05 +01:00
Abdessamad Derraz
68824bf3d4 chore: regenerate database after wave 2 collection
7545 files indexed. Cross-reference: 645 true missing (down
from 1011). All platform baselines at 100% or pre-existing
untested. Only required missing: 3 CLK Enterprise 128 ROMs.
2026-03-28 17:53:39 +01:00
Abdessamad Derraz
31ec407c91 feat: add FBNeo st010/st011/sfiii3na, PCem rivatnt, misc
sfiii3na.zip (CPS3 Asia No-CD), st010.zip and st011.zip
(SNES coprocessors) complete FBNeo required coverage.
rivatnt.bin completes PCem at 139/139. Also add QEMU
firmware (4), FM Towns Marty ROMs (4), Fuse mf1.rom,
Hatari BOOT.ST, PS1 extracts (2), ZC210 data (2).
2026-03-28 17:42:53 +01:00
Abdessamad Derraz
23dd7657aa fix: restore all emulator profiles to ground truth
Previous commit failed to fully revert fuse.yml (agent-added
MD5/SHA1 still present). Also revert qemu.yml and higan_sfc.yml.
All profiles now match their pre-collection state (2666ebd).
Emulator profiles are source-verified ground truth — agents
must never modify them without code verification.
2026-03-28 17:39:31 +01:00
Abdessamad Derraz
fea0338062 fix: revert agent modifications to fuse.yml and qemu.yml
Agents added MD5/SHA1 hashes to fuse.yml computed from downloaded
files, not from source code (ground truth violation: validation
is [size] only). Agents added firmware entries to qemu.yml without
source code verification. Profiles must reflect code, not files.
2026-03-28 17:36:39 +01:00
Abdessamad Derraz
3092d73122 fix: cross-reference checks data/ directories for false positives
_find_in_repo and _name_in_index now scan data/ in addition to
bios/ via database.json. Eliminates 129 false positives from
game data migrated to data/ (OpenTyrian, ScummVM, SDLPAL, Cave
Story, Syobon Action). True missing: 782 -> 653.
2026-03-28 17:31:22 +01:00
Abdessamad Derraz
3d5d5b601c feat: mass BIOS collection (+1020 files, 7520 total)
Add ~1500 BIOS, firmware and game data files sourced from
24 parallel agents. Migrate game engine data (ScummVM,
SDLPAL, OpenTyrian, Cave Story, Syobon Action) from bios/
to data/ via _data_dirs.yml. Add Roland MT-32/CM-32L ROMs
to bios/Roland/. Upload large files (EDK2, SDLPAL, ScummVM,
arcade ZIPs, EasyRPG soundfont) to large-files release.

New coverage: QEMU firmware (67), PCem (137), VICE (53),
Cemu Wii U (58), shadPS4 (14), Amiga (24), MAME ZIPs (724),
O2EM voice (270), Azahar 3DS (4), EightyOne ZX81 (35),
Stella KidVid (14), XRoar (12), and 77 emulators total.
2026-03-28 16:51:43 +01:00
Abdessamad Derraz
7dc8428ac1 refactor: fix cross-reference archive grouping and path resolution
Group archived files by archive unit in find_undeclared_files instead
of reporting individual ROMs. Add path-based fallback for descriptive
names (e.g. "SeaBIOS (128 KB)" resolves via path: bios.bin). Update
_collect_extras to use archive name for pack resolution. Regenerate
database with new bios files. 6 new E2E tests covering archive
in_repo, missing archives, descriptive names, and pack extras.
2026-03-28 14:00:08 +01:00
Abdessamad Derraz
1ee8623b2e feat: add 47 missing bios files and fix cross-reference false positives
Add BIOS files for CLK (Apple II, Mac, Atari ST, Enterprise, MSX,
Commodore, Thomson, PC, Acorn), FBNeo (CPS3 redearthn/sfiii2h),
and QEMU variants. Sources: Asimov mirror, MAME chip extraction,
86Box, Theodore, XRoar upstream, official QEMU repo.

Fix cross-reference in verify.py to group archived files by archive
unit instead of reporting individual ROMs as missing. Add path-based
fallback for descriptive names. Update generate_pack.py extras to
use archive name for resolution. 6 new E2E tests.

MISSING required: 321 -> 6 (zero false positives).
2026-03-28 13:59:48 +01:00
Abdessamad Derraz
543cb504bb fix: skip file/directory path conflicts in pack generation 2026-03-28 12:57:30 +01:00
Abdessamad Derraz
645dc013a4 feat: add qemu bios hash variants 2026-03-28 12:33:03 +01:00
Abdessamad Derraz
5b5b304288 feat: add dragon and coco clone bios variants 2026-03-28 12:33:02 +01:00
Abdessamad Derraz
e07842ac4b chore: add pyproject.toml for uv/pip compatibility 2026-03-28 12:33:00 +01:00
Abdessamad Derraz
366695b764 Update README.md 2026-03-28 12:12:05 +01:00
Abdessamad Derraz
460c49dbcb fix: bizhawk scraper lowercase sha1 + complete bios 2026-03-28 11:49:52 +01:00
Abdessamad Derraz
a5285a9fc0 feat: add ti83 v1.10 and doom2.wad, zero platform missing
TI-83 v1.10 ROM from TOSEC archive (sha1 verified).
DOOM II WAD from archive.org for prboom libretro core.
All platforms now at 0 missing files.
2026-03-28 11:40:06 +01:00
Abdessamad Derraz
84d344450f feat: add vice, qemu, pcem, freedoom, x16, iigs, dsi bios
VICE: basic, chargen, kernal for x64sdl libretro core.
QEMU: SeaBIOS 1.16.3 + VGA BIOS from system package.
PCem: MDA font ROM from PCem v17 ROM set.
Freedoom: freedoom1.wad + freedoom2.wad for prboom.
Commander X16: rom.bin R49 from x16-rom releases.
Apple IIGS: ROM01 from asimov mirrors.
DSi: NDS_Bios7i, NDS_Bios9i, DSi_Firmware, SGB_(JU).
2026-03-28 11:32:58 +01:00
Abdessamad Derraz
dd689bbd2a feat: add name aliases for bizhawk platform resolution
Copy existing BIOS files under BizHawk-expected names:
MCD_eu_100.bin, MCD_jp_100p.bin, PSX_DTLH2000.bin,
MSX_4000bios.rom, MSX_8020-20bios.rom.
2026-03-28 10:29:13 +01:00
Abdessamad Derraz
1f65662fd2 feat: add ~110 missing bios files from archive.org and bizhawk
Sources: emularity_bios_v1, mame-bios-0.282, mame-bios-devices,
mame251, MAME216, bizhawk-firmware-2.11.0, neogeo_bios, retrobat.

Neo Geo: neogeo.zip upgraded 6 to 38 files, ngp.zip added.
Arcade: dsp1-4, jojoa/jojoba/jojobar1, fdsbios, decocass, neocdz.
Multi-platform: 3DO, Atari 7800/Lynx, C64, Channel F, ColecoVision,
Vectrex, Intellivision, GB/GBC/GBA, NDS, N64DD, SNES DSP/Satellaview,
PC Engine CD, PC-FX, SMS, Game Gear, Mega Drive, Mega-CD, Saturn STV,
Videopac, MSX, Amiga, ZX Spectrum, TI-83.
2026-03-28 10:26:30 +01:00
Abdessamad Derraz
b854f71b44 feat: add bizhawk platform config 2026-03-28 09:45:12 +01:00
Abdessamad Derraz
2738a4d326 feat: add bizhawk scraper for FirmwareDatabase.cs 2026-03-28 09:41:52 +01:00
Abdessamad Derraz
b75f2b2a43 feat: add sha1 verification mode for bizhawk 2026-03-28 09:35:13 +01:00
Abdessamad Derraz
f1855641c5 docs: add granular pack options to tools reference 2026-03-28 09:29:31 +01:00
Abdessamad Derraz
2666ebd9b7 chore: regenerate readme and database timestamps 2026-03-28 09:24:01 +01:00
Abdessamad Derraz
22a1e7caf4 fix: sync NstDatabase.xml hashes in platform configs 2026-03-28 09:19:37 +01:00
Abdessamad Derraz
67186448a2 fix: verify and add manifests to split packs 2026-03-28 08:43:41 +01:00
Abdessamad Derraz
70891314d3 fix: include core extras in split packs 2026-03-28 08:29:37 +01:00
Abdessamad Derraz
97b9900f62 chore: update NstDatabase.xml from upstream nestopia
Closes #40
2026-03-28 08:24:13 +01:00
Abdessamad Derraz
a1aa97a70e fix: include core extras in split packs 2026-03-28 08:06:22 +01:00
Abdessamad Derraz
97b1c2c08a feat: add --from-md5 lookup and pack builder 2026-03-28 01:02:25 +01:00
Abdessamad Derraz
0624e9d87e test: add validation tests for pack arg combos 2026-03-28 00:59:02 +01:00
Abdessamad Derraz
3ded72f72b feat: add --group-by manufacturer for split packs 2026-03-28 00:45:12 +01:00
Abdessamad Derraz
94a28f5459 feat: add --split flag for per-system packs 2026-03-28 00:43:20 +01:00
Abdessamad Derraz
837ac80cca refactor: deduplicate manufacturer prefix list 2026-03-28 00:39:28 +01:00
Abdessamad Derraz
43cb7a9884 feat: allow --platform + --system combination 2026-03-28 00:36:51 +01:00
Abdessamad Derraz
020ff148c2 feat: add --required-only flag to generate_pack 2026-03-28 00:32:58 +01:00
Abdessamad Derraz
5cbd461a97 docs: update readme and database for ground truth feature 2026-03-27 23:55:30 +01:00
Abdessamad Derraz
6cbdd4c40c test: add integration tests for ground truth full chain 2026-03-27 23:41:06 +01:00
Abdessamad Derraz
37acc8d0fc feat: add --verbose flag and ground truth rendering 2026-03-27 23:38:43 +01:00
Abdessamad Derraz
2cf1398786 feat: attach ground truth to emulator verification results 2026-03-27 23:33:53 +01:00
Abdessamad Derraz
6b14b5e2b1 feat: attach ground truth to platform verification results 2026-03-27 23:30:49 +01:00
Abdessamad Derraz
6d959ff2b0 feat: add per-emulator ground truth to validation index 2026-03-27 23:25:42 +01:00