Commit Graph

94 Commits

Author SHA1 Message Date
Abdessamad Derraz
74f48b8881 Add emulator profiles and update docs
Add new emulator profiles (ares, BigPEmu, Eden, Model2, PrimeHack, Supermodel, Suyu, Xenia Canary, Yuzu) under emulators/*.yml with metadata, notes, file lists and exclusion/hle details. Update README counts (emulators/systems/files summary and auto-generated timestamp) and bump generated_at in database.json. Adjust mkdocs.yml navigation counts and add Yuzu entry to the Emulators nav.
2026-03-26 13:22:11 +01:00
Abdessamad Derraz
8ac64d6143 docs: add build guide and documentation site section to readme
Quick start commands for generating packs by platform, emulator, or
system. Documentation site section explains what users can find there.
2026-03-26 07:15:01 +01:00
Abdessamad Derraz
bbe4936cf2 feat: add 12 batocera standalone profiles, update 3 existing
New profiles: citron (Switch), clk (multi-system), demul (Hikaru),
eka2l1 (Symbian/N-Gage), gsplus (Apple IIGS), lexaloffle (PICO-8),
openmsx (MSX), ryujinx (Switch), shadps4 (PS4), tsugaru (FM Towns),
x16emu (Commander X16), xroar (Dragon/CoCo), zc210 (Zelda Classic).
Updated: amiberry, fsuae (standalone Amiga), ymir (Saturn standalone).
2026-03-26 07:01:16 +01:00
Abdessamad Derraz
3f676b75e8 feat: standalone emulator support for batocera and multi-platform name mapping
resolve_platform_cores() builds reverse index from profile cores: field,
fixing 17 name mismatches across Batocera, RetroBat, and Recalbox
(genesisplusgx, pce_fast, pcfx, vb, mame078plus, vice cores, etc.).

standalone_path field on file entries + standalone_cores on platform
YAMLs enable mode-aware pack generation. find_undeclared_files() uses
standalone_path for cores the platform runs standalone, filters by
mode: libretro/standalone per file.

batocera.yml gains standalone_cores (92 entries from configgen-defaults).
generate_readme.py dynamically lists platforms from registry.
3 profiles updated for standalone type/path (mame, hatari, mupen64plus_next).
78 E2E tests pass, pipeline verified.
2026-03-26 00:44:21 +01:00
Abdessamad Derraz
d2cc9b8f29 feat: add doom engine wad files, emulatorjs base config 2026-03-25 23:12:53 +01:00
Abdessamad Derraz
38059e0fdf chore: regenerate database, readme, mkdocs nav 2026-03-25 16:49:49 +01:00
Abdessamad Derraz
6e237e3b6d chore: regenerate database, readme, mkdocs nav 2026-03-25 15:54:17 +01:00
Abdessamad Derraz
5529f8474a chore: regenerate database and readme 2026-03-25 15:41:46 +01:00
Abdessamad Derraz
dbc26b11c1 refactor: move fetch_large_file to common, auto-download on db rebuild 2026-03-25 13:19:12 +01:00
Abdessamad Derraz
910428c6f1 fix: resolve large files from cache in database paths 2026-03-25 12:52:20 +01:00
Abdessamad Derraz
47e6174ed4 fix: pack naming, large file preservation, discrepancy reporting 2026-03-25 12:23:40 +01:00
Abdessamad Derraz
ebb55a445b feat: re-profile 40 emulators, harden CI workflows
profile emulators pd777 through tic80, add frozen snapshots
(puae2021, snes9x2002/2005/2010, stella2014/2023).

CI: replace github-script with gh CLI, add test execution,
job-level permissions, propagate changed output, pin jsonschema.
2026-03-25 07:00:17 +01:00
Abdessamad Derraz
94000bdaef fix: align verify and pack validation, pipeline 100% consistent
generate_pack.py now applies emulator-level validation (crc32, sha1,
adler32) matching verify.py behavior. existence mode: validation is
informational (file present = OK). md5 mode: validation downgrades
to UNTESTED. clone resolution moved to common.py resolve_local_file.
all 6 platforms pass consistency check.
2026-03-24 22:21:47 +01:00
Abdessamad Derraz
ae4846550f fix: clone resolution in common.py, move clone map to root
moved _mame_clones.json out of bios/ (was indexed by generate_db.py
as BIOS file). clone resolution now in common.py resolve_local_file
so all tools (verify, pack, cross_reference) resolve clones
transparently. removed duplicate clone code from generate_pack.py.
added error handling on os.remove in dedup.py. consistency check
now passes for Batocera/EmuDeck/Lakka/RetroArch (4/6 platforms).
2026-03-24 21:57:49 +01:00
Abdessamad Derraz
fb1007496d chore: deduplicate bios/ — remove 427 files, save 227 MB
true duplicates (same file in multiple dirs): removed copies, kept
canonical. MAME device clones (different names, identical content):
removed copies, created _mame_clones.json mapping for pack-time
assembly via deterministic ZIP rebuild. generate_pack.py resolves
clones transparently. 95 canonical ZIPs serve 392 clone names.
2026-03-24 21:35:50 +01:00
Abdessamad Derraz
69987cc0bb feat: re-profile 30 emulators, beetle to parallel_n64
source-verified: mesen FdsBios.bin→alias, MesenDB.txt phantom (embedded).
mupen64plus 64DD_IPL.bin→IPL.n64 (code path fix). neocd 000-lo.lo/ng-lo.rom
phantoms (algorithmically generated). new profiles: mesen-s, mupen64plus
next develop/gles2/gles3. regenerate database.
2026-03-24 14:06:20 +01:00
Abdessamad Derraz
ef25f8cebf feat: re-profile 40 emulators from fbalpha2012 to holani
batch re-profiled beetle, holani, kronos, mame variants, mednafen
cores with upstream verification. beetle_pce, mame, mame2010 excluded
(regressions detected, need manual review).
2026-03-24 11:36:58 +01:00
Abdessamad Derraz
4c1caea621 feat: re-profile 40 emulators from fbalpha2012 to holani 2026-03-24 09:33:06 +01:00
Abdessamad Derraz
65b6ab9354 feat: re-profile 14 emulators, add dosbox and ep128emu_core
batch re-profiled dosbox_pure through fbalpha2012_cps1 with upstream
verification. new profiles: dosbox, ep128emu_core. regenerate database.
2026-03-23 15:04:43 +01:00
Abdessamad Derraz
438fb6e9af feat: re-profile 12 emulators, fix dosbox_core regression
batch re-profiled clownmdemu through dosbox_core with upstream
verification. restored dosbox_core mt-32/cm-32l rom halves removed
by batch (14 files), added core_classification. regenerate database.
2026-03-23 12:37:43 +01:00
Abdessamad Derraz
78fa5c3d0f feat: re-profile citra, regenerate database 2026-03-23 11:02:53 +01:00
Abdessamad Derraz
06f5157c35 feat: profile mame, mame2010, mame2015, mame2016, fix aliases
source-verified BIOS root sets for all MAME cores:
- mame (0.286): 99 sets, official_port, new profile
- mame2010 (0.139): 51 sets, remove cvs.zip phantom
- mame2015 (0.160): 69 sets, replace false mame2016 alias
- mame2016 (0.174): 69 sets, fix 3 phantoms, add 2 missing
- mamearcade/mamemess: fix alias_of mame2016 to mame
2026-03-22 22:30:11 +01:00
Abdessamad Derraz
4883520dbf feat: profile mame2000, migrate bios_zips to structured files
Replace bios_zips section with structured files entries (category:
bios_zip, contents: with name/description/size/crc32) across all 4
MAME profiles. Profile mame2000 from source (was false alias of
mame2003_plus). Adds neogeo.zip, decocass.zip, hiscore.dat, cheat.dat.
2026-03-22 19:57:30 +01:00
Abdessamad Derraz
c342b61cdf feat: profile jumpnbump, kronos, lowresnx, lutro, m2000
add core_classification and upstream to all five profiles. kronos:
add ROM cartridge files (mpr-18811-mx.ic1, mpr-19367-mx.ic1), add
archive: field to STV blobs, remove unverified MD5 hashes, fix
cores list (remove yabause_libretro). jumpnbump: fix incorrect
note about embedded data. lowresnx, lutro, m2000: add missing
classification fields.
2026-03-22 14:39:47 +01:00
Abdessamad Derraz
1d350f0578 feat: add emulator/system pack generation, validation checks, path resolution
add --emulator, --system, --standalone, --list-emulators, --list-systems
to verify.py and generate_pack.py. packs are RTU with data directories,
regional BIOS variants, and archive support.

validation: field per file (size, crc32, md5, sha1) with conflict
detection. by_path_suffix index in database.json for regional variant
resolution via dest_hint. restructure GameCube IPL to regional subdirs.

66 E2E tests, full pipeline verified.
2026-03-22 14:02:20 +01:00
Abdessamad Derraz
74f17694c2 feat: add category field to emulator profiles, source missing BIOS
Add category: game_data to sdlpal, nxengine, opentyrian, easyrpg,
mkxp_z profiles. verify.py separates game_data from bios in core
gap metrics for cleaner coverage numbers.

New BIOS files: Cemu fonts (4), QEMU bios-256k + vgabios-stdvga,
GAM4980 ROMs (2), SC-3000 Export variant.
2026-03-21 07:37:22 +01:00
Abdessamad Derraz
13a2938dc7 fix: correct 338 retrodeck hashes (sha1 truncation + stale mame 0.285) 2026-03-20 23:40:09 +01:00
github-actions[bot]
1996135cd1 regenerate database and docs 2026-03-20 19:12:35 +00:00
Abdessamad Derraz
27df5c8fb5 fix: resolve case collisions on case-insensitive filesystems
Move 17 case-colliding BIOS variants to .variants/ so Windows
and macOS clones no longer lose files. Existence-based platforms
keep the primary, MD5-based platforms resolve from .variants/.

Also fix resolve_local_file zipped_file resolution: when multiple
ZIPs share a name, verify the inner ROM exists before accepting
a candidate. Fixes adam_fdc.zip resolving to the wrong archive.
2026-03-20 20:02:42 +01:00
github-actions[bot]
3a7050a8cb regenerate database and docs 2026-03-20 12:10:56 +00:00
Abdessamad Derraz
30c104ac8e fix: correct Jaguar BIOS MD5 in retrodeck platform 2026-03-20 13:05:09 +01:00
Abdessamad Derraz
59ee9088fa chore: add Atari 1450XLD OS Rev 3 (1984-03-27) 2026-03-20 12:52:34 +01:00
Abdessamad Derraz
1c9261d304 chore: add Atari 1450XLD, Arabic 65XE, compiled OS 2026-03-20 10:04:06 +01:00
Abdessamad Derraz
4a76c5ca15 chore: add Atari OS-A NTSC, XEGS, doc format ares 2026-03-20 09:50:59 +01:00
Abdessamad Derraz
7b712ca089 chore: add Game Master, MSX, Atari BIOS files 2026-03-20 07:06:11 +01:00
Abdessamad Derraz
789ff0a32b chore: add MAME and RetroDECK ROM sets 2026-03-20 06:52:57 +01:00
Abdessamad Derraz
21bc225cac fix: resolve truncated md5 in batocera scraper
Batocera upstream has a truncated 29-char MD5 for zx48.rom.
The scraper now resolves truncated hashes via prefix match
against database.json, preventing schema validation failures.
2026-03-19 23:52:25 +01:00
github-actions[bot]
16fd815099 regenerate database and docs 2026-03-19 22:32:31 +00:00
Abdessamad Derraz
6ee162f8fb chore: add MAME and RetroDECK ROM sets 2026-03-19 23:26:49 +01:00
github-actions[bot]
4992af07e7 regenerate database and docs 2026-03-19 15:15:53 +00:00
Abdessamad Derraz
6a21a99c22 feat: platform-core registry for exact pack generation
resolve_platform_cores() links platforms to their cores via
three strategies: all_libretro, explicit list, system ID
fallback. Pack generation always includes core requirements
beyond platform baseline. Case-insensitive dedup prevents
conflicts on Windows/macOS. Data dir strip_components fixes
doubled paths for Dolphin and PPSSPP caches.
2026-03-19 16:10:43 +01:00
Abdessamad Derraz
257ec1a527 fix: round 2 audit fixes, updated emulator profiles
Scripts:
- fix generate_site nav regex destroying mkdocs.yml content
- fix auto_fetch comma-separated MD5 in find_missing
- fix verify print_platform_result conflating untested/missing
- fix validate_pr path traversal and symlink check
- fix batocera_scraper brace counting and escaped quotes in strings
- fix emudeck_scraper hash search crossing function boundaries
- fix pipeline.py cwd to repo root via Path(__file__)
- normalize SHA1 comparison to lowercase in generate_pack

Emulator profiles:
- emux_gb/nes/sms: reclassify from alias to standalone profiles
- ep128emu: remove .info-only files not referenced in source
- fbalpha2012 variants: full source-verified profiles
- fbneo_cps12: add new profile
2026-03-19 15:00:18 +01:00
Abdessamad Derraz
eb21d214ab chore: regenerate database with updated profiles 2026-03-19 13:16:18 +01:00
Abdessamad Derraz
13e5dd37f4 chore: remove redundant tests, keep E2E only (29 tests)
Removed 7 test files (135 tests) fully covered by test_e2e.py.
The E2E creates all fixtures in setUp, exercises every code path
with real files and real functions. Single source of regression.
2026-03-19 12:22:56 +01:00
Abdessamad Derraz
011d0f9441 chore: regenerate database with alias indexes 2026-03-19 11:04:01 +01:00
Abdessamad Derraz
1bde934c45 chore: regenerate database.json with emulator aliases 2026-03-19 09:56:20 +01:00
Abdessamad Derraz
f3db61162c feat: aliases support in resolve and db generation
generate_db.py now reads aliases from emulator YAMLs and indexes
them in database.json by_name. resolve_local_file in common.py
tries all alias names when the primary name fails to match.

beetle_psx alt_names renamed to aliases (was not indexed before).
snes9x BS-X.bios, np2kai FONT.ROM/ide.rom/pci.rom fallbacks,
all now formally declared as aliases and indexed.

verify --all and generate_pack --all pass with 0 regressions.
2026-03-19 08:15:13 +01:00
Abdessamad Derraz
86dbdf28e5 feat: core profiles, data_dirs buildbot, cross_ref fix
profiles: amiberry (new), amiarcadia, atari800, azahar, b2,
bk, blastem, bluemsx, freeintv updated with source refs,
upstream field, mode field, data_directories.

_data_dirs.yml: buildbot source for retroarch platforms,
strip_components for nested ZIPs, freeintv-overlays fixed.

cross_reference.py: data_directories-aware gap analysis,
suppresses false gaps when emulator+platform share refs.

refresh_data_dirs.py: ZIP strip_components support,
for_platforms filter, ETag freshness for buildbot.

scraper: bluemsx single ref, freeintv overlays injection.
generate_pack.py: warning on missing data directory cache.
2026-03-18 21:20:02 +01:00
Abdessamad Derraz
fbb2079f9b feat: complete 81 profile, archive 27 standalone ROMs 2026-03-18 17:52:12 +01:00
Abdessamad Derraz
846640dd7c feat: emulator mode field, archive ZX81 standalone ROMs
emulator profiles support mode: standalone | libretro | both.
cross_reference.py skips standalone-only files for libretro platforms.
81.yml: type standalone + libretro, upstream ref added, files listed
with mode: standalone and source_refs to both codebases.
bios/Sinclair/ZX 81/: zx81.rom (8K) and dkchr.rom (4K) archived.
2026-03-18 17:37:01 +01:00