Commit Graph

475 Commits

Author SHA1 Message Date
Abdessamad Derraz
d0dd05ddf6 docs: add wiki pages for all audiences, fix .old.yml leak
9 new wiki pages: getting-started, faq, troubleshooting,
advanced-usage, verification-modes, adding-a-platform,
adding-a-scraper, testing-guide, release-process.

Updated architecture.md with mermaid diagrams, tools.md with
full pipeline and target/exporter sections, profiling.md with
missing fields, index.md with glossary and nav links.

Expanded CONTRIBUTING.md from stub to full contributor guide.

Filter .old.yml from load_emulator_profiles, generate_db alias
collection, and generate_readme counts. Fix BizHawk sha1 mode
in tools.md, fix RetroPie path, fix export_truth.py typos.
2026-03-30 23:58:12 +02:00
Abdessamad Derraz
038c3d3b40 feat: enrich MAME/FBNeo profiles with upstream BIOS contents
auto-fetched from mamedev/mame 0.287 and finalburnneo/FBNeo v1.0.0.2.
mame: +20 new BIOS root sets, 96 entries enriched with contents.
mamearcade: 47 entries enriched with contents.
mamemess: 20 entries enriched with contents.
fbneo: +13 new ROM entries from upstream BIOS sets.
2026-03-30 21:39:02 +02:00
Abdessamad Derraz
427fef5669 fix: text-based YAML patching preserves formatting
replace yaml.dump with surgical text edits for contents/source_ref.
preserves comments, block scalars, quoting, indentation.
fix FBNeo new entry detection using parsed keys instead of text search.
2026-03-30 21:35:41 +02:00
Abdessamad Derraz
75e34898ee feat: add MAME/FBNeo hash auto-fetch scrapers
sparse clone upstream repos, parse BIOS root sets from C source,
cache as JSON, merge into emulator profiles with backup.
covers macro expansion, version detection, subset profile protection.
2026-03-30 19:11:26 +02:00
Abdessamad Derraz
94c3ac9834 feat: add hash merge for MAME and FBNeo profiles 2026-03-30 18:33:09 +02:00
Abdessamad Derraz
319a1d2041 feat: add MAME source code parser for BIOS root sets 2026-03-30 18:29:31 +02:00
Abdessamad Derraz
00d7b57884 feat: add FBNeo source parser for BIOS sets 2026-03-30 18:29:06 +02:00
Abdessamad Derraz
caf6285a04 fix: skip entries without md5 in batocera and retrobat exports 2026-03-30 17:46:48 +02:00
Abdessamad Derraz
529cb8a915 fix: recalbox paths from scrape, batocera md5 fallback from scrape 2026-03-30 17:35:39 +02:00
Abdessamad Derraz
1146fdf177 fix: rewrite emudeck exporter to match exact checkBIOS.sh format 2026-03-30 17:21:56 +02:00
Abdessamad Derraz
4fbb3571f8 fix: exporters use _dest fallback, merge colliding systems, per-platform subdirs 2026-03-30 17:15:44 +02:00
Abdessamad Derraz
0be68edad0 feat: add exporters for lakka, retropie, emudeck, retrodeck, romm 2026-03-30 17:07:08 +02:00
Abdessamad Derraz
1ffc4f89ca refactor: registry merge is fully flexible, no hardcoded lists 2026-03-30 16:38:13 +02:00
Abdessamad Derraz
f1ebfff5bd refactor: registry merge uses exclusion list instead of hardcoded fields 2026-03-30 16:36:40 +02:00
Abdessamad Derraz
425ea064ae fix: scrapers merge into existing YAML instead of overwriting 2026-03-30 16:31:40 +02:00
Abdessamad Derraz
6818a18a42 feat: load_platform_config merges all metadata from registry 2026-03-30 16:24:40 +02:00
Abdessamad Derraz
c11de6dba6 fix: restore retroarch.yml fields lost by scraper regeneration 2026-03-30 16:22:16 +02:00
Abdessamad Derraz
c4f3192020 fix: system.dat rom quoting, native_ids, acronym display names 2026-03-30 16:17:50 +02:00
Abdessamad Derraz
e2d0510f4e fix: exporters match exact native formats with display names 2026-03-30 16:09:02 +02:00
Abdessamad Derraz
74269bab84 fix: rewrite exporters to match exact native formats 2026-03-30 15:49:33 +02:00
Abdessamad Derraz
1e6b499602 feat: add batocera, recalbox, retrobat native exporters 2026-03-30 15:31:44 +02:00
Abdessamad Derraz
9b785ec785 feat: add missing laseractive sega pac bios files
v1.05 japan and v1.01 japan from archive.org.
v1.04 us variant alias for pioneer-named file.
resolves retrobat laseractive missing files.
2026-03-30 15:16:23 +02:00
Abdessamad Derraz
d415777f2c feat: add PS3UPDAT.PUP to rpcs3 profile
storage: large_file, validated via SCEUF magic + HMAC-SHA1
per entry (PUP.cpp:23-77). also adds missing standard fields
(cores, core_classification, upstream), removes non-standard
firmware_file/validation/firmware_version fields.
2026-03-30 15:06:51 +02:00
Abdessamad Derraz
eafabd20f3 refactor: skip writing generated files when content unchanged
write_if_changed in common.py compares content after stripping
timestamps (generated_at, Auto-generated on, Generated on).
applied to generate_db, generate_readme, generate_site.
eliminates timestamp-only diffs in database.json, README.md,
mkdocs.yml, and 423 docs pages.
2026-03-30 14:33:44 +02:00
Abdessamad Derraz
2aca4927c0 chore: regenerate database, readme, manifests, site 2026-03-30 14:19:00 +02:00
Abdessamad Derraz
17777f315b feat: agnostic bios mode for filename-agnostic emulators
bios_mode: agnostic (profile) and agnostic: true (file) for
emulators that accept any valid BIOS without specific filename.
find_undeclared_files skips agnostic entries, pack extras scan
includes all matching DB files by path prefix + size criteria,
resolve_local_file has agnostic fallback with rename README.
applied to pcsx2, lrps2 (bios_mode), melonds dsi_nand (file).
2026-03-30 14:18:54 +02:00
Abdessamad Derraz
692484d32d refactor: remove false aliases from pcsx2 and melonds profiles
aliases must be same-SHA1 alternative names, not distinct files.
pcsx2: 164 different BIOS dumps are separate DB entries, not aliases.
melonds: 6 regional NAND dumps are separate DB entries, not aliases.
also cleans pcsx2 non-standard fields, fixes display_name.
2026-03-30 12:37:32 +02:00
Abdessamad Derraz
a8430940f9 feat: add regional nand aliases to melonds profile
dsi_nand.bin aliases: DSi_Nand_USA/EUR/JPN/AUS/CHN/KOR.bin
for repo resolution. Code loads a single configurable path
(libretro.cpp:836, EmuInstance.cpp:1036-1050), validates
nocash footer (DSi_NAND.cpp:42-111). size + storage added.
2026-03-30 12:20:26 +02:00
Abdessamad Derraz
1f073f521d fix: preserve batocera version when github fetch fails 2026-03-30 11:55:57 +02:00
Abdessamad Derraz
903c49edcf feat: add tests for registry merge, all_libretro expansion, hash fallback, system normalization 2026-03-30 11:33:59 +02:00
Abdessamad Derraz
d3a2224dd2 chore: regenerate database, readme, manifests, site 2026-03-30 09:42:39 +02:00
Abdessamad Derraz
f898f26847 chore: regenerate retrobat.yml with corrected system slugs 2026-03-30 09:11:00 +02:00
Abdessamad Derraz
2712307420 feat: add tandy vis bios root set to mame profiles
vis.zip (p513bk0b.bin + p513bk1b.bin) from src/mame/trs/vis.cpp.
Driver added in MAME 0.180, not present in older cores.
2026-03-30 09:09:58 +02:00
Abdessamad Derraz
54022e9db1 feat: hash-based matching for cross-reference
expand_platform_declared_names resolves platform file MD5s
through the database to recover canonical names and aliases,
eliminating false positive undeclared files when a platform
renames a file (e.g. Batocera ROM1 vs gsplus ROM).
2026-03-30 08:25:54 +02:00
Abdessamad Derraz
4db9e4350c fix: add missing system slugs to batocera and retrobat scrapers 2026-03-30 07:58:46 +02:00
Abdessamad Derraz
6864ce6584 feat: diff hash fallback detects platform renames 2026-03-30 07:53:59 +02:00
Abdessamad Derraz
12196b6445 feat: add 55 missing cores across 6 platform registries 2026-03-30 07:17:57 +02:00
Abdessamad Derraz
7551e41a7b feat: load_platform_config merges cores from registry 2026-03-30 07:13:15 +02:00
Abdessamad Derraz
7b484605d4 feat: add 47 missing cores to batocera platform config 2026-03-30 07:09:52 +02:00
Abdessamad Derraz
b587381f05 feat: resolve_platform_cores expands all_libretro in list 2026-03-30 07:06:32 +02:00
Abdessamad Derraz
261f3c1671 chore: regenerate database 2026-03-30 06:15:02 +02:00
Abdessamad Derraz
d3bd68f6ad feat: add vsmile, tutor, vg5k, vgmplay profiles and system aliases 2026-03-30 06:14:58 +02:00
Abdessamad Derraz
8f29e0e85c Update DB timestamp; add ti99sim emulator
Bump database.json generated_at timestamp and add a new emulator metadata file emulators/ti99sim.yml. The new YAML registers the TI-99/Sim standalone emulator (ti99sim) with system info, core version, notes on ROM handling, and multiple required/optional ROM entries (with SHA1s, sizes and validation notes) for TI-99/4A support.
2026-03-29 23:33:12 +02:00
Abdessamad Derraz
f39b11955f feat: add ROM_OS_B_NTSC and fs-5500_disk.rom, fix retrodeck
Add Atari 800 OS Rev B NTSC (CRC32 0e86d61d, canonical sysrom.c
match) and National FS-5500 disk controller ROM for openMSX.
Remove ROM_400/800_CUSTOM from retrodeck.yml (config slot key with
forward slash in name, not a real file).
2026-03-29 23:23:52 +02:00
Abdessamad Derraz
83ccf17b11 feat: add ti99 and msx BIOS entries, add fs-5500 disk rom 2026-03-29 23:20:44 +02:00
Abdessamad Derraz
56bff1d013 feat: add ti99 system to mame profile 2026-03-29 23:16:33 +02:00
Abdessamad Derraz
3bd9c0ebef chore: regenerate database, add atari 400/800 BIOS 2026-03-29 23:16:19 +02:00
Abdessamad Derraz
3824193a11 feat: add socrates and vc4000 systems to mame profiles 2026-03-29 23:15:25 +02:00
Abdessamad Derraz
2e21d64a08 refactor: harden codebase and remove unicode artifacts
- fix urllib.parse.quote import (was urllib.request.quote)
- add operator precedence parens in generate_pack dedup check
- narrow bare except to specific types in batocera target scraper
- cache load_platform_config and build_zip_contents_index results
- add selective algorithm support to compute_hashes
- atomic write for fetch_large_file (tmp + rename)
- add response size limit to base scraper fetch
- extract build_target_cores_cache to common.py (dedup verify/pack)
- hoist _build_supplemental_index out of per-platform loop
- migrate function-attribute caches to module-level dicts
- add @abstractmethod to BaseTargetScraper.fetch_targets
- remove backward-compat re-exports from common.py
- replace em-dashes and unicode arrows with ASCII equivalents
- remove decorative section dividers and obvious comments
2026-03-29 23:15:20 +02:00
Abdessamad Derraz
0c5cde83e1 Add TRS-80, RX-78, Sega AI entries; refactor tools
Add many MAME/MESS BIOS entries (TRS-80 family, Bandai RX-78, Sega AI) and update docs/navigation counts (README, mkdocs). Remove empty supplemental file references from database.json and update generated timestamps and totals. Harden and refactor tooling: add MAX_RESPONSE_SIZE limited reader in base_scraper, make target scrapers an abstract base, narrow exception handling in the Batocera targets parser, and switch generate_pack.py and verify.py to use build_target_cores_cache (simplifies target config loading and error handling). verify.py also loads supplemental cross-reference names and accepts them through verify_platform. Update tests to import from updated modules (validation/truth). Misc: small bugfix for case-insensitive path conflict check.
2026-03-29 23:04:30 +02:00