Abdessamad Derraz
03a9fa3276
fix: batocera targets scraper es_systems path and condition parser
...
- Fix es_systems.yml URL (batocera-emulationstation -> batocera-es-system)
- Replace if/endif block parser with select-if condition parser that
matches the actual Config.in structure (select PACKAGE if CONDITION)
- Add Kconfig boolean condition checker (handles &&, ||, !, parentheses)
- Add meta-flag expansion (X86_64_ANY, GLES3, ROCKCHIP_GLES3, etc.)
iterated to fixpoint for chained derivations
- Fix es_systems.yml parser for the actual dict format with requireAnyOf
2026-03-26 09:16:06 +01:00
Abdessamad Derraz
6b86c543af
feat: add emudeck and retropie target scrapers
2026-03-26 08:55:58 +01:00
Abdessamad Derraz
8e91552b16
feat: add batocera per-board emulator target scraper
2026-03-26 08:55:54 +01:00
Abdessamad Derraz
c6ab8e9c3a
feat: add target scraper infra and retroarch buildbot scraper
2026-03-26 08:55:50 +01:00
Abdessamad Derraz
de58f3f28e
feat: add --platform and --target to cross_reference.py
2026-03-26 08:48:41 +01:00
Abdessamad Derraz
89054084b7
feat: target-aware fingerprint in group_identical_platforms
2026-03-26 08:48:40 +01:00
Abdessamad Derraz
ea9cd93e83
feat: propagate --target through pipeline.py
2026-03-26 08:48:34 +01:00
Abdessamad Derraz
5ac14079d6
feat: add --target and --list-targets to generate_pack.py
2026-03-26 08:48:31 +01:00
Abdessamad Derraz
1e939f1470
feat: add --target and --list-targets to verify.py
2026-03-26 08:48:29 +01:00
Abdessamad Derraz
1c34790737
feat: propagate target_cores through find_undeclared_files, find_exclusion_notes, verify_platform, _collect_emulator_extras
2026-03-26 08:44:44 +01:00
Abdessamad Derraz
e17d771710
feat: add target_cores filter to resolve_platform_cores
...
Optional target_cores parameter intersects the resolved core set,
enabling per-target filtering without changing existing call sites.
Includes 2 E2E tests covering intersection and None pass-through.
2026-03-26 08:42:08 +01:00
Abdessamad Derraz
0549b8945e
feat: add load_target_config and list_available_targets to common.py
...
Loads per-platform target YAML files from platforms/targets/,
resolves aliases and applies add_cores/remove_cores from _overrides.yml.
Includes 7 E2E tests covering alias resolution, overrides, and error paths.
2026-03-26 08:41:37 +01:00
Abdessamad Derraz
8ac64d6143
docs: add build guide and documentation site section to readme
...
Quick start commands for generating packs by platform, emulator, or
system. Documentation site section explains what users can find there.
2026-03-26 07:15:01 +01:00
Abdessamad Derraz
3f676b75e8
feat: standalone emulator support for batocera and multi-platform name mapping
...
resolve_platform_cores() builds reverse index from profile cores: field,
fixing 17 name mismatches across Batocera, RetroBat, and Recalbox
(genesisplusgx, pce_fast, pcfx, vb, mame078plus, vice cores, etc.).
standalone_path field on file entries + standalone_cores on platform
YAMLs enable mode-aware pack generation. find_undeclared_files() uses
standalone_path for cores the platform runs standalone, filters by
mode: libretro/standalone per file.
batocera.yml gains standalone_cores (92 entries from configgen-defaults).
generate_readme.py dynamically lists platforms from registry.
3 profiles updated for standalone type/path (mame, hatari, mupen64plus_next).
78 E2E tests pass, pipeline verified.
2026-03-26 00:44:21 +01:00
Abdessamad Derraz
d2cc9b8f29
feat: add doom engine wad files, emulatorjs base config
2026-03-25 23:12:53 +01:00
Roël Couwenberg
32b391ef69
feat: Addition of a RomM package ( #37 )
...
* feat: Initial addition of the RomM scraper and package
* docs: Added RomM to the verification modes table
* chore: Clean up scraper by removing workaround code from main() and an unused system translation map
2026-03-25 19:24:59 +01:00
Abdessamad Derraz
1ad10eddb7
feat: include platform version in pack filenames
2026-03-25 18:55:35 +01:00
Abdessamad Derraz
69131f4ad1
fix: emulator validation is informational, not a platform failure
2026-03-25 17:34:56 +01:00
Abdessamad Derraz
c61d40d8ef
docs: rewrite readme for clarity and discoverability
2026-03-25 17:18:43 +01:00
Abdessamad Derraz
4b09205bc9
fix: zero warnings on mkdocs build, update actions to v4/v5
2026-03-25 16:30:08 +01:00
Abdessamad Derraz
c5b267a6fb
fix: anchor mismatches in platform and emulator index pages
2026-03-25 16:23:25 +01:00
Abdessamad Derraz
851f53ba7f
refactor: extract wiki to source files, use deploy-pages action
2026-03-25 16:02:11 +01:00
Abdessamad Derraz
a6150a43bd
feat: group emulators by classification, slim mkdocs nav, add pymdownx extensions
2026-03-25 15:29:58 +01:00
Abdessamad Derraz
0196fff8c7
feat: improve site UX (quick start, system summary, collapsible sections, wiki index, actionable gaps)
2026-03-25 15:24:38 +01:00
Abdessamad Derraz
904edd65e4
docs: document CI workflows, inheritance, MAME clones, tests, scrapers
2026-03-25 15:14:29 +01:00
Abdessamad Derraz
02a7c58fca
docs: complete wiki coverage, document all scripts and edge cases
2026-03-25 15:02:23 +01:00
Abdessamad Derraz
f8a325260f
feat: add wiki pages (architecture, tools, profiling, data model)
2026-03-25 14:56:37 +01:00
Abdessamad Derraz
313637663a
docs: soften tone, explain methodology without dismissing other sources
2026-03-25 14:51:57 +01:00
Abdessamad Derraz
593466b655
feat: add methodology and ground truth narrative to readme and site
2026-03-25 14:50:09 +01:00
Abdessamad Derraz
23d76d54fd
fix: correct rendering of complex YAML fields in site pages
2026-03-25 14:38:40 +01:00
Abdessamad Derraz
1cd43c3224
feat: exploit 100% of emulator YAML fields in site generation
2026-03-25 14:32:48 +01:00
Abdessamad Derraz
b3c1462a5e
feat: exploit all emulator YAML fields in site generation
2026-03-25 14:28:02 +01:00
Abdessamad Derraz
75bfd04687
feat: full cross-linking web between all site pages
2026-03-25 14:17:10 +01:00
Abdessamad Derraz
3d2762bbc3
feat: cross-reference platform -> core -> systems -> upstream
2026-03-25 14:03:36 +01:00
Abdessamad Derraz
0f4fed2f47
feat: enrich site with full YAML data, cross-references, classification stats
2026-03-25 13:56:28 +01:00
Abdessamad Derraz
dbc26b11c1
refactor: move fetch_large_file to common, auto-download on db rebuild
2026-03-25 13:19:12 +01:00
Abdessamad Derraz
910428c6f1
fix: resolve large files from cache in database paths
2026-03-25 12:52:20 +01:00
Abdessamad Derraz
21465effff
feat: add readme and site generation to pipeline
2026-03-25 12:34:03 +01:00
Abdessamad Derraz
47e6174ed4
fix: pack naming, large file preservation, discrepancy reporting
2026-03-25 12:23:40 +01:00
Abdessamad Derraz
ebb55a445b
feat: re-profile 40 emulators, harden CI workflows
...
profile emulators pd777 through tic80, add frozen snapshots
(puae2021, snes9x2002/2005/2010, stella2014/2023).
CI: replace github-script with gh CLI, add test execution,
job-level permissions, propagate changed output, pin jsonschema.
2026-03-25 07:00:17 +01:00
Abdessamad Derraz
0543165ed2
feat: re-profile 22 emulators, refactor validation to common.py
...
batch re-profiled nekop2 through pokemini. mupen64plus renamed to
mupen64plus_next. new profiles: nes, mupen64plus_next.
validation functions (_build_validation_index, check_file_validation)
consolidated in common.py — single source of truth for verify.py
and generate_pack.py. pipeline 100% consistent on all 6 platforms.
2026-03-24 22:31:22 +01:00
Abdessamad Derraz
94000bdaef
fix: align verify and pack validation, pipeline 100% consistent
...
generate_pack.py now applies emulator-level validation (crc32, sha1,
adler32) matching verify.py behavior. existence mode: validation is
informational (file present = OK). md5 mode: validation downgrades
to UNTESTED. clone resolution moved to common.py resolve_local_file.
all 6 platforms pass consistency check.
2026-03-24 22:21:47 +01:00
Abdessamad Derraz
ae4846550f
fix: clone resolution in common.py, move clone map to root
...
moved _mame_clones.json out of bios/ (was indexed by generate_db.py
as BIOS file). clone resolution now in common.py resolve_local_file
so all tools (verify, pack, cross_reference) resolve clones
transparently. removed duplicate clone code from generate_pack.py.
added error handling on os.remove in dedup.py. consistency check
now passes for Batocera/EmuDeck/Lakka/RetroArch (4/6 platforms).
2026-03-24 21:57:49 +01:00
Abdessamad Derraz
85308edd73
fix: dedup edge cases — preserve non-ZIP different-name files
...
non-ZIP files with different names but same content (64DD_IPL_US.n64
vs IPL_USA.n64) are now preserved — each name may be needed by a
different emulator. only same-name duplicates and MAME ZIP clones
are removed. added empty directory cleanup post-dedup.
2026-03-24 21:39:25 +01:00
Abdessamad Derraz
fb1007496d
chore: deduplicate bios/ — remove 427 files, save 227 MB
...
true duplicates (same file in multiple dirs): removed copies, kept
canonical. MAME device clones (different names, identical content):
removed copies, created _mame_clones.json mapping for pack-time
assembly via deterministic ZIP rebuild. generate_pack.py resolves
clones transparently. 95 canonical ZIPs serve 392 clone names.
2026-03-24 21:35:50 +01:00
Abdessamad Derraz
8fcb86ba35
feat: deterministic MAME ZIP assembly in packs
...
all ZIP files (neogeo.zip, pgm.zip, etc.) are rebuilt with fixed
metadata before packing: sorted filenames, epoch timestamps, fixed
permissions, deflate level 9. same ROM atoms = same ZIP hash, always.
115 internal ZIPs verified identical across two independent builds.
enables version-agnostic ZIP assembly from ROM atoms indexed by CRC32.
2026-03-24 15:17:12 +01:00
Abdessamad Derraz
34e4c36f1c
feat: pack integrity verification, manifests, SHA256SUMS
...
post-generation verification: reopen each ZIP, hash every file,
check against database.json. inject manifest.json inside each pack
(self-documenting: path, sha1, md5, size, status per file).
generate SHA256SUMS.txt alongside packs for download verification.
validation index now uses sets for hashes and sizes to support
multiple valid ROM versions (MT-32 v1.04-v2.07, CM-32L variants).
69 tests pass, pipeline complete.
2026-03-24 14:56:02 +01:00
Abdessamad Derraz
11db9892bf
feat: add sha256 validation support to verify.py
2026-03-24 11:49:58 +01:00
Abdessamad Derraz
685713a7e6
feat: add sect233r1 ECDSA verification for 3DS OTP cert
...
pure python GF(2^233) field arithmetic, binary curve point operations,
and ECDSA-SHA256 on sect233r1. verifies OTP CTCert against nintendo
root CA public key. zero dependencies. sign+verify round-trip tested,
n*G=O verified, wrong key/message rejection confirmed.
2026-03-24 11:45:16 +01:00
Abdessamad Derraz
d4849681a7
feat: add 3DS signature/crypto verification to verify.py
...
pure python RSA-2048 PKCS1v15 SHA256 for SecureInfo_A,
LocalFriendCodeSeed_B, movable.sed. AES-128-CBC + SHA256 for otp.bin.
keys extracted from azahar default_keys.h, added RSA/ECC sections
to aes_keys.txt. sect233r1 ECC not reproducible (binary field curve).
2026-03-24 11:36:29 +01:00
