Commit Graph

511 Commits

Author SHA1 Message Date
Abdessamad Derraz
ae71a41b32 feat: verify_pack checks data dirs and rebuilt ZIPs
verify_pack now verifies files against data directory caches and
validates rebuilt MAME ZIPs by comparing inner ROM CRC32s against
source. Reduces false untracked count from 6242 to 0 for RetroArch.
2026-03-29 14:04:46 +02:00
Abdessamad Derraz
dee37c2530 fix: truth generation uses platform config not registry 2026-03-29 13:41:49 +02:00
Abdessamad Derraz
3bf6e5c961 feat: add truth and diff integration test 2026-03-29 13:27:32 +02:00
Abdessamad Derraz
a800ded477 chore: remove 4 remaining bios/data duplicates 2026-03-29 13:25:33 +02:00
Abdessamad Derraz
ac2502d767 feat: add --with-truth and --with-export to pipeline 2026-03-29 13:22:20 +02:00
Abdessamad Derraz
bb257f52ce feat: add export_native.py CLI 2026-03-29 13:21:08 +02:00
Abdessamad Derraz
e86d8d68af feat: add exporter plugin architecture 2026-03-29 13:19:38 +02:00
Abdessamad Derraz
2ce8db1754 feat: add native_id to scraper output 2026-03-29 13:15:57 +02:00
Abdessamad Derraz
5247512bea feat: add diff_truth.py CLI 2026-03-29 13:12:29 +02:00
Abdessamad Derraz
2aab7420d7 feat: add diff_platform_truth function and tests 2026-03-29 13:09:08 +02:00
Abdessamad Derraz
97eb4835be feat: add load_from field for non-system_dir files
Replaces mode: standalone hack with load_from: save_dir on Panda3DS
files. The load_from field documents which libretro directory callback
provides the base path (system_dir default, save_dir, content_dir).
Pack generator and cross-reference skip files not targeting system_dir.
2026-03-29 13:07:30 +02:00
Abdessamad Derraz
2dd2b724c1 feat: add generate_truth.py CLI 2026-03-29 13:05:51 +02:00
Abdessamad Derraz
525f51d5fb feat: add truth mode filtering and dedup tests 2026-03-29 13:01:37 +02:00
Abdessamad Derraz
d38f2b7180 fix: exclude save_dir paths from pack multi-dest 2026-03-29 12:02:30 +02:00
Abdessamad Derraz
abca70cb5b feat: add generate_platform_truth to common.py 2026-03-29 11:22:35 +02:00
Abdessamad Derraz
5ac48ef703 fix: recursive basename search in data dir fallback 2026-03-29 11:18:03 +02:00
Abdessamad Derraz
44145d41cd chore: remove 399 bios/ files now sourced from buildbot
Files verified by MD5 to be identical to their buildbot-fetched
copies in data/. resolve_local_file data directory fallback ensures
they remain resolvable for verify and pack generation.
2026-03-29 11:13:08 +02:00
Abdessamad Derraz
c513d6c0ad feat: resolve_local_file data directory fallback 2026-03-29 11:08:31 +02:00
Abdessamad Derraz
500e33d1ff feat: add --check-buildbot pipeline step 2026-03-29 11:04:04 +02:00
Abdessamad Derraz
b9be8b9783 feat: add buildbot system change detection 2026-03-29 11:03:15 +02:00
Abdessamad Derraz
44cb444a7b chore: regenerate install manifests after path fixes 2026-03-29 10:13:02 +02:00
Abdessamad Derraz
e13e8734b9 chore: standardize notes: to note: in file entries
240 file-level entries used notes: instead of the canonical note:
field. verify.py and cross_reference.py only read note:, so these
were silently ignored.
2026-03-29 10:02:20 +02:00
Abdessamad Derraz
b7528a71e7 fix: correct type and path fields across 58 emulator profiles
49 libretro cores had type: game/utility/test instead of type: libretro,
breaking the all_libretro filtering in resolve_platform_cores and
excluding them from platform packs (e.g. cannonball missing from
RetroArch). core_classification already carries the descriptive role.

9 profiles with subdirectory-loading cores (cannonball/, nxengine/,
Citra/sysdata/, mame2003/, mame2003-plus/, mame2010/) now have path:
fields so cross-reference places files at the correct destination.

resolve_local_file now tries basename when name contains a path
separator (e.g. res/tilemap.bin -> tilemap.bin), fixing resolution
of files with subdirectory names.
2026-03-29 09:53:46 +02:00
Abdessamad Derraz
36f6ad1379 fix: correct stale scummvm.zip and cpc464.rom hashes
RetroDECK had stale buildbot hash a17e0e01 for scummvm.zip
(from old 9.5MB build, current is 79MB) copied to cpc464.rom.
RomM had same stale hash. Updated to current verified values.
All 8 platforms now 100% OK, 0 untested.
2026-03-29 09:52:32 +02:00
Abdessamad Derraz
718ea2192d feat: add FDS original BIOS and GameCube DSP hardware dumps
disksys.rom non-Rev1 variant (SHA256 fdc1a76e, ares-compatible)
from Myrient No-Intro. GameCube dsp_rom.bin + dsp_coef.bin real
hardware dumps (adler32 match Dolphin) from Redump collection.
All placed as .variants/ — primaries unchanged.
2026-03-29 09:35:48 +02:00
Abdessamad Derraz
ce6c218942 fix: restore original ZIPs to match platform MD5 hashes
Agents enriched neogeo/pgm/stvbios/neocdz/nmk004/channelf
ZIPs with extra ROMs, breaking platform MD5 verification.
Restore originals from git history. RetroBat: 0 untested.
RetroDECK 2 untested remain (upstream scraper bug).
2026-03-29 09:28:52 +02:00
Abdessamad Derraz
536300984d feat: add howard.o Lynx development bootloader
513-byte BS93-format bootloader reconstructed from Handy
emulator source code analysis. Header verified: BRA +8,
load_addr 0x0212, magic BS93, SEI at code entry. CRam
reset simulation passed.
2026-03-29 09:18:30 +02:00
Abdessamad Derraz
7492777b47 fix: skip storage: release files in cross-reference
Files with storage: release are in GitHub release assets,
not in bios/. Eliminates donpachi/sfz3mix/twotiger false
positives. 149/149 tests pass. Cross-ref: 10 -> 7.
2026-03-29 07:28:12 +02:00
Abdessamad Derraz
a369defc15 fix: skip path: null entries in cross-reference
Files with explicit path: null are UI-imported (Dolphin NAND,
Hatari cartridge) and not resolvable by pack placement. Skip
them in find_undeclared_files and cross_reference. Also add
desc.dat (SDLPAL fan-made descriptions) to data/. 149/149 OK.
2026-03-29 07:26:40 +02:00
Abdessamad Derraz
c241a44c33 fix: fbneo data dir paths (hiscore, samples) 2026-03-29 06:50:36 +02:00
Abdessamad Derraz
1381a17f9c feat: complete fbneo samples (49 small + 3 large) 2026-03-29 01:24:25 +01:00
Abdessamad Derraz
bf72543d8f feat: add SquirrelJME test JARs built from source
squirreljme-0.3.0-test.jar and squirreljme-test.jar compiled
from SquirrelJME trunk using Gradle + JDK 17. No prebuilt
artifacts exist (v0.3.0 never released). Built with
romTestSpringCoatRelease target.
2026-03-29 01:20:22 +01:00
Abdessamad Derraz
ee033e152a feat: add Toaplan fireshrk and vimana WAV samples
ThunderMAME samples from original MediaFire links (still
alive). fireshrk.zip (180MB, = samesame.zip parent name)
and vimana.zip (279MB). Upload to large-files release.
2026-03-29 01:14:19 +01:00
Abdessamad Derraz
6830ba3db3 feat: add QEMU MIPS and LEON3 U-Boot firmware
mips_bios.bin and mipsel_bios.bin cross-compiled from U-Boot
v2024.10 (malta/maltael_defconfig). u-boot.bin for LEON3 SPARC
cross-compiled from U-Boot v2016.11 (grsim_defconfig) using
Gaisler sparc-gaisler-linux5.10 toolchain. All built in Docker.
2026-03-29 01:13:01 +01:00
Abdessamad Derraz
d0b3d59fa7 feat: add Canon A1100 barebox ROM for QEMU DIGIC4
4MB barebox bootloader from frantony/barebox GitHub repo.
Only publicly available ROM for QEMU canon-a1100 machine.
Matches FLASH_K8P3215UQB_SIZE in hw/arm/digic_boards.c.
2026-03-29 01:10:21 +01:00
Abdessamad Derraz
e92f6a5409 feat: add RISC-V Linux kernel for RVVM
Pre-built rv64 Linux Image (22MB) from Bootlin buildroot
stable 2025.08. S-mode kernel payload for fw_jump.bin mode.
2026-03-29 01:04:26 +01:00
Abdessamad Derraz
48598d61b0 feat: add Wii keys.bin from archive.org NAND dump
1024-byte OTP/SEEPROM dump for Dolphin/PrimeHack/Ishiiruka
NAND import. From archive.org nand_20240722 collection.
2026-03-29 01:01:47 +01:00
Abdessamad Derraz
a37bf11524 fix: add path fields for correct pack subdirectory placement
Cores that load files from system_dir subdirectories (same_cdi/bios/,
neocd/, cannonball/, Citra/sysdata/, mame2003/, etc.) need path: on
each file entry so cross-reference and pack generation place files at
the correct destination. Also fixes neocd.yml using non-standard dest:
field instead of path:.

Closes #41
2026-03-29 00:59:57 +01:00
Abdessamad Derraz
87d3c4c645 chore: regenerate database and revert agent profile violations
16 emulator profiles reverted to ground truth (agents modified
cannonball, citra, fbalpha2012, mame variants, neocd, nxengine,
same_cdi without source code verification). OpenTyrian residuals
moved from bios/ to data/. Database regenerated: 7616 files.
2026-03-29 00:54:44 +01:00
Abdessamad Derraz
b01a138f1c feat: add fbneo samples data directory 2026-03-29 00:53:27 +01:00
Abdessamad Derraz
459a745527 feat: add RVVM firmware and Hatari diagnostic cartridge
fw_payload.bin (OpenSBI + U-Boot 2023.4) from RVVM v0.5
release. ST Diagnostic v4.4 cartridge ROM for Hatari from
SidecarTridge ROM library.
2026-03-29 00:52:37 +01:00
Abdessamad Derraz
32a23a14ae chore: move OpenTyrian residuals from bios/ to data/
Agent placed Tyrian 2000 files in bios/Game Engines/ instead
of data/opentyrian/. Synced and removed duplicate. OpenTyrian
data lives in data/ via _data_dirs.yml, not in bios/.
2026-03-29 00:38:57 +01:00
Abdessamad Derraz
40b7badc1d feat: add Sharp X1 kanji font, PC-88 fonts, PC-98 PCI ROM
FNT1616.X1 (306KB) from NeoKobe X1turboZ BIOS via HTTP range
request on 934MB archive. font2/font3.rom split from M88 6KB
font.rom. pci.rom (32KB) from MAME 0.278 pc9821cx3.zip. All
previously declared "impossible" by multiple agents.
2026-03-29 00:33:22 +01:00
Abdessamad Derraz
c835336999 fix: correct OpenTyrian shapes profiling error
Remove 30 phantom shapes[2-v].dat entries. shapeFile[34] in
lvlmast.c maps to newsh?.shp (enemy sprites), NOT shapes?.dat
(level tilesets). Only 5 shapes exist: ), W, X, Y, Z — the
characters actually referenced by level data. Verified against
trapexit/libretro-opentyrian source. Cross-ref: 54 -> 24.
2026-03-29 00:30:06 +01:00
Abdessamad Derraz
a306f93a1b feat: add Amiga CD32 FMV cartridge ROM
cd32fmv.rom (v40.030, SHA1 03ca81c7) extracted from Amiga
Forever 11.0.22 ROM pack on archive.org. Verified against
WinUAE rommgr.cpp ROMTYPE_CD32CART entry. Previously
declared "impossible" by 4 separate agents.
2026-03-29 00:23:19 +01:00
Abdessamad Derraz
5413c6a5ac feat: add Cemu Wii U sound RPLs and Lexaloffle Voxatron
snd_user.rpl + snduser2.rpl from Cafe SDK 2.12.13 archive.org.
Voxatron v0.3.5b binaries (vox, vox_dyn, vox.dat) from
archive.org pico-8-tron-and-voxatron. libHoloPlayCore.so from
Looking Glass SDK GitHub. All 6 "impossible" files found.
2026-03-29 00:21:55 +01:00
Abdessamad Derraz
329deb82fd feat: add fbneo cheats data directory 2026-03-29 00:21:53 +01:00
Abdessamad Derraz
99c8fd62f0 feat: derive Amiga rom.key via XOR cryptanalysis
Cloanto decryption key (2069 bytes) derived by XOR-ing
encrypted and plain Kickstart ROM pairs from archive.org
commodore-amiga-firmware collection. Unlocks 5 emulators
(Amiberry, PUAE, PUAE2021, FS-UAE, uae4arm).
2026-03-29 00:20:00 +01:00
Abdessamad Derraz
abc997525b feat: wire fbneo-hiscore data dir to arcade system 2026-03-29 00:18:06 +01:00
Abdessamad Derraz
e91caa6444 feat: add Saturn AR4MP firmware and QEMU Virtex DTB
satar4mp.bin (256KB) extracted from existing ar_bios.zip
MAME ROM. virtex-ml507.dtb compiled from Linux 4.19 DTS
with dtc. Both resolve long-standing optional missing files.
2026-03-29 00:07:40 +01:00