write_if_changed in common.py compares content after stripping
timestamps (generated_at, Auto-generated on, Generated on).
applied to generate_db, generate_readme, generate_site.
eliminates timestamp-only diffs in database.json, README.md,
mkdocs.yml, and 423 docs pages.
bios_mode: agnostic (profile) and agnostic: true (file) for
emulators that accept any valid BIOS without specific filename.
find_undeclared_files skips agnostic entries, pack extras scan
includes all matching DB files by path prefix + size criteria,
resolve_local_file has agnostic fallback with rename README.
applied to pcsx2, lrps2 (bios_mode), melonds dsi_nand (file).
aliases must be same-SHA1 alternative names, not distinct files.
pcsx2: 164 different BIOS dumps are separate DB entries, not aliases.
melonds: 6 regional NAND dumps are separate DB entries, not aliases.
also cleans pcsx2 non-standard fields, fixes display_name.
expand_platform_declared_names resolves platform file MD5s
through the database to recover canonical names and aliases,
eliminating false positive undeclared files when a platform
renames a file (e.g. Batocera ROM1 vs gsplus ROM).
Bump database.json generated_at timestamp and add a new emulator metadata file emulators/ti99sim.yml. The new YAML registers the TI-99/Sim standalone emulator (ti99sim) with system info, core version, notes on ROM handling, and multiple required/optional ROM entries (with SHA1s, sizes and validation notes) for TI-99/4A support.
Add Atari 800 OS Rev B NTSC (CRC32 0e86d61d, canonical sysrom.c
match) and National FS-5500 disk controller ROM for openMSX.
Remove ROM_400/800_CUSTOM from retrodeck.yml (config slot key with
forward slash in name, not a real file).
Add many MAME/MESS BIOS entries (TRS-80 family, Bandai RX-78, Sega AI) and update docs/navigation counts (README, mkdocs). Remove empty supplemental file references from database.json and update generated timestamps and totals. Harden and refactor tooling: add MAX_RESPONSE_SIZE limited reader in base_scraper, make target scrapers an abstract base, narrow exception handling in the Batocera targets parser, and switch generate_pack.py and verify.py to use build_target_cores_cache (simplifies target config loading and error handling). verify.py also loads supplemental cross-reference names and accepts them through verify_platform. Update tests to import from updated modules (validation/truth). Misc: small bugfix for case-insensitive path conflict check.
Add Casio PV-2000 BIOS entry (pv2000.zip) to MAME and MESS profiles and update system lists/counts. Add Funtech Super A'Can BIOS entries (supracan.zip and umc6650.zip) with ROM contents to mamemess. Simplify and condense Vita3K emulator profile (rename fields, update profiled_date, add PSVUPDAT.PUP and optional PSP2UPDAT.PUP file entries, and clarify install/partition behavior). Bump database generated_at timestamp and add a system alias mapping "psvita" -> "sony-playstation-vita" in scripts/common.py.
Regenerate database.json and update README counts/timestamps; add and normalize numerous BIOS entries and hashes. Key changes: update generated_at timestamp and system count (355→357) in README; add OpenBIOS / HLE fallback and additional aliases to beetle_psx, include beetle_psx core name and profiled_date update; add laseractive to ares systems; adjust atari800 systems and source_ref line numbers; mark dinothawr as a system and expand its note; update gsplus upstream/profile date, add apple-iie system and tweak source_refs; add pcsx2 core to lrps2; refresh mame profiled_date and add multiple systems and BIOS root sets. Miscellaneous script changes and other JSON normalization to reflect newly discovered/merged BIOS files.
Add support for Coleco Adam, Entex Adventure Vision and APF M-1000 BIOS/ROM sets in MAME and MESS metadata (multiple Adam device MCU ROMs and optional FDC/SPI variants, Advision and APF BIOS entries). Update generated metadata across the repo: README coverage numbers and per-platform coverage rows, database generated timestamp and totals (total_files 7245), and various install manifests (notably batocera.json) with new timestamps, adjusted file counts/sizes, SHA1s, repo_path fixes and an added adam_fdc_320kb.zip entry. Also update notes to reflect the new system ROM sets in the emulators entries.
resolve_local_file step 2 (pure MD5 lookup) now verifies that the
found file's name matches the requested name or is a .variants/
derivative. Prevents serving wrong files when an unrelated file
shares the same MD5 in the index (e.g. spi.zip returned for
a7ports.zip because RetroDECK expected an MD5 we don't have).
verify_pack now verifies files against data directory caches and
validates rebuilt MAME ZIPs by comparing inner ROM CRC32s against
source. Reduces false untracked count from 6242 to 0 for RetroArch.
Replaces mode: standalone hack with load_from: save_dir on Panda3DS
files. The load_from field documents which libretro directory callback
provides the base path (system_dir default, save_dir, content_dir).
Pack generator and cross-reference skip files not targeting system_dir.
Files verified by MD5 to be identical to their buildbot-fetched
copies in data/. resolve_local_file data directory fallback ensures
they remain resolvable for verify and pack generation.
240 file-level entries used notes: instead of the canonical note:
field. verify.py and cross_reference.py only read note:, so these
were silently ignored.
49 libretro cores had type: game/utility/test instead of type: libretro,
breaking the all_libretro filtering in resolve_platform_cores and
excluding them from platform packs (e.g. cannonball missing from
RetroArch). core_classification already carries the descriptive role.
9 profiles with subdirectory-loading cores (cannonball/, nxengine/,
Citra/sysdata/, mame2003/, mame2003-plus/, mame2010/) now have path:
fields so cross-reference places files at the correct destination.
resolve_local_file now tries basename when name contains a path
separator (e.g. res/tilemap.bin -> tilemap.bin), fixing resolution
of files with subdirectory names.
RetroDECK had stale buildbot hash a17e0e01 for scummvm.zip
(from old 9.5MB build, current is 79MB) copied to cpc464.rom.
RomM had same stale hash. Updated to current verified values.
All 8 platforms now 100% OK, 0 untested.
disksys.rom non-Rev1 variant (SHA256 fdc1a76e, ares-compatible)
from Myrient No-Intro. GameCube dsp_rom.bin + dsp_coef.bin real
hardware dumps (adler32 match Dolphin) from Redump collection.
All placed as .variants/ — primaries unchanged.
Files with storage: release are in GitHub release assets,
not in bios/. Eliminates donpachi/sfz3mix/twotiger false
positives. 149/149 tests pass. Cross-ref: 10 -> 7.
Files with explicit path: null are UI-imported (Dolphin NAND,
Hatari cartridge) and not resolvable by pack placement. Skip
them in find_undeclared_files and cross_reference. Also add
desc.dat (SDLPAL fan-made descriptions) to data/. 149/149 OK.
squirreljme-0.3.0-test.jar and squirreljme-test.jar compiled
from SquirrelJME trunk using Gradle + JDK 17. No prebuilt
artifacts exist (v0.3.0 never released). Built with
romTestSpringCoatRelease target.
mips_bios.bin and mipsel_bios.bin cross-compiled from U-Boot
v2024.10 (malta/maltael_defconfig). u-boot.bin for LEON3 SPARC
cross-compiled from U-Boot v2016.11 (grsim_defconfig) using
Gaisler sparc-gaisler-linux5.10 toolchain. All built in Docker.
4MB barebox bootloader from frantony/barebox GitHub repo.
Only publicly available ROM for QEMU canon-a1100 machine.
Matches FLASH_K8P3215UQB_SIZE in hw/arm/digic_boards.c.
Cores that load files from system_dir subdirectories (same_cdi/bios/,
neocd/, cannonball/, Citra/sysdata/, mame2003/, etc.) need path: on
each file entry so cross-reference and pack generation place files at
the correct destination. Also fixes neocd.yml using non-standard dest:
field instead of path:.
Closes#41
Agent placed Tyrian 2000 files in bios/Game Engines/ instead
of data/opentyrian/. Synced and removed duplicate. OpenTyrian
data lives in data/ via _data_dirs.yml, not in bios/.
FNT1616.X1 (306KB) from NeoKobe X1turboZ BIOS via HTTP range
request on 934MB archive. font2/font3.rom split from M88 6KB
font.rom. pci.rom (32KB) from MAME 0.278 pc9821cx3.zip. All
previously declared "impossible" by multiple agents.
Remove 30 phantom shapes[2-v].dat entries. shapeFile[34] in
lvlmast.c maps to newsh?.shp (enemy sprites), NOT shapes?.dat
(level tilesets). Only 5 shapes exist: ), W, X, Y, Z — the
characters actually referenced by level data. Verified against
trapexit/libretro-opentyrian source. Cross-ref: 54 -> 24.
cd32fmv.rom (v40.030, SHA1 03ca81c7) extracted from Amiga
Forever 11.0.22 ROM pack on archive.org. Verified against
WinUAE rommgr.cpp ROMTYPE_CD32CART entry. Previously
declared "impossible" by 4 separate agents.
satar4mp.bin (256KB) extracted from existing ar_bios.zip
MAME ROM. virtex-ml507.dtb compiled from Linux 4.19 DTS
with dtc. Both resolve long-standing optional missing files.
CLK os.rom (Acorn Electron MOS), exos10.bin and basic10.bin
(Enterprise EXOS/BASIC 1.0) from TOSEC/Myrient — zero
required missing across all platforms. Also add Enterprise
ROMs, ZX Spectrum 128/+2/+3, MSX CLK variants, FBA2012
samples from progettosnaps.
Abdessamad Derraz