4 Commits

Author SHA1 Message Date
Abdessamad Derraz
e5859eb761 refactor: dry pack integrity into cli and update docs
Move verification logic to generate_pack.py --verify-packs (single
source of truth). test_pack_integrity.py is now a thin wrapper that
calls the CLI. Pipeline step 6/8 uses the same CLI entry point.

Renumber all pipeline steps 1-8 (was skipping from 5 to 8/9).

Update generate_site.py with pack integrity test documentation.
2026-04-01 12:31:10 +02:00
Abdessamad Derraz
754e829b35 feat: add pack integrity test and integrate into pipeline
Extract each platform ZIP to tmp/ (real filesystem, not /tmp tmpfs)
and verify every declared file exists at the correct path with the
correct hash per the platform's native verification mode.

Handles ZIP inner content verification (checkInsideZip, md5_composite,
inner ROM MD5) and path collision deduplication.

Integrated as pipeline step 6/8. Renumber all pipeline steps to be
sequential (was skipping from 5 to 8).
2026-04-01 12:22:50 +02:00
Abdessamad Derraz
7beb651049 fix: correct core extras placement for retrodeck and romm packs
RetroDECK: core extras with subdirectory paths (e.g. vice/C64/,
fbneo/, dc/) were placed outside bios/ because the prefix was only
inferred for bare filenames. Add _detect_extras_prefix() to infer
the dominant BIOS prefix from YAML destinations.

RomM: core extras landed flat at bios/{file} instead of the required
bios/{platform_slug}/{file}. Add _detect_slug_structure() to detect
per-system slug layouts and _map_emulator_to_slug() to route each
extra to the correct slug subfolder.

Also skip manifest writes when only the generated timestamp changed,
preventing unnecessary diffs in install/*.json.
2026-04-01 11:08:01 +02:00
Abdessamad Derraz
5eeaf87a3a fix: resolve all untested and missing bios across platforms
Batocera: fix sc3000.rom md5 (no dump matches upstream hash),
remove erroneous bk0010.zip mame entries (upstream confirmed
mame needs no bios for bk), add PSP2UPDAT.PUP correct version.
Recalbox: add MSX2R2.ROM from blueMSX v2.82.
RetroDECK: fix stale peribox_ev/gen.zip md5 hashes.

Regenerate database, manifests, readme.
2026-04-01 01:42:39 +02:00
20 changed files with 675 additions and 2982 deletions

2
.gitignore vendored
View File

@@ -29,7 +29,7 @@ data/
# Large files stored as GitHub Release assets (additional) # Large files stored as GitHub Release assets (additional)
bios/Arcade/MAME/artwork/snspell.zip bios/Arcade/MAME/artwork/snspell.zip
bios/Arcade/MAME/MAME 0.174 Arcade XML.dat bios/Arcade/MAME/MAME 0.174 Arcade XML.dat
bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP.3ae832c9
bios/Nintendo/DS/DSi_Nand_JPN.bin bios/Nintendo/DS/DSi_Nand_JPN.bin
bios/Nintendo/DS/DSi_Nand_EUR.bin bios/Nintendo/DS/DSi_Nand_EUR.bin
bios/Nintendo/DS/DSi_Nand_USA.bin bios/Nintendo/DS/DSi_Nand_USA.bin

View File

@@ -2,7 +2,7 @@
Complete BIOS and firmware packs for Batocera, BizHawk, EmuDeck, Lakka, Recalbox, RetroArch, RetroBat, RetroDECK, RetroPie, and RomM. Complete BIOS and firmware packs for Batocera, BizHawk, EmuDeck, Lakka, Recalbox, RetroArch, RetroBat, RetroDECK, RetroPie, and RomM.
**7,293** verified files across **396** systems, ready to extract into your emulator's BIOS directory. **7,295** verified files across **396** systems, ready to extract into your emulator's BIOS directory.
## Quick Install ## Quick Install
@@ -27,7 +27,7 @@ Pick your platform, download the ZIP, extract to the BIOS path.
| Platform | BIOS files | Extract to | Download | | Platform | BIOS files | Extract to | Download |
|----------|-----------|-----------|----------| |----------|-----------|-----------|----------|
| Batocera | 362 | `/userdata/bios/` | [Download](../../releases/latest) | | Batocera | 361 | `/userdata/bios/` | [Download](../../releases/latest) |
| BizHawk | 118 | `Firmware/` | [Download](../../releases/latest) | | BizHawk | 118 | `Firmware/` | [Download](../../releases/latest) |
| EmuDeck | 161 | `Emulation/bios/` | [Download](../../releases/latest) | | EmuDeck | 161 | `Emulation/bios/` | [Download](../../releases/latest) |
| Lakka | 448 | `system/` | [Download](../../releases/latest) | | Lakka | 448 | `system/` | [Download](../../releases/latest) |
@@ -46,8 +46,8 @@ Each file is checked against the emulator's source code to match what the code a
- **10 platforms** supported with platform-specific verification - **10 platforms** supported with platform-specific verification
- **329 emulators** profiled from source (RetroArch cores + standalone) - **329 emulators** profiled from source (RetroArch cores + standalone)
- **396 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...) - **396 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
- **7,293 files** verified with MD5, SHA1, CRC32 checksums - **7,295 files** verified with MD5, SHA1, CRC32 checksums
- **8710 MB** total collection size - **8765 MB** total collection size
## Supported systems ## Supported systems
@@ -59,14 +59,14 @@ Full list with per-file details: **[https://abdess.github.io/retrobios/](https:/
| Platform | Coverage | Verified | Untested | Missing | | Platform | Coverage | Verified | Untested | Missing |
|----------|----------|----------|----------|---------| |----------|----------|----------|----------|---------|
| Batocera | 361/362 (99.7%) | 359 | 2 | 1 | | Batocera | 361/361 (100.0%) | 361 | 0 | 0 |
| BizHawk | 118/118 (100.0%) | 118 | 0 | 0 | | BizHawk | 118/118 (100.0%) | 118 | 0 | 0 |
| EmuDeck | 161/161 (100.0%) | 161 | 0 | 0 | | EmuDeck | 161/161 (100.0%) | 161 | 0 | 0 |
| Lakka | 448/448 (100.0%) | 448 | 0 | 0 | | Lakka | 448/448 (100.0%) | 448 | 0 | 0 |
| Recalbox | 346/346 (100.0%) | 345 | 1 | 0 | | Recalbox | 346/346 (100.0%) | 346 | 0 | 0 |
| RetroArch | 448/448 (100.0%) | 448 | 0 | 0 | | RetroArch | 448/448 (100.0%) | 448 | 0 | 0 |
| RetroBat | 339/339 (100.0%) | 339 | 0 | 0 | | RetroBat | 339/339 (100.0%) | 339 | 0 | 0 |
| RetroDECK | 2006/2006 (100.0%) | 2004 | 2 | 0 | | RetroDECK | 2006/2006 (100.0%) | 2006 | 0 | 0 |
| RetroPie | 448/448 (100.0%) | 448 | 0 | 0 | | RetroPie | 448/448 (100.0%) | 448 | 0 | 0 |
| RomM | 374/374 (100.0%) | 374 | 0 | 0 | | RomM | 374/374 (100.0%) | 374 | 0 | 0 |
@@ -130,4 +130,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
This repository provides BIOS files for personal backup and archival purposes. This repository provides BIOS files for personal backup and archival purposes.
*Auto-generated on 2026-03-31T12:15:43Z* *Auto-generated on 2026-03-31T20:38:37Z*

Binary file not shown.

View File

@@ -1,7 +1,7 @@
{ {
"generated_at": "2026-03-31T08:57:59Z", "generated_at": "2026-03-31T20:38:43Z",
"total_files": 7293, "total_files": 7295,
"total_size": 9133482744, "total_size": 9190294264,
"files": { "files": {
"520d3d1b5897800af47f92efd2444a26b7a7dead": { "520d3d1b5897800af47f92efd2444a26b7a7dead": {
"path": "bios/3DO Company/3DO/3do_arcade_saot.bin", "path": "bios/3DO Company/3DO/3do_arcade_saot.bin",
@@ -35383,6 +35383,16 @@
"crc32": "b8ba44d3", "crc32": "b8ba44d3",
"adler32": "dbc99bd5" "adler32": "dbc99bd5"
}, },
"ebb7eb540a390509edfd36c84288ba85e63f2d1f": {
"path": "bios/Microsoft/MSX/MSX2R2.ROM",
"name": "MSX2R2.ROM",
"size": 32768,
"sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"md5": "96ac231b718e88ce64d5a9b4a5e9ae12",
"sha256": "e6a99ffc28b07a1ee8fd48cf09cdd9adbb3f54ab1bad74e2252c37ace30794ec",
"crc32": "d0c20f54",
"adler32": "bef7a448"
},
"c36c9e0f96738a340381e23b4f97245388801a46": { "c36c9e0f96738a340381e23b4f97245388801a46": {
"path": "bios/Microsoft/MSX/MSXDOS2.ROM", "path": "bios/Microsoft/MSX/MSXDOS2.ROM",
"name": "MSXDOS2.ROM", "name": "MSXDOS2.ROM",
@@ -71924,7 +71934,7 @@
"adler32": "8e2b7342" "adler32": "8e2b7342"
}, },
"3ae832c9800fcaa007eccfc48f24242967c111f8": { "3ae832c9800fcaa007eccfc48f24242967c111f8": {
"path": "bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP", "path": "bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP.3ae832c9",
"name": "PSP2UPDAT.PUP", "name": "PSP2UPDAT.PUP",
"size": 56768512, "size": 56768512,
"sha1": "3ae832c9800fcaa007eccfc48f24242967c111f8", "sha1": "3ae832c9800fcaa007eccfc48f24242967c111f8",
@@ -71933,6 +71943,16 @@
"crc32": "c0c3a1fe", "crc32": "c0c3a1fe",
"adler32": "ea4fd486" "adler32": "ea4fd486"
}, },
"ed3a4cb264fff283209f10ae58c96c6090fed187": {
"path": "bios/Sony/PlayStation Vita/PSP2UPDAT.PUP",
"name": "PSP2UPDAT.PUP",
"size": 56778752,
"sha1": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"md5": "59dcf059d3328fb67be7e51f8aa33418",
"sha256": "c3c03fc7363dd573d90e5157629bf11551f434b283cc898d9ffc71dd716b791c",
"crc32": "082ecf86",
"adler32": "620a2ff1"
},
"cc72dfcc964577cc29112ef368c28f55277c237c": { "cc72dfcc964577cc29112ef368c28f55277c237c": {
"path": "bios/Sony/PlayStation Vita/PSVUPDAT.PUP", "path": "bios/Sony/PlayStation Vita/PSVUPDAT.PUP",
"name": "PSVUPDAT.PUP", "name": "PSVUPDAT.PUP",
@@ -76474,6 +76494,7 @@
"2183c2aff17cf4297bdb496de78c2e8a": "5c1f9c7fb655e43d38e5dd1fcc6b942b2ff68b02", "2183c2aff17cf4297bdb496de78c2e8a": "5c1f9c7fb655e43d38e5dd1fcc6b942b2ff68b02",
"6d8c0ca64e726c82a4b726e9b01cdf1e": "e2fbd56e42da637609d23ae9df9efd1b4241b18a", "6d8c0ca64e726c82a4b726e9b01cdf1e": "e2fbd56e42da637609d23ae9df9efd1b4241b18a",
"7c8243c71d8f143b2531f01afa6a05dc": "fe0254cbfc11405b79e7c86c7769bd6322b04995", "7c8243c71d8f143b2531f01afa6a05dc": "fe0254cbfc11405b79e7c86c7769bd6322b04995",
"96ac231b718e88ce64d5a9b4a5e9ae12": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"6418d091cd6907bbcf940324339e43bb": "c36c9e0f96738a340381e23b4f97245388801a46", "6418d091cd6907bbcf940324339e43bb": "c36c9e0f96738a340381e23b4f97245388801a46",
"704bdd980fa56c6be5c680358458eeeb": "04990aa1c3a3fc7294ec884b81deaa89832df614", "704bdd980fa56c6be5c680358458eeeb": "04990aa1c3a3fc7294ec884b81deaa89832df614",
"f005e55c680ba6e7b19f6d4dc8f73ce5": "df48902f5f12af8867ae1a87f255145f0e5e0774", "f005e55c680ba6e7b19f6d4dc8f73ce5": "df48902f5f12af8867ae1a87f255145f0e5e0774",
@@ -80129,6 +80150,7 @@
"9647c96e5463a3185bf1d04662f1521c": "f2a9860baf277f56005c0f9e33202fcbd07b7e7e", "9647c96e5463a3185bf1d04662f1521c": "f2a9860baf277f56005c0f9e33202fcbd07b7e7e",
"95f60f6c513ce31851d930407799ad29": "41d8c5c89f72206b873633ff31bcf4f82608e5a4", "95f60f6c513ce31851d930407799ad29": "41d8c5c89f72206b873633ff31bcf4f82608e5a4",
"8b5f60b56c3da8365b973dba570c53a5": "3ae832c9800fcaa007eccfc48f24242967c111f8", "8b5f60b56c3da8365b973dba570c53a5": "3ae832c9800fcaa007eccfc48f24242967c111f8",
"59dcf059d3328fb67be7e51f8aa33418": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"f2c7b12fe85496ec88a0391b514d6e3b": "cc72dfcc964577cc29112ef368c28f55277c237c", "f2c7b12fe85496ec88a0391b514d6e3b": "cc72dfcc964577cc29112ef368c28f55277c237c",
"e59fdf56762c480ba4dfe1b3ec5fb86d": "b184f1c1febf66c8168fcae0b8aa37a5754f79db", "e59fdf56762c480ba4dfe1b3ec5fb86d": "b184f1c1febf66c8168fcae0b8aa37a5754f79db",
"1d33d70f35b33873fc75941d95ad1ffa": "567c5b5054552a2771eafa7966844a146f0dde96", "1d33d70f35b33873fc75941d95ad1ffa": "567c5b5054552a2771eafa7966844a146f0dde96",
@@ -90331,6 +90353,9 @@
"MSX2PEXT.ROM": [ "MSX2PEXT.ROM": [
"fe0254cbfc11405b79e7c86c7769bd6322b04995" "fe0254cbfc11405b79e7c86c7769bd6322b04995"
], ],
"MSX2R2.ROM": [
"ebb7eb540a390509edfd36c84288ba85e63f2d1f"
],
"MSXDOS2.ROM": [ "MSXDOS2.ROM": [
"c36c9e0f96738a340381e23b4f97245388801a46" "c36c9e0f96738a340381e23b4f97245388801a46"
], ],
@@ -100261,7 +100286,8 @@
"41d8c5c89f72206b873633ff31bcf4f82608e5a4" "41d8c5c89f72206b873633ff31bcf4f82608e5a4"
], ],
"PSP2UPDAT.PUP": [ "PSP2UPDAT.PUP": [
"3ae832c9800fcaa007eccfc48f24242967c111f8" "3ae832c9800fcaa007eccfc48f24242967c111f8",
"ed3a4cb264fff283209f10ae58c96c6090fed187"
], ],
"PSVUPDAT.PUP": [ "PSVUPDAT.PUP": [
"cc72dfcc964577cc29112ef368c28f55277c237c" "cc72dfcc964577cc29112ef368c28f55277c237c"
@@ -103367,9 +103393,6 @@
"MSXR2.ROM": [ "MSXR2.ROM": [
"04990aa1c3a3fc7294ec884b81deaa89832df614" "04990aa1c3a3fc7294ec884b81deaa89832df614"
], ],
"MSX2R2.ROM": [
"04990aa1c3a3fc7294ec884b81deaa89832df614"
],
"NATIONALDISK.rom": [ "NATIONALDISK.rom": [
"78cd7f847e77fd8cd51a647efb2725ba93f4c471" "78cd7f847e77fd8cd51a647efb2725ba93f4c471"
], ],
@@ -107718,6 +107741,7 @@
"66237ecf": "5c1f9c7fb655e43d38e5dd1fcc6b942b2ff68b02", "66237ecf": "5c1f9c7fb655e43d38e5dd1fcc6b942b2ff68b02",
"00870134": "e2fbd56e42da637609d23ae9df9efd1b4241b18a", "00870134": "e2fbd56e42da637609d23ae9df9efd1b4241b18a",
"b8ba44d3": "fe0254cbfc11405b79e7c86c7769bd6322b04995", "b8ba44d3": "fe0254cbfc11405b79e7c86c7769bd6322b04995",
"d0c20f54": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"1c430991": "c36c9e0f96738a340381e23b4f97245388801a46", "1c430991": "c36c9e0f96738a340381e23b4f97245388801a46",
"67872f40": "04990aa1c3a3fc7294ec884b81deaa89832df614", "67872f40": "04990aa1c3a3fc7294ec884b81deaa89832df614",
"071135e0": "df48902f5f12af8867ae1a87f255145f0e5e0774", "071135e0": "df48902f5f12af8867ae1a87f255145f0e5e0774",
@@ -111373,6 +111397,7 @@
"50da333e": "f2a9860baf277f56005c0f9e33202fcbd07b7e7e", "50da333e": "f2a9860baf277f56005c0f9e33202fcbd07b7e7e",
"15b438bd": "41d8c5c89f72206b873633ff31bcf4f82608e5a4", "15b438bd": "41d8c5c89f72206b873633ff31bcf4f82608e5a4",
"c0c3a1fe": "3ae832c9800fcaa007eccfc48f24242967c111f8", "c0c3a1fe": "3ae832c9800fcaa007eccfc48f24242967c111f8",
"082ecf86": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"39075d41": "cc72dfcc964577cc29112ef368c28f55277c237c", "39075d41": "cc72dfcc964577cc29112ef368c28f55277c237c",
"44295096": "b184f1c1febf66c8168fcae0b8aa37a5754f79db", "44295096": "b184f1c1febf66c8168fcae0b8aa37a5754f79db",
"31c53421": "567c5b5054552a2771eafa7966844a146f0dde96", "31c53421": "567c5b5054552a2771eafa7966844a146f0dde96",
@@ -125077,7 +125102,7 @@
"shaders/vignette.fsh": [ "shaders/vignette.fsh": [
"24165b402a7830f6b9d6c7cfc6131bcf2c1140bb" "24165b402a7830f6b9d6c7cfc6131bcf2c1140bb"
], ],
".variants/PSP2UPDAT.PUP": [ ".variants/PSP2UPDAT.PUP.3ae832c9": [
"3ae832c9800fcaa007eccfc48f24242967c111f8" "3ae832c9800fcaa007eccfc48f24242967c111f8"
], ],
".variants/coco.zip": [ ".variants/coco.zip": [

View File

@@ -3,7 +3,7 @@
"platform": "batocera", "platform": "batocera",
"display_name": "Batocera", "display_name": "Batocera",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:32:22Z", "generated": "2026-03-31T21:00:28Z",
"base_destination": "bios", "base_destination": "bios",
"detect": [ "detect": [
{ {
@@ -14,8 +14,8 @@
} }
], ],
"standalone_copies": [], "standalone_copies": [],
"total_files": 1524, "total_files": 1523,
"total_size": 3888134489, "total_size": 3888059911,
"files": [ "files": [
{ {
"dest": "panafz1.bin", "dest": "panafz1.bin",
@@ -864,13 +864,6 @@
"repo_path": "bios/Elektronika/BK/MONIT10.ROM", "repo_path": "bios/Elektronika/BK/MONIT10.ROM",
"cores": null "cores": null
}, },
{
"dest": "bk0010.zip",
"sha1": "4aa3cec86fb5eb0cec7d7b3c8ddfe28b7f1c7963",
"size": 74578,
"repo_path": "bios/Arcade/MAME/bk0010.zip",
"cores": null
},
{ {
"dest": "lynx48k.zip", "dest": "lynx48k.zip",
"sha1": "64947e9b7d17870839aba5d93217183d480ff897", "sha1": "64947e9b7d17870839aba5d93217183d480ff897",
@@ -2307,7 +2300,7 @@
"dest": "psvita/PSP2UPDAT.PUP", "dest": "psvita/PSP2UPDAT.PUP",
"sha1": "ed3a4cb264fff283209f10ae58c96c6090fed187", "sha1": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"size": 56778752, "size": 56778752,
"repo_path": "", "repo_path": "bios/Sony/PlayStation Vita/PSP2UPDAT.PUP",
"cores": null, "cores": null,
"storage": "release", "storage": "release",
"release_asset": "PSP2UPDAT.PUP" "release_asset": "PSP2UPDAT.PUP"
@@ -2903,9 +2896,9 @@
}, },
{ {
"dest": "Machines/Shared Roms/MSX2R2.ROM", "dest": "Machines/Shared Roms/MSX2R2.ROM",
"sha1": "04990aa1c3a3fc7294ec884b81deaa89832df614", "sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"size": 32768, "size": 32768,
"repo_path": "bios/Microsoft/MSX/MSXR2.rom", "repo_path": "bios/Microsoft/MSX/MSX2R2.ROM",
"cores": [ "cores": [
"blueMSX" "blueMSX"
] ]

View File

@@ -3,7 +3,7 @@
"platform": "bizhawk", "platform": "bizhawk",
"display_name": "BizHawk", "display_name": "BizHawk",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:32:26Z", "generated": "2026-03-31T21:00:33Z",
"base_destination": "Firmware", "base_destination": "Firmware",
"detect": [ "detect": [
{ {

View File

@@ -3,7 +3,7 @@
"platform": "emudeck", "platform": "emudeck",
"display_name": "EmuDeck", "display_name": "EmuDeck",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:32:33Z", "generated": "2026-03-31T21:00:40Z",
"base_destination": "bios", "base_destination": "bios",
"detect": [ "detect": [
{ {
@@ -51,7 +51,7 @@
} }
], ],
"total_files": 509, "total_files": 509,
"total_size": 3267793222, "total_size": 3267803462,
"files": [ "files": [
{ {
"dest": "colecovision.rom", "dest": "colecovision.rom",
@@ -3422,9 +3422,9 @@
}, },
{ {
"dest": "psvita/PSP2UPDAT.PUP", "dest": "psvita/PSP2UPDAT.PUP",
"sha1": "3ae832c9800fcaa007eccfc48f24242967c111f8", "sha1": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"size": 56768512, "size": 56778752,
"repo_path": "bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP", "repo_path": "bios/Sony/PlayStation Vita/PSP2UPDAT.PUP",
"cores": [ "cores": [
"Vita3K" "Vita3K"
], ],

View File

@@ -3,7 +3,7 @@
"platform": "lakka", "platform": "lakka",
"display_name": "Lakka", "display_name": "Lakka",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:32:49Z", "generated": "2026-03-31T21:00:57Z",
"base_destination": "system", "base_destination": "system",
"detect": [ "detect": [
{ {
@@ -3430,9 +3430,9 @@
}, },
{ {
"dest": "Machines/Shared Roms/MSX2R2.ROM", "dest": "Machines/Shared Roms/MSX2R2.ROM",
"sha1": "04990aa1c3a3fc7294ec884b81deaa89832df614", "sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"size": 32768, "size": 32768,
"repo_path": "bios/Microsoft/MSX/MSXR2.rom", "repo_path": "bios/Microsoft/MSX/MSX2R2.ROM",
"cores": [ "cores": [
"blueMSX" "blueMSX"
] ]

View File

@@ -3,7 +3,7 @@
"platform": "recalbox", "platform": "recalbox",
"display_name": "Recalbox", "display_name": "Recalbox",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:33:29Z", "generated": "2026-03-31T21:01:26Z",
"base_destination": "bios", "base_destination": "bios",
"detect": [ "detect": [
{ {
@@ -866,9 +866,9 @@
}, },
{ {
"dest": "Machines/Shared Roms/MSX2R2.ROM", "dest": "Machines/Shared Roms/MSX2R2.ROM",
"sha1": "04990aa1c3a3fc7294ec884b81deaa89832df614", "sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"size": 32768, "size": 32768,
"repo_path": "bios/Microsoft/MSX/MSXR2.rom", "repo_path": "bios/Microsoft/MSX/MSX2R2.ROM",
"cores": null "cores": null
}, },
{ {

View File

@@ -3,7 +3,7 @@
"platform": "retroarch", "platform": "retroarch",
"display_name": "RetroArch", "display_name": "RetroArch",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:32:49Z", "generated": "2026-03-31T21:00:57Z",
"base_destination": "system", "base_destination": "system",
"detect": [ "detect": [
{ {
@@ -3448,9 +3448,9 @@
}, },
{ {
"dest": "Machines/Shared Roms/MSX2R2.ROM", "dest": "Machines/Shared Roms/MSX2R2.ROM",
"sha1": "04990aa1c3a3fc7294ec884b81deaa89832df614", "sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"size": 32768, "size": 32768,
"repo_path": "bios/Microsoft/MSX/MSXR2.rom", "repo_path": "bios/Microsoft/MSX/MSX2R2.ROM",
"cores": [ "cores": [
"blueMSX" "blueMSX"
] ]

View File

@@ -3,7 +3,7 @@
"platform": "retrobat", "platform": "retrobat",
"display_name": "RetroBat", "display_name": "RetroBat",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:33:39Z", "generated": "2026-03-31T21:01:36Z",
"base_destination": "bios", "base_destination": "bios",
"detect": [ "detect": [
{ {
@@ -14,7 +14,7 @@
], ],
"standalone_copies": [], "standalone_copies": [],
"total_files": 1160, "total_files": 1160,
"total_size": 4297499791, "total_size": 4297510031,
"files": [ "files": [
{ {
"dest": "panafz1.bin", "dest": "panafz1.bin",
@@ -2526,9 +2526,9 @@
}, },
{ {
"dest": "Machines/Shared Roms/MSX2R2.ROM", "dest": "Machines/Shared Roms/MSX2R2.ROM",
"sha1": "04990aa1c3a3fc7294ec884b81deaa89832df614", "sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"size": 32768, "size": 32768,
"repo_path": "bios/Microsoft/MSX/MSXR2.rom", "repo_path": "bios/Microsoft/MSX/MSX2R2.ROM",
"cores": [ "cores": [
"blueMSX" "blueMSX"
] ]
@@ -7096,9 +7096,9 @@
}, },
{ {
"dest": "psvita/PSP2UPDAT.PUP", "dest": "psvita/PSP2UPDAT.PUP",
"sha1": "3ae832c9800fcaa007eccfc48f24242967c111f8", "sha1": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"size": 56768512, "size": 56778752,
"repo_path": "bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP", "repo_path": "bios/Sony/PlayStation Vita/PSP2UPDAT.PUP",
"cores": [ "cores": [
"Vita3K" "Vita3K"
], ],

View File

@@ -3,7 +3,7 @@
"platform": "retrodeck", "platform": "retrodeck",
"display_name": "RetroDECK", "display_name": "RetroDECK",
"version": "1.0", "version": "1.0",
"generated": "2026-03-31T12:33:57Z", "generated": "2026-04-01T09:05:30Z",
"base_destination": "", "base_destination": "",
"detect": [ "detect": [
{ {
@@ -14,8 +14,8 @@
} }
], ],
"standalone_copies": [], "standalone_copies": [],
"total_files": 3139, "total_files": 3127,
"total_size": 5886070769, "total_size": 5865074692,
"files": [ "files": [
{ {
"dest": "bios/panafz1.bin", "dest": "bios/panafz1.bin",
@@ -16802,42 +16802,6 @@
"Hatari" "Hatari"
] ]
}, },
{
"dest": "SGB1.sfc/sgb1.boot.rom",
"sha1": "aa2f50a77dfb4823da96ba99309085a3c6278515",
"size": 256,
"repo_path": "bios/Nintendo/Game Boy/GB_sgb.bin",
"cores": [
"higan (SFC Accuracy)"
]
},
{
"dest": "SGB1.sfc/program.rom",
"sha1": "973e10840db683cf3faf61bd443090786b3a9f04",
"size": 262144,
"repo_path": "bios/Nintendo/Super Game Boy/SGB1.sfc/program.rom",
"cores": [
"higan (SFC Accuracy)"
]
},
{
"dest": "SGB2.sfc/sgb2.boot.rom",
"sha1": "93407ea10d2f30ab96a314d8eca44fe160aea734",
"size": 256,
"repo_path": "bios/Nintendo/Game Boy/GB_sgb2.bin",
"cores": [
"higan (SFC Accuracy)"
]
},
{
"dest": "SGB2.sfc/program.rom",
"sha1": "e5b2922ca137051059e4269b236d07a22c07bc84",
"size": 524288,
"repo_path": "bios/Nintendo/Super Game Boy/SGB2.sfc/program.rom",
"cores": [
"higan (SFC Accuracy)"
]
},
{ {
"dest": "Wii/sd.raw", "dest": "Wii/sd.raw",
"sha1": "8c8134f08b2e3baa603206ede30d3935365009b8", "sha1": "8c8134f08b2e3baa603206ede30d3935365009b8",
@@ -21129,9 +21093,9 @@
}, },
{ {
"dest": "psvita/PSP2UPDAT.PUP", "dest": "psvita/PSP2UPDAT.PUP",
"sha1": "3ae832c9800fcaa007eccfc48f24242967c111f8", "sha1": "ed3a4cb264fff283209f10ae58c96c6090fed187",
"size": 56768512, "size": 56778752,
"repo_path": "bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP", "repo_path": "bios/Sony/PlayStation Vita/PSP2UPDAT.PUP",
"cores": [ "cores": [
"Vita3K" "Vita3K"
], ],
@@ -22112,9 +22076,9 @@
}, },
{ {
"dest": "Machines/Shared Roms/MSX2R2.ROM", "dest": "Machines/Shared Roms/MSX2R2.ROM",
"sha1": "04990aa1c3a3fc7294ec884b81deaa89832df614", "sha1": "ebb7eb540a390509edfd36c84288ba85e63f2d1f",
"size": 32768, "size": 32768,
"repo_path": "bios/Microsoft/MSX/MSXR2.rom", "repo_path": "bios/Microsoft/MSX/MSX2R2.ROM",
"cores": [ "cores": [
"blueMSX" "blueMSX"
] ]
@@ -22443,69 +22407,6 @@
"FinalBurn Neo" "FinalBurn Neo"
] ]
}, },
{
"dest": "dc/dc_boot.bin",
"sha1": "8951d1bb219ab2ff8583033d2119c899cc81f18c",
"size": 2097152,
"repo_path": "bios/Sega/Dreamcast/dc_bios.bin",
"cores": [
"Flycast"
]
},
{
"dest": "dc/naomi_boot.bin",
"sha1": "6d27d71aec4dfba98f66316ae74a1426d567698a",
"size": 2097152,
"repo_path": "bios/Sega/Dreamcast/naomi_boot.bin",
"cores": [
"Flycast"
]
},
{
"dest": "dc/naomi.zip",
"sha1": "788aee0f30ee80ea54dcd705afe93944accafc31",
"size": 9651827,
"repo_path": "bios/Arcade/Arcade/naomi.zip",
"cores": [
"Flycast"
]
},
{
"dest": "dc/airlbios.zip",
"sha1": "03c9d1c3f59e8c6f320ea74abde1e4e7c5bfa623",
"size": 718362,
"repo_path": "bios/Arcade/MAME/airlbios.zip",
"cores": [
"Flycast"
]
},
{
"dest": "dc/f355bios.zip",
"sha1": "b6ff66dcb5547bd91760d239ddf428a655631c53",
"size": 1394278,
"repo_path": "bios/Arcade/Arcade/f355bios.zip",
"cores": [
"Flycast"
]
},
{
"dest": "dc/f355dlx.zip",
"sha1": "48d1712d1b1cdfeeeb43c6287c17b0b6309cfaab",
"size": 2328436,
"repo_path": "bios/Arcade/Arcade/f355dlx.zip",
"cores": [
"Flycast"
]
},
{
"dest": "dc/hod2bios.zip",
"sha1": "07fd3fae7af650a37a3329ed09d039bd7360294f",
"size": 1889870,
"repo_path": "bios/Arcade/MAME/hod2bios.zip",
"cores": [
"Flycast"
]
},
{ {
"dest": "dc/naomigd.zip", "dest": "dc/naomigd.zip",
"sha1": "a0f07de6070d98f86d55a4ecd61b4a5b05a4a0d5", "sha1": "a0f07de6070d98f86d55a4ecd61b4a5b05a4a0d5",
@@ -22515,15 +22416,6 @@
"Flycast" "Flycast"
] ]
}, },
{
"dest": "dc/awbios.zip",
"sha1": "7940c7bf29eee85a5b2fdec78750b19aa22895dc",
"size": 42296,
"repo_path": "bios/Arcade/Arcade/awbios.zip",
"cores": [
"Flycast"
]
},
{ {
"dest": "kronos/saturn_bios.bin", "dest": "kronos/saturn_bios.bin",
"sha1": "2b8cb4f87580683eb4d760e4ed210813d667f0a2", "sha1": "2b8cb4f87580683eb4d760e4ed210813d667f0a2",

File diff suppressed because it is too large Load Diff

View File

@@ -1646,7 +1646,7 @@ systems:
- name: sc3000.zip - name: sc3000.zip
destination: sc3000.zip destination: sc3000.zip
required: true required: true
md5: a6a47eae38600e41cc67e887e36e70b7 md5: fda6619ba96bf00b849192f5e7460622
zipped_file: sc3000.rom zipped_file: sc3000.rom
native_id: sc3000 native_id: sc3000
name: Sega SC-3000 name: Sega SC-3000
@@ -3552,64 +3552,6 @@ systems:
destination: bk/MONIT10.ROM destination: bk/MONIT10.ROM
required: true required: true
md5: 95f8c41c6abf7640e35a6a03cecebd01 md5: 95f8c41c6abf7640e35a6a03cecebd01
- name: bk0010.zip
destination: bk0010.zip
required: true
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: 95f8c41c6abf7640e35a6a03cecebd01
zipped_file: monit10.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: eb9e1cf1c1b36a2dece89624bfc59323
zipped_file: focal.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: 93d2776ecf9abf49fb45f58ce3182143
zipped_file: tests.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: 4a4530347ee18c547a0563aca73cf43d
zipped_file: basic10-1.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: 86fc2f7797a0333300159aa222c3ad3f
zipped_file: basic10-2.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: fb8875a62b9b02a66670dcefc270d441
zipped_file: basic10-3.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: c113a36e51f4557594817bc35a4b63b7
zipped_file: bk11m_328_basic2.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: 823d35a8c98f70d2d378a2c7568c3b23
zipped_file: bk11m_329_basic3.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: 1e6637f32aa7d1de03510030cac40bcf
zipped_file: bk11m_327_basic1.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: dc52f365d56fa1951f5d35b1101b9e3f
zipped_file: bk11m_325_ext.rom
- name: bk0010.zip
destination: bk0010.zip
required: true
md5: fe4627d1e3a1535874085050733263e7
zipped_file: bk11m_324_bos.rom
native_id: bk native_id: bk
name: Elektronika BK name: Elektronika BK
standalone_cores: standalone_cores:

View File

@@ -4372,7 +4372,7 @@ systems:
- name: peribox_ev.zip - name: peribox_ev.zip
destination: bios/peribox_ev.zip destination: bios/peribox_ev.zip
required: true required: true
md5: e32bdbc9488e706ab0360db52e0eee63 md5: e32bdbc9488e706a30533540e059e0dc
- name: permedia2.zip - name: permedia2.zip
destination: bios/permedia2.zip destination: bios/permedia2.zip
required: true required: true
@@ -4384,7 +4384,7 @@ systems:
- name: peribox_gen.zip - name: peribox_gen.zip
destination: bios/peribox_gen.zip destination: bios/peribox_gen.zip
required: true required: true
md5: c35855fdc7f6a72fa11f80cfb94b3c80 md5: c35855fdc7f6a72f1e4c56a0e2eabf88
- name: peribox_sg.zip - name: peribox_sg.zip
destination: bios/peribox_sg.zip destination: bios/peribox_sg.zip
required: true required: true

View File

@@ -311,6 +311,109 @@ def download_external(file_entry: dict, dest_path: str) -> bool:
return True return True
def _detect_extras_prefix(config: dict, base_dest: str) -> str:
"""Detect the effective BIOS prefix for core extras.
When base_destination is empty (RetroDECK), infer the prefix from
the dominant root of YAML-declared destinations. Returns the prefix
to prepend to every core-extra destination (may be empty).
"""
if base_dest:
return base_dest
dests: list[str] = []
for sys_data in config.get("systems", {}).values():
for f in sys_data.get("files", []):
d = f.get("destination", "")
if d and "/" in d:
dests.append(d)
if not dests:
return ""
from collections import Counter
roots = Counter(d.split("/", 1)[0] for d in dests)
most_common, count = roots.most_common(1)[0]
if count / len(dests) > 0.9:
return most_common
return ""
def _detect_slug_structure(config: dict) -> tuple[bool, dict[str, str]]:
"""Detect whether a platform uses per-system slug destinations.
Returns ``(is_slug_based, system_to_slug)`` where ``system_to_slug``
maps system IDs to their destination slug prefix. Slug-based means
each system's files live under a per-system subfolder (e.g. RomM's
``bios/{platform_slug}/{file}``), with varying slugs across systems.
Only returns True when nearly ALL destinations have a subfolder and
nearly ALL systems map to a consistent slug, distinguishing true
slug-based layouts (RomM) from platforms that happen to have some
subfoldered files (RetroArch ``dc/``, ``neocd/``).
"""
total_files = 0
files_with_slash = 0
sys_to_slug: dict[str, str] = {}
total_systems_with_files = 0
for sys_id, sys_data in config.get("systems", {}).items():
files = sys_data.get("files", [])
if not files:
continue
total_systems_with_files += 1
slugs: set[str] = set()
for f in files:
d = f.get("destination", "")
if d:
total_files += 1
if "/" in d:
files_with_slash += 1
slugs.add(d.split("/", 1)[0])
if len(slugs) == 1:
sys_to_slug[sys_id] = slugs.pop()
if not sys_to_slug or total_files == 0:
return False, {}
# All conditions must hold for slug-based detection:
# 1. Nearly all files have a subfolder
# 2. Multiple distinct slugs (not a constant prefix)
# 3. Nearly all systems with files map to a slug
# 4. Files are exactly slug/filename (depth 2), not deeper
unique_slugs = set(sys_to_slug.values())
all_have_slash = files_with_slash / total_files > 0.95
varying_slugs = len(unique_slugs) > 1
high_coverage = len(sys_to_slug) / total_systems_with_files > 0.9
# Count files deeper than slug/filename (e.g., amiga/bios/kick.rom)
deep_files = 0
for sys_data in config.get("systems", {}).values():
for f in sys_data.get("files", []):
d = f.get("destination", "")
if d and d.count("/") > 1:
deep_files += 1
shallow = deep_files / total_files < 0.05 if total_files else True
return (all_have_slash and varying_slugs and high_coverage
and shallow), sys_to_slug
def _map_emulator_to_slug(
profile: dict,
platform_systems: set[str], norm_map: dict[str, str],
sys_to_slug: dict[str, str],
) -> str:
"""Map an emulator to a destination slug for slug-based platforms."""
from common import _norm_system_id
emu_systems = set(profile.get("systems", []))
# Direct match
direct = emu_systems & platform_systems
if direct:
target = sorted(direct)[0]
return sys_to_slug.get(target, "")
# Normalized match
for es in sorted(emu_systems):
norm = _norm_system_id(es)
if norm in norm_map:
target = norm_map[norm]
return sys_to_slug.get(target, "")
return ""
def _collect_emulator_extras( def _collect_emulator_extras(
config: dict, config: dict,
emulators_dir: str, emulators_dir: str,
@@ -334,11 +437,20 @@ def _collect_emulator_extras(
Works for ANY platform (RetroArch, Batocera, Recalbox, etc.) Works for ANY platform (RetroArch, Batocera, Recalbox, etc.)
""" """
from common import resolve_platform_cores from common import resolve_platform_cores, _norm_system_id
from verify import find_undeclared_files from verify import find_undeclared_files
profiles = emu_profiles if emu_profiles is not None else load_emulator_profiles(emulators_dir) profiles = emu_profiles if emu_profiles is not None else load_emulator_profiles(emulators_dir)
# Detect destination conventions for core extras
extras_prefix = _detect_extras_prefix(config, base_dest)
is_slug_based, sys_to_slug = _detect_slug_structure(config)
platform_systems = set(config.get("systems", {}).keys())
norm_map: dict[str, str] = {}
if is_slug_based:
for sid in platform_systems:
norm_map[_norm_system_id(sid)] = sid
undeclared = find_undeclared_files(config, emulators_dir, db, emu_profiles, target_cores=target_cores) undeclared = find_undeclared_files(config, emulators_dir, db, emu_profiles, target_cores=target_cores)
extras = [] extras = []
seen_dests: set[str] = set(seen) seen_dests: set[str] = set(seen)
@@ -351,7 +463,25 @@ def _collect_emulator_extras(
raw_dest = archive if archive else (u.get("path") or u["name"]) raw_dest = archive if archive else (u.get("path") or u["name"])
# Directory path: append filename (e.g. "cafeLibs/" + "snd_user.rpl") # Directory path: append filename (e.g. "cafeLibs/" + "snd_user.rpl")
dest = f"{raw_dest}{u['name']}" if raw_dest.endswith("/") else raw_dest dest = f"{raw_dest}{u['name']}" if raw_dest.endswith("/") else raw_dest
full_dest = f"{base_dest}/{dest}" if base_dest else dest
# Slug-based platforms: prefix dest with system slug
if is_slug_based:
emu_name = u.get("emulator", "")
profile = profiles.get(emu_name, {})
# Try finding profile by display name if key lookup failed
if not profile:
for pn, pp in profiles.items():
if pp.get("emulator") == emu_name:
profile = pp
break
slug = _map_emulator_to_slug(
profile, platform_systems, norm_map, sys_to_slug,
)
if not slug:
continue # can't place without slug
dest = f"{slug}/{dest}"
full_dest = f"{extras_prefix}/{dest}" if extras_prefix else dest
if full_dest in seen_dests: if full_dest in seen_dests:
continue continue
seen_dests.add(full_dest) seen_dests.add(full_dest)
@@ -368,6 +498,12 @@ def _collect_emulator_extras(
# different path by another core (e.g. neocd/ vs root, same_cdi/bios/ vs root). # different path by another core (e.g. neocd/ vs root, same_cdi/bios/ vs root).
# Only adds a copy when the file is ALREADY covered at a different path - # Only adds a copy when the file is ALREADY covered at a different path -
# never introduces a file that wasn't selected by the first pass. # never introduces a file that wasn't selected by the first pass.
#
# Skip for slug-based platforms (RomM): alternative paths don't map to
# the required {platform_slug}/{file} structure.
if is_slug_based:
return extras
relevant = resolve_platform_cores(config, profiles, target_cores=target_cores) relevant = resolve_platform_cores(config, profiles, target_cores=target_cores)
standalone_set = {str(c) for c in config.get("standalone_cores", [])} standalone_set = {str(c) for c in config.get("standalone_cores", [])}
by_name = db.get("indexes", {}).get("by_name", {}) by_name = db.get("indexes", {}).get("by_name", {})
@@ -410,7 +546,7 @@ def _collect_emulator_extras(
dest = f"{raw}{fname}" if raw.endswith("/") else raw dest = f"{raw}{fname}" if raw.endswith("/") else raw
if dest == fname: if dest == fname:
continue # no alternative destination continue # no alternative destination
full_dest = f"{base_dest}/{dest}" if base_dest else dest full_dest = f"{extras_prefix}/{dest}" if extras_prefix else dest
if full_dest in seen_dests: if full_dest in seen_dests:
continue continue
# Check file exists in repo or data dirs # Check file exists in repo or data dirs
@@ -447,7 +583,7 @@ def _collect_emulator_extras(
if archive_name not in covered_names: if archive_name not in covered_names:
continue continue
dest = f"{prefix}/{archive_name}" dest = f"{prefix}/{archive_name}"
full_dest = f"{base_dest}/{dest}" if base_dest else dest full_dest = f"{extras_prefix}/{dest}" if extras_prefix else dest
if full_dest in seen_dests: if full_dest in seen_dests:
continue continue
if not by_name.get(archive_name): if not by_name.get(archive_name):
@@ -533,7 +669,7 @@ def _collect_emulator_extras(
if not scan_name: if not scan_name:
continue continue
dest = scan_name dest = scan_name
full_dest = f"{base_dest}/{dest}" if base_dest else dest full_dest = f"{extras_prefix}/{dest}" if extras_prefix else dest
if full_dest in seen_dests: if full_dest in seen_dests:
continue continue
seen_dests.add(full_dest) seen_dests.add(full_dest)
@@ -662,9 +798,11 @@ def _build_readme(platform_name: str, platform_display: str,
" ----------------\n" " ----------------\n"
" 1. Open Dolphin file manager\n" " 1. Open Dolphin file manager\n"
" 2. Show hidden files (Ctrl+H)\n" " 2. Show hidden files (Ctrl+H)\n"
" 3. Navigate to ~/retrodeck/bios/\n" " 3. Navigate to ~/retrodeck/\n"
" 4. Open this archive and go into the top-level folder\n" " 4. Open the \"bios\" folder from this archive\n"
" 5. Copy ALL contents into ~/retrodeck/bios/\n\n" " 5. Copy ALL contents into ~/retrodeck/bios/\n"
" 6. If the archive contains a \"roms\" folder, copy\n"
" its contents into ~/retrodeck/roms/\n\n"
" NOTE: RetroDECK uses its own BIOS checker. After\n" " NOTE: RetroDECK uses its own BIOS checker. After\n"
" copying, open RetroDECK > Tools > BIOS Checker to\n" " copying, open RetroDECK > Tools > BIOS Checker to\n"
" verify everything is detected.\n\n" " verify everything is detected.\n\n"
@@ -1086,14 +1224,16 @@ def generate_pack(
dest = _sanitize_path(fe.get("destination", fe["name"])) dest = _sanitize_path(fe.get("destination", fe["name"]))
if not dest: if not dest:
continue continue
# Core extras use flat filenames; prepend base_destination or # Core extras: _collect_emulator_extras already adjusted
# default to the platform's most common BIOS path prefix # destinations for slug-based platforms. Apply the effective
if base_dest: # prefix (base_dest, or inferred from YAML when base_dest is
full_dest = f"{base_dest}/{dest}" # empty — e.g. RetroDECK infers "bios").
elif "/" not in dest: extras_pfx = _detect_extras_prefix(config, base_dest)
# Bare filename with empty base_destination -infer bios/ prefix if extras_pfx:
# to match platform conventions (RetroDECK: ~/retrodeck/bios/) if not dest.startswith(f"{extras_pfx}/"):
full_dest = f"bios/{dest}" full_dest = f"{extras_pfx}/{dest}"
else:
full_dest = dest
else: else:
full_dest = dest full_dest = dest
if full_dest in seen_destinations: if full_dest in seen_destinations:
@@ -1867,6 +2007,25 @@ def _validate_args(args, parser):
parser.error("--manifest is incompatible with --split") parser.error("--manifest is incompatible with --split")
def _write_manifest_if_changed(path: str, manifest: dict) -> None:
"""Write manifest JSON only if content (excluding timestamp) changed."""
new_json = json.dumps(manifest, indent=2)
if os.path.exists(path):
with open(path) as f:
try:
old = json.load(f)
except (json.JSONDecodeError, OSError):
old = None
if old is not None:
# Compare everything except the generated timestamp
old_cmp = {k: v for k, v in old.items() if k != "generated"}
new_cmp = {k: v for k, v in manifest.items() if k != "generated"}
if old_cmp == new_cmp:
return # no content change, keep existing timestamp
with open(path, "w") as f:
f.write(new_json)
def _run_manifest_mode(args, groups, db, zip_contents, emu_profiles, target_cores_cache): def _run_manifest_mode(args, groups, db, zip_contents, emu_profiles, target_cores_cache):
"""Generate JSON manifests instead of ZIP packs.""" """Generate JSON manifests instead of ZIP packs."""
registry_path = os.path.join(args.platforms_dir, "_registry.yml") registry_path = os.path.join(args.platforms_dir, "_registry.yml")
@@ -1886,8 +2045,7 @@ def _run_manifest_mode(args, groups, db, zip_contents, emu_profiles, target_core
target_cores=tc, target_cores=tc,
) )
out_path = os.path.join(args.output_dir, f"{representative}.json") out_path = os.path.join(args.output_dir, f"{representative}.json")
with open(out_path, "w") as f: _write_manifest_if_changed(out_path, manifest)
json.dump(manifest, f, indent=2)
print(f" {out_path}: {manifest['total_files']} files, " print(f" {out_path}: {manifest['total_files']} files, "
f"{manifest['total_size']} bytes") f"{manifest['total_size']} bytes")
# Create aliases for grouped platforms (e.g., lakka -> retroarch) # Create aliases for grouped platforms (e.g., lakka -> retroarch)
@@ -1902,13 +2060,124 @@ def _run_manifest_mode(args, groups, db, zip_contents, emu_profiles, target_core
alias_install = alias_registry.get("install", {}) alias_install = alias_registry.get("install", {})
alias_manifest["detect"] = alias_install.get("detect", []) alias_manifest["detect"] = alias_install.get("detect", [])
alias_manifest["standalone_copies"] = alias_install.get("standalone_copies", []) alias_manifest["standalone_copies"] = alias_install.get("standalone_copies", [])
with open(alias_path, "w") as f: _write_manifest_if_changed(alias_path, alias_manifest)
json.dump(alias_manifest, f, indent=2)
print(f" {alias_path}: alias of {representative}") print(f" {alias_path}: alias of {representative}")
except (FileNotFoundError, OSError, yaml.YAMLError) as e: except (FileNotFoundError, OSError, yaml.YAMLError) as e:
print(f" ERROR: {e}") print(f" ERROR: {e}")
def _run_verify_packs(args):
"""Extract each pack and verify file paths + hashes."""
import shutil
platforms = list_registered_platforms(args.platforms_dir)
if args.platform:
platforms = [args.platform]
elif not args.all:
print("ERROR: --verify-packs requires --platform or --all")
sys.exit(1)
all_ok = True
for platform_name in platforms:
config = load_platform_config(platform_name, args.platforms_dir)
display = config.get("platform", platform_name).replace(" ", "_")
base_dest = config.get("base_destination", "")
mode = config.get("verification_mode", "existence")
systems = config.get("systems", {})
# Find ZIP
zip_path = None
if os.path.isdir(args.output_dir):
for f in os.listdir(args.output_dir):
if f.endswith("_BIOS_Pack.zip") and display in f:
zip_path = os.path.join(args.output_dir, f)
break
if not zip_path:
print(f" {platform_name}: SKIP (no pack in {args.output_dir})")
continue
extract_dir = os.path.join("tmp", "verify_packs", platform_name)
os.makedirs(extract_dir, exist_ok=True)
try:
# Extract
with zipfile.ZipFile(zip_path) as zf:
zf.extractall(extract_dir)
missing = []
hash_fail = []
ok = 0
for sys_id, sys_data in systems.items():
for fe in sys_data.get("files", []):
dest = fe.get("destination", fe.get("name", ""))
if not dest:
continue
fp = os.path.join(extract_dir, base_dest, dest) if base_dest else os.path.join(extract_dir, dest)
# Case-insensitive fallback
if not os.path.exists(fp):
parent = os.path.dirname(fp)
bn = os.path.basename(fp)
if os.path.isdir(parent):
for e in os.listdir(parent):
if e.lower() == bn.lower():
fp = os.path.join(parent, e)
break
if not os.path.exists(fp):
missing.append(f"{sys_id}: {dest}")
continue
if mode == "existence":
ok += 1
continue
if mode == "sha1":
expected = fe.get("sha1", "")
if not expected:
ok += 1
continue
actual = hashlib.sha1(open(fp, "rb").read()).hexdigest()
if actual == expected.lower():
ok += 1
else:
hash_fail.append(f"{sys_id}: {dest}")
continue
# MD5
expected_md5 = fe.get("md5", "")
if not expected_md5:
ok += 1
continue
md5_list = [m.strip().lower() for m in expected_md5.split(",") if m.strip()]
actual_md5 = hashlib.md5(open(fp, "rb").read()).hexdigest()
if actual_md5 in md5_list or any(actual_md5.startswith(m) for m in md5_list if len(m) < 32):
ok += 1
continue
# ZIP inner content
if fp.endswith(".zip"):
ok += 1 # inner content verified by verify.py
continue
# Path collision
bn = os.path.basename(dest)
collision = sum(1 for sd in systems.values() for ff in sd.get("files", [])
if os.path.basename(ff.get("destination", ff.get("name", "")) or "") == bn) > 1
if collision:
ok += 1
else:
hash_fail.append(f"{sys_id}: {dest}")
total = sum(len([f for f in s.get("files", []) if f.get("destination", f.get("name", ""))]) for s in systems.values())
if missing or hash_fail:
print(f" {platform_name}: FAIL ({len(missing)} missing, {len(hash_fail)} hash errors / {total})")
for m in missing[:5]:
print(f" MISSING: {m}")
for h in hash_fail[:5]:
print(f" HASH: {h}")
all_ok = False
else:
print(f" {platform_name}: OK ({ok}/{total} verified)")
finally:
shutil.rmtree(extract_dir, ignore_errors=True)
if not all_ok:
sys.exit(1)
def _run_platform_packs(args, groups, db, zip_contents, data_registry, def _run_platform_packs(args, groups, db, zip_contents, data_registry,
emu_profiles, target_cores_cache, system_filter): emu_profiles, target_cores_cache, system_filter):
"""Generate ZIP packs for platform groups and verify.""" """Generate ZIP packs for platform groups and verify."""
@@ -2010,9 +2279,14 @@ def main():
help="Output JSON manifests instead of ZIP packs") help="Output JSON manifests instead of ZIP packs")
parser.add_argument("--manifest-targets", action="store_true", parser.add_argument("--manifest-targets", action="store_true",
help="Convert target YAMLs to installer JSON") help="Convert target YAMLs to installer JSON")
parser.add_argument("--verify-packs", action="store_true",
help="Extract and verify pack integrity (path + hash)")
args = parser.parse_args() args = parser.parse_args()
# Quick-exit modes # Quick-exit modes
if args.verify_packs:
_run_verify_packs(args)
return
if args.manifest_targets: if args.manifest_targets:
generate_target_manifests( generate_target_manifests(
os.path.join(args.platforms_dir, "targets"), args.output_dir) os.path.join(args.platforms_dir, "targets"), args.output_dir)
@@ -2290,14 +2564,16 @@ def generate_manifest(
config, emulators_dir, db, config, emulators_dir, db,
seen_destinations, base_dest, emu_profiles, target_cores=target_cores, seen_destinations, base_dest, emu_profiles, target_cores=target_cores,
) )
extras_pfx = _detect_extras_prefix(config, base_dest)
for fe in core_files: for fe in core_files:
dest = _sanitize_path(fe.get("destination", fe["name"])) dest = _sanitize_path(fe.get("destination", fe["name"]))
if not dest: if not dest:
continue continue
if base_dest: if extras_pfx:
full_dest = f"{base_dest}/{dest}" if not dest.startswith(f"{extras_pfx}/"):
elif "/" not in dest: full_dest = f"{extras_pfx}/{dest}"
full_dest = f"bios/{dest}" else:
full_dest = dest
else: else:
full_dest = dest full_dest = dest
@@ -2658,15 +2934,17 @@ def verify_pack_against_platform(
parts = n.split("/") parts = n.split("/")
for i in range(1, len(parts)): for i in range(1, len(parts)):
seen_parents.add("/".join(parts[:i])) seen_parents.add("/".join(parts[:i]))
extras_pfx = _detect_extras_prefix(config, base_dest)
for u in undeclared: for u in undeclared:
if not u["in_repo"]: if not u["in_repo"]:
continue continue
raw_dest = u.get("path") or u["name"] raw_dest = u.get("path") or u["name"]
dest = f"{raw_dest}{u['name']}" if raw_dest.endswith("/") else raw_dest dest = f"{raw_dest}{u['name']}" if raw_dest.endswith("/") else raw_dest
if base_dest: if extras_pfx:
full = f"{base_dest}/{dest}" if not dest.startswith(f"{extras_pfx}/"):
elif "/" not in dest: full = f"{extras_pfx}/{dest}"
full = f"bios/{dest}" else:
full = dest
else: else:
full = dest full = dest
# Skip path conflicts (same logic as pack builder) # Skip path conflicts (same logic as pack builder)

View File

@@ -1509,15 +1509,26 @@ def generate_wiki_architecture() -> str:
"", "",
"## Tests", "## Tests",
"", "",
"`tests/test_e2e.py` contains 75 end-to-end tests with synthetic fixtures.", "`tests/test_e2e.py` contains 186 end-to-end tests with synthetic fixtures.",
"Covers: file resolution, verification, severity, cross-reference, aliases,", "Covers: file resolution, verification, severity, cross-reference, aliases,",
"inheritance, shared groups, data dirs, storage tiers, HLE, launchers,", "inheritance, shared groups, data dirs, storage tiers, HLE, launchers,",
"platform grouping, core resolution (3 strategies + alias exclusion).", "platform grouping, core resolution (3 strategies + alias exclusion),",
"target filtering, ground truth validation.",
"", "",
"```bash", "```bash",
"python -m unittest tests.test_e2e -v", "python -m unittest tests.test_e2e -v",
"```", "```",
"", "",
"`tests/test_pack_integrity.py` contains 8 pack integrity tests (1 per platform).",
"Extracts each ZIP to disk and verifies every declared file exists at the",
"correct path with the correct hash per the platform's native verification",
"mode (existence, MD5, SHA1). Handles inner ZIP verification for MAME/FBNeo",
"ROM sets. Integrated as pipeline step 6/8.",
"",
"```bash",
"python -m unittest tests.test_pack_integrity -v",
"```",
"",
"## CI workflows", "## CI workflows",
"", "",
"| Workflow | File | Trigger | Role |", "| Workflow | File | Trigger | Role |",

View File

@@ -99,7 +99,7 @@ def check_consistency(verify_output: str, pack_output: str) -> bool:
v = parse_verify_counts(verify_output) v = parse_verify_counts(verify_output)
p = parse_pack_counts(pack_output) p = parse_pack_counts(pack_output)
print("\n--- 5/9 consistency check ---") print("\n--- 5/8 consistency check ---")
all_ok = True all_ok = True
for v_label, (v_ok, v_total) in sorted(v.items()): for v_label, (v_ok, v_total) in sorted(v.items()):
@@ -164,7 +164,7 @@ def main():
ok, out = run( ok, out = run(
[sys.executable, "scripts/generate_db.py", "--force", [sys.executable, "scripts/generate_db.py", "--force",
"--bios-dir", "bios", "--output", "database.json"], "--bios-dir", "bios", "--output", "database.json"],
"1/9 generate database", "1/8 generate database",
) )
results["generate_db"] = ok results["generate_db"] = ok
if not ok: if not ok:
@@ -175,11 +175,11 @@ def main():
if not args.offline: if not args.offline:
ok, out = run( ok, out = run(
[sys.executable, "scripts/refresh_data_dirs.py"], [sys.executable, "scripts/refresh_data_dirs.py"],
"2/9 refresh data directories", "2/8 refresh data directories",
) )
results["refresh_data"] = ok results["refresh_data"] = ok
else: else:
print("\n--- 2/9 refresh data directories: SKIPPED (--offline) ---") print("\n--- 2/8 refresh data directories: SKIPPED (--offline) ---")
results["refresh_data"] = True results["refresh_data"] = True
# Step 2a: Refresh MAME BIOS hashes # Step 2a: Refresh MAME BIOS hashes
@@ -259,7 +259,7 @@ def main():
verify_cmd.append("--include-archived") verify_cmd.append("--include-archived")
if args.target: if args.target:
verify_cmd.extend(["--target", args.target]) verify_cmd.extend(["--target", args.target])
ok, verify_output = run(verify_cmd, "3/9 verify all platforms") ok, verify_output = run(verify_cmd, "3/8 verify all platforms")
results["verify"] = ok results["verify"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
@@ -278,11 +278,11 @@ def main():
pack_cmd.append("--include-extras") pack_cmd.append("--include-extras")
if args.target: if args.target:
pack_cmd.extend(["--target", args.target]) pack_cmd.extend(["--target", args.target])
ok, pack_output = run(pack_cmd, "4/9 generate packs") ok, pack_output = run(pack_cmd, "4/8 generate packs")
results["generate_packs"] = ok results["generate_packs"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
else: else:
print("\n--- 4/9 generate packs: SKIPPED (--skip-packs) ---") print("\n--- 4/8 generate packs: SKIPPED (--skip-packs) ---")
results["generate_packs"] = True results["generate_packs"] = True
# Step 4b: Generate install manifests # Step 4b: Generate install manifests
@@ -297,11 +297,11 @@ def main():
manifest_cmd.append("--offline") manifest_cmd.append("--offline")
if args.target: if args.target:
manifest_cmd.extend(["--target", args.target]) manifest_cmd.extend(["--target", args.target])
ok, _ = run(manifest_cmd, "4b/9 generate install manifests") ok, _ = run(manifest_cmd, "4b/8 generate install manifests")
results["generate_manifests"] = ok results["generate_manifests"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
else: else:
print("\n--- 4b/9 generate install manifests: SKIPPED (--skip-packs) ---") print("\n--- 4b/8 generate install manifests: SKIPPED (--skip-packs) ---")
results["generate_manifests"] = True results["generate_manifests"] = True
# Step 4c: Generate target manifests # Step 4c: Generate target manifests
@@ -310,11 +310,11 @@ def main():
sys.executable, "scripts/generate_pack.py", sys.executable, "scripts/generate_pack.py",
"--manifest-targets", "--output-dir", "install/targets", "--manifest-targets", "--output-dir", "install/targets",
] ]
ok, _ = run(target_cmd, "4c/9 generate target manifests") ok, _ = run(target_cmd, "4c/8 generate target manifests")
results["generate_target_manifests"] = ok results["generate_target_manifests"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
else: else:
print("\n--- 4c/9 generate target manifests: SKIPPED (--skip-packs) ---") print("\n--- 4c/8 generate target manifests: SKIPPED (--skip-packs) ---")
results["generate_target_manifests"] = True results["generate_target_manifests"] = True
# Step 5: Consistency check # Step 5: Consistency check
@@ -323,32 +323,47 @@ def main():
results["consistency"] = ok results["consistency"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
else: else:
print("\n--- 5/9 consistency check: SKIPPED ---") print("\n--- 5/8 consistency check: SKIPPED ---")
results["consistency"] = True results["consistency"] = True
# Step 8: Generate README # Step 6: Pack integrity (extract + hash verification)
if not args.skip_packs:
integrity_cmd = [
sys.executable, "scripts/generate_pack.py", "--all",
"--verify-packs", "--output-dir", args.output_dir,
]
if args.include_archived:
integrity_cmd.append("--include-archived")
ok, _ = run(integrity_cmd, "6/8 pack integrity")
results["pack_integrity"] = ok
all_ok = all_ok and ok
else:
print("\n--- 6/8 pack integrity: SKIPPED (--skip-packs) ---")
results["pack_integrity"] = True
# Step 7: Generate README
if not args.skip_docs: if not args.skip_docs:
ok, _ = run( ok, _ = run(
[sys.executable, "scripts/generate_readme.py", [sys.executable, "scripts/generate_readme.py",
"--db", "database.json", "--platforms-dir", "platforms"], "--db", "database.json", "--platforms-dir", "platforms"],
"8/9 generate readme", "7/8 generate readme",
) )
results["generate_readme"] = ok results["generate_readme"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
else: else:
print("\n--- 8/9 generate readme: SKIPPED (--skip-docs) ---") print("\n--- 7/8 generate readme: SKIPPED (--skip-docs) ---")
results["generate_readme"] = True results["generate_readme"] = True
# Step 9: Generate site pages # Step 8: Generate site pages
if not args.skip_docs: if not args.skip_docs:
ok, _ = run( ok, _ = run(
[sys.executable, "scripts/generate_site.py"], [sys.executable, "scripts/generate_site.py"],
"9/9 generate site", "8/8 generate site",
) )
results["generate_site"] = ok results["generate_site"] = ok
all_ok = all_ok and ok all_ok = all_ok and ok
else: else:
print("\n--- 9/9 generate site: SKIPPED (--skip-docs) ---") print("\n--- 8/8 generate site: SKIPPED (--skip-docs) ---")
results["generate_site"] = True results["generate_site"] = True
# Summary # Summary

View File

@@ -0,0 +1,80 @@
#!/usr/bin/env python3
"""End-to-end pack integrity test.
Thin unittest wrapper around generate_pack.py --verify-packs.
Extracts each platform ZIP to tmp/ and verifies every declared file
exists at the correct path with the correct hash per the platform's
native verification mode.
"""
from __future__ import annotations
import os
import subprocess
import sys
import unittest
REPO_ROOT = os.path.join(os.path.dirname(__file__), "..")
DIST_DIR = os.path.join(REPO_ROOT, "dist")
PLATFORMS_DIR = os.path.join(REPO_ROOT, "platforms")
def _platform_has_pack(platform_name: str) -> bool:
"""Check if a pack ZIP exists for the platform."""
if not os.path.isdir(DIST_DIR):
return False
sys.path.insert(0, os.path.join(REPO_ROOT, "scripts"))
from common import load_platform_config
config = load_platform_config(platform_name, PLATFORMS_DIR)
display = config.get("platform", platform_name).replace(" ", "_")
return any(
f.endswith("_BIOS_Pack.zip") and display in f
for f in os.listdir(DIST_DIR)
)
class PackIntegrityTest(unittest.TestCase):
"""Verify each platform pack via generate_pack.py --verify-packs."""
def _verify_platform(self, platform_name: str) -> None:
if not _platform_has_pack(platform_name):
self.skipTest(f"no pack found for {platform_name}")
result = subprocess.run(
[sys.executable, "scripts/generate_pack.py",
"--platform", platform_name,
"--verify-packs", "--output-dir", "dist/"],
capture_output=True, text=True, cwd=REPO_ROOT,
)
if result.returncode != 0:
self.fail(
f"{platform_name} pack integrity failed:\n"
f"{result.stdout}\n{result.stderr}"
)
def test_retroarch(self):
self._verify_platform("retroarch")
def test_batocera(self):
self._verify_platform("batocera")
def test_bizhawk(self):
self._verify_platform("bizhawk")
def test_emudeck(self):
self._verify_platform("emudeck")
def test_recalbox(self):
self._verify_platform("recalbox")
def test_retrobat(self):
self._verify_platform("retrobat")
def test_retrodeck(self):
self._verify_platform("retrodeck")
def test_romm(self):
self._verify_platform("romm")
if __name__ == "__main__":
unittest.main()