kmoody/libretro
1
0 Fork 0
mirror of https://github.com/Abdess/retroarch_system.git synced 2026-10-10 13:33:24 -05:00
Code Issues Packages Projects Releases Wiki Activity

feat: make the native export round-trip a platform file

Browse Source
This commit is contained in:
Abdessamad Derraz committed 2026-09-05 17:32:48 +02:00
1 parent 7b93285e1d
commit 691ccbfca7
65 files changed
+18417 -2115
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/validate.yml
+8
View File
@@ -115,6 +115,14 @@ jobs:
- name: Install dependencies
run: pip install pyyaml jsonschema==4.23.0
# The exporter fidelity cases compare what we hand back against the
# file each platform publishes, so they need those files. They skip
# without them rather than fail, which is why an unreachable upstream
# must not stop the run: the rest of the suite still has to report.
- name: Fetch the platforms' own files
continue-on-error: true
run: python scripts/export_native.py --all --fetch --output-dir "$RUNNER_TEMP/native"
- name: Run test suite
run: python -m unittest discover tests -v
README.md
+5 -5
View File
@@ -63,7 +63,7 @@ These are the files an emulator loads from disk instead of carrying inside itsel
Each file is checked the way your platform checks it. Most compare a checksum, the fingerprint of a file's contents, which catches a corrupt or unexpected copy. RetroArch, Lakka and RetroPie only look for the filename, because that is all their code does: the Coverage table says which applies to you. Independently of that, the collection records five fingerprints per file, and wherever an emulator's code states an expected size or hash, that value is read from its source and rechecked here.
- **6 files** the platforms' emulators load are still to be found, and 15 more cannot be sourced at all (per-user keys, user-filled slots, dumps nobody has made); both are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/)
- **6 files** the platforms' emulators load are still to be found, and 21 more cannot be sourced at all (per-user keys, user-filled slots, dumps nobody has made); both are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/)
- **12 platforms** supported with platform-specific verification
- **449 emulators** profiled from source (RetroArch cores + standalone)
- **470 systems** handled by those emulators (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
@@ -83,15 +83,15 @@ Full list with per-file details: **[https://abdess.github.io/retrobios/](https:/
|----------|-----------------:|-------------------------:|------------|
| Batocera | 353/353 | 1,299/1,316 | MD5 hash |
| BizHawk | 118/118 | 370/375 | SHA1 hash |
| EmuDeck | 161/161 | 427/427 | MD5 hash |
| EmuDeck | 166/168 | 427/427 | MD5 hash |
| Lakka | 530/530 | 1,206/1,224 | file presence |
| MiSTer FPGA | 72/72 | - | MD5 hash |
| ROCKNIX | 38/38 | 1,636/1,654 | MD5 hash |
| Recalbox | 346/346 | 828/843 | MD5 hash |
| Recalbox | 351/351 | 827/842 | MD5 hash |
| RetroArch | 530/530 | 1,206/1,224 | file presence |
| RetroBat | 343/343 | 938/953 | MD5 hash |
| RetroDECK | 2,008/2,008 | 1,250/1,269 | MD5 hash |
| RetroPie * | 530/530 | 1,206/1,224 | file presence |
| RetroPie * | 530/530 | 1,529/1,554 | file presence |
| RomM | 381/381 | 284/293 | MD5 hash |
Each fraction is what the pack has over what is needed, counting required and optional files alike since both ship. The first column is the BIOS list the platform publishes. The second counts files its emulators load that this list never mentions, found by reading their source code, and it is routinely several times larger. A short fraction means files are still missing, and they are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/).
@@ -174,4 +174,4 @@ The scripts and tooling are released under the [MIT License](LICENSE).
The BIOS and firmware files are not covered by that license: they are third-party system software, preserved and provided for personal backup, archival, and interoperability with emulation software. [NOTICE](NOTICE) sets out their status and how to ask for a file to be removed.
The reasoning, and where it is weakest, is in the [FAQ](https://abdess.github.io/retrobios/wiki/faq/#is-this-legal).
*Auto-generated on 2026-09-05T07:33:35Z*
*Auto-generated on 2026-09-05T11:57:09Z*
database.json
+19 -28
View File
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"generated_at": "2026-09-05T07:19:53Z",
"generated_at": "2026-09-05T09:26:20Z",
"total_files": 10330,
"total_size": 13322705446,
"files": {
@@ -148048,12 +148048,18 @@
"bios7.bin": [
"24f67bdea115a2c847c8813a262502ee1607b7df"
],
"nintendo-ds:df692a80a5b1bc90728bc3dfc76cd948": [
"24f67bdea115a2c847c8813a262502ee1607b7df"
],
"nds9.bin": [
"bfaac75f101c135e32e2aaf541de6b1be4c8c62d"
],
"bios9.bin": [
"bfaac75f101c135e32e2aaf541de6b1be4c8c62d"
],
"nintendo-ds:a392174eb3e572fed6447e956bde4b25": [
"bfaac75f101c135e32e2aaf541de6b1be4c8c62d"
],
"cx4.data.rom": [
"a002f4efba42775a31185d443f3ed1790b0e949a"
],
@@ -149308,6 +149314,9 @@
"dc_boot.bin": [
"8951d1bb219ab2ff8583033d2119c899cc81f18c"
],
"sega-dreamcast:e10c53c2f8b90bab96ead2d368858623": [
"8951d1bb219ab2ff8583033d2119c899cc81f18c"
],
"boot.bin": [
"8951d1bb219ab2ff8583033d2119c899cc81f18c"
],
@@ -150273,6 +150282,15 @@
"sega-saturn:0306c0e408d6682dd2d86324bd4ac661": [
"8c031bf9908fd0142fdd10a9cdd79389f8a3f2fc"
],
"sega-dreamcast:0a93f7940c455905bea6e392dfde92a4": [
"94d44d7f9529ec1642ba3771ed3c5f756d5bc872"
],
"flash.bin": [
"94d44d7f9529ec1642ba3771ed3c5f756d5bc872"
],
"nintendo-ds:145eaef5bd3037cbc247c213bb3da1b3": [
"8497afdd16af92d6887b71582846c40514e9039f"
],
"bsx_bios.rom": [
"4891d739a8a8b67923681bad4fb67edab2e90e50"
],
@@ -150333,9 +150351,6 @@
"squirreljme-slow.jar": [
"8a2ddc37df2366d4e205e9904cd8bac22db2afd5"
],
"flash.bin": [
"94d44d7f9529ec1642ba3771ed3c5f756d5bc872"
],
"a1000kbd_fr.zip": [
"93d36de4d5d376c521a9ca99722f659e60d177f2"
],
@@ -151693,30 +151708,6 @@
"naomi2_bios.bin.bin": [
"a8b7dce572b74e02e65cb949b2c366c87625157f"
],
"amrdecoder_64.dll": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"amrdecoder_arm64.dll": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"libamrdecoder.so": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"libamrdecoder_64.so": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"libamrdecoder_arm32.so": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"libamrdecoder_arm64.so": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"libamrdecoder.dylib": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"libamrdecoder_arm64.dylib": [
"aedd333c565ccf05fb062289e5fab4722883712c"
],
"FluidR3_GM.sf2": [
"22bcfeb5b6c6481569b90db96aa3f4b5f06c8848"
],
install/batocera.json
+12 -42
View File
@@ -5,7 +5,7 @@
"platform": "batocera",
"display_name": "Batocera",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "bios",
"detect": [
{
@@ -237,9 +237,9 @@
}
}
],
"total_files": 1833,
"total_size": 4315260164,
"total_omitted": 11,
"total_files": 1832,
"total_size": 4314998020,
"total_omitted": 9,
"omitted_files": [
{
"dest": "Enterprise/basic21.bin",
@@ -282,13 +282,13 @@
]
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"required": true,
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"nxengine"
"virtualjaguar"
]
},
{
@@ -322,26 +322,6 @@
"required": true,
"reason": "hash_mismatch",
"cores": null
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
}
],
"files": [
@@ -3465,16 +3445,6 @@
"beetle_psx"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bk/TERAK.ROM",
"sha1": "273a9933b68a290c5aedcd6d69faa7b1d22c0344",
@@ -6047,10 +6017,10 @@
},
{
"dest": "MSX/MSX.ROM",
"sha1": "e998f0c441f4f1800ef44e42cd1659150206cf79",
"sha256": "1c85dac5536fa3ba6f2cb70deba02ff680b34ac6cc787d2977258bd663a99555",
"sha1": "409e82adac40f6bdd18eb6c84e8b2fbdc7fb5498",
"sha256": "999564a371dd2fdf7fbe8d853e82a68d557c27b7d87417639b2fa17704b83f78",
"size": 32768,
"repo_path": "bios/Microsoft/MSX/MSX_8020-20bios.rom",
"repo_path": "bios/Microsoft/MSX/MSX.ROM",
"cores": [
"clk"
]
install/bizhawk.json
+16 -15
View File
@@ -5,7 +5,7 @@
"platform": "bizhawk",
"display_name": "BizHawk",
"version": "1.0",
"generated": "2026-08-12T10:39:36Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "Firmware",
"detect": [
{
@@ -20,10 +20,21 @@
}
],
"standalone_copies": [],
"total_files": 540,
"total_size": 2328953247,
"total_omitted": 0,
"omitted_files": [],
"total_files": 539,
"total_size": 2328691103,
"total_omitted": 1,
"omitted_files": [
{
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"virtualjaguar"
]
}
],
"files": [
{
"dest": "panafz1.bin",
@@ -1221,16 +1232,6 @@
"beetle_saturn"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "dsp1.program.rom",
"sha1": "c5d283604beebb3d832f15a0b3fc60d10dd4bddc",
install/lakka.json
+8 -98
View File
@@ -5,7 +5,7 @@
"platform": "lakka",
"display_name": "Lakka",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "system",
"detect": [
{
@@ -16,98 +16,18 @@
}
],
"standalone_copies": [],
"total_files": 1872,
"total_size": 5865708838,
"total_omitted": 9,
"total_files": 1871,
"total_size": 5865446694,
"total_omitted": 1,
"omitted_files": [
{
"dest": "mkxp-z/Fonts/Fonts",
"name": "Fonts",
"system": "rpgmaker",
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVX/RPGVX",
"name": "RPGVX",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVXAce/RPGVXAce",
"name": "RPGVXAce",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/Standard/Standard",
"name": "Standard",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Postload/Postload",
"name": "Postload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Preload/Preload",
"name": "Preload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
}
],
@@ -4608,16 +4528,6 @@
"beetle_psx"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bk/TERAK.ROM",
"sha1": "273a9933b68a290c5aedcd6d69faa7b1d22c0344",
install/recalbox.json
+53 -53
View File
@@ -5,7 +5,7 @@
"platform": "recalbox",
"display_name": "Recalbox",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "bios",
"detect": [
{
@@ -237,9 +237,9 @@
}
}
],
"total_files": 1326,
"total_size": 3898770140,
"total_omitted": 5,
"total_files": 1328,
"total_size": 3900250204,
"total_omitted": 4,
"omitted_files": [
{
"dest": "Machines/MSX2+ - European/Msx2pe.rom",
@@ -258,33 +258,21 @@
"cores": null
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"dest": "geolith/neocd.zip",
"name": "neocd.zip",
"system": "snk-neogeo-cd",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
"reason": "hash_mismatch",
"cores": null
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
}
],
@@ -2417,14 +2405,6 @@
"repo_path": "bios/Sega/Saturn/sega_100.bin",
"cores": null
},
{
"dest": "saturn/stvbios.zip",
"sha1": "7783ab663bb1a8d62ff60d583674355c2af556b1",
"sha256": "38ceb2792c72770f56aadb1865cf66dc33a3017db1ee05e11c2b2dff397ad39e",
"size": 2955630,
"repo_path": "bios/Arcade/MAME/stvbios.zip",
"cores": null
},
{
"dest": "saturn/sega_101.bin",
"sha1": "df94c5b4d47eb3cc404d88b33a8fda237eaf4720",
@@ -2689,6 +2669,14 @@
"repo_path": "bios/SNK/Neo Geo CD/ng-lo.rom",
"cores": null
},
{
"dest": "geolith/neocdz.zip",
"sha1": "80c9a25ccf39c6b85d1a7c9c0d6fe527ab45ec1c",
"sha256": "ce9df4f0e7706c485713a143802ddcc43e13c533944ef29054bfe1cbab5f0c5b",
"size": 598775,
"repo_path": "bios/SNK/Neo Geo CD/neocdz.zip",
"cores": null
},
{
"dest": "scph5500.bin",
"sha1": "b05def971d8ec59f346f2d9ac21fb742e3eb6917",
@@ -2841,6 +2829,38 @@
"repo_path": "bios/Microsoft/MSX/SVI - Spectravideo SVI-328/svi328.rom",
"cores": null
},
{
"dest": "stv/epr-20091.ic8",
"sha1": "eff0f54c70bce05ff3a289bf30b1027e1c8cd117",
"sha256": "ac778ec04aaa4df296d30743536da3de31281f8ae5c94d7be433dcc84e25d85b",
"size": 524288,
"repo_path": "bios/Sega/ST-V/epr-20091.ic8",
"cores": null
},
{
"dest": "stv/epr-17952a.ic8",
"sha1": "eaf1c3e5d602e1139d2090a78d7e19f04f916794",
"sha256": "bac5a52794cf424271f073df228e0b0eb042dede6a3b829eb49abf155e7e0137",
"size": 524288,
"repo_path": "bios/Sega/ST-V/epr-17952a.ic8",
"cores": null
},
{
"dest": "stv/epr-17954a.ic8",
"sha1": "af79cff317e5b57d49e463af16a9f616ed1eee08",
"sha256": "3e6f91506031badc4ebdf7fe5b4f33180222a369b575522861688d3b27322a68",
"size": 524288,
"repo_path": "bios/Sega/ST-V/epr-17954a.ic8",
"cores": null
},
{
"dest": "stv/stvbios.zip",
"sha1": "7783ab663bb1a8d62ff60d583674355c2af556b1",
"sha256": "38ceb2792c72770f56aadb1865cf66dc33a3017db1ee05e11c2b2dff397ad39e",
"size": 2955630,
"repo_path": "bios/Arcade/MAME/stvbios.zip",
"cores": null
},
{
"dest": "sufami/STBIOS.bin",
"sha1": "ef86ea192eed03d5c413fdbbfd46043be1d7a127",
@@ -4081,16 +4101,6 @@
"beetle_psx"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bk/BASIC10.ROM",
"sha1": "af7eae90c398ec12942a408fa5ac2722e331738c",
@@ -6439,16 +6449,6 @@
"genesis_plus_gx"
]
},
{
"dest": "neocd.zip",
"sha1": "cef6a6e78989a595700337c15b79155523935333",
"sha256": "a0cdb87bb7753c9a1a1d99b83f58ace4bcd54dee41211d9d2c96fe06098b3590",
"size": 429431,
"repo_path": "bios/SNK/Neo Geo CD/neocd.zip",
"cores": [
"geolith"
]
},
{
"dest": "irrmaze.zip",
"sha1": "0a8aa300042cdc8fdeb825ddaa840b59fad63f92",
install/retroarch.json
+8 -98
View File
@@ -5,7 +5,7 @@
"platform": "retroarch",
"display_name": "RetroArch",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "system",
"detect": [
{
@@ -255,98 +255,18 @@
}
}
],
"total_files": 1872,
"total_size": 5865708838,
"total_omitted": 9,
"total_files": 1871,
"total_size": 5865446694,
"total_omitted": 1,
"omitted_files": [
{
"dest": "mkxp-z/Fonts/Fonts",
"name": "Fonts",
"system": "rpgmaker",
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVX/RPGVX",
"name": "RPGVX",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVXAce/RPGVXAce",
"name": "RPGVXAce",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/Standard/Standard",
"name": "Standard",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Postload/Postload",
"name": "Postload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Preload/Preload",
"name": "Preload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
}
],
@@ -4847,16 +4767,6 @@
"beetle_psx"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bk/TERAK.ROM",
"sha1": "273a9933b68a290c5aedcd6d69faa7b1d22c0344",
install/retrobat.json
+6 -26
View File
@@ -5,7 +5,7 @@
"platform": "retrobat",
"display_name": "RetroBat",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "bios",
"detect": [
{
@@ -238,7 +238,7 @@
],
"total_files": 1464,
"total_size": 4733583386,
"total_omitted": 5,
"total_omitted": 3,
"omitted_files": [
{
"dest": "SEGA_KakuGothic-DB-Roman_12.abc",
@@ -261,33 +261,13 @@
]
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
}
],
install/retrodeck.json
+13 -103
View File
@@ -5,7 +5,7 @@
"platform": "retrodeck",
"display_name": "RetroDECK",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "",
"detect": [
{
@@ -237,9 +237,9 @@
}
}
],
"total_files": 3517,
"total_size": 6656707765,
"total_omitted": 11,
"total_files": 3515,
"total_size": 6656314549,
"total_omitted": 4,
"omitted_files": [
{
"dest": "bios/dc/naomi2.zip",
@@ -250,93 +250,21 @@
"cores": null
},
{
"dest": "bios/mkxp-z/Fonts/Fonts",
"name": "Fonts",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "bios/mkxp-z/RTP/RPGVX/RPGVX",
"name": "RPGVX",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "bios/mkxp-z/RTP/RPGVXAce/RPGVXAce",
"name": "RPGVXAce",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "bios/mkxp-z/RTP/Standard/Standard",
"name": "Standard",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "bios/mkxp-z/Scripts/Postload/Postload",
"name": "Postload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "bios/mkxp-z/Scripts/Preload/Preload",
"name": "Preload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "bios/nxengine/data/data",
"name": "data/",
"system": "cave-story",
"dest": "bios/dsifirmware.bin",
"name": "dsifirmware.bin",
"system": "nintendo-ds",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
"reason": "hash_mismatch",
"cores": null
},
{
"dest": "bios/rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"dest": "bios/jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "bios/rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
},
{
@@ -14421,14 +14349,6 @@
"repo_path": "bios/Nintendo/Nintendo DS/biosdsi9.bin",
"cores": null
},
{
"dest": "bios/dsifirmware.bin",
"sha1": "3773f52559d5ac4fc6d8aefe35bce58730ae8181",
"sha256": "a52e281c4dfc8f2da61e7a193d0f40e15eff6f6142ba80bf48ffe755624e2017",
"size": 131072,
"repo_path": "bios/Nintendo/Nintendo DS/dsifirmware.bin",
"cores": null
},
{
"dest": "bios/gb_bios.bin",
"sha1": "4ed31ec6b0b175bb109c0eb5fd3d193da823339f",
@@ -16739,16 +16659,6 @@
"beetle_psx"
]
},
{
"dest": "bios/satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bios/bk/MONIT10.ROM",
"sha1": "4e83a94ae5155bbea14d7331a5a8db82457bd5ae",
install/retropie.json
+8 -98
View File
@@ -5,7 +5,7 @@
"platform": "retropie",
"display_name": "RetroPie",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "system",
"detect": [
{
@@ -15,98 +15,18 @@
}
],
"standalone_copies": [],
"total_files": 1872,
"total_size": 5865708838,
"total_omitted": 9,
"total_files": 1871,
"total_size": 5865446694,
"total_omitted": 1,
"omitted_files": [
{
"dest": "mkxp-z/Fonts/Fonts",
"name": "Fonts",
"system": "rpgmaker",
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVX/RPGVX",
"name": "RPGVX",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVXAce/RPGVXAce",
"name": "RPGVXAce",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/Standard/Standard",
"name": "Standard",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Postload/Postload",
"name": "Postload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Preload/Preload",
"name": "Preload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
}
],
@@ -4607,16 +4527,6 @@
"beetle_psx"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bk/TERAK.ROM",
"sha1": "273a9933b68a290c5aedcd6d69faa7b1d22c0344",
install/rocknix.json
+8 -98
View File
@@ -5,7 +5,7 @@
"platform": "rocknix",
"display_name": "ROCKNIX",
"version": "1.0",
"generated": "2026-09-03T20:45:52Z",
"generated": "2026-09-05T09:26:20Z",
"base_destination": "bios",
"detect": [
{
@@ -17,98 +17,18 @@
}
],
"standalone_copies": [],
"total_files": 1739,
"total_size": 5743738994,
"total_omitted": 9,
"total_files": 1738,
"total_size": 5743476850,
"total_omitted": 1,
"omitted_files": [
{
"dest": "mkxp-z/Fonts/Fonts",
"name": "Fonts",
"system": "rpgmaker",
"dest": "jagboot.rom",
"name": "jagboot.rom",
"system": "atari-jaguar",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVX/RPGVX",
"name": "RPGVX",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/RPGVXAce/RPGVXAce",
"name": "RPGVXAce",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/RTP/Standard/Standard",
"name": "Standard",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Postload/Postload",
"name": "Postload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "mkxp-z/Scripts/Preload/Preload",
"name": "Preload",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"mkxp_z"
]
},
{
"dest": "nxengine/data/data",
"name": "data/",
"system": "cave-story",
"required": true,
"reason": "not_found",
"cores": [
"nxengine"
]
},
{
"dest": "rtp/2000/RPG Maker 2000 RTP",
"name": "RPG Maker 2000 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
]
},
{
"dest": "rtp/2003/RPG Maker 2003 RTP",
"name": "RPG Maker 2003 RTP",
"system": "rpgmaker",
"required": false,
"reason": "not_found",
"cores": [
"easyrpg"
"virtualjaguar"
]
}
],
@@ -911,16 +831,6 @@
"beetle_psx"
]
},
{
"dest": "satar4mp.bin",
"sha1": "11f9e62cf299f72184835b7b2a70a16333fc0d88",
"sha256": "87cddb1f499e32758de20145e73031a84bab299e3f6e5c8487e76d02b2ee9d16",
"size": 262144,
"repo_path": "bios/Commodore/Amiga/amiga-os-120.rom",
"cores": [
"beetle_saturn"
]
},
{
"dest": "bk/MONIT10.ROM",
"sha1": "4e83a94ae5155bbea14d7331a5a8db82457bd5ae",
platforms/_registry.yml
+4 -2
View File
@@ -956,8 +956,10 @@ platforms:
config: retropie.yml
status: archived
logo: https://avatars.githubusercontent.com/u/11378204
scraper: null
cores: all_libretro
scraper: retropie
source: https://github.com/RetroPie/RetroPie-Setup
# No cores here: the registry overrides the config, and RetroPie's own
# scriptmodules are what say which packages it ships.
schedule: null
target_scraper: retropie_targets
target_source: https://retropie.org.uk/stats/pkgflags/
platforms/batocera.yml
+2076
View File
File diff suppressed because it is too large. Load diff
platforms/bizhawk.yml
+121
View File
@@ -41,24 +41,29 @@ systems:
required: true
sha1: dbebd76a448447cb6e524ac3cb0fd19fc065d944
size: 256
native_system: 32X
- name: 32X_M_BIOS.BIN
destination: 32X_M_BIOS.BIN
required: true
sha1: 1e5b0b2441a4979b6966d942b20cc76c413b8c5e
size: 2048
native_system: 32X
- name: 32X_S_BIOS.BIN
destination: 32X_S_BIOS.BIN
required: true
sha1: 4103668c1bbd66c5e24558e73d4f3f92061a109a
size: 1024
native_system: 32X
nintendo-3ds:
files:
- name: aes_keys.txt
destination: aes_keys.txt
required: true
native_system: 3DS
- name: seeddb.bin
destination: seeddb.bin
required: true
native_system: 3DS
atari-7800:
files:
- name: A78_PAL_BIOS.bin
@@ -66,16 +71,19 @@ systems:
required: true
sha1: 5a140136a16d1d83e4ff32a19409ca376a8df874
size: 16384
native_system: A78
- name: A78_highscore.bin
destination: A78_highscore.bin
required: true
sha1: a3af676991391a6dd716c79022d4947206b78164
size: 4096
native_system: A78
- name: A78_NTSC_bios.bin
destination: A78_NTSC_bios.bin
required: true
sha1: d9d134bb6b36907c615a594cc7688f7bfcef5b43
size: 4096
native_system: A78
commodore-c64:
files:
- name: C64_Kernal.bin
@@ -83,26 +91,31 @@ systems:
required: true
sha1: 1d503e56df85a62fee696e7618dc5b4e781df1bb
size: 8192
native_system: C64
- name: C64_Basic.bin
destination: C64_Basic.bin
required: true
sha1: 79015323128650c742a3694c9429aa91f355905e
size: 8192
native_system: C64
- name: C64_Chargen.bin
destination: C64_Chargen.bin
required: true
sha1: adc7c31e18c7c7413d54802ef2f4193da14711aa
size: 4096
native_system: C64
- name: C64_Drive-1541.bin
destination: C64_Drive-1541.bin
required: true
sha1: ab16f56989b27d89babe5f89c5a8cb3da71a82f0
size: 16384
native_system: C64
- name: C64_Drive-1541ii.bin
destination: C64_Drive-1541ii.bin
required: true
sha1: d3b78c3dbac55f5199f33f3fe0036439811f7fb3
size: 16384
native_system: C64
fairchild-channel-f:
files:
- name: ChannelF_SL31253.rom
@@ -110,16 +123,19 @@ systems:
required: true
sha1: 81193965a374d77b99b4743d317824b53c3e3c78
size: 1024
native_system: ChannelF
- name: ChannelF_SL31254.rom
destination: ChannelF_SL31254.rom
required: true
sha1: 8f70d1b74483ba3a37e86cf16c849d601a8c3d2c
size: 1024
native_system: ChannelF
- name: ChannelF_sl90025.rom
destination: ChannelF_sl90025.rom
required: true
sha1: 759e2ed31fbde4a2d8daf8b9f3e0dffebc90dae2
size: 1024
native_system: ChannelF
coleco-colecovision:
files:
- name: Coleco_Bios.bin
@@ -127,6 +143,7 @@ systems:
required: true
sha1: 45bedc4cbdeac66c7df59e9e599195c778d86a92
size: 8192
native_system: Coleco
doom:
files:
- name: doom2.wad
@@ -134,6 +151,7 @@ systems:
required: true
sha1: 7ec7652fcfce8ddc6e801839291f0e28ef1d5ae7
size: 14604584
native_system: Doom
sega-mega-drive:
files:
- name: GEN_boot.md
@@ -141,21 +159,25 @@ systems:
required: true
sha1: 3f50b76b0529db7f79c396b5e808cc0786ffc311
size: 2048
native_system: GEN
- name: MCD_eu_100.bin
destination: MCD_eu_100.bin
required: true
sha1: f891e0ea651e2232af0c5c4cb46a0cae2ee8f356
size: 131072
native_system: GEN
- name: MCD_jp_100p.bin
destination: MCD_jp_100p.bin
required: true
sha1: 4846f448160059a7da0215a5df12ca160f26dd69
size: 131072
native_system: GEN
- name: SCD_m1_us_110.bin
destination: SCD_m1_us_110.bin
required: true
sha1: f4f315adcef9b8feb0364c21ab7f0eaf5457f3ed
size: 131072
native_system: GEN
sega-game-gear:
files:
- name: GG_majesco.gg
@@ -163,6 +185,7 @@ systems:
required: true
sha1: 914aa165e3d879f060be77870d345b60cfeb4ede
size: 1024
native_system: GG
mattel-intellivision:
files:
- name: INTV_EROM.bin
@@ -170,11 +193,13 @@ systems:
required: true
sha1: 5a65b922b562cb1f57dab51b73151283f0e20c7a
size: 8192
native_system: INTV
- name: INTV_GROM.bin
destination: INTV_GROM.bin
required: true
sha1: f9608bb4ad1cfe3640d02844c7ad8e0bcd974917
size: 2048
native_system: INTV
atari-lynx:
files:
- name: Atari_LYNX_boot.img
@@ -182,6 +207,7 @@ systems:
required: true
sha1: e4ed47fae31693e016b081c6bda48da5b70d7ccb
size: 512
native_system: Lynx
microsoft-msx:
files:
- name: MSX_cbios_main_msx1.rom
@@ -189,31 +215,37 @@ systems:
required: true
sha1: 2f997e8a57528518c82ab3693fdae243dbbcc508
size: 32768
native_system: MSX
- name: MSX_8020-20bios.rom
destination: MSX_8020-20bios.rom
required: true
sha1: e998f0c441f4f1800ef44e42cd1659150206cf79
size: 16384
native_system: MSX
- name: MSX_4000bios.rom
destination: MSX_4000bios.rom
required: true
sha1: df48902f5f12af8867ae1a87f255145f0e5e0774
size: 16384
native_system: MSX
- name: MSX.rom
destination: MSX.rom
required: true
sha1: 409e82adac40f6bdd18eb6c84e8b2fbdc7fb5498
size: 32768
native_system: MSX
- name: MSX_1.0.rom
destination: MSX_1.0.rom
required: true
sha1: 3656bb3bbc17d280d2016fe4f6ff3cded3082a41
size: 32768
native_system: MSX
- name: MSX_jpn.rom
destination: MSX_jpn.rom
required: true
sha1: 302afb5d8be26c758309ca3df611ae69cced2821
size: 32768
native_system: MSX
nintendo-64dd:
files:
- name: 64DD_IPL_DEV.bin
@@ -221,16 +253,19 @@ systems:
required: true
sha1: 10c4173c2a7eb09c6579818f72ef18fa0b6d32de
size: 4194304
native_system: N64DD
- name: 64DD_IPL_USA.bin
destination: 64DD_IPL_USA.bin
required: true
sha1: 3c5b93ca231550c68693a14f03cea8d5dbd1be9e
size: 4194304
native_system: N64DD
- name: 64DD_IPL_v12_JPN.bin
destination: 64DD_IPL_v12_JPN.bin
required: true
sha1: bf861922dcb78c316360e3e742f4f70ff63c9bc3
size: 4194304
native_system: N64DD
nintendo-ds:
files:
- name: NDS_Bios7.bin
@@ -238,49 +273,60 @@ systems:
required: true
sha1: 24f67bdea115a2c847c8813a262502ee1607b7df
size: 16384
native_system: NDS
- name: NDS_Bios9.bin
destination: NDS_Bios9.bin
required: true
sha1: bfaac75f101c135e32e2aaf541de6b1be4c8c62d
size: 4096
native_system: NDS
- name: NDS_Bios7i.bin
destination: NDS_Bios7i.bin
required: true
sha1: c7c7570bfe51c3c7c5da3b01331b94e7e7cb4f53
size: 65536
native_system: NDS
- name: NDS_Bios9i.bin
destination: NDS_Bios9i.bin
required: true
sha1: 719b9eef33692406d7170ff526069615759c4dfc
size: 65536
native_system: NDS
- name: DSi_Firmware.bin
destination: DSi_Firmware.bin
required: true
size: 131072
native_system: NDS
- name: DSi_Nand_JPN.bin
destination: DSi_Nand_JPN.bin
required: true
size: 251658304
native_system: NDS
- name: DSi_Nand_EUR.bin
destination: DSi_Nand_EUR.bin
required: true
size: 251658304
native_system: NDS
- name: DSi_Nand_USA.bin
destination: DSi_Nand_USA.bin
required: true
size: 251658304
native_system: NDS
- name: DSi_Nand_AUS.bin
destination: DSi_Nand_AUS.bin
required: true
size: 251658304
native_system: NDS
- name: DSi_Nand_CHN.bin
destination: DSi_Nand_CHN.bin
required: true
size: 251658304
native_system: NDS
- name: DSi_Nand_KOR.bin
destination: DSi_Nand_KOR.bin
required: true
size: 251658304
native_system: NDS
philips-videopac:
files:
- name: O2_Odyssey2.bin
@@ -288,16 +334,19 @@ systems:
required: true
sha1: b2e1955d957a475de2411770452eff4ea19f4cee
size: 1024
native_system: O2
- name: O2_PhillipsC52.bin
destination: O2_PhillipsC52.bin
required: true
sha1: a6120aed50831c9c0d95dbdf707820f601d9452e
size: 1024
native_system: O2
- name: O2_G7400_bios.bin
destination: O2_G7400_bios.bin
required: true
sha1: 5130243429b40b01a14e1304d0394b8459a6fbae
size: 1024
native_system: O2
nec-pc-engine-cd:
files:
- name: "PCECD_3.0-(J).pce"
@@ -305,11 +354,13 @@ systems:
required: true
sha1: 79f5ff55dd10187c7fd7b8daab0b3ffbd1f56a2c
size: 262144
native_system: PCECD
- name: PCECD_gecard.pce
destination: PCECD_gecard.pce
required: true
sha1: 014881a959e045e00f4db8f52955200865d40280
size: 32768
native_system: PCECD
sega-saturn:
files:
- name: SAT_KoF95.bin
@@ -317,41 +368,49 @@ systems:
required: true
sha1: a67cd4f550751f8b91de2b8b74528ab4e0c11c77
size: 2097152
native_system: SAT
- name: SAT_Ultraman.bin
destination: SAT_Ultraman.bin
required: true
sha1: 56c1b93da6b660bf393fbf48ca47569000ef4047
size: 2097152
native_system: SAT
- name: "SAT_1.01-(J).bin"
destination: "SAT_1.01-(J).bin"
required: true
sha1: df94c5b4d47eb3cc404d88b33a8fda237eaf4720
size: 524288
native_system: SAT
- name: "SAT_1.00-(U+E).bin"
destination: "SAT_1.00-(U+E).bin"
required: true
sha1: faa8ea183a6d7bbe5d4e03bb1332519800d3fbc3
size: 524288
native_system: SAT
- name: "SAT_1.00-(U+E).bin"
destination: "SAT_1.00-(U+E).bin"
required: true
sha1: faa8ea183a6d7bbe5d4e03bb1332519800d3fbc3
size: 524288
native_system: SAT
- name: "SAT_STV_20091-(J).bin"
destination: "SAT_STV_20091-(J).bin"
required: true
sha1: eff0f54c70bce05ff3a289bf30b1027e1c8cd117
size: 524288
native_system: SAT
- name: "SAT_STV_17952A-(U).bin"
destination: "SAT_STV_17952A-(U).bin"
required: true
sha1: eaf1c3e5d602e1139d2090a78d7e19f04f916794
size: 524288
native_system: SAT
- name: "SAT_STV_17954A-(E).bin"
destination: "SAT_STV_17954A-(E).bin"
required: true
sha1: af79cff317e5b57d49e463af16a9f616ed1eee08
size: 524288
native_system: SAT
nintendo-snes:
files:
- name: SNES_cx4.rom
@@ -359,56 +418,67 @@ systems:
required: true
sha1: a002f4efba42775a31185d443f3ed1790b0e949a
size: 3072
native_system: SNES
- name: SNES_dsp1.rom
destination: SNES_dsp1.rom
required: true
sha1: 188d471fefea71eb53f0ee7064697ff0971b1014
size: 8192
native_system: SNES
- name: SNES_dsp1b.rom
destination: SNES_dsp1b.rom
required: true
sha1: 78b724811f5f18d8c67669d9390397eb1a47a5e2
size: 8192
native_system: SNES
- name: SNES_dsp2.rom
destination: SNES_dsp2.rom
required: true
sha1: 198c4b1c3bfc6d69e734c5957af3dbfa26238dfb
size: 8192
native_system: SNES
- name: SNES_dsp3.rom
destination: SNES_dsp3.rom
required: true
sha1: 558da7cb3bd3876a6ca693661ffc6c110e948cf9
size: 8192
native_system: SNES
- name: SNES_dsp4.rom
destination: SNES_dsp4.rom
required: true
sha1: af6478aecb6f1b67177e79c82ca04c56250a8c72
size: 8192
native_system: SNES
- name: SNES_st010.rom
destination: SNES_st010.rom
required: true
sha1: 6472828403de3589433a906e2c3f3d274c0ff008
size: 53248
native_system: SNES
- name: SNES_st011.rom
destination: SNES_st011.rom
required: true
sha1: fecbae2cec76c710422486baa186ffa7ca1cf925
size: 53248
native_system: SNES
- name: SNES_st018.rom
destination: SNES_st018.rom
required: true
sha1: 91383b92745cc7cc4f15409ac5bc2c2f699a43f1
size: 163840
native_system: SNES
- name: "SNES_SGB_(JU).sfc"
destination: "SNES_SGB_(JU).sfc"
required: true
sha1: 6ed55c4368333b57f6a2f8bbd70ccd87ed48058e
size: 262144
native_system: SNES
- name: "SNES_SGB2_(J).sfc"
destination: "SNES_SGB2_(J).sfc"
required: true
sha1: e5b2922ca137051059e4269b236d07a22c07bc84
size: 524288
native_system: SNES
nintendo-satellaview:
files:
- name: Satellaview_BS-X.sfc
@@ -416,46 +486,55 @@ systems:
required: true
sha1: 604556b2e62860af18db5a77f2956ebc75450020
size: 1048576
native_system: BSX
- name: Satellaview_Mahjong.sfc
destination: Satellaview_Mahjong.sfc
required: true
sha1: 8417556b5ce22fb9bc4026fe28f7aee348a76fd9
size: 1048576
native_system: BSX
- name: Satellaview_GNEXT.sfc
destination: Satellaview_GNEXT.sfc
required: true
sha1: 356a3d60e9fabff4b4cec66277c9a6b717c958ce
size: 1572864
native_system: BSX
- name: Satellaview_RPG_Tsukuru.sfc
destination: Satellaview_RPG_Tsukuru.sfc
required: true
sha1: 95f31dfb31200e31e80007cc66a32e2c1a7b02be
size: 2097152
native_system: BSX
- name: Satellaview_SameGame.sfc
destination: Satellaview_SameGame.sfc
required: true
sha1: a6ad0f89c3e75327d70aa0f0239c7b06f151e0a4
size: 1048576
native_system: BSX
- name: Satellaview_Derby_Stallion_96.sfc
destination: Satellaview_Derby_Stallion_96.sfc
required: true
sha1: 24af9a5a78635ec493bbbc2540ab98bd161c5798
size: 3145728
native_system: BSX
- name: Satellaview_Ongaku_Tsukuru.sfc
destination: Satellaview_Ongaku_Tsukuru.sfc
required: true
sha1: 337e0c4bdbf544787b543a1fb83afba7526191b9
size: 1048576
native_system: BSX
- name: Satellaview_SoundNovel_Tsukuru.sfc
destination: Satellaview_SoundNovel_Tsukuru.sfc
required: true
sha1: a9b124948a19bb5a295ec208fca8180476bdfebb
size: 3145728
native_system: BSX
- name: Satellaview_Bass_Tsuri.sfc
destination: Satellaview_Bass_Tsuri.sfc
required: true
sha1: 357ac4826297a6496035e3951cacda55dcae4b1b
size: 4194304
native_system: BSX
gce-vectrex:
files:
- name: VEC_Bios.bin
@@ -463,11 +542,13 @@ systems:
required: true
sha1: b9bbf5bb0eac52d039a4a993a2d8064b862c9e28
size: 4096
native_system: VEC
- name: VEC_Minestorm.vec
destination: VEC_Minestorm.vec
required: true
sha1: 65d07426b520ddd3115d40f255511e0fd2e20ae7
size: 8192
native_system: VEC
sinclair-zx-spectrum:
files:
- name: ZX_pentagon.rom
@@ -475,11 +556,13 @@ systems:
required: true
sha1: a584272f21dc82c14b7d4f1ed440e23a976e71f0
size: 32768
native_system: ZXSpectrum
- name: ZX_trdos.rom
destination: ZX_trdos.rom
required: true
sha1: 282eb7bc819aad2a12fd954e76f7838a4e1a7929
size: 16384
native_system: ZXSpectrum
3do:
files:
- name: panafz1.bin
@@ -487,71 +570,85 @@ systems:
required: true
sha1: 34bf189111295f74d7b7dfc1f304d98b8d36325a
size: 1048576
native_system: 3DO
- name: panafz1e.bin
destination: panafz1e.bin
required: true
sha1: 1d0db81e171ebc1d07cefc8ce8ab082306186e56
size: 1048576
native_system: 3DO
- name: panafz1j.bin
destination: panafz1j.bin
required: true
sha1: ec7ec62d60ec0459a14ed56ebc66761ef3c80efc
size: 1048576
native_system: 3DO
- name: panafz10.bin
destination: panafz10.bin
required: true
sha1: 3c912300775d1ad730dc35757e279c274c0acaad
size: 1048576
native_system: 3DO
- name: panafz10e-anvil.bin
destination: panafz10e-anvil.bin
required: true
sha1: a900371f0cdcdc03f79557f11d406fd71251a5fd
size: 1048576
native_system: 3DO
- name: panafz10j.bin
destination: panafz10j.bin
required: true
sha1: fe7f9c9c6a98910013bf13f2cf798de9fea52acd
size: 1048576
native_system: 3DO
- name: goldstar.bin
destination: goldstar.bin
required: true
sha1: c4a2e5336f77fb5f743de1eea2cda43675ee2de7
size: 1048576
native_system: 3DO
- name: goldstar_fc1_enc.bin
destination: goldstar_fc1_enc.bin
required: true
sha1: 8ef7503c948314d242da47b7fdc272f68dac2aee
size: 1048576
native_system: 3DO
- name: sanyotry.bin
destination: sanyotry.bin
required: true
sha1: b01c53da256dde43ffec4ad3fc3adfa8d635e943
size: 1048576
native_system: 3DO
- name: sanyo_hc21_b3_unenc.bin
destination: sanyo_hc21_b3_unenc.bin
required: true
sha1: c389af32bcadf0d86826927dc3d20b7072f90069
size: 1048576
native_system: 3DO
- name: 3do_arcade_saot.bin
destination: 3do_arcade_saot.bin
required: true
sha1: 520d3d1b5897800af47f92efd2444a26b7a7dead
size: 524288
native_system: 3DO
- name: 3do_devkit_1.0fc2.bin
destination: 3do_devkit_1.0fc2.bin
required: true
sha1: bd325c869e1dde8a3872fc21565e0646a3d5b525
size: 1048576
native_system: 3DO
- name: panafz1-kanji.bin
destination: panafz1-kanji.bin
required: true
sha1: acd39a8fee1b9d2950d5ab447846c11fb31af63e
size: 933636
native_system: 3DO
- name: panafz10ja-anvil-kanji.bin
destination: panafz10ja-anvil-kanji.bin
required: true
sha1: 2e857b957803d0331fd229328df01f3ffab69eee
size: 1048576
native_system: 3DO
commodore-amiga:
files:
- name: amiga-os-130.rom
@@ -559,31 +656,37 @@ systems:
required: true
sha1: 891e9a547772fe0c6c19b610baf8bc4ea7fcb785
size: 262144
native_system: Amiga
- name: amiga-os-205-a600.rom
destination: amiga-os-205-a600.rom
required: true
sha1: 02843c4253bbd29aba535b0aa3bd9a85034ecde4
size: 524288
native_system: Amiga
- name: amiga-os-310-a1200.rom
destination: amiga-os-310-a1200.rom
required: true
sha1: e21545723fe8374e91342617604f1b3d703094f1
size: 524288
native_system: Amiga
- name: amiga-os-310-a4000.rom
destination: amiga-os-310-a4000.rom
required: true
sha1: 5fe04842d04a489720f0f4bb0e46948199406f49
size: 524288
native_system: Amiga
- name: amiga-os-310-cd32.rom
destination: amiga-os-310-cd32.rom
required: true
sha1: 3525be8887f79b5929e017b42380a79edfee542d
size: 524288
native_system: Amiga
- name: amiga-os-310-cd32-ext.rom
destination: amiga-os-310-cd32-ext.rom
required: true
sha1: 5bef3d628ce59cc02a66e6e4ae0da48f60e78f7f
size: 524288
native_system: Amiga
apple-ii:
files:
- name: AppleIIe.rom
@@ -591,11 +694,13 @@ systems:
required: true
sha1: b8ea90abe135a0031065e01697c4a3a20d51198b
size: 16384
native_system: AppleII
- name: AppleIIe_DiskII.rom
destination: AppleIIe_DiskII.rom
required: true
sha1: d4181c9f046aafc3fb326b381baac809d9e38d16
size: 256
native_system: AppleII
nintendo-gba:
files:
- name: GBA_bios.rom
@@ -603,6 +708,7 @@ systems:
required: true
sha1: 300c20df6731a33952ded8c436f7f186d25d3492
size: 16384
native_system: GBA
nintendo-gb:
files:
- name: GB_dmg.bin
@@ -610,16 +716,19 @@ systems:
required: true
sha1: 4ed31ec6b0b175bb109c0eb5fd3d193da823339f
size: 256
native_system: GB
- name: GB_sgb.bin
destination: GB_sgb.bin
required: true
sha1: aa2f50a77dfb4823da96ba99309085a3c6278515
size: 256
native_system: GB
- name: GB_sgb2.bin
destination: GB_sgb2.bin
required: true
sha1: 93407ea10d2f30ab96a314d8eca44fe160aea734
size: 256
native_system: GB
nintendo-gbc:
files:
- name: GBC_cgb.bin
@@ -627,11 +736,13 @@ systems:
required: true
sha1: 1293d68bf9643bc4f36954c1e80e38f39864528d
size: 2304
native_system: GBC
- name: GBC_agb.bin
destination: GBC_agb.bin
required: true
sha1: fa5287e24b0fa533b3b5ef2b28a81245346c1a0f
size: 2304
native_system: GBC
nintendo-nes:
files:
- name: FDS_disksys-nintendo.rom
@@ -639,6 +750,7 @@ systems:
required: true
sha1: 57fe1bdee955bb48d357e463ccbf129496930b62
size: 8192
native_system: NES
nec-pc-fx:
files:
- name: PCFX_bios.bin
@@ -646,11 +758,13 @@ systems:
required: true
sha1: 1a77fd83e337f906aecab27a1604db064cf10074
size: 1048576
native_system: PCFX
- name: PCFX_fx-scsi.rom
destination: PCFX_fx-scsi.rom
required: true
sha1: 65482a23ac5c10a6095aee1db5824cca54ead6e5
size: 524288
native_system: PCFX
sony-playstation:
files:
- name: PSX_DTLH2000.bin
@@ -658,16 +772,19 @@ systems:
required: true
sha1: 1a8d6f9453111b1d317bb7dae300495fbf54600c
size: 524288
native_system: PSX
- name: PSX_DTLH2000.bin
destination: PSX_DTLH2000.bin
required: true
sha1: 1a8d6f9453111b1d317bb7dae300495fbf54600c
size: 524288
native_system: PSX
- name: PSX_DTLH2000.bin
destination: PSX_DTLH2000.bin
required: true
sha1: 1a8d6f9453111b1d317bb7dae300495fbf54600c
size: 524288
native_system: PSX
sega-master-system:
files:
- name: SMS_us_1.3.sms
@@ -675,16 +792,19 @@ systems:
required: true
sha1: c315672807d8ddb8d91443729405c766dd95cae7
size: 8192
native_system: SMS
- name: SMS_jp_2.1.sms
destination: SMS_jp_2.1.sms
required: true
sha1: a8c1b39a2e41137835eda6a5de6d46dd9fadbaf2
size: 8192
native_system: SMS
- name: SMS_kr.sms
destination: SMS_kr.sms
required: true
sha1: 2feafd8f1c40fdf1bd5668f8c5c02e5560945b17
size: 131072
native_system: SMS
texas-instruments-ti-83:
files:
- name: TI83_110.rom
@@ -692,3 +812,4 @@ systems:
required: true
sha1: 25b373b58523647bb7b904001d391615e0b79bee
size: 262144
native_system: TI83
platforms/emudeck.yml
+198
View File
@@ -39,652 +39,850 @@ systems:
- name: scph5500.bin
destination: scph5500.bin
required: true
native_system: sony-playstation
- name: scph5501.bin
destination: scph5501.bin
required: true
native_system: sony-playstation
- name: scph5502.bin
destination: scph5502.bin
required: true
native_system: sony-playstation
- name: "sony-playstation:239665b1a3dade1b5a52c06338011044"
destination: ''
required: true
md5: 239665b1a3dade1b5a52c06338011044
native_system: sony-playstation
- name: "sony-playstation:2118230527a9f51bd9216e32fa912842"
destination: ''
required: true
md5: 2118230527a9f51bd9216e32fa912842
native_system: sony-playstation
- name: "sony-playstation:849515939161e62f6b866f6853006780"
destination: ''
required: true
md5: 849515939161e62f6b866f6853006780
native_system: sony-playstation
- name: "sony-playstation:dc2b9bf8da62ec93e868cfd29f0d067d"
destination: ''
required: true
md5: dc2b9bf8da62ec93e868cfd29f0d067d
native_system: sony-playstation
- name: "sony-playstation:54847e693405ffeb0359c6287434cbef"
destination: ''
required: true
md5: 54847e693405ffeb0359c6287434cbef
native_system: sony-playstation
- name: "sony-playstation:cba733ceeff5aef5c32254f1d617fa62"
destination: ''
required: true
md5: cba733ceeff5aef5c32254f1d617fa62
native_system: sony-playstation
- name: "sony-playstation:da27e8b6dab242d8f91a9b25d80c63b8"
destination: ''
required: true
md5: da27e8b6dab242d8f91a9b25d80c63b8
native_system: sony-playstation
- name: "sony-playstation:417b34706319da7cf001e76e40136c23"
destination: ''
required: true
md5: 417b34706319da7cf001e76e40136c23
native_system: sony-playstation
- name: "sony-playstation:57a06303dfa9cf9351222dfcbb4a29d9"
destination: ''
required: true
md5: 57a06303dfa9cf9351222dfcbb4a29d9
native_system: sony-playstation
- name: "sony-playstation:81328b966e6dcf7ea1e32e55e1c104bb"
destination: ''
required: true
md5: 81328b966e6dcf7ea1e32e55e1c104bb
native_system: sony-playstation
- name: "sony-playstation:924e392ed05558ffdb115408c263dccf"
destination: ''
required: true
md5: 924e392ed05558ffdb115408c263dccf
native_system: sony-playstation
- name: "sony-playstation:e2110b8a2b97a8e0b857a45d32f7e187"
destination: ''
required: true
md5: e2110b8a2b97a8e0b857a45d32f7e187
native_system: sony-playstation
- name: "sony-playstation:ca5cfc321f916756e3f0effbfaeba13b"
destination: ''
required: true
md5: ca5cfc321f916756e3f0effbfaeba13b
native_system: sony-playstation
- name: "sony-playstation:8dd7d5296a650fac7319bce665a6a53c"
destination: ''
required: true
md5: 8dd7d5296a650fac7319bce665a6a53c
native_system: sony-playstation
- name: "sony-playstation:490f666e1afb15b7362b406ed1cea246"
destination: ''
required: true
md5: 490f666e1afb15b7362b406ed1cea246
native_system: sony-playstation
- name: "sony-playstation:32736f17079d0b2b7024407c39bd3050"
destination: ''
required: true
md5: 32736f17079d0b2b7024407c39bd3050
native_system: sony-playstation
- name: "sony-playstation:8e4c14f567745eff2f0408c8129f72a6"
destination: ''
required: true
md5: 8e4c14f567745eff2f0408c8129f72a6
native_system: sony-playstation
- name: "sony-playstation:b84be139db3ee6cbd075630aa20a6553"
destination: ''
required: true
md5: b84be139db3ee6cbd075630aa20a6553
native_system: sony-playstation
- name: "sony-playstation:1e68c231d0896b7eadcad1d7d8e76129"
destination: ''
required: true
md5: 1e68c231d0896b7eadcad1d7d8e76129
native_system: sony-playstation
- name: "sony-playstation:b9d9a0286c33dc6b7237bb13cd46fdee"
destination: ''
required: true
md5: b9d9a0286c33dc6b7237bb13cd46fdee
native_system: sony-playstation
- name: "sony-playstation:8abc1b549a4a80954addc48ef02c4521"
destination: ''
required: true
md5: 8abc1b549a4a80954addc48ef02c4521
native_system: sony-playstation
- name: "sony-playstation:9a09ab7e49b422c007e6d54d7c49b965"
destination: ''
required: true
md5: 9a09ab7e49b422c007e6d54d7c49b965
native_system: sony-playstation
- name: "sony-playstation:b10f5e0e3d9eb60e5159690680b1e774"
destination: ''
required: true
md5: b10f5e0e3d9eb60e5159690680b1e774
native_system: sony-playstation
- name: "sony-playstation:6e3735ff4c7dc899ee98981385f6f3d0"
destination: ''
required: true
md5: 6e3735ff4c7dc899ee98981385f6f3d0
native_system: sony-playstation
- name: "sony-playstation:de93caec13d1a141a40a79f5c86168d6"
destination: ''
required: true
md5: de93caec13d1a141a40a79f5c86168d6
native_system: sony-playstation
- name: "sony-playstation:c53ca5908936d412331790f4426c6c33"
destination: ''
required: true
md5: c53ca5908936d412331790f4426c6c33
native_system: sony-playstation
- name: "sony-playstation:476d68a94ccec3b9c8303bbd1daf2810"
destination: ''
required: true
md5: 476d68a94ccec3b9c8303bbd1daf2810
native_system: sony-playstation
- name: "sony-playstation:d8f485717a5237285e4d7c5f881b7f32"
destination: ''
required: true
md5: d8f485717a5237285e4d7c5f881b7f32
native_system: sony-playstation
- name: "sony-playstation:fbb5f59ec332451debccf1e377017237"
destination: ''
required: true
md5: fbb5f59ec332451debccf1e377017237
native_system: sony-playstation
- name: "sony-playstation:81bbe60ba7a3d1cea1d48c14cbcc647b"
destination: ''
required: true
md5: 81bbe60ba7a3d1cea1d48c14cbcc647b
native_system: sony-playstation
sony-playstation-2:
files:
- name: SCPH-70004_BIOS_V12_EUR_200.BIN
destination: SCPH-70004_BIOS_V12_EUR_200.BIN
required: true
native_system: sony-playstation-2
- name: SCPH-70004_BIOS_V12_EUR_200.EROM
destination: SCPH-70004_BIOS_V12_EUR_200.EROM
required: true
native_system: sony-playstation-2
- name: SCPH-70004_BIOS_V12_EUR_200.ROM1
destination: SCPH-70004_BIOS_V12_EUR_200.ROM1
required: true
native_system: sony-playstation-2
- name: SCPH-70004_BIOS_V12_EUR_200.ROM2
destination: SCPH-70004_BIOS_V12_EUR_200.ROM2
required: true
native_system: sony-playstation-2
- name: "sony-playstation-2:32f2e4d5ff5ee11072a6bc45530f5765"
destination: ''
required: true
md5: 32f2e4d5ff5ee11072a6bc45530f5765
native_system: sony-playstation-2
- name: "sony-playstation-2:acf4730ceb38ac9d8c7d8e21f2614600"
destination: ''
required: true
md5: acf4730ceb38ac9d8c7d8e21f2614600
native_system: sony-playstation-2
- name: "sony-playstation-2:acf9968c8f596d2b15f42272082513d1"
destination: ''
required: true
md5: acf9968c8f596d2b15f42272082513d1
native_system: sony-playstation-2
- name: "sony-playstation-2:b1459d7446c69e3e97e6ace3ae23dd1c"
destination: ''
required: true
md5: b1459d7446c69e3e97e6ace3ae23dd1c
native_system: sony-playstation-2
- name: "sony-playstation-2:d3f1853a16c2ec18f3cd1ae655213308"
destination: ''
required: true
md5: d3f1853a16c2ec18f3cd1ae655213308
native_system: sony-playstation-2
- name: "sony-playstation-2:63e6fd9b3c72e0d7b920e80cf76645cd"
destination: ''
required: true
md5: 63e6fd9b3c72e0d7b920e80cf76645cd
native_system: sony-playstation-2
- name: "sony-playstation-2:a20c97c02210f16678ca3010127caf36"
destination: ''
required: true
md5: a20c97c02210f16678ca3010127caf36
native_system: sony-playstation-2
- name: "sony-playstation-2:8db2fbbac7413bf3e7154c1e0715e565"
destination: ''
required: true
md5: 8db2fbbac7413bf3e7154c1e0715e565
native_system: sony-playstation-2
- name: "sony-playstation-2:91c87cb2f2eb6ce529a2360f80ce2457"
destination: ''
required: true
md5: 91c87cb2f2eb6ce529a2360f80ce2457
native_system: sony-playstation-2
- name: "sony-playstation-2:3016b3dd42148a67e2c048595ca4d7ce"
destination: ''
required: true
md5: 3016b3dd42148a67e2c048595ca4d7ce
native_system: sony-playstation-2
- name: "sony-playstation-2:b7fa11e87d51752a98b38e3e691cbf17"
destination: ''
required: true
md5: b7fa11e87d51752a98b38e3e691cbf17
native_system: sony-playstation-2
- name: "sony-playstation-2:f63bc530bd7ad7c026fcd6f7bd0d9525"
destination: ''
required: true
md5: f63bc530bd7ad7c026fcd6f7bd0d9525
native_system: sony-playstation-2
- name: "sony-playstation-2:cee06bd68c333fc5768244eae77e4495"
destination: ''
required: true
md5: cee06bd68c333fc5768244eae77e4495
native_system: sony-playstation-2
- name: "sony-playstation-2:0bf988e9c7aaa4c051805b0fa6eb3387"
destination: ''
required: true
md5: 0bf988e9c7aaa4c051805b0fa6eb3387
native_system: sony-playstation-2
- name: "sony-playstation-2:8accc3c49ac45f5ae2c5db0adc854633"
destination: ''
required: true
md5: 8accc3c49ac45f5ae2c5db0adc854633
native_system: sony-playstation-2
- name: "sony-playstation-2:6f9a6feb749f0533aaae2cc45090b0ed"
destination: ''
required: true
md5: 6f9a6feb749f0533aaae2cc45090b0ed
native_system: sony-playstation-2
- name: "sony-playstation-2:838544f12de9b0abc90811279ee223c8"
destination: ''
required: true
md5: 838544f12de9b0abc90811279ee223c8
native_system: sony-playstation-2
- name: "sony-playstation-2:bb6bbc850458fff08af30e969ffd0175"
destination: ''
required: true
md5: bb6bbc850458fff08af30e969ffd0175
native_system: sony-playstation-2
- name: "sony-playstation-2:815ac991d8bc3b364696bead3457de7d"
destination: ''
required: true
md5: 815ac991d8bc3b364696bead3457de7d
native_system: sony-playstation-2
- name: "sony-playstation-2:b107b5710042abe887c0f6175f6e94bb"
destination: ''
required: true
md5: b107b5710042abe887c0f6175f6e94bb
native_system: sony-playstation-2
- name: "sony-playstation-2:ab55cceea548303c22c72570cfd4dd71"
destination: ''
required: true
md5: ab55cceea548303c22c72570cfd4dd71
native_system: sony-playstation-2
- name: "sony-playstation-2:18bcaadb9ff74ed3add26cdf709fff2e"
destination: ''
required: true
md5: 18bcaadb9ff74ed3add26cdf709fff2e
native_system: sony-playstation-2
- name: "sony-playstation-2:491209dd815ceee9de02dbbc408c06d6"
destination: ''
required: true
md5: 491209dd815ceee9de02dbbc408c06d6
native_system: sony-playstation-2
- name: "sony-playstation-2:7200a03d51cacc4c14fcdfdbc4898431"
destination: ''
required: true
md5: 7200a03d51cacc4c14fcdfdbc4898431
native_system: sony-playstation-2
- name: "sony-playstation-2:8359638e857c8bc18c3c18ac17d9cc3c"
destination: ''
required: true
md5: 8359638e857c8bc18c3c18ac17d9cc3c
native_system: sony-playstation-2
- name: "sony-playstation-2:352d2ff9b3f68be7e6fa7e6dd8389346"
destination: ''
required: true
md5: 352d2ff9b3f68be7e6fa7e6dd8389346
native_system: sony-playstation-2
- name: "sony-playstation-2:d5ce2c7d119f563ce04bc04dbc3a323e"
destination: ''
required: true
md5: d5ce2c7d119f563ce04bc04dbc3a323e
native_system: sony-playstation-2
- name: "sony-playstation-2:0d2228e6fd4fb639c9c39d077a9ec10c"
destination: ''
required: true
md5: 0d2228e6fd4fb639c9c39d077a9ec10c
native_system: sony-playstation-2
- name: "sony-playstation-2:72da56fccb8fcd77bba16d1b6f479914"
destination: ''
required: true
md5: 72da56fccb8fcd77bba16d1b6f479914
native_system: sony-playstation-2
- name: "sony-playstation-2:5b1f47fbeb277c6be2fccdd6344ff2fd"
destination: ''
required: true
md5: 5b1f47fbeb277c6be2fccdd6344ff2fd
native_system: sony-playstation-2
- name: "sony-playstation-2:315a4003535dfda689752cb25f24785c"
destination: ''
required: true
md5: 315a4003535dfda689752cb25f24785c
native_system: sony-playstation-2
- name: "sony-playstation-2:312ad4816c232a9606e56f946bc0678a"
destination: ''
required: true
md5: 312ad4816c232a9606e56f946bc0678a
native_system: sony-playstation-2
- name: "sony-playstation-2:666018ffec65c5c7e04796081295c6c7"
destination: ''
required: true
md5: 666018ffec65c5c7e04796081295c6c7
native_system: sony-playstation-2
- name: "sony-playstation-2:6e69920fa6eef8522a1d688a11e41bc6"
destination: ''
required: true
md5: 6e69920fa6eef8522a1d688a11e41bc6
native_system: sony-playstation-2
- name: "sony-playstation-2:eb960de68f0c0f7f9fa083e9f79d0360"
destination: ''
required: true
md5: eb960de68f0c0f7f9fa083e9f79d0360
native_system: sony-playstation-2
- name: "sony-playstation-2:8aa12ce243210128c5074552d3b86251"
destination: ''
required: true
md5: 8aa12ce243210128c5074552d3b86251
native_system: sony-playstation-2
- name: "sony-playstation-2:240d4c5ddd4b54069bdc4a3cd2faf99d"
destination: ''
required: true
md5: 240d4c5ddd4b54069bdc4a3cd2faf99d
native_system: sony-playstation-2
- name: "sony-playstation-2:1c6cd089e6c83da618fbf2a081eb4888"
destination: ''
required: true
md5: 1c6cd089e6c83da618fbf2a081eb4888
native_system: sony-playstation-2
- name: "sony-playstation-2:463d87789c555a4a7604e97d7db545d1"
destination: ''
required: true
md5: 463d87789c555a4a7604e97d7db545d1
native_system: sony-playstation-2
- name: "sony-playstation-2:35461cecaa51712b300b2d6798825048"
destination: ''
required: true
md5: 35461cecaa51712b300b2d6798825048
native_system: sony-playstation-2
- name: "sony-playstation-2:bd6415094e1ce9e05daabe85de807666"
destination: ''
required: true
md5: bd6415094e1ce9e05daabe85de807666
native_system: sony-playstation-2
- name: "sony-playstation-2:2e70ad008d4ec8549aada8002fdf42fb"
destination: ''
required: true
md5: 2e70ad008d4ec8549aada8002fdf42fb
native_system: sony-playstation-2
- name: "sony-playstation-2:b53d51edc7fc086685e31b811dc32aad"
destination: ''
required: true
md5: b53d51edc7fc086685e31b811dc32aad
native_system: sony-playstation-2
- name: "sony-playstation-2:1b6e631b536247756287b916f9396872"
destination: ''
required: true
md5: 1b6e631b536247756287b916f9396872
native_system: sony-playstation-2
- name: "sony-playstation-2:00da1b177096cfd2532c8fa22b43e667"
destination: ''
required: true
md5: 00da1b177096cfd2532c8fa22b43e667
native_system: sony-playstation-2
- name: "sony-playstation-2:afde410bd026c16be605a1ae4bd651fd"
destination: ''
required: true
md5: afde410bd026c16be605a1ae4bd651fd
native_system: sony-playstation-2
- name: "sony-playstation-2:81f4336c1de607dd0865011c0447052e"
destination: ''
required: true
md5: 81f4336c1de607dd0865011c0447052e
native_system: sony-playstation-2
- name: "sony-playstation-2:0eee5d1c779aa50e94edd168b4ebf42e"
destination: ''
required: true
md5: 0eee5d1c779aa50e94edd168b4ebf42e
native_system: sony-playstation-2
- name: "sony-playstation-2:d333558cc14561c1fdc334c75d5f37b7"
destination: ''
required: true
md5: d333558cc14561c1fdc334c75d5f37b7
native_system: sony-playstation-2
- name: "sony-playstation-2:dc752f160044f2ed5fc1f4964db2a095"
destination: ''
required: true
md5: dc752f160044f2ed5fc1f4964db2a095
native_system: sony-playstation-2
- name: "sony-playstation-2:63ead1d74893bf7f36880af81f68a82d"
destination: ''
required: true
md5: 63ead1d74893bf7f36880af81f68a82d
native_system: sony-playstation-2
- name: "sony-playstation-2:3e3e030c0f600442fa05b94f87a1e238"
destination: ''
required: true
md5: 3e3e030c0f600442fa05b94f87a1e238
native_system: sony-playstation-2
- name: "sony-playstation-2:1ad977bb539fc9448a08ab276a836bbc"
destination: ''
required: true
md5: 1ad977bb539fc9448a08ab276a836bbc
native_system: sony-playstation-2
- name: "sony-playstation-2:eb4f40fcf4911ede39c1bbfe91e7a89a"
destination: ''
required: true
md5: eb4f40fcf4911ede39c1bbfe91e7a89a
native_system: sony-playstation-2
- name: "sony-playstation-2:9959ad7a8685cad66206e7752ca23f8b"
destination: ''
required: true
md5: 9959ad7a8685cad66206e7752ca23f8b
native_system: sony-playstation-2
- name: "sony-playstation-2:929a14baca1776b00869f983aa6e14d2"
destination: ''
required: true
md5: 929a14baca1776b00869f983aa6e14d2
native_system: sony-playstation-2
- name: "sony-playstation-2:573f7d4a430c32b3cc0fd0c41e104bbd"
destination: ''
required: true
md5: 573f7d4a430c32b3cc0fd0c41e104bbd
native_system: sony-playstation-2
- name: "sony-playstation-2:df63a604e8bff5b0599bd1a6c2721bd0"
destination: ''
required: true
md5: df63a604e8bff5b0599bd1a6c2721bd0
native_system: sony-playstation-2
- name: "sony-playstation-2:5b1ba4bb914406fae75ab8e38901684d"
destination: ''
required: true
md5: 5b1ba4bb914406fae75ab8e38901684d
native_system: sony-playstation-2
- name: "sony-playstation-2:cb801b7920a7d536ba07b6534d2433ca"
destination: ''
required: true
md5: cb801b7920a7d536ba07b6534d2433ca
native_system: sony-playstation-2
- name: "sony-playstation-2:af60e6d1a939019d55e5b330d24b1c25"
destination: ''
required: true
md5: af60e6d1a939019d55e5b330d24b1c25
native_system: sony-playstation-2
- name: "sony-playstation-2:549a66d0c698635ca9fa3ab012da7129"
destination: ''
required: true
md5: 549a66d0c698635ca9fa3ab012da7129
native_system: sony-playstation-2
- name: "sony-playstation-2:5de9d0d730ff1e7ad122806335332524"
destination: ''
required: true
md5: 5de9d0d730ff1e7ad122806335332524
native_system: sony-playstation-2
- name: "sony-playstation-2:21fe4cad111f7dc0f9af29477057f88d"
destination: ''
required: true
md5: 21fe4cad111f7dc0f9af29477057f88d
native_system: sony-playstation-2
- name: "sony-playstation-2:40c11c063b3b9409aa5e4058e984e30c"
destination: ''
required: true
md5: 40c11c063b3b9409aa5e4058e984e30c
native_system: sony-playstation-2
- name: "sony-playstation-2:80bbb237a6af9c611df43b16b930b683"
destination: ''
required: true
md5: 80bbb237a6af9c611df43b16b930b683
native_system: sony-playstation-2
- name: "sony-playstation-2:c37bce95d32b2be480f87dd32704e664"
destination: ''
required: true
md5: c37bce95d32b2be480f87dd32704e664
native_system: sony-playstation-2
- name: "sony-playstation-2:80ac46fa7e77b8ab4366e86948e54f83"
destination: ''
required: true
md5: 80ac46fa7e77b8ab4366e86948e54f83
native_system: sony-playstation-2
- name: "sony-playstation-2:21038400dc633070a78ad53090c53017"
destination: ''
required: true
md5: 21038400dc633070a78ad53090c53017
native_system: sony-playstation-2
- name: "sony-playstation-2:dc69f0643a3030aaa4797501b483d6c4"
destination: ''
required: true
md5: dc69f0643a3030aaa4797501b483d6c4
native_system: sony-playstation-2
- name: "sony-playstation-2:30d56e79d89fbddf10938fa67fe3f34e"
destination: ''
required: true
md5: 30d56e79d89fbddf10938fa67fe3f34e
native_system: sony-playstation-2
- name: "sony-playstation-2:93ea3bcee4252627919175ff1b16a1d9"
destination: ''
required: true
md5: 93ea3bcee4252627919175ff1b16a1d9
native_system: sony-playstation-2
- name: "sony-playstation-2:d3e81e95db25f5a86a7b7474550a2155"
destination: ''
required: true
md5: d3e81e95db25f5a86a7b7474550a2155
native_system: sony-playstation-2
sega-mega-cd:
files:
- name: bios_CD_E.bin
destination: bios_CD_E.bin
required: true
native_system: sega-mega-cd
- name: bios_CD_U.bin
destination: bios_CD_U.bin
required: true
native_system: sega-mega-cd
- name: bios_CD_J.bin
destination: bios_CD_J.bin
required: true
native_system: sega-mega-cd
- name: "sega-mega-cd:bc6ae4e1db01a2f349d9af392bf7e2bd"
destination: ''
required: true
md5: bc6ae4e1db01a2f349d9af392bf7e2bd
native_system: sega-mega-cd
- name: "sega-mega-cd:29ad9ce848b49d0f9cefc294137f653c"
destination: ''
required: true
md5: 29ad9ce848b49d0f9cefc294137f653c
native_system: sega-mega-cd
- name: "sega-mega-cd:cc049159d7e744c15eee080c241273b4"
destination: ''
required: true
md5: cc049159d7e744c15eee080c241273b4
native_system: sega-mega-cd
- name: "sega-mega-cd:278a9397d192149e84e820ac621a8edd"
destination: ''
required: true
md5: 278a9397d192149e84e820ac621a8edd
native_system: sega-mega-cd
- name: "sega-mega-cd:a3ddcc8483b0368141adfd99d9a1e466"
destination: ''
required: true
md5: a3ddcc8483b0368141adfd99d9a1e466
native_system: sega-mega-cd
- name: "sega-mega-cd:bdeb4c47da613946d422d97d98b21cda"
destination: ''
required: true
md5: bdeb4c47da613946d422d97d98b21cda
native_system: sega-mega-cd
- name: "sega-mega-cd:96ea588d647f2ab1f291279fc691663c"
destination: ''
required: true
md5: 96ea588d647f2ab1f291279fc691663c
native_system: sega-mega-cd
- name: "sega-mega-cd:2efd74e3232ff260e371b99f84024f7f"
destination: ''
required: true
md5: 2efd74e3232ff260e371b99f84024f7f
native_system: sega-mega-cd
- name: "sega-mega-cd:e66fa1dc5820d254611fdcdba0662372"
destination: ''
required: true
md5: e66fa1dc5820d254611fdcdba0662372
native_system: sega-mega-cd
- name: "sega-mega-cd:683a8a9e273662561172468dfa2858eb"
destination: ''
required: true
md5: 683a8a9e273662561172468dfa2858eb
native_system: sega-mega-cd
- name: "sega-mega-cd:310a9081d2edf2d316ab38813136725e"
destination: ''
required: true
md5: 310a9081d2edf2d316ab38813136725e
native_system: sega-mega-cd
- name: "sega-mega-cd:9b562ebf2d095bf1dabadbc1881f519a"
destination: ''
required: true
md5: 9b562ebf2d095bf1dabadbc1881f519a
native_system: sega-mega-cd
- name: "sega-mega-cd:854b9150240a198070150e4566ae1290"
destination: ''
required: true
md5: 854b9150240a198070150e4566ae1290
native_system: sega-mega-cd
- name: "sega-mega-cd:b10c0a97abc57b758497d3fae6ab35a4"
destination: ''
required: true
md5: b10c0a97abc57b758497d3fae6ab35a4
native_system: sega-mega-cd
- name: "sega-mega-cd:ecc837c31d77b774c6e27e38f828aa9a"
destination: ''
required: true
md5: ecc837c31d77b774c6e27e38f828aa9a
native_system: sega-mega-cd
- name: "sega-mega-cd:baca1df271d7c11fe50087c0358f4eb5"
destination: ''
required: true
md5: baca1df271d7c11fe50087c0358f4eb5
native_system: sega-mega-cd
sega-saturn:
files:
- name: sega_101.bin
destination: sega_101.bin
required: true
native_system: sega-saturn
- name: mpr-17933.bin
destination: mpr-17933.bin
required: true
native_system: sega-saturn
- name: saturn_bios.bin
destination: saturn_bios.bin
required: true
native_system: sega-saturn
- name: "sega-saturn:af5828fdff51384f99b3c4926be27762"
destination: ''
required: true
md5: af5828fdff51384f99b3c4926be27762
native_system: sega-saturn
- name: "sega-saturn:85ec9ca47d8f6807718151cbcca8b964"
destination: ''
required: true
md5: 85ec9ca47d8f6807718151cbcca8b964
native_system: sega-saturn
- name: "sega-saturn:f273555d7d91e8a5a6bfd9bcf066331c"
destination: ''
required: true
md5: f273555d7d91e8a5a6bfd9bcf066331c
native_system: sega-saturn
- name: "sega-saturn:3240872c70984b6cbfda1586cab68dbe"
destination: ''
required: true
md5: 3240872c70984b6cbfda1586cab68dbe
native_system: sega-saturn
- name: "sega-saturn:ac4e4b6522e200c0d23d371a8cecbfd3"
destination: ''
required: true
md5: ac4e4b6522e200c0d23d371a8cecbfd3
native_system: sega-saturn
- name: "sega-saturn:3ea3202e2634cb47cb90f3a05c015010"
destination: ''
required: true
md5: 3ea3202e2634cb47cb90f3a05c015010
native_system: sega-saturn
- name: "sega-saturn:cb2cebc1b6e573b7c44523d037edcd45"
destination: ''
required: true
md5: cb2cebc1b6e573b7c44523d037edcd45
native_system: sega-saturn
- name: "sega-saturn:0306c0e408d6682dd2d86324bd4ac661"
destination: ''
required: true
md5: 0306c0e408d6682dd2d86324bd4ac661
native_system: sega-saturn
sega-dreamcast:
files:
- name: dc_boot.bin
destination: dc/dc_boot.bin
required: true
native_system: sega-dreamcast
- name: dc_flash.bin
destination: dc/dc_flash.bin
required: true
native_system: sega-dreamcast
- name: "sega-dreamcast:d407fcf70b56acb84b8c77c93b0e5327"
destination: ''
required: true
md5: d407fcf70b56acb84b8c77c93b0e5327
native_system: sega-dreamcast
- name: "sega-dreamcast:e10c53c2f8b90bab96ead2d368858623"
destination: ''
required: true
md5: e10c53c2f8b90bab96ead2d368858623
native_system: sega-dreamcast
- name: "sega-dreamcast:93a9766f14159b403178ac77417c6b68"
destination: ''
required: true
md5: 93a9766f14159b403178ac77417c6b68
native_system: sega-dreamcast
- name: "sega-dreamcast:0a93f7940c455905bea6e392dfde92a4"
destination: ''
required: true
md5: 0a93f7940c455905bea6e392dfde92a4
native_system: sega-dreamcast
nintendo-ds:
files:
- name: bios7.bin
destination: bios7.bin
required: true
native_system: nintendo-ds
- name: bios9.bin
destination: bios9.bin
required: true
native_system: nintendo-ds
- name: firmware.bin
destination: firmware.bin
required: true
native_system: nintendo-ds
- name: "nintendo-ds:145eaef5bd3037cbc247c213bb3da1b3"
destination: ''
required: true
md5: 145eaef5bd3037cbc247c213bb3da1b3
native_system: nintendo-ds
- name: "nintendo-ds:df692a80a5b1bc90728bc3dfc76cd948"
destination: ''
required: true
md5: df692a80a5b1bc90728bc3dfc76cd948
native_system: nintendo-ds
- name: "nintendo-ds:a392174eb3e572fed6447e956bde4b25"
destination: ''
required: true
md5: a392174eb3e572fed6447e956bde4b25
native_system: nintendo-ds
snk-neogeo:
files:
- name: neogeo.zip
destination: neogeo.zip
required: true
native_system: snk-neogeo
- name: neocdz.zip
destination: neocdz.zip
required: true
native_system: snk-neogeo
panasonic-3do:
files:
- name: panafz1.bin
destination: panafz1.bin
required: true
native_system: panasonic-3do
nintendo-nes:
files:
- name: disksys.rom
destination: disksys.rom
required: true
native_system: nintendo-nes
sega-dreamcast-arcade:
files:
- name: naomi.zip
destination: dc/naomi.zip
required: true
native_system: sega-dreamcast-arcade
sony-psp:
files:
- name: ppsspp.zip
destination: ppsspp.zip
required: true
native_system: sony-psp
nintendo-3ds:
files:
- name: aes_keys.txt
destination: aes_keys.txt
required: true
native_system: nintendo-3ds
nintendo-64:
files:
- name: 64DD_IPL_US.n64
destination: 64DD_IPL_US.n64
required: true
native_system: nintendo-64
- name: 64DD_IPL_JP.n64
destination: 64DD_IPL_JP.n64
required: true
native_system: nintendo-64
- name: 64DD_IPL_DEV.n64
destination: 64DD_IPL_DEV.n64
required: true
native_system: nintendo-64
nintendo-gamecube:
files:
- name: IPL.bin
destination: IPL.bin
required: true
native_system: nintendo-gamecube
nintendo-switch:
files:
- name: prod.keys
destination: prod.keys
required: true
native_system: nintendo-switch
nintendo-wii-u:
files:
- name: keys.txt
destination: keys.txt
required: true
native_system: nintendo-wii-u
snk-neogeo-cd:
files:
- name: neocdz.zip
destination: neocdz.zip
required: true
native_system: snk-neogeo-cd
- name: neogeo.zip
destination: neogeo.zip
required: true
native_system: snk-neogeo-cd
xbox:
files:
- name: mcpx_1.0.bin
destination: mcpx_1.0.bin
required: true
native_system: xbox
- name: Complex_4627v1.03.bin
destination: Complex_4627v1.03.bin
required: true
native_system: xbox
coleco-colecovision:
files:
- name: colecovision.rom
destination: colecovision.rom
required: true
native_system: coleco-colecovision
platforms/misterfpga.yml
+72
View File
@@ -14,11 +14,13 @@ systems:
required: true
md5: f47264dd47fe30f73ab3c010015c155b
size: 1048576
native_system: 3DO
- name: kanji.rom
destination: 3DO/kanji.rom
required: true
md5: c23fb5d5e6bb1c240d02cf968972be37
size: 1048576
native_system: 3DO
native_id: 3DO
name: 3DO Interactive Multiplayer
docs: "https://github.com/MiSTer-devel/3DO_MiSTer"
@@ -29,11 +31,13 @@ systems:
required: true
md5: ba89edf2729a28a17cd9e0f7a0ac9a39
size: 262144
native_system: Apple-IIgs
- name: boot1.rom
destination: Apple-IIgs/boot1.rom
required: true
md5: 20a0334c447cb069a040ae5be1d938df
size: 131072
native_system: Apple-IIgs
native_id: Apple-IIgs
name: Apple IIGS
docs: "https://github.com/MiSTer-devel/Apple-IIgs_MiSTer"
@@ -44,16 +48,19 @@ systems:
required: true
md5: 6e844759720226e58d55ecaf33608a13
size: 131072
native_system: Jaguar
- name: boot1.rom
destination: Jaguar/boot1.rom
required: true
md5: 18f0741bdb8bb9b6bb99393cb90426a2
size: 262144
native_system: Jaguar
- name: boot2.rom
destination: Jaguar/boot2.rom
required: true
md5: 4af00f1c26898cf04585e1693d25faba
size: 131072
native_system: Jaguar
native_id: Jaguar
name: Atari Jaguar
docs: "https://github.com/MiSTer-devel/Jaguar_MiSTer"
@@ -64,6 +71,7 @@ systems:
required: true
md5: fcd403db69f54290b51035d82f835e7b
size: 512
native_system: AtariLynx
native_id: AtariLynx
name: Atari Lynx
docs: "https://github.com/MiSTer-devel/AtariLynx_MiSTer"
@@ -74,6 +82,7 @@ systems:
required: true
md5: 7d25a26e5c4841b364cfe6b1735eaf03
size: 8192
native_system: Astrocade
native_id: Astrocade
name: Bally Astrocade
docs: "https://github.com/MiSTer-devel/Astrocade_MiSTer"
@@ -84,6 +93,7 @@ systems:
required: true
md5: 558540c5c6f776d88a897f2f3b8fec8f
size: 16384
native_system: Casio_PV-2000
native_id: Casio_PV-2000
name: Casio PV-2000
docs: "https://github.com/MiSTer-devel/Casio_PV-2000_MiSTer"
@@ -94,6 +104,7 @@ systems:
required: true
md5: 4dfadfb1158f84e2df2b85b13f303986
size: 16384
native_system: CreatiVision
native_id: CreatiVision
name: VTech CreatiVision
docs: "https://github.com/MiSTer-devel/CreatiVision_MiSTer"
@@ -104,6 +115,7 @@ systems:
required: true
md5: b596975be2e0360232bbeb1e492ab873
size: 6144
native_system: SCV
native_id: SCV
name: Epoch Super Cassette Vision
docs: "https://github.com/MiSTer-devel/SuperCassetteVision_MiSTer"
@@ -114,6 +126,7 @@ systems:
required: true
md5: ef67993a94503c4b7798b5901c7dda52
size: 4096
native_system: Gamate
native_id: Gamate
name: Bit Corporation Gamate
docs: "https://github.com/MiSTer-devel/Gamate_MiSTer"
@@ -124,6 +137,7 @@ systems:
required: true
md5: 1c7e3cb2df4af8ce3eff187c4155d1a7
size: 524288
native_system: GameKing
native_id: GameKing
name: gameking
interact:
@@ -133,6 +147,7 @@ systems:
required: true
md5: aa9fb0e9697a009dfb9d876351dd8f48
size: 2048
native_system: Interact
native_id: Interact
name: Interact Home Computer
docs: "https://github.com/MiSTer-devel/Interact_MiSTer"
@@ -143,21 +158,25 @@ systems:
required: true
md5: 62e761035cb657903761800f4437b8af
size: 8192
native_system: Intellivision
- name: boot1.rom
destination: Intellivision/boot1.rom
required: true
md5: 0cd5946c6473e42e8e4c2137785e427f
size: 2048
native_system: Intellivision
- name: boot2.rom
destination: Intellivision/boot2.rom
required: true
md5: d5530f74681ec6e0f282dab42e6b1c5f
size: 2048
native_system: Intellivision
- name: boot3.rom
destination: Intellivision/boot3.rom
required: true
md5: 2e72a9a2b897d330a35c8b07a6146c52
size: 24576
native_system: Intellivision
native_id: Intellivision
name: Mattel Intellivision
docs: "https://github.com/MiSTer-devel/Intv_MiSTer"
@@ -168,6 +187,7 @@ systems:
required: true
md5: a0452dbf5ace7d2e49d0a8029efed09a
size: 32768
native_system: MSX1
native_id: MSX1
name: MSX1
docs: "https://github.com/MiSTer-devel/MSX1_MiSTer"
@@ -178,6 +198,7 @@ systems:
required: true
md5: f1ce4d5f83717093982e5f75516b8f3c
size: 393216
native_system: PC8801
native_id: PC8801
name: NEC PC-8801
docs: "https://github.com/MiSTer-devel/PC88_MiSTer"
@@ -188,6 +209,7 @@ systems:
required: true
md5: 98d43a097a165b03df170fd5c2ad2c2f
size: 262144
native_system: TGFX16-CD
native_id: TGFX16-CD
name: PC Engine CD / TurboGrafx-CD
docs: "https://github.com/MiSTer-devel/TurboGrafx16_MiSTer"
@@ -198,11 +220,13 @@ systems:
required: true
md5: 3655aacebb43a963607d4cf821fc1301
size: 65536
native_system: NGPC
- name: boot1.rom
destination: NGPC/boot1.rom
required: true
md5: d87d876c4391935c9d48ef352a3ff02d
size: 65536
native_system: NGPC
native_id: NGPC
name: ngpc
nintendo-64:
@@ -212,26 +236,31 @@ systems:
required: true
md5: 5c124e7948ada85da603a522782940d0
size: 1984
native_system: N64
- name: boot1.rom
destination: N64/boot1.rom
required: true
md5: d4232dc935cad0650ac2664d52281f3a
size: 1984
native_system: N64
- name: boot3.rom
destination: N64/boot3.rom
required: true
md5: 8d3d9f294b6e174bc7b1d2fd1c727530
size: 4194304
native_system: N64
- name: boot4.rom
destination: N64/boot4.rom
required: true
md5: aad37b1492886b892f1821f37fd3ae34
size: 4194304
native_system: N64
- name: boot5.rom
destination: N64/boot5.rom
required: true
md5: 37c36e4286d36892a9fc70eafe4104be
size: 4194304
native_system: N64
native_id: N64
name: Nintendo 64
docs: "https://github.com/MiSTer-devel/N64_MiSTer"
@@ -242,6 +271,7 @@ systems:
required: true
md5: dbfce9db9deaa2567f6a84fde55f9680
size: 2304
native_system: GAMEBOY
native_id: GAMEBOY
name: Game Boy / Game Boy Color
docs: "https://github.com/MiSTer-devel/Gameboy_MiSTer"
@@ -252,6 +282,7 @@ systems:
required: true
md5: a860e8c0b6d573d191e4ec7db1b1e4f6
size: 16384
native_system: GBA
native_id: GBA
name: Game Boy Advance
docs: "https://github.com/MiSTer-devel/GBA_MiSTer"
@@ -262,6 +293,7 @@ systems:
required: true
md5: ca30b50f880eb660a320674ed365ef7a
size: 8192
native_system: NES
native_id: NES
name: NES / Famicom Disk System
docs: "https://github.com/MiSTer-devel/NES_MiSTer"
@@ -272,6 +304,7 @@ systems:
required: true
md5: 1e4fb124a3a886865acb574f388c803d
size: 4096
native_system: PokemonMini
native_id: PokemonMini
name: Pokemon Mini
docs: "https://github.com/MiSTer-devel/PokemonMini_MiSTer"
@@ -282,11 +315,13 @@ systems:
required: true
md5: 8ecd73eb4edf7ed7e81aef1be80031d5
size: 524288
native_system: SGB
- name: Super Game Boy.sfc
destination: SGB/Super Game Boy.sfc
required: true
md5: b15ddb15721c657d82c5bab6db982ee9
size: 262144
native_system: SGB
native_id: SGB
name: Super Game Boy
docs: "https://github.com/MiSTer-devel/SGB_MiSTer"
@@ -297,6 +332,7 @@ systems:
required: true
md5: 96cf17bf589fcbfa6f8de2dc84f19fa2
size: 1048576
native_system: SNES
native_id: SNES
name: Super NES / Super Famicom
docs: "https://github.com/MiSTer-devel/SNES_MiSTer"
@@ -307,16 +343,19 @@ systems:
required: true
md5: ac0d468be366779c9df633be98da250a
size: 524288
native_system: CD-i
- name: boot1.rom
destination: CD-i/boot1.rom
required: true
md5: 3d20cf7550f1b723158b42a1fd5bac62
size: 8192
native_system: CD-i
- name: boot2.rom
destination: CD-i/boot2.rom
required: true
md5: 9694c466f9b65c1990a81b7a6280546b
size: 131072
native_system: CD-i
native_id: CD-i
name: Philips CD-i
docs: "https://github.com/MiSTer-devel/CDi_MiSTer"
@@ -327,11 +366,13 @@ systems:
required: true
md5: 54b915694731cc22e07d3fb8a00ee2db
size: 4096
native_system: PocketChallengeV2
- name: boot1.rom
destination: PocketChallengeV2/boot1.rom
required: true
md5: 880893bd5a7d53fff826bd76a83d566e
size: 8192
native_system: PocketChallengeV2
native_id: PocketChallengeV2
name: Benesse Pocket Challenge V2
docs: "https://github.com/MiSTer-devel/WonderSwan_MiSTer"
@@ -342,21 +383,25 @@ systems:
required: true
md5: 9b562ebf2d095bf1dabadbc1881f519a
size: 131072
native_system: MegaCD
- name: cd_bios.rom
destination: MegaCD/Japan/cd_bios.rom
required: true
md5: 683a8a9e273662561172468dfa2858eb
size: 131072
native_system: MegaCD
- name: cd_bios.rom
destination: MegaCD/USA/cd_bios.rom
required: true
md5: 310a9081d2edf2d316ab38813136725e
size: 131072
native_system: MegaCD
- name: boot.rom
destination: MegaCD/boot.rom
required: true
md5: 14db9657bbaa6fbb9249752424dc0ce4
size: 131072
native_system: MegaCD
native_id: MegaCD
name: Sega CD / Mega-CD
docs: "https://github.com/MiSTer-devel/MegaCD_MiSTer"
@@ -367,6 +412,7 @@ systems:
required: true
md5: 0306c0e408d6682dd2d86324bd4ac661
size: 524288
native_system: Saturn
native_id: Saturn
name: Sega Saturn
docs: "https://github.com/MiSTer-devel/Saturn_MiSTer"
@@ -377,26 +423,31 @@ systems:
required: true
md5: fc7599f3f871578fe9a0453662d1c966
size: 131072
native_system: NEOGEO
- name: neo-epo.sp1
destination: NEOGEO/neo-epo.sp1
required: true
md5: b11751ad42879c461d64ad2b7b2b0129
size: 131072
native_system: NEOGEO
- name: sfix.sfix
destination: NEOGEO/sfix.sfix
required: true
md5: aa2b5d0eae4158ffc0d7d63481c7830b
size: 131072
native_system: NEOGEO
- name: sp-s2.sp1
destination: NEOGEO/sp-s2.sp1
required: true
md5: 2968f59f44bf328639aa79391aeeeab4
size: 131072
native_system: NEOGEO
- name: uni-bios.rom
destination: NEOGEO/uni-bios.rom
required: true
md5: 4f0aeda8d2d145f596826b62d563c4ef
size: 131072
native_system: NEOGEO
native_id: NEOGEO
name: Neo Geo AES / MVS
docs: "https://github.com/MiSTer-devel/NeoGeo_MiSTer"
@@ -407,16 +458,19 @@ systems:
required: true
md5: f39572af7584cb5b3f70ae8cc848aba2
size: 524288
native_system: NeoGeo-CD
- name: top-sp1.bin
destination: NeoGeo-CD/top-sp1.bin
required: true
md5: 122aee210324c72e8a11116e6ef9c0d0
size: 524288
native_system: NeoGeo-CD
- name: uni-bioscd.rom
destination: NeoGeo-CD/uni-bioscd.rom
required: true
md5: 08ca8b2dba6662e8024f9e789711c6fc
size: 524288
native_system: NeoGeo-CD
native_id: NeoGeo-CD
name: Neo Geo CD
docs: "https://github.com/MiSTer-devel/NeoGeo_MiSTer"
@@ -427,21 +481,25 @@ systems:
required: true
md5: 1e68c231d0896b7eadcad1d7d8e76129
size: 524288
native_system: PSX
- name: boot1.rom
destination: PSX/boot1.rom
required: true
md5: 8e4c14f567745eff2f0408c8129f72a6
size: 524288
native_system: PSX
- name: boot2.rom
destination: PSX/boot2.rom
required: true
md5: b9d9a0286c33dc6b7237bb13cd46fdee
size: 524288
native_system: PSX
- name: sbi.zip
destination: PSX/sbi.zip
required: true
md5: 706d60ab2dfdbfdd53be2069cb85a1fe
size: 218953
native_system: PSX
native_id: PSX
name: Sony PlayStation
docs: "https://github.com/MiSTer-devel/PSX_MiSTer"
@@ -452,26 +510,31 @@ systems:
required: true
md5: b37205bf19b197682f00619d05da194b
size: 2048
native_system: Studio-II
- name: boot1.rom
destination: Studio-II/boot1.rom
required: true
md5: a6b94e449bc9ec58a30e1f75d590c558
size: 4096
native_system: Studio-II
- name: boot2.rom
destination: Studio-II/boot2.rom
required: true
md5: 849a484aa4b2784ece5c35c39d9d51a8
size: 4096
native_system: Studio-II
- name: boot3.rom
destination: Studio-II/boot3.rom
required: true
md5: aeec6fe3934481e20eb7db6d5ff56a54
size: 2048
native_system: Studio-II
- name: boot4.rom
destination: Studio-II/boot4.rom
required: true
md5: 9f037435b6721be9ee91dc93293e52ce
size: 768
native_system: Studio-II
native_id: Studio-II
name: studio-ii
tandy-coco:
@@ -481,16 +544,19 @@ systems:
required: true
md5: 7233c6c429f3ce1c7392f28a933e0b6f
size: 32768
native_system: COCO3
- name: boot1.rom
destination: COCO3/boot1.rom
required: true
md5: 8cab28f4b7311b8df63c07bb3b59bfd5
size: 8192
native_system: COCO3
- name: boot2.rom
destination: COCO3/boot2.rom
required: true
md5: 171fb7a5d3b808565295b45c773abaaf
size: 8192
native_system: COCO3
native_id: COCO3
name: Tandy Color Computer 3
docs: "https://github.com/MiSTer-devel/CoCo3_MiSTer"
@@ -501,6 +567,7 @@ systems:
required: true
md5: a75bb208176c0aed1b6a04b2dcf4770c
size: 196608
native_system: TI-99_4A
native_id: TI-99_4A
name: Texas Instruments TI-99/4A
docs: "https://github.com/MiSTer-devel/TI-99_4A_MiSTer"
@@ -511,6 +578,7 @@ systems:
required: true
md5: 3f2178d717dd309ed7424d49ef92987f
size: 262144
native_system: GameCom
native_id: GameCom
name: Tiger Game.com
wonderswan:
@@ -520,11 +588,13 @@ systems:
required: true
md5: 54b915694731cc22e07d3fb8a00ee2db
size: 4096
native_system: WonderSwan
- name: boot1.rom
destination: WonderSwan/boot1.rom
required: true
md5: 880893bd5a7d53fff826bd76a83d566e
size: 8192
native_system: WonderSwan
native_id: WonderSwan
name: Bandai WonderSwan
docs: "https://github.com/MiSTer-devel/WonderSwan_MiSTer"
@@ -535,11 +605,13 @@ systems:
required: true
md5: 54b915694731cc22e07d3fb8a00ee2db
size: 4096
native_system: WonderSwanColor
- name: boot1.rom
destination: WonderSwanColor/boot1.rom
required: true
md5: 880893bd5a7d53fff826bd76a83d566e
size: 8192
native_system: WonderSwanColor
native_id: WonderSwanColor
name: Bandai WonderSwan Color
docs: "https://github.com/MiSTer-devel/WonderSwan_MiSTer"
platforms/recalbox.yml
+1594 -9
View File
File diff suppressed because it is too large. Load diff
platforms/retroarch.yml
+763
View File
File diff suppressed because it is too large. Load diff
platforms/retrobat.yml
+1065
View File
File diff suppressed because it is too large. Load diff
platforms/retrodeck.yml
+7821
View File
File diff suppressed because it is too large. Load diff
platforms/retropie.yml
+184 -3
View File
@@ -1,6 +1,187 @@
inherits: retroarch
platform: RetroPie
homepage: "https://retropie.org.uk"
base_destination: "BIOS"
# Version is inherited from retroarch.yml (fetched dynamically by scraper)
# RetroPie uses the same cores as RetroArch but with BIOS/ base path
source: "https://codeload.github.com/RetroPie/RetroPie-Setup/tar.gz/refs/heads/master"
base_destination: BIOS
cores:
- "81"
- advmame
- advmame-0.94
- advmame-1.4
- ags
- alephone
- amiberry
- atari800
- basilisk
- beetle-lynx
- beetle-ngp
- beetle-pce
- beetle-pce-fast
- beetle-pcfx
- beetle-psx
- beetle-saturn
- beetle-supergrafx
- beetle-vb
- beetle-wswan
- bennugd
- bluemsx
- bsnes
- cannonball
- caprice32
- capricerpi
- cdogs-sdl
- cgenius
- coolcv
- daphne
- darkplaces-quake
- desmume
- desmume2015
- dgen
- dinothawr
- dirksimple
- dolphin
- dosbox
- dosbox-pure
- dosbox-sdl2
- dosbox-staging
- drastic
- dsda-doom
- dxx-rebirth
- eduke32
- ep128emu
- fbalpha2012
- fbneo
- fbzx
- fceumm
- flycast
- flycast-dev
- fmsx
- freechaf
- freeintv
- frotz
- fs-uae
- fuse
- gambatte
- geargrafx
- gearsystem
- gemrb
- genesis-plus-gx
- giana
- gngeopi
- gpsp
- gw
- gzdoom
- handy
- hatari
- hypseus
- ionfury
- ioquake3
- jumpnbump
- jzintv
- kodi
- kronos
- linapple
- lincity-ng
- love
- love-0.10.2
- lzdoom
- mame
- mame2000
- mame2003
- mame2003-plus
- mame2010
- mame2015
- mame2016
- mame4all
- mesen
- mess
- mess2016
- mgba
- micropolis
- minecraft
- minivmac
- mrboom
- mupen64plus
- mupen64plus-next
- mysticmine
- neocd
- nestopia
- np2kai
- np2pi
- nxengine
- o2em
- openblok
- openbor
- openmsx
- openpht
- openttd
- opentyrian
- opera
- oricutron
- osmose
- parallel-n64
- pcsx-rearmed
- pcsx2
- picodrive
- pifba
- pisnes
- pokemini
- ppsspp
- ppsspp-1.5.4
- prboom
- prosystem
- puae
- puae2021
- px68k
- quake3
- quasi88
- quicknes
- redream
- reicast
- retro8
- retroarch
- rpix86
- scummvm
- scummvm-sdl1
- sdlpop
- sdltrs
- simcoupe
- smsplus-gx
- smw
- snes9x
- snes9x2002
- snes9x2005
- snes9x2010
- solarus
- splitwolf
- srb2
- steamlink
- stella
- stella2014
- stratagus
- superflappybirds
- supermodel3
- supertux
- tgbdual
- theodore
- ti99sim
- ti99sim-sdl1
- tic80
- tyrquake
- uae4all
- uae4arm
- uqm
- vba-next
- vecx
- vice
- virtualjaguar
- vvvvvv
- wolf4sdl
- x1
- xm7
- xrick
- xroar
- yabasanshiro
- yabause
- yquake2
- zesarux
platforms/rocknix.yml
+76
View File
@@ -13,10 +13,14 @@ systems:
destination: xemu/bios/mcpx_1.0.bin
required: true
md5: d49c52a4102f6df7bcf8d0617ac475ed
native_system: xbox
native_name: Microsoft Xbox
- name: Complex_4627v1.03.bin
destination: xemu/bios/Complex_4627v1.03.bin
required: true
md5: 21445c6f28fca7285b0f167ea770d1e5
native_system: xbox
native_name: Microsoft Xbox
native_id: xbox
name: Microsoft Xbox
nintendo-gamecube:
@@ -25,14 +29,20 @@ systems:
destination: GC/EUR/IPL.bin
required: true
md5: db92574caab77a7ec99d4605fd6f2450
native_system: gamecube
native_name: Nintendo GameCube
- name: IPL.bin
destination: GC/JAP/IPL.bin
required: true
md5: b17148254a5799684c7d783206504926
native_system: gamecube
native_name: Nintendo GameCube
- name: IPL.bin
destination: GC/USA/IPL.bin
required: true
md5: b17148254a5799684c7d783206504926
native_system: gamecube
native_name: Nintendo GameCube
native_id: gamecube
name: Nintendo GameCube
nintendo-ds:
@@ -41,30 +51,44 @@ systems:
destination: firmware.bin
required: true
md5: 94bc5094607c5e6598d50472c52f27f2
native_system: nds
native_name: Nintendo DS
- name: bios7.bin
destination: bios7.bin
required: true
md5: df692a80a5b1bc90728bc3dfc76cd948
native_system: nds
native_name: Nintendo DS
- name: bios9.bin
destination: bios9.bin
required: true
md5: a392174eb3e572fed6447e956bde4b25
native_system: nds
native_name: Nintendo DS
- name: dsi_bios7.bin
destination: dsi_bios7.bin
required: true
md5: 559dae4ea78eb9d67702c56c1d791e81
native_system: nds
native_name: Nintendo DS
- name: dsi_bios9.bin
destination: dsi_bios9.bin
required: true
md5: 87b665fce118f76251271c3732532777
native_system: nds
native_name: Nintendo DS
- name: dsi_firmware.bin
destination: dsi_firmware.bin
required: true
md5: 74f23348012d7b3e1cc216c47192ffeb
native_system: nds
native_name: Nintendo DS
- name: dsi_nand.bin
destination: dsi_nand.bin
required: true
md5: d71edf897ddd06bf335feeb68edeb272
native_system: nds
native_name: Nintendo DS
native_id: nds
name: Nintendo DS
nintendo-gb:
@@ -73,6 +97,8 @@ systems:
destination: gb_bios.bin
required: true
md5: 32fbbd84168d3482956eb3c5051637f5
native_system: gb
native_name: Nintendo Gameboy
native_id: gb
name: Nintendo Gameboy
nintendo-gba:
@@ -81,6 +107,8 @@ systems:
destination: gba_bios.bin
required: true
md5: a860e8c0b6d573d191e4ec7db1b1e4f6
native_system: gba
native_name: Nintendo Gameboy Advance
native_id: gba
name: Nintendo Gameboy Advance
nintendo-gbc:
@@ -89,6 +117,8 @@ systems:
destination: gbc_bios.bin
required: true
md5: dbfce9db9deaa2567f6a84fde55f9680
native_system: gbc
native_name: Nintendo Gameboy Color
native_id: gbc
name: Nintendo Gameboy Color
sega-dreamcast:
@@ -97,14 +127,20 @@ systems:
destination: dc/dc_boot.bin
required: true
md5: e10c53c2f8b90bab96ead2d368858623
native_system: dreamcast
native_name: Sega Dreamcast
- name: dc_flash.bin
destination: dc/dc_flash.bin
required: true
md5: 0a93f7940c455905bea6e392dfde92a4
native_system: dreamcast
native_name: Sega Dreamcast
- name: naomi_boot.bin
destination: dc/naomi_boot.bin
required: true
md5: 3bffafac42a7767d8dcecf771f5552ba
native_system: dreamcast
native_name: Sega Dreamcast
native_id: dreamcast
name: Sega Dreamcast
sega-game-gear:
@@ -113,6 +149,8 @@ systems:
destination: bios.gg
required: true
md5: 672e104c3be3a238301aceffc3b23fd6
native_system: gamegear
native_name: Sega Game Gear
native_id: gamegear
name: Sega Game Gear
sega-master-system:
@@ -121,6 +159,8 @@ systems:
destination: bios.sms
required: true
md5: 840481177270d5642a14ca71ee72844c
native_system: mastersystem
native_name: Sega Master System
native_id: mastersystem
name: Sega Master System
sega-mega-cd:
@@ -129,14 +169,20 @@ systems:
destination: bios_CD_E.bin
required: true
md5: e66fa1dc5820d254611fdcdba0662372
native_system: segacd
native_name: Sega CD
- name: bios_CD_U.bin
destination: bios_CD_U.bin
required: true
md5: 2efd74e3232ff260e371b99f84024f7f
native_system: segacd
native_name: Sega CD
- name: bios_CD_J.bin
destination: bios_CD_J.bin
required: true
md5: 278a9397d192149e84e820ac621a8edd
native_system: segacd
native_name: Sega CD
native_id: segacd
name: Sega CD
sega-saturn:
@@ -145,22 +191,32 @@ systems:
destination: sega_101.bin
required: true
md5: 85ec9ca47d8f6807718151cbcca8b964
native_system: saturn
native_name: Sega Saturn
- name: mpr-17933.bin
destination: mpr-17933.bin
required: true
md5: 3240872c70984b6cbfda1586cab68dbe
native_system: saturn
native_name: Sega Saturn
- name: mpr-18811-mx.ic1
destination: mpr-18811-mx.ic1
required: true
md5: 255113ba943c92a54facd25a10fd780c
native_system: saturn
native_name: Sega Saturn
- name: mpr-19367-mx.ic1
destination: mpr-19367-mx.ic1
required: true
md5: 1cd19988d1d72a3e7caa0b73234c96b4
native_system: saturn
native_name: Sega Saturn
- name: saturn_bios.bin
destination: saturn_bios.bin
required: true
md5: af5828fdff51384f99b3c4926be27762
native_system: saturn
native_name: Sega Saturn
native_id: saturn
name: Sega Saturn
sony-playstation:
@@ -169,30 +225,44 @@ systems:
destination: psxonpsp660.bin
required: true
md5: c53ca5908936d412331790f4426c6c33
native_system: psx
native_name: PSX
- name: scph101.bin
destination: scph101.bin
required: true
md5: 6e3735ff4c7dc899ee98981385f6f3d0
native_system: psx
native_name: PSX
- name: scph1001.bin
destination: scph1001.bin
required: true
md5: dc2b9bf8da62ec93e868cfd29f0d067d
native_system: psx
native_name: PSX
- name: scph5500.bin
destination: scph5500.bin
required: true
md5: 8dd7d5296a650fac7319bce665a6a53c
native_system: psx
native_name: PSX
- name: scph5501.bin
destination: scph5501.bin
required: true
md5: 490f666e1afb15b7362b406ed1cea246
native_system: psx
native_name: PSX
- name: scph5502.bin
destination: scph5502.bin
required: true
md5: 32736f17079d0b2b7024407c39bd3050
native_system: psx
native_name: PSX
- name: scph7001.bin
destination: scph7001.bin
required: true
md5: 1e68c231d0896b7eadcad1d7d8e76129
native_system: psx
native_name: PSX
native_id: psx
name: PSX
sony-playstation-2:
@@ -201,10 +271,14 @@ systems:
destination: pcsx2/bios/ps2-0230a-20080220.bin
required: true
md5: 21038400dc633070a78ad53090c53017
native_system: ps2
native_name: PS2
- name: ps2-0230a-20080220.bin
destination: aethersx2/bios/ps2-0230a-20080220.bin
required: true
md5: 21038400dc633070a78ad53090c53017
native_system: ps2
native_name: PS2
native_id: ps2
name: PS2
sony-playstation-3:
@@ -213,5 +287,7 @@ systems:
destination: PS3UPDAT.PUP
required: true
md5: a2b60b6ac1d2e49e230144345616927c
native_system: ps3
native_name: PS3
native_id: ps3
name: PS3
platforms/romm.yml
+381
View File
File diff suppressed because it is too large. Load diff
schemas/platform.schema.json
+140 -15
View File
@@ -5,7 +5,9 @@
"description": "Schema for retrogaming platform BIOS configuration files",
"type": "object",
"additionalProperties": false,
"required": ["platform"],
"required": [
"platform"
],
"properties": {
"inherits": {
"type": "string",
@@ -19,16 +21,28 @@
"type": "string",
"description": "Platform version"
},
"dat_version": { "type": "string" },
"dat_version": {
"type": "string"
},
"cores": {
"oneOf": [
{ "type": "string" },
{ "type": "array", "items": { "type": "string" }, "uniqueItems": true }
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
},
"uniqueItems": true
}
]
},
"standalone_cores": {
"type": "array",
"items": { "type": "string" },
"items": {
"type": "string"
},
"uniqueItems": true
},
"homepage": {
@@ -46,12 +60,21 @@
},
"hash_type": {
"type": "string",
"enum": ["sha1", "md5", "crc32", "sha256"],
"enum": [
"sha1",
"md5",
"crc32",
"sha256"
],
"description": "Primary hash algorithm used by this platform"
},
"verification_mode": {
"type": "string",
"enum": ["existence", "md5", "sha1"],
"enum": [
"existence",
"md5",
"sha1"
],
"default": "existence",
"description": "How the platform verifies BIOS at runtime: existence (file exists), md5 (checks MD5 hash), sha1 (checks SHA1 hash)"
},
@@ -89,8 +112,12 @@
"type": "string",
"description": "Hardware manufacturer"
},
"name": { "type": "string" },
"native_id": { "type": "string" },
"name": {
"type": "string"
},
"native_id": {
"type": "string"
},
"dat_version": {
"type": "string",
"description": "Version of the DAT file used as source for BIOS hashes"
@@ -112,7 +139,9 @@
},
"includes": {
"type": "array",
"items": { "type": "string" },
"items": {
"type": "string"
},
"description": "Shared BIOS groups to include (from _shared.yml)"
},
"data_directories": {
@@ -120,10 +149,16 @@
"items": {
"type": "object",
"additionalProperties": false,
"required": ["ref"],
"required": [
"ref"
],
"properties": {
"ref": { "type": "string" },
"destination": { "type": "string" }
"ref": {
"type": "string"
},
"destination": {
"type": "string"
}
}
}
},
@@ -133,13 +168,45 @@
"$ref": "#/$defs/bios_file"
},
"description": "BIOS files required for this system"
},
"native_check": {
"type": "object",
"additionalProperties": false,
"description": "EmuDeck: the shape of the shell check written for this system",
"properties": {
"kind": {
"type": "string",
"enum": [
"md5",
"file-exists"
]
},
"function": {
"type": "string"
},
"var": {
"type": "string"
},
"array": {
"type": "string"
},
"firmware": {
"type": "string"
},
"keys": {
"type": "string"
}
}
}
}
},
"bios_file": {
"type": "object",
"additionalProperties": false,
"required": ["name", "destination"],
"required": [
"name",
"destination"
],
"properties": {
"name": {
"type": "string",
@@ -186,7 +253,13 @@
},
"storage": {
"type": "string",
"enum": ["embedded", "external", "user_provided", "large_file", "release"],
"enum": [
"embedded",
"external",
"user_provided",
"large_file",
"release"
],
"default": "embedded",
"description": "Storage tier: embedded (in repo), external (download at build), user_provided (manual)"
},
@@ -202,6 +275,58 @@
"zipped_file": {
"type": "string",
"description": "If set, the MD5 hash refers to this file inside the ZIP, not the ZIP itself"
},
"native_system": {
"type": "string",
"description": "System id the platform itself uses for this file"
},
"native_name": {
"type": "string",
"description": "Display name the platform gives its own system"
},
"alt_md5": {
"type": "string",
"pattern": "^[a-fA-F0-9]{32}(,[a-fA-F0-9]{32})*$",
"description": "Second accepted MD5 (Batocera altmd5, ROCKNIX altmd5)"
},
"core": {
"type": "string",
"description": "Cores the platform names for this file (comma-separated)"
},
"note": {
"type": "string",
"description": "Free-text note the platform attaches to this file"
},
"description": {
"type": "string",
"description": "Description the platform attaches to this file"
},
"hash_match_mandatory": {
"type": "boolean",
"description": "Recalbox: a hash mismatch is an error, not a warning"
},
"mandatory_declared": {
"type": "boolean",
"description": "Recalbox: mandatory was stated explicitly, not defaulted"
},
"required_label": {
"type": "string",
"description": "RetroDECK: the requirement wording, which is free text"
},
"component": {
"type": "string",
"description": "RetroDECK: the component whose manifest declares this file"
},
"alt_paths": {
"type": "array",
"items": {
"type": "string"
},
"description": "Further paths the platform accepts for the same file"
},
"native_path": {
"type": "string",
"description": "How the platform itself writes this file reference"
}
}
}
scripts/diff_truth.py
+1 -2
View File
@@ -168,12 +168,11 @@ def main() -> None:
)
parser.add_argument("--truth-dir", default="dist/truth")
parser.add_argument("--platforms-dir", default="platforms")
parser.add_argument("--include-archived", action="store_true")
args = parser.parse_args()
if args.all:
platforms = list_registered_platforms(
args.platforms_dir, include_archived=args.include_archived
args.platforms_dir, include_archived=True
)
else:
platforms = [args.platform]
scripts/export_native.py
+253 -74
View File
@@ -1,40 +1,223 @@
"""Export truth data to native platform formats."""
#!/usr/bin/env python3
"""Rewrite each platform's own BIOS file, corrected by the ground truth.
The output is the file the platform maintains, not a rendering of our data
in its syntax: what the truth can prove is applied, what it says nothing
about is left alone, and what it knows and the platform lacks is added. The
formats that carry code are patched rather than regenerated, so the checker
a platform ships keeps working.
Usage:
python scripts/export_native.py --all --fetch
python scripts/export_native.py --platform recalbox --upstream-dir up/
"""
from __future__ import annotations
import argparse
import sys
import urllib.error
import urllib.request
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import yaml
from common import list_registered_platforms, load_platform_config, yaml_load
from exporter import discover_exporters
from exporter.baseline import build_native_model
OUTPUT_FILENAMES: dict[str, str] = {
"retroarch": "System.dat",
"lakka": "System.dat",
"retropie": "System.dat",
"batocera": "batocera-systems",
"recalbox": "es_bios.xml",
"retrobat": "batocera-systems.json",
"emudeck": "checkBIOS.sh",
"retrodeck": "component_manifest.json",
"romm": "known_bios_files.json",
}
DEFAULT_CACHE = ".cache/upstream-native"
_USER_AGENT = "retrobios-exporter/1.0"
_MAX_BYTES = 64 * 1024 * 1024
def output_path(platform: str, output_dir: str) -> str:
"""Return the full output path for a platform's native export.
def fetch(url: str, destination: Path) -> bytes:
"""Download an original once, then read it from the cache."""
if destination.exists():
return destination.read_bytes()
request = urllib.request.Request(url, headers={"User-Agent": _USER_AGENT})
with urllib.request.urlopen(request, timeout=60) as response:
payload = response.read(_MAX_BYTES + 1)
if len(payload) > _MAX_BYTES:
raise ValueError(f"{url}: response larger than {_MAX_BYTES} bytes")
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_bytes(payload)
return payload
Each platform gets its own subdirectory to avoid filename collisions
(e.g. retroarch, lakka, retropie all produce System.dat).
def pinned_base(wanted: dict[str, str], scraped: dict | None) -> str:
"""The revision our transcription came from, when the YAML records it.
A scraper pins a stable tag (batocera-43.1, BizHawk 2.11.1) and writes
that URL into the platform YAML. Patching the branch tip instead would
correct a file our data never described.
"""
filename = OUTPUT_FILENAMES.get(platform, f"{platform}_bios.dat")
plat_dir = Path(output_dir) / platform
plat_dir.mkdir(parents=True, exist_ok=True)
return str(plat_dir / filename)
source = _raw_url(str((scraped or {}).get("source", "")))
for relative in wanted:
if source.endswith("/" + relative):
return source[: -len(relative)]
return ""
def _raw_url(url: str) -> str:
"""A GitHub blob page names a revision but serves HTML; raw serves bytes."""
marker = "/blob/"
if url.startswith("https://github.com/") and marker in url:
owner_repo, _, path = url[len("https://github.com/") :].partition(marker)
return f"https://raw.githubusercontent.com/{owner_repo}/{path}"
return url
def collect_originals(
exporter: object,
systems: dict,
upstream_dir: Path,
allow_fetch: bool,
scraped: dict | None = None,
) -> tuple[dict[str, str], list[str]]:
"""Gather the platform's own files, from disk or from upstream."""
wanted = dict(exporter.native_sources())
components = getattr(exporter, "components", None)
if callable(components):
for component in components(systems):
wanted[f"{component}/component_manifest.json"] = exporter.component_url(
component
)
base = pinned_base(wanted, scraped)
if base:
wanted = {relative: base + relative for relative in wanted}
root = upstream_dir / exporter.platform_name()
originals: dict[str, str] = {}
missing: list[str] = []
for relative, url in wanted.items():
path = root / relative
payload: bytes | None = None
if path.exists():
payload = path.read_bytes()
elif allow_fetch:
try:
payload = fetch(url, path)
except (urllib.error.URLError, urllib.error.HTTPError, OSError) as exc:
missing.append(f"{relative}: {exc}")
continue
else:
missing.append(f"{relative}: absent and fetching is off")
continue
unpack = getattr(exporter, "unpack", None)
if callable(unpack) and relative.endswith((".zip", ".tar.gz")):
originals.update(unpack(payload))
else:
originals[relative] = payload.decode("utf-8", errors="replace")
return originals, missing
def export_platform(
platform: str,
exporter_class: type,
truth_dir: Path,
output_dir: Path,
platforms_dir: str,
upstream_dir: Path,
allow_fetch: bool,
) -> tuple[bool, list[str]]:
"""Write one platform's corrected file. Returns (ok, messages)."""
messages: list[str] = []
truth_file = truth_dir / f"{platform}.yml"
truth: dict = {}
if truth_file.exists():
with open(truth_file) as handle:
truth = yaml_load(handle) or {}
else:
messages.append(f"no truth for {platform}, only the platform's own data")
try:
scraped = load_platform_config(platform, platforms_dir)
except (FileNotFoundError, OSError):
scraped = None
systems, report = build_native_model(truth, scraped)
if not systems:
return False, ["nothing to write: neither the platform nor the truth has data"]
exporter = exporter_class()
originals, missing = collect_originals(
exporter, systems, upstream_dir, allow_fetch, scraped
)
if missing and exporter.needs_original():
return False, [f"the platform's own file is required: {m}" for m in missing]
messages.extend(
f"original unavailable, written from our data: {m}" for m in missing
)
try:
produced = exporter.render(systems, report, originals, scraped)
except ValueError as exc:
return False, [str(exc)]
if not produced and not exporter.may_write_nothing():
return False, ["the exporter produced no file"]
issues = exporter.validate(systems, produced)
platform_dir = output_dir / platform
for relative, content in produced.items():
path = platform_dir / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
# Only what the format can state: a correction to a field the file has
# no place for is not a change the maintainer will find in the diff, and
# counting it would announce work the export did not do.
carried = exporter.carries()
applied = [
correction
for correction in report.hashes_corrected
if correction.rsplit(" ", 1)[-1] in carried
]
requirements = len(report.required_corrected) if "required" in carried else 0
landed = 0
refused = 0
lost = 0
for system in systems.values():
for entry in system.files:
if not entry.name:
continue
if exporter.writable(entry):
landed += entry.platform is None
elif entry.platform is None:
refused += 1
else:
# The platform declares it and the format still cannot state
# it, so their own file loses a line. Worth saying out loud.
lost += 1
summary = exporter.outcome(systems, produced)
if summary is None:
summary = (
f"{report.files_kept} kept, {landed} added, "
f"{len(applied)} hashes corrected, {requirements} requirements corrected"
)
if refused:
summary += f", {refused} the format cannot state"
if lost:
summary += f", {lost} of theirs dropped"
messages.append(summary)
for correction in applied[:5]:
messages.append(f"hash corrected: {correction}")
if len(applied) > 5:
messages.append(f"and {len(applied) - 5} more hash corrections")
messages.extend(f"INVALID: {issue}" for issue in issues[:10])
if len(issues) > 10:
messages.append(f"and {len(issues) - 10} more validation failures")
return not issues, messages
def run(
@@ -42,87 +225,83 @@ def run(
truth_dir: str,
output_dir: str,
platforms_dir: str,
upstream_dir: str,
allow_fetch: bool,
) -> int:
"""Export truth to native formats, return exit code."""
exporters = discover_exporters()
errors = 0
failures = 0
skipped: list[str] = []
for platform in sorted(platforms):
exporter_cls = exporters.get(platform)
if not exporter_cls:
print(f" SKIP {platform}: no exporter available")
exporter_class = exporters.get(platform)
if not exporter_class:
skipped.append(platform)
print(f" SKIP {platform}: no exporter")
continue
truth_file = Path(truth_dir) / f"{platform}.yml"
if not truth_file.exists():
print(f" SKIP {platform}: {truth_file} not found")
continue
ok, messages = export_platform(
platform,
exporter_class,
Path(truth_dir),
Path(output_dir),
platforms_dir,
Path(upstream_dir),
allow_fetch,
)
label = "OK " if ok else "FAIL"
print(f" {label} {platform}")
for message in messages:
print(f" {message}")
if not ok:
failures += 1
with open(truth_file) as f:
truth_data = yaml_load(f) or {}
if skipped:
print(f"\n{len(skipped)} platform(s) without an exporter: {', '.join(skipped)}")
failures += len(skipped)
scraped: dict | None = None
try:
scraped = load_platform_config(platform, platforms_dir)
except (FileNotFoundError, OSError):
pass
dest = output_path(platform, output_dir)
exporter = exporter_cls()
exporter.export(truth_data, dest, scraped_data=scraped)
issues = exporter.validate(truth_data, dest)
if issues:
print(f" WARN {platform}: {len(issues)} validation issue(s)")
for issue in issues:
print(f" {issue}")
errors += 1
else:
print(f" OK {platform} -> {dest}")
return 1 if errors else 0
return 1 if failures else 0
def main() -> None:
parser = argparse.ArgumentParser(
description="Export truth data to native platform formats.",
description="Rewrite each platform's own BIOS file, corrected.",
)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("--all", action="store_true", help="export all platforms")
group.add_argument("--platform", help="export a single platform")
group.add_argument("--all", action="store_true", help="every platform")
group.add_argument("--platform", help="a single platform")
parser.add_argument("--output-dir", default="dist/upstream")
parser.add_argument("--truth-dir", default="dist/truth")
parser.add_argument("--platforms-dir", default="platforms")
parser.add_argument(
"--output-dir",
default="dist/upstream",
help="output directory",
"--upstream-dir",
default=DEFAULT_CACHE,
help="where the platforms' own files are read and cached",
)
parser.add_argument(
"--truth-dir",
default="dist/truth",
help="truth YAML directory",
)
parser.add_argument(
"--platforms-dir",
default="platforms",
help="platform configs directory",
)
parser.add_argument(
"--include-archived",
"--fetch",
action="store_true",
help="include archived platforms",
help="download a platform's file when it is not in the cache",
)
args = parser.parse_args()
if args.all:
platforms = list_registered_platforms(
args.platforms_dir,
include_archived=args.include_archived,
include_archived=True,
)
else:
platforms = [args.platform]
code = run(platforms, args.truth_dir, args.output_dir, args.platforms_dir)
sys.exit(code)
sys.exit(
run(
platforms,
args.truth_dir,
args.output_dir,
args.platforms_dir,
args.upstream_dir,
args.fetch,
)
)
if __name__ == "__main__":
scripts/exporter/base_exporter.py
+141 -56
View File
@@ -1,81 +1,166 @@
"""Abstract base class for platform exporters."""
"""Contract shared by the platform exporters.
An exporter answers one question: what would this platform's own file look
like if it were corrected. It is handed the platform's file when we have it,
because several of these formats carry code, and a generator that emits only
the data block hands the maintainer something that no longer runs.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from .baseline import NativeFile, NativeSystem, Report
class BaseExporter(ABC):
"""Base class for exporting truth data to native platform formats."""
"""Base class for writing a platform's own BIOS file, corrected."""
@staticmethod
@abstractmethod
def platform_name() -> str:
"""Return the platform identifier this exporter targets."""
@staticmethod
@abstractmethod
def export(
def native_filename() -> str:
"""Return the name the platform gives this file."""
@staticmethod
def carries() -> frozenset[str]:
"""Fields this format has somewhere to put.
A correction the file cannot state is not a correction it delivers,
and counting it would announce a change the maintainer will not find
in the diff.
"""
return frozenset({"md5"})
@staticmethod
def native_sources() -> dict[str, str]:
"""Return {relative path: URL} of the originals this exporter patches.
An exporter that can build its file from nothing returns an empty
mapping. One whose format carries code cannot, and names the file it
needs so the caller can fetch it.
"""
return {}
@staticmethod
def needs_original() -> bool:
"""Whether a faithful export requires the platform's own file."""
return False
@staticmethod
def may_write_nothing() -> bool:
"""Whether producing no file is an outcome rather than a failure.
True only where the unit is a correction rather than a document:
RetroPie has no BIOS list to rewrite, so a run with nothing to
correct writes nothing and is right to.
"""
return False
@abstractmethod
def render(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
"""Export truth data to the native platform format."""
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
"""Return {relative output path: file content}."""
@abstractmethod
def validate(self, truth_data: dict, output_path: str) -> list[str]:
"""Validate exported file against truth data, return list of issues."""
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
"""Check the produced files against what was asked of them."""
# Shared helpers
@classmethod
def exportable(
cls,
systems: dict[str, NativeSystem],
require: str = "",
) -> list[tuple[NativeSystem, list[NativeFile]]]:
"""Systems paired with the files this format can actually carry.
`require` names a hash the format cannot omit. An entry the format
has no way to express is dropped here and counted by the caller,
never written as a half entry the platform would read as a file that
can never verify.
"""
result: list[tuple[NativeSystem, list[NativeFile]]] = []
for system in systems.values():
files = [fe for fe in system.files if fe.name and cls.writable(fe, require)]
if files:
result.append((system, files))
return result
@staticmethod
def _is_pattern(name: str) -> bool:
"""Check if a filename is a placeholder pattern (not a real file)."""
return "<" in name or ">" in name or "*" in name
def requires() -> str:
"""The hash this format cannot write a new entry without."""
return ""
@staticmethod
def _dest(fe: dict) -> str:
"""Get destination path for a file entry, falling back to name."""
return fe.get("path") or fe.get("destination") or fe.get("name", "")
def can_add() -> bool:
"""Whether a file the platform does not declare can be stated at all.
False where an entry is a declaration in code rather than a row:
BizHawk wires each firmware into an option list and a status that
only its source expresses, and writing C# for one is not something
an exporter can do safely.
"""
return True
@classmethod
def writable(cls, fe: NativeFile, require: str = "") -> bool:
"""Whether this format can carry the entry.
An entry the platform already declares is always carried, hash or
no hash: it is in their file today, and an export that drops it
hands back a file poorer than the one it corrects. The requirement
only gates what we would be adding.
"""
if fe.platform is not None:
return True
if not cls.can_add():
return False
require = require or cls.requires()
return not require or bool(fe.hash(require))
def outcome(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> str | None:
"""What this export did, when counting file entries would not say it.
Most formats state one entry per file, so the caller's own count is
the answer. RetroPie states a sentence per package, and a count of
file entries would describe work it did not do.
"""
return None
@staticmethod
def _display_name(
sys_id: str,
scraped_sys: dict | None = None,
) -> str:
"""Get display name for a system from scraped data or slug."""
if scraped_sys:
name = scraped_sys.get("name")
if name:
return name
# Fallback: convert slug to display name with acronym handling
def display_name(system: NativeSystem) -> str:
"""The name the platform shows for a system."""
if system.name:
return system.name
for fe in system.files:
native = fe.native("native_name", "")
if native:
return str(native)
_UPPER = {
"3do",
"cdi",
"cpc",
"cps1",
"cps2",
"cps3",
"dos",
"gba",
"gbc",
"hle",
"msx",
"nes",
"nds",
"ngp",
"psp",
"psx",
"sms",
"snes",
"stv",
"tvc",
"vb",
"zx",
"3do", "cdi", "cpc", "cps1", "cps2", "cps3", "dos", "gba", "gbc",
"hle", "msx", "nes", "nds", "ngp", "psp", "psx", "sms", "snes",
"stv", "tvc", "vb", "zx",
}
parts = sys_id.replace("-", " ").split()
result = []
for p in parts:
if p.lower() in _UPPER:
result.append(p.upper())
else:
result.append(p.capitalize())
return " ".join(result)
parts = system.native_id.replace("-", " ").replace("_", " ").split()
return " ".join(
p.upper() if p.lower() in _UPPER else p.capitalize() for p in parts
)
scripts/exporter/baseline.py
+373
View File
@@ -0,0 +1,373 @@
"""Reconciliation of a platform's own declarations with the ground truth.
An export is not the truth rendered in a native syntax. It is the platform's
file, corrected: what the truth can prove is applied, what it says nothing
about is left alone, and what it knows and the platform lacks is added. A
platform that loses two thirds of its systems to an export cannot use it.
Systems are keyed by the identifier the platform itself uses. Several of our
slugs collapse onto one native id (Recalbox files pcengine, pcenginecd and
supergrafx under one slug) and one slug can carry several native ids, so the
grouping is rebuilt from the per-file native_system the scrapers record.
"""
from __future__ import annotations
import sys
from dataclasses import dataclass, field
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from common import _norm_system_id
HASH_FIELDS = ("sha1", "md5", "sha256", "crc32")
def _hash_values(entry: dict, field_name: str) -> list[str]:
"""Return a hash field as a list, whatever shape it was written in."""
raw = entry.get(field_name)
if not raw:
return []
if isinstance(raw, list):
return [str(v).strip().lower() for v in raw if str(v).strip()]
return [v.strip().lower() for v in str(raw).split(",") if v.strip()]
def _is_placeholder(name: str) -> bool:
return "<" in name or ">" in name or "*" in name
@dataclass
class NativeFile:
"""One file as the platform will read it, after correction."""
name: str
destination: str
native_system: str
platform: dict | None = None
truth: dict | None = None
corrections: list[str] = field(default_factory=list)
@property
def origin(self) -> str:
if self.platform is not None and self.truth is not None:
return "both"
return "platform" if self.platform is not None else "truth"
def hashes(self, field_name: str) -> list[str]:
"""Accepted values for a hash, truth first when it has an opinion.
The truth is read from the emulator's source; the platform list is a
secondary source. When both speak and disagree, the truth decides and
the divergence is recorded, never silently merged: an emulator that
rejects a file will reject it whatever the platform declares.
"""
truth_values = _hash_values(self.truth or {}, field_name)
platform_values = _hash_values(self.platform or {}, field_name)
if truth_values and platform_values and not set(truth_values) & set(
platform_values
):
return truth_values
if truth_values:
# Keep the platform's extra accepted revisions alongside ours.
merged = list(truth_values)
merged.extend(v for v in platform_values if v not in merged)
return merged
return platform_values
def hash(self, field_name: str) -> str:
values = self.hashes(field_name)
return values[0] if values else ""
@property
def required(self) -> bool:
if self.truth is not None and self.truth.get("required") is not None:
return bool(self.truth["required"])
if self.platform is not None and self.platform.get("required") is not None:
return bool(self.platform["required"])
return True
@property
def priority(self) -> int | None:
"""Where the code looks for this file, lowest first.
DuckStation's FindBIOSImageInDirectory keeps the image whose
priority is lower, and a search list a core walks in order maps
onto it as 1, 2, 3. Where cores disagree this is the best rank any
of them gives: nothing is dropped on it, so a file that is some
core's first choice is read early. None when nothing states one.
"""
for entry in (self.truth, self.platform):
if entry and entry.get("priority") is not None:
return int(entry["priority"])
return None
def size(self) -> int | None:
for entry in (self.truth, self.platform):
if entry and entry.get("size"):
return int(entry["size"])
return None
def native(self, key: str, default: object = "") -> object:
"""Read a field the platform declares and we only carry through."""
for entry in (self.platform, self.truth):
if entry and entry.get(key) not in (None, ""):
return entry[key]
return default
def cores(self) -> list[str]:
"""Cores that want this file, the platform's naming preferred."""
declared = self.native("core", "")
if declared:
return [c.strip() for c in str(declared).split(",") if c.strip()]
if self.truth:
return [f"libretro/{c}" for c in self.truth.get("_cores", [])]
return []
@dataclass
class NativeSystem:
"""One system as the platform names it."""
native_id: str
name: str = ""
files: list[NativeFile] = field(default_factory=list)
from_platform: bool = False
@property
def origin(self) -> str:
return "platform" if self.from_platform else "truth"
@dataclass
class Report:
"""What the reconciliation changed, so the caller can say it out loud."""
systems_kept: int = 0
systems_added: int = 0
files_kept: int = 0
files_added: int = 0
hashes_corrected: list[str] = field(default_factory=list)
required_corrected: list[str] = field(default_factory=list)
@property
def corrections(self) -> int:
return len(self.hashes_corrected) + len(self.required_corrected)
def _native_id_of(sys_key: str, sys_data: dict, file_entry: dict) -> str:
"""The platform's own id for the system a file belongs to."""
return (
file_entry.get("native_system")
or sys_data.get("native_id")
or sys_key
)
def _match_key(entry: dict) -> tuple[str, str]:
dest = str(entry.get("destination") or entry.get("path") or entry.get("name", ""))
return dest.casefold(), str(entry.get("name", "")).casefold()
def build_native_model(
truth: dict,
scraped: dict | None,
) -> tuple[dict[str, NativeSystem], Report]:
"""Rebuild the platform's systems, corrected by the truth.
Returns the systems keyed by native id, in the platform's own order
first and truth-only additions after, plus what changed.
"""
report = Report()
systems: dict[str, NativeSystem] = {}
scraped_systems = (scraped or {}).get("systems", {})
# Pass 1: the platform's own file, grouped as the platform groups it.
for sys_key, sys_data in scraped_systems.items():
for file_entry in sys_data.get("files", []):
native_id = _native_id_of(sys_key, sys_data, file_entry)
system = systems.get(native_id)
if system is None:
system = NativeSystem(
native_id=native_id,
name=str(file_entry.get("native_name") or sys_data.get("name", "")),
from_platform=True,
)
systems[native_id] = system
report.systems_kept += 1
name = str(file_entry.get("name", ""))
destination = str(file_entry.get("destination") or name)
system.files.append(
NativeFile(
name=name,
destination=destination,
native_system=native_id,
platform=file_entry,
)
)
report.files_kept += 1
# Which native ids a truth system may contribute to.
norm_to_scraped: dict[str, str] = {
_norm_system_id(key): key for key in scraped_systems
}
native_by_norm: dict[str, str] = {}
for system in systems.values():
native_by_norm.setdefault(_norm_system_id(system.native_id), system.native_id)
def _target_native_ids(truth_sid: str) -> list[str]:
scraped_key = truth_sid if truth_sid in scraped_systems else None
if scraped_key is None:
scraped_key = norm_to_scraped.get(_norm_system_id(truth_sid))
if scraped_key is not None:
sys_data = scraped_systems[scraped_key]
ids = {
_native_id_of(scraped_key, sys_data, fe)
for fe in sys_data.get("files", [])
}
if not ids:
return [sys_data.get("native_id") or scraped_key]
# A file the platform does not declare joins the system's primary
# id, not whichever of its native ids sorts first: Recalbox files
# pcengine, pcenginecd and supergrafx under one slug, and an
# addition belongs to the one the system is named for.
primary = sys_data.get("native_id")
ordered = sorted(ids)
if primary in ids:
ordered.remove(primary)
ordered.insert(0, primary)
return ordered
direct = native_by_norm.get(_norm_system_id(truth_sid))
return [direct] if direct else [truth_sid]
# Pass 2: apply the truth onto that grouping.
for truth_sid in sorted(truth.get("systems", {})):
truth_sys = truth["systems"][truth_sid]
truth_files = truth_sys.get("files", [])
if not truth_files:
continue
target_ids = _target_native_ids(truth_sid)
for truth_entry in truth_files:
name = str(truth_entry.get("name", ""))
if not name or name.startswith("_") or _is_placeholder(name):
continue
t_dest, t_name = _match_key(truth_entry)
t_hashes = {
value
for field_name in HASH_FIELDS
for value in _hash_values(truth_entry, field_name)
}
candidates = [
candidate
for native_id in target_ids
for candidate in systems.get(
native_id, NativeSystem(native_id)
).files
if candidate.truth is None
]
def by_destination(candidate: NativeFile) -> bool:
theirs = _match_key(candidate.platform or {})[0]
return bool(t_dest) and theirs == t_dest
def by_name(candidate: NativeFile) -> bool:
theirs = _match_key(candidate.platform or {})[1]
return bool(t_name) and theirs == t_name
def by_hash(candidate: NativeFile) -> bool:
if not t_hashes:
return False
theirs = {
value
for field_name in HASH_FIELDS
for value in _hash_values(candidate.platform or {}, field_name)
}
return bool(t_hashes & theirs)
# Tried in order across every candidate, not per candidate: with
# three IPL.bin under one system, separated only by their path, a
# first-match-wins scan would attach the truth to whichever came
# first and correct the wrong region's file.
matched: NativeFile | None = None
for test in (by_destination, by_name, by_hash):
matched = next((c for c in candidates if test(c)), None)
if matched is not None:
break
if matched is not None:
matched.truth = truth_entry
for field_name in HASH_FIELDS:
ours = set(_hash_values(truth_entry, field_name))
theirs = set(_hash_values(matched.platform or {}, field_name))
if ours and theirs and not ours & theirs:
matched.corrections.append(field_name)
report.hashes_corrected.append(
f"{matched.native_system}/{matched.name} {field_name}"
)
t_req = truth_entry.get("required")
p_req = (matched.platform or {}).get("required")
if (
t_req is not None
and p_req is not None
and bool(t_req) != bool(p_req)
):
matched.corrections.append("required")
report.required_corrected.append(
f"{matched.native_system}/{matched.name}"
)
continue
# The truth knows a file the platform does not declare.
native_id = target_ids[0]
system = systems.get(native_id)
if system is None:
system = NativeSystem(native_id=native_id, from_platform=False)
systems[native_id] = system
report.systems_added += 1
destination = str(
truth_entry.get("path") or truth_entry.get("destination") or name
)
system.files.append(
NativeFile(
name=name,
destination=destination,
native_system=native_id,
truth=truth_entry,
)
)
report.files_added += 1
# A reader takes the files in the order the list gives, so the one the
# code looks for first is named first. Only what we add is ordered: what
# the platform already wrote keeps the place the platform gave it.
for system in systems.values():
head = [fe for fe in system.files if fe.platform is not None]
tail = [fe for fe in system.files if fe.platform is None]
system.files = head + search_order(tail)
return systems, report
def search_order(files: list[NativeFile]) -> list[NativeFile]:
"""Order files the way the code looks for them, best first.
`priority:` is that order where the source states it, lowest first.
Where it does not, the order the entries were declared in is the order
the code walks, so it is left alone.
"""
ranked = [(fe.priority, position, fe) for position, fe in enumerate(files)]
return [
fe
for _, _, fe in sorted(
ranked,
key=lambda item: (
(0, item[0]) if item[0] is not None else (1, item[1])
),
)
]
scripts/exporter/batocera_exporter.py
+274 -82
View File
@@ -1,109 +1,301 @@
"""Exporter for Batocera batocera-systems format.
"""Exporter for Batocera's batocera-systems.
Produces a Python dict matching the exact format of
batocera-linux/batocera-scripts/scripts/batocera-systems.
batocera-systems is an executable script: the systems dict is one block
inside it, the rest is the checker Batocera runs. Only the block is
rewritten, entry by entry, so the comments the maintainers wrote between
entries and the code around them survive untouched.
"""
from __future__ import annotations
from pathlib import Path
import re
from .base_exporter import BaseExporter
from .baseline import NativeFile, NativeSystem, Report
SOURCE_URL = (
"https://raw.githubusercontent.com/batocera-linux/batocera.linux/master"
"/package/batocera/core/batocera-scripts/scripts/batocera-systems"
)
_ENTRY_START = re.compile(r'^(\s{4})"([^"]+)":\s*\{')
_INDENT = " " * 4
class Exporter(BaseExporter):
"""Export truth data to Batocera batocera-systems format."""
"""Write Batocera's batocera-systems, corrected."""
@staticmethod
def platform_name() -> str:
return "batocera"
def export(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
# Build native_id and display name maps from scraped data
native_map: dict[str, str] = {}
if scraped_data:
for sys_id, sys_data in scraped_data.get("systems", {}).items():
nid = sys_data.get("native_id")
if nid:
native_map[sys_id] = nid
@staticmethod
def native_filename() -> str:
return "batocera-systems"
lines: list[str] = ["systems = {", ""]
@staticmethod
def requires() -> str:
return "md5"
systems = truth_data.get("systems", {})
for sys_id in sorted(systems):
sys_data = systems[sys_id]
files = sys_data.get("files", [])
if not files:
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"md5"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"batocera-systems": SOURCE_URL}
@staticmethod
def needs_original() -> bool:
# The data block is a fraction of the file; the rest is the checker.
return True
def _bios_files(self, files: list[NativeFile]) -> str:
return ", ".join(self._bios_items(files))
def _bios_items(self, files: list[NativeFile]) -> list[str]:
parts: list[str] = []
for fe in files:
# The platform states an unhashed file as an empty md5 rather
# than leaving it out, and so do we.
item = [f'"md5": "{fe.hash("md5")}"']
alt = fe.native("alt_md5", "")
if alt:
item.append(f'"altmd5": "{alt}"')
declared = fe.native("native_path", "")
path = str(declared) if declared else f"bios/{fe.destination}"
item.append(f'"file": "{path}"')
zipped = fe.native("zipped_file", "")
if zipped:
item.append(f'"zippedFile": "{zipped}"')
parts.append("{ " + ", ".join(item) + " }")
return parts
def _entry_line(self, system: NativeSystem, files: list[NativeFile]) -> str:
return (
f'{_INDENT}"{system.native_id}": '
f'{{ "name": "{self.display_name(system)}", '
f'"biosFiles": [ {self._bios_files(files)} ] }},'
)
@staticmethod
def _item_notes(lines: list[str]) -> tuple[dict[str, str], dict[str, list[str]]]:
"""The notes a maintainer wrote about each file, keyed by its path.
Rewriting an entry as one line would drop them, and a note like
"ideally - 94bc50..." is the only record of why a hash is blank.
A comment on its own line belongs to the file below it; a comment at
the end of a line belongs to the file on it.
"""
trailing: dict[str, str] = {}
leading: dict[str, list[str]] = {}
pending: list[str] = []
for line in lines:
stripped = line.strip()
hash_at = line.find("#")
if stripped.startswith("#"):
pending.append(stripped)
continue
path = re.search(r'"file"\s*:\s*"([^"]+)"', line)
if not path:
continue
key = path.group(1)
if pending:
leading[key] = pending
pending = []
if 0 <= hash_at and hash_at > line.index(key):
trailing[key] = line[hash_at:].rstrip()
return trailing, leading
native_id = native_map.get(sys_id, sys_id)
scraped_sys = (
scraped_data.get("systems", {}).get(sys_id) if scraped_data else None
def _entry_block(
self,
system: NativeSystem,
files: list[NativeFile],
original: list[str],
) -> list[str]:
"""One entry, keeping the layout and the notes it was written with."""
trailing, leading = self._item_notes(original)
items = self._bios_items(files)
one_line = len(original) == 1 and not trailing and not leading
if one_line:
return [self._entry_line(system, files)]
head = (
f'{_INDENT}"{system.native_id}": '
f'{{ "name": "{self.display_name(system)}", "biosFiles": ['
)
pad = " " * (len(_INDENT) + 4)
lines = [head]
for index, item in enumerate(items):
key = self._item_path(item)
lines.extend(f"{pad}{note}" for note in leading.get(key, []))
separator = "," if index < len(items) - 1 else ""
note = trailing.get(key, "")
suffix = f" {note}" if note else ""
lines.append(f"{pad}{item}{separator}{suffix}")
lines.append(f"{_INDENT}] }},")
return lines
@staticmethod
def _item_path(item: str) -> str:
match = re.search(r'"file": "([^"]+)"', item)
return match.group(1) if match else ""
@staticmethod
def _split_original(original: str) -> tuple[list[str], list[str], list[str]]:
"""Cut the script into what precedes the dict, the dict, what follows."""
lines = original.split("\n")
start = next(
(i for i, line in enumerate(lines) if line.startswith("systems = {")),
None,
)
if start is None:
raise ValueError("batocera-systems: no systems dict found")
end = next(
(i for i in range(start + 1, len(lines)) if lines[i].startswith("}")),
None,
)
if end is None:
raise ValueError("batocera-systems: the systems dict is not closed")
return lines[: start + 1], lines[start + 1 : end], lines[end:]
@staticmethod
def _entry_spans(body: list[str]) -> dict[str, tuple[int, int]]:
"""Locate each top-level entry, which may wrap over several lines."""
spans: dict[str, tuple[int, int]] = {}
index = 0
while index < len(body):
match = _ENTRY_START.match(body[index])
if not match:
index += 1
continue
key = match.group(2)
depth = 0
end = index
for cursor in range(index, len(body)):
depth += body[cursor].count("{") + body[cursor].count("[")
depth -= body[cursor].count("}") + body[cursor].count("]")
if depth <= 0:
end = cursor
break
else:
end = len(body) - 1
spans[key] = (index, end)
index = end + 1
return spans
@staticmethod
def _parse_entry(lines: list[str]) -> dict | None:
"""Evaluate one entry so two spellings of the same data compare equal."""
text = "\n".join(lines).strip().rstrip(",")
namespace: dict[str, object] = {}
try:
exec(f"entry = {{{text}}}", {}, namespace) # noqa: S102
except (SyntaxError, ValueError, TypeError):
return None
entry = namespace.get("entry")
return entry if isinstance(entry, dict) else None
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
exportable = dict(
(system.native_id, (system, files))
for system, files in self.exportable(systems, require="md5")
)
original = originals.get(self.native_filename(), "")
if not original:
raise ValueError(
f"{self.native_filename()} cannot be written without the "
"platform's own file: the systems dict is a fraction of a "
"script, and the rest of it is the checker"
)
display_name = self._display_name(sys_id, scraped_sys)
# Build md5 lookup from scraped data for this system
scraped_md5: dict[str, str] = {}
if scraped_data:
s_sys = scraped_data.get("systems", {}).get(sys_id, {})
for sf in s_sys.get("files", []):
sname = sf.get("name", "").lower()
smd5 = sf.get("md5", "")
if sname and smd5:
scraped_md5[sname] = smd5
head, body, tail = self._split_original(original)
spans = self._entry_spans(body)
# Build biosFiles entries as compact single-line dicts
# Original format ALWAYS has md5 — use scraped md5 as fallback
bios_parts: list[str] = []
for fe in files:
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
continue
dest = self._dest(fe)
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
if not md5:
md5 = scraped_md5.get(name.lower(), "")
rebuilt: list[str] = []
written: set[str] = set()
index = 0
for key, (start, end) in sorted(spans.items(), key=lambda kv: kv[1][0]):
rebuilt.extend(body[index:start])
pair = exportable.get(key)
index = end + 1
if pair is None:
# The truth has nothing to say and no file left to declare.
rebuilt.extend(body[start:end + 1])
continue
written.add(key)
original_lines = body[start:end + 1]
replacement = self._entry_line(*pair)
before = self._parse_entry(original_lines)
after = self._parse_entry([replacement])
if before is not None and before == after:
# Nothing changed: keep the maintainer's own lines, comments
# and alignment included, so the diff shows only corrections.
rebuilt.extend(original_lines)
else:
rebuilt.extend(self._entry_block(*pair, original_lines))
rebuilt.extend(body[index:])
# Original format requires md5 for every entry — skip without
if not md5:
continue
bios_parts.append(f'{{ "md5": "{md5}", "file": "bios/{dest}" }}')
added = [
self._entry_line(system, files)
for native_id, (system, files) in exportable.items()
if native_id not in written
]
if added:
while rebuilt and not rebuilt[-1].strip():
rebuilt.pop()
rebuilt.append("")
rebuilt.extend(sorted(added))
rebuilt.append("")
bios_str = ", ".join(bios_parts)
line = (
f' "{native_id}": '
f'{{ "name": "{display_name}", '
f'"biosFiles": [ {bios_str} ] }},'
)
lines.append(line)
return {self.native_filename(): "\n".join([*head, *rebuilt, *tail])}
lines.append("")
lines.append("}")
lines.append("")
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
def validate(self, truth_data: dict, output_path: str) -> list[str]:
content = Path(output_path).read_text(encoding="utf-8")
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
content = produced[self.native_filename()]
issues: list[str] = []
for sys_data in truth_data.get("systems", {}).values():
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
continue
# Skip entries without md5 (not exportable in this format)
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
if not md5:
continue
dest = self._dest(fe)
if dest not in content and name not in content:
issues.append(f"missing: {name}")
namespace: dict[str, object] = {}
block = content.split("\nsystems = {", 1)
if len(block) != 2:
return ["no systems dict in the output"]
end = block[1].find("\n}")
if end < 0:
return ["the systems dict is not closed"]
try:
exec("systems = {" + block[1][:end] + "\n}", {}, namespace) # noqa: S102
except SyntaxError as exc:
return [f"the systems dict does not parse: {exc}"]
exported = namespace.get("systems", {})
if not isinstance(exported, dict):
return ["the systems dict did not evaluate to a dict"]
for system, files in self.exportable(systems, require="md5"):
entry = exported.get(system.native_id)
if entry is None:
issues.append(f"system absent: {system.native_id}")
continue
declared = {bios.get("file", "") for bios in entry.get("biosFiles", [])}
for fe in files:
path = str(fe.native("native_path", "")) or f"bios/{fe.destination}"
if path not in declared:
issues.append(f"absent: {system.native_id}/{fe.name}")
for native_id, entry in exported.items():
if not entry.get("biosFiles"):
issues.append(f"empty entry: {native_id}")
if "def checkBios(" not in content:
issues.append("the checker the script exists for is missing")
return issues
scripts/exporter/bizhawk_exporter.py
+153
View File
@@ -0,0 +1,153 @@
"""Exporter for BizHawk's FirmwareDatabase.cs.
The database is C#: every firmware is a call whose arguments are the SHA1,
the size, the file name and a description, wired into option lists and
status flags that only the source expresses. The calls are rewritten in
place, and nothing else in the file is touched.
"""
from __future__ import annotations
import re
from .base_exporter import BaseExporter
from .baseline import NativeFile, NativeSystem, Report
SOURCE_URL = (
"https://raw.githubusercontent.com/TASEmulators/BizHawk/master"
"/src/BizHawk.Emulation.Common/Database/FirmwareDatabase.cs"
)
# File("<sha1>", <size>, "<name>", ...) and the same three arguments inside
# FirmwareAndOption(<sha1>, <size>, <system>, <id>, <name>, ...).
_FILE_CALL = re.compile(
r'(File\(\s*")([0-9A-Fa-f]{40})("\s*,\s*)(\d+)(\s*,\s*")([^"]+)(")'
)
_FIRMWARE_AND_OPTION = re.compile(
r'(FirmwareAndOption\(\s*")([0-9A-Fa-f]{40})("\s*,\s*)(\d+)'
r'(\s*,\s*"[^"]*"\s*,\s*"[^"]*"\s*,\s*")([^"]+)(")'
)
class Exporter(BaseExporter):
"""Write BizHawk's FirmwareDatabase.cs, corrected."""
@staticmethod
def platform_name() -> str:
return "bizhawk"
@staticmethod
def native_filename() -> str:
return "FirmwareDatabase.cs"
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"sha1", "size"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"FirmwareDatabase.cs": SOURCE_URL}
@staticmethod
def can_add() -> bool:
# A firmware is a call wired into an option list and a status; the
# exporter corrects the calls that exist, it does not write C#.
return False
@staticmethod
def needs_original() -> bool:
# The database is code: option lists, statuses and the systems they
# hang off exist nowhere else.
return True
@staticmethod
def _unambiguous(systems: dict[str, NativeSystem]) -> dict[str, NativeFile]:
"""Files whose name identifies exactly one entry with a SHA1.
BizHawk names a firmware by file name inside a system, and the same
name recurs across systems. Correcting on a name that resolves to
two different sets of bytes would corrupt the database, so only the
names that resolve to one are touched.
"""
seen: dict[str, list[NativeFile]] = {}
for system in systems.values():
for fe in system.files:
if fe.hash("sha1"):
seen.setdefault(fe.name.casefold(), []).append(fe)
resolved: dict[str, NativeFile] = {}
for name, entries in seen.items():
hashes = {fe.hash("sha1").lower() for fe in entries}
if len(hashes) == 1:
resolved[name] = entries[0]
return resolved
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
source = originals.get(self.native_filename(), "")
if not source:
raise ValueError(
"FirmwareDatabase.cs cannot be written without BizHawk's own "
"file: the database is C#, not data"
)
index = self._unambiguous(systems)
def commented_out(text: str, position: int) -> bool:
"""Whether the call sits on a line the compiler never sees.
BizHawk keeps disabled entries in place behind //, and a hash
written into one of those is a change to a comment.
"""
line_start = text.rfind("\n", 0, position) + 1
return text[line_start:position].lstrip().startswith("//")
def rewrite(match: re.Match[str], name_group: int) -> str:
if commented_out(match.string, match.start()):
return match.group(0)
name = match.group(name_group)
fe = index.get(name.casefold())
if fe is None:
return match.group(0)
sha1 = fe.hash("sha1").upper()
size = fe.size() or int(match.group(4))
groups = list(match.groups())
groups[1] = sha1
groups[3] = str(size)
return "".join(groups)
patched = _FILE_CALL.sub(lambda m: rewrite(m, 6), source)
patched = _FIRMWARE_AND_OPTION.sub(lambda m: rewrite(m, 6), patched)
return {self.native_filename(): patched}
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
content = produced[self.native_filename()]
issues: list[str] = []
if "FirmwareDatabase" not in content:
issues.append("the class the database lives in is missing")
if content.count("{") != content.count("}"):
issues.append("braces are unbalanced, the file would not compile")
index = self._unambiguous(systems)
declared: dict[str, str] = {}
for pattern in (_FILE_CALL, _FIRMWARE_AND_OPTION):
for match in pattern.finditer(content):
line_start = content.rfind("\n", 0, match.start()) + 1
if content[line_start : match.start()].lstrip().startswith("//"):
continue
declared[match.group(6).casefold()] = match.group(2).lower()
for name, fe in index.items():
written = declared.get(name)
if written is not None and written != fe.hash("sha1").lower():
issues.append(f"hash not applied: {fe.name}")
return issues
scripts/exporter/emudeck_exporter.py
+135 -184
View File
@@ -1,215 +1,166 @@
"""Exporter for EmuDeck checkBIOS.sh format.
"""Exporter for EmuDeck's checkBIOS.sh.
Produces a bash script matching the exact pattern of EmuDeck's
functions/checkBIOS.sh: per-system check functions with MD5 arrays
inside the function body, iterating over $biosPath/* files.
Two patterns:
- MD5 pattern: systems with known hashes, loop $biosPath/*, md5sum each, match
- File-exists pattern: systems with specific paths, check -f
checkBIOS.sh is a shell library: each system is a function EmuDeck calls by
name, and the only data in it is the MD5 list each function matches against.
Writing the functions from a table would publish a file missing whichever
checks the table forgot, so the original is patched instead and every
function it defines keeps its shape, its scan directory and its output.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from scraper.emudeck_scraper import FUNCTION_HASH_MAP, _RE_FUNC, _RE_LOCAL_HASHES
from .base_exporter import BaseExporter
from .baseline import NativeSystem, Report
# Map our system IDs to EmuDeck function naming conventions
_SYSTEM_CONFIG: dict[str, dict] = {
"sony-playstation": {
"func": "checkPS1BIOS",
"var": "PSXBIOS",
"array": "PSBios",
"pattern": "md5",
},
"sony-playstation-2": {
"func": "checkPS2BIOS",
"var": "PS2BIOS",
"array": "PS2Bios",
"pattern": "md5",
},
"sega-mega-cd": {
"func": "checkSegaCDBios",
"var": "SEGACDBIOS",
"array": "CDBios",
"pattern": "md5",
},
"sega-saturn": {
"func": "checkSaturnBios",
"var": "SATURNBIOS",
"array": "SaturnBios",
"pattern": "md5",
},
"sega-dreamcast": {
"func": "checkDreamcastBios",
"var": "BIOS",
"array": "hashes",
"pattern": "md5",
},
"nintendo-ds": {
"func": "checkDSBios",
"var": "BIOS",
"array": "hashes",
"pattern": "md5",
},
"nintendo-switch": {
"func": "checkCitronBios",
"pattern": "file-exists",
"firmware_path": "$biosPath/citron/firmware",
"keys_path": "$biosPath/citron/keys/prod.keys",
},
}
def _make_md5_function(cfg: dict, md5s: list[str]) -> list[str]:
"""Generate a MD5-checking function matching EmuDeck's exact pattern."""
func = cfg["func"]
var = cfg["var"]
array = cfg["array"]
md5_str = " ".join(md5s)
return [
f"{func}(){{",
"",
f'\t{var}="NULL"',
"",
'\tfor entry in "$biosPath/"*',
"\tdo",
'\t\tif [ -f "$entry" ]; then',
'\t\t\tmd5=($(md5sum "$entry"))',
f'\t\t\tif [[ "${var}" != true ]]; then',
f"\t\t\t\t{array}=({md5_str})",
f'\t\t\t\tfor i in "${{{array}[@]}}"',
"\t\t\t\tdo",
'\t\t\t\tif [[ "$md5" == *"${i}"* ]]; then',
f"\t\t\t\t\t{var}=true",
"\t\t\t\t\tbreak",
"\t\t\t\telse",
f"\t\t\t\t\t{var}=false",
"\t\t\t\tfi",
"\t\t\t\tdone",
"\t\t\tfi",
"\t\tfi",
"\tdone",
"",
"",
f"\tif [ ${var} == true ]; then",
'\t\techo "$entry true";',
"\telse",
'\t\techo "false";',
"\tfi",
"}",
]
def _make_file_exists_function(cfg: dict) -> list[str]:
"""Generate a file-exists function matching EmuDeck's pattern."""
func = cfg["func"]
firmware = cfg.get("firmware_path", "")
keys = cfg.get("keys_path", "")
return [
f"{func}(){{",
"",
f'\tlocal FIRMWARE="{firmware}"',
f'\tlocal KEYS="{keys}"',
'\tif [[ -f "$KEYS" ]] && [[ "$( ls -A "$FIRMWARE")" ]]; then',
'\t\t\techo "true";',
"\telse",
'\t\t\techo "false";',
"\tfi",
"}",
]
SOURCE_URL = (
"https://raw.githubusercontent.com/dragoonDorise/EmuDeck/main"
"/functions/checkBIOS.sh"
)
_MD5 = re.compile(r"^[0-9a-f]{32}$")
class Exporter(BaseExporter):
"""Export truth data to EmuDeck checkBIOS.sh format."""
"""Write EmuDeck's checkBIOS.sh, corrected."""
@staticmethod
def platform_name() -> str:
return "emudeck"
def export(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
lines: list[str] = ["#!/bin/bash"]
@staticmethod
def native_filename() -> str:
return "checkBIOS.sh"
systems = truth_data.get("systems", {})
@staticmethod
def requires() -> str:
return "md5"
for sys_id, cfg in sorted(_SYSTEM_CONFIG.items(), key=lambda x: x[1]["func"]):
sys_data = systems.get(sys_id)
if not sys_data:
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"md5"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"checkBIOS.sh": SOURCE_URL}
@staticmethod
def needs_original() -> bool:
# The checks are code, and EmuDeck calls them by name.
return True
@staticmethod
def can_add() -> bool:
"""A hash is corrected in place, never added to an array.
EmuDeck's frontend calls one check for several emulators
(EmulatorsDetailPage.jsx: 'ra' asks checkPS1BIOS, checkSegaCDBios,
checkSaturnBios, checkDSBios and checkDreamcastBios, and
'duckstation' asks checkPS1BIOS as well), and nothing in
checkBIOS.sh names them. Growing an array therefore changes answers
for consumers the array does not list: DuckStation boots from a PS2
image, the RetroArch PSX cores do not, so adding the ones
DuckStation accepts would report a BIOS to a card that has none.
Correcting a value in place changes no consumer's set.
"""
return False
@classmethod
def _md5s(cls, systems: dict[str, NativeSystem], system_id: str) -> list[str]:
"""Every MD5 the system accepts, in a stable order, deduplicated."""
seen: list[str] = []
for system in systems.values():
if system.native_id != system_id:
continue
for fe in system.files:
if not cls.writable(fe):
continue
for value in fe.hashes("md5"):
if _MD5.match(value) and value not in seen:
seen.append(value)
return seen
lines.append("")
def _function_spans(self, script: str) -> list[tuple[str, int, int]]:
"""Name and byte span of every check the script defines."""
matches = list(_RE_FUNC.finditer(script))
spans: list[tuple[str, int, int]] = []
for index, match in enumerate(matches):
end = (
matches[index + 1].start()
if index + 1 < len(matches)
else len(script)
)
spans.append((match.group(1), match.start(), end))
return spans
if cfg["pattern"] == "md5":
md5s: list[str] = []
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if self._is_pattern(name) or name.startswith("_"):
continue
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5s.extend(
m for m in md5 if m and re.fullmatch(r"[a-f0-9]{32}", m)
)
elif md5 and re.fullmatch(r"[a-f0-9]{32}", md5):
md5s.append(md5)
if md5s:
lines.extend(_make_md5_function(cfg, md5s))
elif cfg["pattern"] == "file-exists":
lines.extend(_make_file_exists_function(cfg))
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
script = originals.get(self.native_filename(), "")
if not script:
raise ValueError(
"checkBIOS.sh cannot be written without EmuDeck's own file: "
"the checks are code, not data"
)
lines.append("")
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
pieces: list[str] = []
cursor = 0
for name, start, end in self._function_spans(script):
pieces.append(script[cursor:start])
body = script[start:end]
system_id = FUNCTION_HASH_MAP.get(name)
md5s = self._md5s(systems, system_id) if system_id else []
match = _RE_LOCAL_HASHES.search(body)
if md5s and match:
# An array compared by membership says nothing about order,
# so the same set is left as the maintainer wrote it.
if set(md5s) != set(match.group(1).split()):
body = (
body[: match.start(1)]
+ " ".join(md5s)
+ body[match.end(1) :]
)
pieces.append(body)
cursor = end
pieces.append(script[cursor:])
def validate(self, truth_data: dict, output_path: str) -> list[str]:
content = Path(output_path).read_text(encoding="utf-8")
return {self.native_filename(): "".join(pieces)}
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
content = produced[self.native_filename()]
issues: list[str] = []
systems = truth_data.get("systems", {})
for sys_id, cfg in _SYSTEM_CONFIG.items():
if cfg["pattern"] != "md5":
defined = {name for name, _, _ in self._function_spans(content)}
for name, system_id in FUNCTION_HASH_MAP.items():
if name not in defined:
issues.append(f"check absent from the output: {name}")
continue
sys_data = systems.get(sys_id)
if not sys_data:
md5s = self._md5s(systems, system_id)
if not md5s:
continue
for fe in sys_data.get("files", []):
# export skips placeholders and private entries, so looking
# for them here makes the exporter reject its own output.
name = fe.get("name", "")
if self._is_pattern(name) or name.startswith("_"):
continue
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
if md5 and re.fullmatch(r"[a-f0-9]{32}", md5) and md5 not in content:
issues.append(f"missing md5: {md5} ({name})")
for sys_id, cfg in _SYSTEM_CONFIG.items():
func = cfg["func"]
if func in content:
body = next(
content[start:end]
for fname, start, end in self._function_spans(content)
if fname == name
)
if not _RE_LOCAL_HASHES.search(body):
# A check with no hash list is a path check, not a hash check.
continue
sys_data = systems.get(sys_id)
if not sys_data or not sys_data.get("files"):
continue
# Only flag if the system has usable data for the function type
if cfg["pattern"] == "md5":
has_md5 = any(
fe.get("md5")
and isinstance(fe.get("md5"), str)
and re.fullmatch(r"[a-f0-9]{32}", fe["md5"])
for fe in sys_data["files"]
)
if has_md5:
issues.append(f"missing function: {func}")
elif cfg["pattern"] == "file-exists":
issues.append(f"missing function: {func}")
for md5 in md5s:
if md5 not in body:
issues.append(f"absent from {name}: {md5}")
return issues
scripts/exporter/lakka_exporter.py
+2 -6
View File
@@ -1,8 +1,4 @@
"""Exporter for Lakka (System.dat format, same as RetroArch).
Lakka inherits RetroArch cores and uses the same System.dat format.
Delegates to systemdat_exporter for export and validation.
"""
"""Exporter for Lakka, which reads RetroArch's System.dat unchanged."""
from __future__ import annotations
@@ -10,7 +6,7 @@ from .systemdat_exporter import Exporter as SystemDatExporter
class Exporter(SystemDatExporter):
"""Export truth data to Lakka System.dat format."""
"""Write Lakka's System.dat, corrected."""
@staticmethod
def platform_name() -> str:
scripts/exporter/misterfpga_exporter.py
+125
View File
@@ -0,0 +1,125 @@
"""Exporter for MiSTer's BiosDB (bios_db.json, shipped zipped).
A Downloader database entry carries the download URL, the install path and
the tag ids alongside the hash, and none of those are ours to invent. Only
the hash and the size are rewritten, in MiSTer's own database.
"""
from __future__ import annotations
import io
import json
import zipfile
from collections import OrderedDict
from .base_exporter import BaseExporter
from .baseline import NativeSystem, Report
SOURCE_URL = (
"https://raw.githubusercontent.com/ajgowans/BiosDB_MiSTer/db/bios_db.json.zip"
)
_DB_NAME = "bios_db.json"
class Exporter(BaseExporter):
"""Write MiSTer's bios_db.json, corrected."""
@staticmethod
def platform_name() -> str:
return "misterfpga"
@staticmethod
def native_filename() -> str:
return _DB_NAME
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"md5", "size"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"bios_db.json.zip": SOURCE_URL}
@staticmethod
def can_add() -> bool:
# Every entry carries the URL MiSTer installs it from, and that is
# not ours to invent.
return False
@staticmethod
def needs_original() -> bool:
# Entries carry a URL and a tag vocabulary the database owns.
return True
@staticmethod
def unpack(raw: bytes) -> dict[str, str]:
"""Read the database out of the archive MiSTer publishes."""
with zipfile.ZipFile(io.BytesIO(raw)) as archive:
return {_DB_NAME: archive.read(_DB_NAME).decode("utf-8")}
def _by_path(self, systems: dict[str, NativeSystem]) -> dict[str, object]:
indexed: dict[str, object] = {}
for system in systems.values():
for fe in system.files:
if fe.destination:
indexed[f"games/{fe.destination}"] = fe
return indexed
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
original = originals.get(_DB_NAME) or originals.get("bios_db.json.zip")
if not original:
raise ValueError(
"bios_db.json cannot be written without MiSTer's own database: "
"every entry carries a URL and tag ids that are not ours"
)
database = json.loads(original, object_pairs_hook=OrderedDict)
indexed = self._by_path(systems)
for path, entry in database.get("files", {}).items():
fe = indexed.get(path)
if fe is None:
continue
md5 = fe.hash("md5")
if md5:
entry["hash"] = md5
size = fe.size()
if size:
entry["size"] = size
return {_DB_NAME: json.dumps(database, indent=2, ensure_ascii=False) + "\n"}
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
try:
database = json.loads(produced[_DB_NAME])
except json.JSONDecodeError as exc:
return [f"the database does not parse: {exc}"]
issues: list[str] = []
if not database.get("db_id"):
issues.append("the database lost its db_id")
files = database.get("files", {})
if not files:
issues.append("the database has no files left")
for path, entry in files.items():
if not entry.get("hash"):
issues.append(f"entry without a hash: {path}")
if not entry.get("url"):
issues.append(f"entry without a URL, uninstallable: {path}")
indexed = self._by_path(systems)
for path, fe in indexed.items():
declared = files.get(path)
md5 = fe.hash("md5")
if declared is not None and md5 and declared.get("hash") != md5:
issues.append(f"hash not applied: {path}")
return issues
scripts/exporter/recalbox_exporter.py
+134 -104
View File
@@ -1,135 +1,165 @@
"""Exporter for Recalbox es_bios.xml format.
"""Exporter for Recalbox's es_bios.xml.
Produces XML matching the exact format of recalbox's es_bios.xml:
- XML namespace declaration
- <system fullname="..." platform="...">
- <bios path="system/file" md5="..." core="..." /> with optional mandatory, hashMatchMandatory, note
- mandatory absent = true (only explicit when false)
- 2-space indentation
The file is validated by es_bios.xsd, which makes path, md5 and core
required on every bios element. An entry we cannot give all three to is not
written: Recalbox would reject the file whole.
mandatory and hashMatchMandatory are separate axes. Recalbox reads a missing
attribute as true for both, so each is written only when it is false, or
when the platform stated it explicitly.
"""
from __future__ import annotations
import sys
from pathlib import Path
from xml.etree.ElementTree import ParseError
from xml.sax.saxutils import quoteattr
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from common import parse_untrusted_xml
from .base_exporter import BaseExporter
from .baseline import NativeFile, NativeSystem, Report
SOURCE_URL = (
"https://gitlab.com/recalbox/recalbox/-/raw/master/board/recalbox/fsoverlay"
"/recalbox/share_init/system/.emulationstation/es_bios.xml"
)
SCHEMA_URL = (
"https://gitlab.com/recalbox/recalbox/-/raw/master/board/recalbox/fsoverlay"
"/recalbox/share_init/system/.emulationstation/es_bios.xsd"
)
class Exporter(BaseExporter):
"""Export truth data to Recalbox es_bios.xml format."""
"""Write Recalbox's es_bios.xml, corrected."""
@staticmethod
def platform_name() -> str:
return "recalbox"
def export(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
native_map: dict[str, str] = {}
if scraped_data:
for sys_id, sys_data in scraped_data.get("systems", {}).items():
nid = sys_data.get("native_id")
if nid:
native_map[sys_id] = nid
@staticmethod
def native_filename() -> str:
return "es_bios.xml"
lines: list[str] = [
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"md5", "required"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"es_bios.xml": SOURCE_URL, "es_bios.xsd": SCHEMA_URL}
def _path(self, fe: NativeFile, native_id: str) -> str:
"""The path Recalbox reads, pipe-joined when it accepts several.
A path Recalbox already states is reproduced exactly: several of its
entries sit at the BIOS root with no directory at all, and prefixing
them with the system would point the frontend somewhere else.
"""
if fe.platform is not None:
path = str(fe.platform.get("destination") or fe.name)
alternatives = fe.platform.get("alt_paths") or []
if alternatives:
return "|".join([path, *[str(a) for a in alternatives]])
return path
dest = fe.destination or fe.name
return dest if "/" in dest else f"{native_id}/{dest}"
def _bios_element(self, fe: NativeFile, native_id: str) -> str:
attrs = [f"path={quoteattr(self._path(fe, native_id))}"]
attrs.append(f'md5={quoteattr(",".join(fe.hashes("md5")))}')
attrs.append(f'core={quoteattr(",".join(fe.cores()))}')
if not fe.required:
attrs.append('mandatory="false"')
elif fe.native("mandatory_declared", None) is True:
attrs.append('mandatory="true"')
hash_match = fe.native("hash_match_mandatory", None)
if hash_match is False:
attrs.append('hashMatchMandatory="false"')
elif hash_match is True:
attrs.append('hashMatchMandatory="true"')
note = " ".join(str(fe.native("note", "")).split())
if note:
attrs.append(f"note={quoteattr(note)}")
return f" <bios {' '.join(attrs)} />"
@classmethod
def writable(cls, fe: NativeFile, require: str = "") -> bool:
"""es_bios.xsd makes md5 and core required; without them, no element.
An entry Recalbox already ships has both, so this only ever gates
what we would be adding.
"""
return bool(fe.hashes("md5")) and bool(fe.cores())
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
lines = [
'<?xml version="1.0" encoding="UTF-8"?>',
'<biosList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"'
' xsi:noNamespaceSchemaLocation="es_bios.xsd">',
]
systems = truth_data.get("systems", {})
for sys_id in sorted(systems):
sys_data = systems[sys_id]
files = sys_data.get("files", [])
if not files:
for system in sorted(systems.values(), key=lambda s: s.native_id):
writable = [fe for fe in system.files if self.writable(fe)]
if not writable:
continue
native_id = native_map.get(sys_id, sys_id)
scraped_sys = (
scraped_data.get("systems", {}).get(sys_id) if scraped_data else None
)
display_name = self._display_name(sys_id, scraped_sys)
lines.append(f' <system fullname="{display_name}" platform="{native_id}">')
# Build path lookup from scraped data for this system
scraped_paths: dict[str, str] = {}
if scraped_data:
s_sys = scraped_data.get("systems", {}).get(sys_id, {})
for sf in s_sys.get("files", []):
sname = sf.get("name", "").lower()
spath = sf.get("destination", sf.get("name", ""))
if sname and spath:
scraped_paths[sname] = spath
for fe in files:
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
continue
# Use scraped path when available (preserves original format)
path = scraped_paths.get(name.lower())
if not path:
dest = self._dest(fe)
path = f"{native_id}/{dest}" if "/" not in dest else dest
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = ",".join(md5)
required = fe.get("required", True)
# Build cores string from _cores
cores_list = fe.get("_cores", [])
core_str = (
",".join(f"libretro/{c}" for c in cores_list) if cores_list else ""
)
attrs = [f'path="{path}"']
if md5:
attrs.append(f'md5="{md5}"')
if not required:
attrs.append('mandatory="false"')
if not required:
attrs.append('hashMatchMandatory="true"')
if core_str:
attrs.append(f'core="{core_str}"')
lines.append(f" <bios {' '.join(attrs)} />")
fullname = quoteattr(self.display_name(system))
platform = quoteattr(system.native_id)
lines.append(f" <system fullname={fullname} platform={platform}>")
for fe in writable:
lines.append(self._bios_element(fe, system.native_id))
lines.append(" </system>")
lines.append("</biosList>")
lines.append("")
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
def validate(self, truth_data: dict, output_path: str) -> list[str]:
from xml.etree.ElementTree import parse as xml_parse
tree = xml_parse(output_path)
root = tree.getroot()
exported_paths: set[str] = set()
for bios_el in root.iter("bios"):
path = bios_el.get("path", "")
if path:
exported_paths.add(path.lower())
exported_paths.add(path.split("/")[-1].lower())
return {self.native_filename(): "\n".join(lines)}
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
content = produced[self.native_filename()]
issues: list[str] = []
for sys_data in truth_data.get("systems", {}).values():
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
try:
root = parse_untrusted_xml(content, self.native_filename())
except (ParseError, ValueError) as exc:
return [f"the XML does not parse: {exc}"]
for element in root.iter("bios"):
for attribute in ("path", "md5", "core"):
if not element.get(attribute):
issues.append(
f"es_bios.xsd requires {attribute}: "
f"{element.get('path', '?')}"
)
for element in root.iter("system"):
if not list(element):
issues.append(f"empty system: {element.get('platform', '?')}")
for attribute in ("fullname", "platform"):
if not element.get(attribute):
issues.append(f"es_bios.xsd requires {attribute} on system")
exported = {
element.get("path", "").casefold() for element in root.iter("bios")
}
for system in systems.values():
for fe in system.files:
if not self.writable(fe):
continue
dest = self._dest(fe)
if (
name.lower() not in exported_paths
and dest.lower() not in exported_paths
):
issues.append(f"missing: {name}")
if self._path(fe, system.native_id).casefold() not in exported:
issues.append(f"absent: {system.native_id}/{fe.name}")
return issues
scripts/exporter/retrobat_exporter.py
+90 -88
View File
@@ -1,116 +1,118 @@
"""Exporter for RetroBat batocera-systems.json format.
"""Exporter for RetroBat's batocera-systems.json.
Produces JSON matching the exact format of
RetroBat-Official/emulatorlauncher/batocera-systems/Resources/batocera-systems.json:
- System keys with "name" and "biosFiles" fields
- Each biosFile has "md5" before "file" (matching original key order)
Pure data: a system key carrying a name and a biosFiles array whose entries
state md5 then file, in that order.
"""
from __future__ import annotations
import json
from collections import OrderedDict
from pathlib import Path
from .base_exporter import BaseExporter
from .baseline import NativeSystem, Report
SOURCE_URL = (
"https://raw.githubusercontent.com/RetroBat-Official/emulatorlauncher/master"
"/batocera-systems/Resources/batocera-systems.json"
)
class Exporter(BaseExporter):
"""Export truth data to RetroBat batocera-systems.json format."""
"""Write RetroBat's batocera-systems.json, corrected."""
@staticmethod
def platform_name() -> str:
return "retrobat"
def export(
@staticmethod
def native_filename() -> str:
return "batocera-systems.json"
@staticmethod
def requires() -> str:
return "md5"
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"md5"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"batocera-systems.json": SOURCE_URL}
def render(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
native_map: dict[str, str] = {}
if scraped_data:
for sys_id, sys_data in scraped_data.get("systems", {}).items():
nid = sys_data.get("native_id")
if nid:
native_map[sys_id] = nid
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
# Keep the platform's own key order when we have its file, so the
# diff a maintainer reads is the corrections and nothing else.
order: list[str] = []
original = originals.get(self.native_filename(), "")
if original:
try:
order = list(json.loads(original))
except json.JSONDecodeError:
order = []
output: OrderedDict[str, dict] = OrderedDict()
systems = truth_data.get("systems", {})
for sys_id in sorted(systems):
sys_data = systems[sys_id]
files = sys_data.get("files", [])
if not files:
continue
native_id = native_map.get(sys_id, sys_id)
scraped_sys = (
scraped_data.get("systems", {}).get(sys_id) if scraped_data else None
)
display_name = self._display_name(sys_id, scraped_sys)
bios_files: list[OrderedDict] = []
exportable = {
system.native_id: (system, files)
for system, files in self.exportable(systems, require="md5")
}
keys = [k for k in order if k in exportable]
keys.extend(sorted(k for k in exportable if k not in keys))
output: OrderedDict[str, object] = OrderedDict()
for key in keys:
system, files = exportable[key]
bios_files = []
for fe in files:
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
continue
dest = self._dest(fe)
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
# Original format requires md5 for every entry
if not md5:
continue
entry: OrderedDict[str, str] = OrderedDict()
entry["md5"] = md5
entry["file"] = f"bios/{dest}"
entry["md5"] = fe.hash("md5")
declared = fe.native("native_path", "")
entry["file"] = str(declared) if declared else f"bios/{fe.destination}"
bios_files.append(entry)
system_entry: OrderedDict[str, object] = OrderedDict()
system_entry["name"] = self.display_name(system)
system_entry["biosFiles"] = bios_files
output[key] = system_entry
if bios_files:
if native_id in output:
existing_files = {
e.get("file") for e in output[native_id]["biosFiles"]
}
for entry in bios_files:
if entry.get("file") not in existing_files:
output[native_id]["biosFiles"].append(entry)
else:
sys_entry: OrderedDict[str, object] = OrderedDict()
sys_entry["name"] = display_name
sys_entry["biosFiles"] = bios_files
output[native_id] = sys_entry
text = json.dumps(output, indent=2, ensure_ascii=False) + "\n"
return {self.native_filename(): text}
Path(output_path).write_text(
json.dumps(output, indent=2, ensure_ascii=False) + "\n",
encoding="utf-8",
)
def validate(self, truth_data: dict, output_path: str) -> list[str]:
data = json.loads(Path(output_path).read_text(encoding="utf-8"))
exported_files: set[str] = set()
for sys_data in data.values():
for bf in sys_data.get("biosFiles", []):
path = bf.get("file", "")
stripped = path.removeprefix("bios/")
exported_files.add(stripped)
basename = path.split("/")[-1] if "/" in path else path
exported_files.add(basename)
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
try:
data = json.loads(produced[self.native_filename()])
except json.JSONDecodeError as exc:
return [f"the JSON does not parse: {exc}"]
issues: list[str] = []
for sys_data in truth_data.get("systems", {}).values():
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
continue
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
if not md5:
continue
dest = self._dest(fe)
if name not in exported_files and dest not in exported_files:
issues.append(f"missing: {name}")
for key, entry in data.items():
if not entry.get("name"):
issues.append(f"system without a name: {key}")
if not entry.get("biosFiles"):
issues.append(f"empty entry: {key}")
for bios in entry.get("biosFiles", []):
# RetroBat states an unhashed file with an empty md5, so only
# a missing path makes an entry unusable.
if "md5" not in bios or not bios.get("file"):
issues.append(f"incomplete entry: {key}/{bios.get('file', '?')}")
for system, files in self.exportable(systems, require="md5"):
entry = data.get(system.native_id)
if entry is None:
issues.append(f"system absent: {system.native_id}")
continue
declared = {bios.get("file") for bios in entry.get("biosFiles", [])}
for fe in files:
path = str(fe.native("native_path", "")) or f"bios/{fe.destination}"
if path not in declared:
issues.append(f"absent: {system.native_id}/{fe.name}")
return issues
scripts/exporter/retrodeck_exporter.py
+152 -180
View File
@@ -1,210 +1,182 @@
"""Exporter for RetroDECK component_manifest.json format.
"""Exporter for RetroDECK's component manifests.
Produces a JSON file compatible with RetroDECK's component manifests.
Each system maps to a component with BIOS entries containing filename,
md5 (comma-separated if multiple), paths ($bios_path default), and
required status.
Path tokens: $bios_path for bios/, $roms_path for roms/.
Entries without an explicit path default to $bios_path.
RetroDECK has no single BIOS file. Each component carries its own
component_manifest.json, and the BIOS list sits inside it next to the
component's name, description and presets, at one of three keys. Only that
list is rewritten, in the component's own file, so everything else the
manifest drives is left alone.
"""
from __future__ import annotations
import json
import re
from collections import OrderedDict
from pathlib import Path
from .base_exporter import BaseExporter
from .baseline import NativeFile, NativeSystem, Report
# retrobios slug -> RetroDECK system ID (reverse of scraper SYSTEM_SLUG_MAP)
_REVERSE_SLUG: dict[str, str] = {
"nintendo-nes": "nes",
"nintendo-snes": "snes",
"nintendo-64": "n64",
"nintendo-64dd": "n64dd",
"nintendo-gamecube": "gc",
"nintendo-wii": "wii",
"nintendo-wii-u": "wiiu",
"nintendo-switch": "switch",
"nintendo-gb": "gb",
"nintendo-gbc": "gbc",
"nintendo-gba": "gba",
"nintendo-ds": "nds",
"nintendo-3ds": "3ds",
"nintendo-fds": "fds",
"nintendo-sgb": "sgb",
"nintendo-virtual-boy": "virtualboy",
"nintendo-pokemon-mini": "pokemini",
"sony-playstation": "psx",
"sony-playstation-2": "ps2",
"sony-playstation-3": "ps3",
"sony-psp": "psp",
"sony-psvita": "psvita",
"sega-mega-drive": "megadrive",
"sega-mega-cd": "megacd",
"sega-saturn": "saturn",
"sega-dreamcast": "dreamcast",
"sega-dreamcast-arcade": "naomi",
"sega-game-gear": "gamegear",
"sega-master-system": "mastersystem",
"nec-pc-engine": "pcengine",
"nec-pc-fx": "pcfx",
"nec-pc-98": "pc98",
"nec-pc-88": "pc88",
"3do": "3do",
"amstrad-cpc": "amstradcpc",
"arcade": "arcade",
"atari-400-800": "atari800",
"atari-5200": "atari5200",
"atari-7800": "atari7800",
"atari-jaguar": "atarijaguar",
"atari-lynx": "atarilynx",
"atari-st": "atarist",
"commodore-c64": "c64",
"commodore-amiga": "amiga",
"philips-cdi": "cdimono1",
"fairchild-channel-f": "channelf",
"coleco-colecovision": "colecovision",
"mattel-intellivision": "intellivision",
"microsoft-msx": "msx",
"microsoft-xbox": "xbox",
"doom": "doom",
"j2me": "j2me",
"apple-macintosh-ii": "macintosh",
"apple-ii": "apple2",
"apple-iigs": "apple2gs",
"enterprise-64-128": "enterprise",
"tiger-game-com": "gamecom",
"hartung-game-master": "gmaster",
"epoch-scv": "scv",
"watara-supervision": "supervision",
"bandai-wonderswan": "wonderswan",
"snk-neogeo-cd": "neogeocd",
"tandy-coco": "coco",
"tandy-trs-80": "trs80",
"dragon-32-64": "dragon",
"pico8": "pico8",
"wolfenstein-3d": "wolfenstein",
"sinclair-zx-spectrum": "zxspectrum",
}
def _dest_to_path_token(destination: str) -> str:
"""Convert a truth destination path to a RetroDECK path token."""
if destination.startswith("roms/"):
return "$roms_path/" + destination.removeprefix("roms/")
if destination.startswith("bios/"):
return "$bios_path/" + destination.removeprefix("bios/")
# Default: bios path
return "$bios_path/" + destination
COMPONENTS_REPO = "RetroDECK/components"
COMPONENTS_BRANCH = "main"
RAW_BASE = f"https://raw.githubusercontent.com/{COMPONENTS_REPO}/{COMPONENTS_BRANCH}"
MANIFEST = "component_manifest.json"
class Exporter(BaseExporter):
"""Export truth data to RetroDECK component_manifest.json format."""
"""Write RetroDECK's component manifests, corrected."""
@staticmethod
def platform_name() -> str:
return "retrodeck"
def export(
@staticmethod
def native_filename() -> str:
return MANIFEST
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"md5", "sha256", "required"})
@staticmethod
def needs_original() -> bool:
# A manifest is mostly presets and launch configuration; rebuilding
# one from BIOS data alone would throw the component away.
return True
@staticmethod
def component_url(component: str) -> str:
return f"{RAW_BASE}/{component}/{MANIFEST}"
def components(self, systems: dict[str, NativeSystem]) -> list[str]:
"""Components the corrected data touches."""
found: set[str] = set()
for system in systems.values():
for fe in system.files:
component = str(fe.native("component", ""))
if component:
found.add(component)
return sorted(found)
@staticmethod
def _entry(fe: NativeFile) -> OrderedDict:
entry: OrderedDict[str, object] = OrderedDict()
entry["filename"] = fe.name
md5 = ",".join(fe.hashes("md5"))
if md5:
entry["md5"] = md5
sha256 = fe.hash("sha256")
if sha256:
entry["sha256"] = sha256
entry["system"] = fe.native_system
description = fe.native("description", "")
if description:
entry["description"] = str(description)
# RetroDECK words the requirement in prose ("Required", "At least one
# BIOS file required"), so the platform's own wording is kept and a
# boolean is only rendered when there is none to keep.
label = fe.native("required_label", "")
if label:
entry["required"] = str(label)
elif fe.required:
entry["required"] = "Required"
destination = fe.destination
if destination and destination not in (fe.name, f"bios/{fe.name}"):
directory = destination.rsplit("/", 1)[0]
entry["paths"] = "$bios_path/" + directory.removeprefix("bios/")
return entry
def _by_component(
self, systems: dict[str, NativeSystem]
) -> dict[str, list[NativeFile]]:
grouped: dict[str, list[NativeFile]] = {}
for system in systems.values():
for fe in system.files:
component = str(fe.native("component", ""))
if component:
grouped.setdefault(component, []).append(fe)
return grouped
@staticmethod
def _bios_holder(component_value: dict) -> tuple[dict, str] | None:
"""Where in a manifest the BIOS list lives, if it has one."""
if "bios" in component_value:
return component_value, "bios"
for key in ("preset_actions", "cores"):
nested = component_value.get(key)
if isinstance(nested, dict) and "bios" in nested:
return nested, "bios"
return None
def render(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
native_map: dict[str, str] = {}
if scraped_data:
for sys_id, sys_data in scraped_data.get("systems", {}).items():
nid = sys_data.get("native_id")
if nid:
native_map[sys_id] = nid
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
grouped = self._by_component(systems)
produced: dict[str, str] = {}
manifest: OrderedDict[str, dict] = OrderedDict()
systems = truth_data.get("systems", {})
for sys_id in sorted(systems):
sys_data = systems[sys_id]
files = sys_data.get("files", [])
if not files:
for component, files in sorted(grouped.items()):
path = f"{component}/{MANIFEST}"
original = originals.get(path)
if not original:
continue
try:
manifest = json.loads(original, object_pairs_hook=OrderedDict)
except json.JSONDecodeError:
continue
native_id = native_map.get(sys_id, _REVERSE_SLUG.get(sys_id, sys_id))
bios_entries: list[OrderedDict] = []
for fe in files:
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
entries = [self._entry(fe) for fe in files]
for component_value in manifest.values():
if not isinstance(component_value, dict):
continue
dest = self._dest(fe)
path_token = _dest_to_path_token(dest)
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = ",".join(m for m in md5 if m)
required = fe.get("required", True)
entry: OrderedDict[str, object] = OrderedDict()
entry["filename"] = name
if md5:
# Validate MD5 entries
parts = [
m.strip().lower()
for m in str(md5).split(",")
if re.fullmatch(r"[0-9a-f]{32}", m.strip())
]
if parts:
entry["md5"] = ",".join(parts) if len(parts) > 1 else parts[0]
entry["paths"] = path_token
entry["required"] = required
system_val = native_id
entry["system"] = system_val
bios_entries.append(entry)
if bios_entries:
if native_id in manifest:
# Merge into existing component (multiple truth systems
# may map to the same native ID)
existing_names = {
e["filename"] for e in manifest[native_id]["bios"]
}
for entry in bios_entries:
if entry["filename"] not in existing_names:
manifest[native_id]["bios"].append(entry)
holder = self._bios_holder(component_value)
if holder is None:
component_value["bios"] = entries
else:
component = OrderedDict()
component["system"] = native_id
component["bios"] = bios_entries
manifest[native_id] = component
container, key = holder
container[key] = entries
break
Path(output_path).write_text(
json.dumps(manifest, indent=2, ensure_ascii=False) + "\n",
encoding="utf-8",
)
produced[path] = json.dumps(manifest, indent=2, ensure_ascii=False) + "\n"
def validate(self, truth_data: dict, output_path: str) -> list[str]:
data = json.loads(Path(output_path).read_text(encoding="utf-8"))
exported_names: set[str] = set()
for comp_data in data.values():
bios = comp_data.get("bios", [])
if isinstance(bios, list):
for entry in bios:
fn = entry.get("filename", "")
if fn:
exported_names.add(fn)
return produced
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
issues: list[str] = []
for sys_data in truth_data.get("systems", {}).values():
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
grouped = self._by_component(systems)
for component, files in grouped.items():
path = f"{component}/{MANIFEST}"
if path not in produced:
issues.append(f"manifest not written: {path}")
continue
try:
manifest = json.loads(produced[path])
except json.JSONDecodeError as exc:
issues.append(f"{path} does not parse: {exc}")
continue
declared: set[str] = set()
for component_value in manifest.values():
if not isinstance(component_value, dict):
continue
if name not in exported_names:
issues.append(f"missing: {name}")
holder = self._bios_holder(component_value)
if holder is None:
continue
container, key = holder
for entry in container[key]:
declared.add(entry.get("filename", ""))
if not component_value.get("name") and not component_value.get(
"system"
):
issues.append(f"{path}: the component lost its identity")
for fe in files:
if fe.name not in declared:
issues.append(f"absent from {path}: {fe.name}")
return issues
scripts/exporter/retropie_exporter.py
+301 -6
View File
@@ -1,17 +1,312 @@
"""Exporter for RetroPie (System.dat format, same as RetroArch).
"""Exporter for RetroPie's scriptmodules.
RetroPie inherits RetroArch cores and uses the same System.dat format.
Delegates to systemdat_exporter for export and validation.
RetroPie ships no BIOS list. What it maintains is one shell script per
package, and the BIOS files a package needs are named in its
`rp_module_help` string, in prose a person reads before copying files.
platforms.cfg carries extensions and full names only.
So the correctable unit is that sentence, and the correction is a file name
missing from it. A name RetroPie already writes is never removed, and no
sentence is invented where a maintainer wrote none: a package we would have
to document from scratch is reported, not drafted.
"""
from __future__ import annotations
from .systemdat_exporter import Exporter as SystemDatExporter
import io
import re
import sys
import tarfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from common import load_emulator_profiles
from .base_exporter import BaseExporter
from .baseline import NativeFile, NativeSystem, Report, search_order
SOURCE_URL = (
"https://codeload.github.com/RetroPie/RetroPie-Setup/tar.gz/refs/heads/master"
)
ARCHIVE = "RetroPie-Setup.tar.gz"
# Resolved from the file rather than the working directory: the profiles
# are what say which files a package needs.
EMULATORS_DIR = Path(__file__).resolve().parents[2] / "emulators"
# The longest list RetroPie writes is six names (lr-atari800). Past that the
# sentence stops being something a reader uses, so the names are reported
# instead of appended.
MAX_NAMES = 6
_MODULE_ID = re.compile(r'rp_module_id="([^"]+)"')
_MODULE_HELP = re.compile(r'rp_module_help="((?:[^"\\]|\\.)*)"')
_FILENAME = re.compile(r"[A-Za-z0-9][\w.+-]*\.[A-Za-z0-9]{1,5}\b")
# RetroPie words the instruction several ways ("Copy the required BIOS files
# a.bin and b.bin to $biosdir", "The Sega CD requires the BIOS files a.bin,
# b.bin copied to $biosdir"), so the clause is found by the word BIOS rather
# than by a sentence template, and the names in it are the list to extend.
_BIOS_CLAUSE = re.compile(r"BIOS\b.*?(?=\\n|$)", re.DOTALL)
_BIOS_NOUN = re.compile(r"BIOS (files?)\b")
class Exporter(SystemDatExporter):
"""Export truth data to RetroPie System.dat format."""
class Exporter(BaseExporter):
"""Write RetroPie's scriptmodules, corrected."""
@staticmethod
def platform_name() -> str:
return "retropie"
@staticmethod
def native_filename() -> str:
return "scriptmodules"
@staticmethod
def carries() -> frozenset[str]:
# The help names files; it states no hash and no requirement flag.
return frozenset({"name"})
@staticmethod
def native_sources() -> dict[str, str]:
return {ARCHIVE: SOURCE_URL}
@staticmethod
def needs_original() -> bool:
# The declaration is a sentence inside a shell script.
return True
@staticmethod
def may_write_nothing() -> bool:
# Nothing to correct is a result, not a failure.
return True
@staticmethod
def unpack(raw: bytes) -> dict[str, str]:
"""Keep the scriptmodules that say anything about BIOS files."""
found: dict[str, str] = {}
with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as archive:
for member in archive:
if not member.isfile() or not member.name.endswith(".sh"):
continue
relative = member.name.split("/", 1)[-1]
if not relative.startswith("scriptmodules/"):
continue
handle = archive.extractfile(member)
if handle is None:
continue
text = handle.read().decode("utf-8", errors="replace")
if "BIOS" in text:
found[relative] = text
return found
def _core_index(self) -> dict[str, str]:
"""Module id (without its lr- prefix) to the profile it stands for."""
index: dict[str, str] = {}
for key, profile in load_emulator_profiles(str(EMULATORS_DIR)).items():
index[key.replace("-", "_").lower()] = key
for name in profile.get("cores", []) or []:
index[str(name).replace("-", "_").lower()] = key
return index
@staticmethod
def _files_by_core(systems: dict[str, NativeSystem]) -> dict[str, list[NativeFile]]:
"""Which files each core asks for, as the truth read its source."""
grouped: dict[str, list[NativeFile]] = {}
for system in systems.values():
for fe in system.files:
for core in (fe.truth or {}).get("_cores", []):
grouped.setdefault(str(core), []).append(fe)
return grouped
@staticmethod
def _names_in(text: str) -> list[re.Match[str]]:
"""File names in a fragment, skipping what only looks like one.
A token preceded by a dot is the tail of a ROM extension list
(.atr.gz), and one preceded by a separator is part of a path
($biosdir/Machines/COL/coleco.rom): neither is a name in a list.
"""
found: list[re.Match[str]] = []
for match in _FILENAME.finditer(text):
before = text[match.start() - 1 : match.start()]
if before in (".", "/", "\\"):
continue
found.append(match)
return found
@classmethod
def _listed(cls, help_text: str) -> set[str]:
"""File names the help already writes, wherever in the string."""
plain = help_text.replace("\\n", " ")
return {match.group(0).lower() for match in cls._names_in(plain)}
@classmethod
def _insertion_point(cls, help_text: str) -> int | None:
"""Where a name joins the list, or None when there is no list."""
clause = _BIOS_CLAUSE.search(help_text)
if clause is None:
return None
names = cls._names_in(clause.group(0))
return clause.start() + names[-1].end() if names else None
@staticmethod
def _in_search_order(candidates: list[NativeFile]) -> list[str]:
"""The names in the order the code looks for them, best first.
Someone reading the sentence copies the files in the order it
gives, so the one the emulator prefers is named first. The model
already holds them in that order; this only removes the duplicates
a name can pick up from several cores.
"""
names: list[str] = []
for fe in search_order(candidates):
if fe.name not in names:
names.append(fe.name)
return names
@staticmethod
def _join(names: list[str]) -> str:
"""RetroPie's own idiom: a, b and c."""
if len(names) == 1:
return names[0]
return ", ".join(names[:-1]) + " and " + names[-1]
def modules(self, originals: dict[str, str]) -> dict[str, tuple[str, str]]:
"""Module id and help string of every script that mentions BIOS."""
found: dict[str, tuple[str, str]] = {}
for relative, text in originals.items():
if not relative.startswith("scriptmodules/"):
continue
module = _MODULE_ID.search(text)
help_text = _MODULE_HELP.search(text)
if not module or not help_text or "BIOS" not in help_text.group(1):
continue
found[relative] = (module.group(1), help_text.group(1))
return found
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
modules = self.modules(originals)
if not modules:
raise ValueError(
"the scriptmodules cannot be written without RetroPie's own "
"repository: the BIOS list is a sentence inside a shell script"
)
index = self._core_index()
by_core = self._files_by_core(systems)
# Aliases count as names we know: RetroPie writes dc_flash.bin where
# flycast's profile files it as an alias of another primary.
known: set[str] = set()
for system in systems.values():
for fe in system.files:
known.add(fe.name.lower())
known.update(
str(a).lower() for a in (fe.native("aliases", []) or [])
)
self._skipped: dict[str, list[str]] = {}
produced: dict[str, str] = {}
def skip(reason: str, module: str) -> None:
self._skipped.setdefault(reason, []).append(module)
for relative, (module_id, help_text) in sorted(modules.items()):
core = index.get(module_id.removeprefix("lr-").replace("-", "_").lower())
files = by_core.get(core or "", [])
if not files:
skip("no core of ours packages it", module_id)
continue
listed = self._listed(help_text)
unknown = sorted(listed - known)
if unknown:
skip(
f"names a file we have never seen ({', '.join(unknown)})",
module_id,
)
# A file already listed under one of its other names is not
# missing: proposing the primary would name the same bytes twice.
candidates = [
fe
for fe in files
if fe.required
and not (
{fe.name.lower()}
| {str(a).lower() for a in (fe.native("aliases", []) or [])}
)
& listed
]
missing = self._in_search_order(candidates)
if not missing:
continue
insert_at = self._insertion_point(help_text)
if insert_at is None:
# No enumeration to extend, and a sentence we would have to
# write ourselves is a documentation change, not a correction.
skip("names no file to extend", module_id)
continue
if len(listed) + len(missing) > MAX_NAMES:
skip("more names than the help enumerates", module_id)
continue
new_help = (
help_text[:insert_at]
+ ", "
+ self._join(missing)
+ help_text[insert_at:]
)
if len(listed) + len(missing) > 1:
new_help = _BIOS_NOUN.sub("BIOS files", new_help, count=1)
produced[relative] = originals[relative].replace(
f'rp_module_help="{help_text}"',
f'rp_module_help="{new_help}"',
1,
)
return produced
def outcome(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> str:
"""Packages are the unit here, not file entries."""
skipped: dict[str, list[str]] = getattr(self, "_skipped", {})
parts = [f"{len(produced)} packages corrected"]
for reason, modules in sorted(skipped.items()):
shown = ", ".join(sorted(modules)[:4])
if len(modules) > 4:
shown += f" and {len(modules) - 4} more"
parts.append(f"{len(modules)} {reason} ({shown})")
return "; ".join(parts)
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
issues: list[str] = []
for relative, text in produced.items():
module = _MODULE_ID.search(text)
help_text = _MODULE_HELP.search(text)
if not module:
issues.append(f"{relative}: the module lost its id")
if not help_text:
issues.append(f"{relative}: the help string is not closed")
continue
if "BIOS" not in help_text.group(1):
issues.append(f"{relative}: the BIOS sentence is gone")
names = self._listed(help_text.group(1))
if len(names) > MAX_NAMES:
issues.append(
f"{relative}: {len(names)} names, past what the help enumerates"
)
return issues
scripts/exporter/rocknix_exporter.py
+30
View File
@@ -0,0 +1,30 @@
"""Exporter for ROCKNIX's rocknix-systems.
Same shape as Batocera's script: a systems mapping inside a checker ROCKNIX
runs, so only the mapping is rewritten.
"""
from __future__ import annotations
from .batocera_exporter import Exporter as BatoceraExporter
SOURCE_URL = (
"https://raw.githubusercontent.com/ROCKNIX/distribution/next/projects/ROCKNIX"
"/packages/rocknix/sources/scripts/rocknix-systems"
)
class Exporter(BatoceraExporter):
"""Write ROCKNIX's rocknix-systems, corrected."""
@staticmethod
def platform_name() -> str:
return "rocknix"
@staticmethod
def native_filename() -> str:
return "rocknix-systems"
@staticmethod
def native_sources() -> dict[str, str]:
return {"rocknix-systems": SOURCE_URL}
scripts/exporter/romm_exporter.py
+105 -132
View File
@@ -1,160 +1,133 @@
"""Exporter for RomM known_bios_files.json format.
"""Exporter for RomM's known_bios_files.json.
Produces JSON matching the exact format of
rommapp/romm/backend/models/fixtures/known_bios_files.json:
- Keys are "igdb_slug:filename"
- Values contain size, crc, md5, sha1 (all optional but at least one hash)
- Hashes are lowercase hex strings
- Size is an integer
Keys are "<igdb slug>:<filename>". RomM verifies a firmware file with
`file_size_bytes == int(entry.get("size", 0))` and then one hash among md5,
sha1 and crc, so an entry without a size can never match and an entry
without a hash can never match either. Neither is written.
"""
from __future__ import annotations
import json
import sys
from collections import OrderedDict
from pathlib import Path
from .base_exporter import BaseExporter
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
# retrobios slug -> IGDB slug (reverse of scraper SLUG_MAP)
_REVERSE_SLUG: dict[str, str] = {
"3do": "3do",
"nintendo-64dd": "64dd",
"amstrad-cpc": "acpc",
"commodore-amiga": "amiga",
"arcade": "arcade",
"atari-st": "atari-st",
"atari-5200": "atari5200",
"atari-7800": "atari7800",
"atari-400-800": "atari8bit",
"coleco-colecovision": "colecovision",
"sega-dreamcast": "dc",
"doom": "doom",
"enterprise-64-128": "enterprise",
"fairchild-channel-f": "fairchild-channel-f",
"nintendo-fds": "fds",
"sega-game-gear": "gamegear",
"nintendo-gb": "gb",
"nintendo-gba": "gba",
"nintendo-gbc": "gbc",
"sega-mega-drive": "genesis",
"mattel-intellivision": "intellivision",
"j2me": "j2me",
"atari-lynx": "lynx",
"apple-macintosh-ii": "mac",
"microsoft-msx": "msx",
"nintendo-ds": "nds",
"snk-neogeo-cd": "neo-geo-cd",
"nintendo-nes": "nes",
"nintendo-gamecube": "ngc",
"magnavox-odyssey2": "odyssey-2-slash-videopac-g7000",
"nec-pc-98": "pc-9800-series",
"nec-pc-fx": "pc-fx",
"nintendo-pokemon-mini": "pokemon-mini",
"sony-playstation-2": "ps2",
"sony-psp": "psp",
"sony-playstation": "psx",
"nintendo-satellaview": "satellaview",
"sega-saturn": "saturn",
"scummvm": "scummvm",
"sega-mega-cd": "segacd",
"sharp-x68000": "sharp-x68000",
"sega-master-system": "sms",
"nintendo-snes": "snes",
"nintendo-sufami-turbo": "sufami-turbo",
"nintendo-sgb": "super-gb",
"nec-pc-engine": "tg16",
"videoton-tvc": "tvc",
"philips-videopac": "videopac-g7400",
"wolfenstein-3d": "wolfenstein",
"sharp-x1": "x1",
"microsoft-xbox": "xbox",
"sinclair-zx-spectrum": "zxs",
}
from scraper.romm_scraper import SLUG_MAP
from .base_exporter import BaseExporter
from .baseline import NativeFile, NativeSystem, Report
SOURCE_URL = (
"https://raw.githubusercontent.com/rommapp/romm/master/backend/models"
"/fixtures/known_bios_files.json"
)
class Exporter(BaseExporter):
"""Export truth data to RomM known_bios_files.json format."""
"""Write RomM's known_bios_files.json, corrected."""
@staticmethod
def platform_name() -> str:
return "romm"
def export(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
native_map: dict[str, str] = {}
if scraped_data:
for sys_id, sys_data in scraped_data.get("systems", {}).items():
nid = sys_data.get("native_id")
if nid:
native_map[sys_id] = nid
@staticmethod
def native_filename() -> str:
return "known_bios_files.json"
output: OrderedDict[str, dict] = OrderedDict()
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"size", "crc32", "md5", "sha1"})
systems = truth_data.get("systems", {})
for sys_id in sorted(systems):
sys_data = systems[sys_id]
files = sys_data.get("files", [])
if not files:
continue
@staticmethod
def native_sources() -> dict[str, str]:
return {"known_bios_files.json": SOURCE_URL}
igdb_slug = native_map.get(sys_id, _REVERSE_SLUG.get(sys_id, sys_id))
for fe in files:
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
continue
key = f"{igdb_slug}:{name}"
entry: OrderedDict[str, object] = OrderedDict()
size = fe.get("size")
if size is not None:
entry["size"] = int(size)
crc = fe.get("crc32", "")
if crc:
entry["crc"] = str(crc).strip().lower()
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
if md5:
entry["md5"] = str(md5).strip().lower()
sha1 = fe.get("sha1", "")
if isinstance(sha1, list):
sha1 = sha1[0] if sha1 else ""
if sha1:
entry["sha1"] = str(sha1).strip().lower()
output[key] = entry
Path(output_path).write_text(
json.dumps(output, indent=2, ensure_ascii=False) + "\n",
encoding="utf-8",
@staticmethod
def _verifiable(fe: NativeFile) -> bool:
return bool(fe.size()) and any(
fe.hash(h) for h in ("md5", "sha1", "crc32")
)
def validate(self, truth_data: dict, output_path: str) -> list[str]:
data = json.loads(Path(output_path).read_text(encoding="utf-8"))
@staticmethod
def _known_platform(native_id: str) -> bool:
"""RomM keys by IGDB platform slug, and only looks up its own.
exported_names: set[str] = set()
for key in data:
if ":" in key:
_, filename = key.split(":", 1)
exported_names.add(filename)
A key spelled with one of our slugs (capcom-cps3, snk-neogeo-mvs)
matches nothing on their side, so it is reported rather than
written.
"""
return native_id in SLUG_MAP
@classmethod
def writable(cls, fe: NativeFile, require: str = "") -> bool:
"""What RomM already ships stays; the conditions gate additions.
An entry of theirs that could never verify is still theirs, and the
round trip is not the place to decide otherwise.
"""
if fe.platform is not None:
return True
return cls._verifiable(fe) and cls._known_platform(fe.native_system)
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
output: OrderedDict[str, dict] = OrderedDict()
for system in sorted(systems.values(), key=lambda s: s.native_id):
for fe in sorted(system.files, key=lambda f: f.name):
if not self.writable(fe):
continue
entry: OrderedDict[str, str] = OrderedDict()
# The fixture states every value as a string, size included.
entry["size"] = str(fe.size())
crc = fe.hash("crc32")
if crc:
entry["crc"] = crc
md5 = fe.hash("md5")
if md5:
entry["md5"] = md5
sha1 = fe.hash("sha1")
if sha1:
entry["sha1"] = sha1
output[f"{system.native_id}:{fe.name}"] = entry
text = json.dumps(output, indent=2, ensure_ascii=False) + "\n"
return {self.native_filename(): text}
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
try:
data = json.loads(produced[self.native_filename()])
except json.JSONDecodeError as exc:
return [f"the JSON does not parse: {exc}"]
issues: list[str] = []
for sys_data in truth_data.get("systems", {}).values():
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
for key, entry in data.items():
slug = key.split(":", 1)[0] if ":" in key else ""
if not slug:
issues.append(f"key without a platform slug: {key}")
elif not self._known_platform(slug):
issues.append(f"platform slug RomM does not know: {slug}")
if not entry.get("size"):
issues.append(f"entry without a size, never verifiable: {key}")
if not any(entry.get(h) for h in ("md5", "sha1", "crc")):
issues.append(f"entry without a hash, never verifiable: {key}")
for system in systems.values():
for fe in system.files:
if not self.writable(fe):
continue
if name not in exported_names:
issues.append(f"missing: {name}")
if f"{system.native_id}:{fe.name}" not in data:
issues.append(f"absent: {system.native_id}/{fe.name}")
return issues
scripts/exporter/systemdat_exporter.py
+106 -75
View File
@@ -1,7 +1,7 @@
"""Exporter for libretro System.dat (clrmamepro DAT format).
Produces a single 'game' block with all ROMs grouped by system,
matching the exact format of libretro-database/dat/System.dat.
One 'game' block, systems separated by a comment line carrying the name
libretro gives them, matching libretro-database/dat/System.dat.
"""
from __future__ import annotations
@@ -14,43 +14,71 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from scraper.dat_parser import parse_dat
from .base_exporter import BaseExporter
from .baseline import NativeSystem, Report
SOURCE_URL = (
"https://raw.githubusercontent.com/libretro/libretro-database/master/dat/System.dat"
)
def _slug_to_native(slug: str) -> str:
"""Convert a system slug to 'Manufacturer - Console' format."""
parts = slug.split("-", 1)
if len(parts) == 1:
return parts[0].title()
manufacturer = parts[0].replace("-", " ").title()
console = parts[1].replace("-", " ").title()
return f"{manufacturer} - {console}"
def _quote(name: str) -> str:
"""Quote a ROM name the way the original does: only when it must be."""
return f'"{name}"' if any(c in name for c in ' ()') else name
class Exporter(BaseExporter):
"""Export truth data to libretro System.dat format."""
"""Write libretro's System.dat, corrected."""
@staticmethod
def platform_name() -> str:
return "retroarch"
def export(
self,
truth_data: dict,
output_path: str,
scraped_data: dict | None = None,
) -> None:
native_map: dict[str, str] = {}
if scraped_data:
for sys_id, sys_data in scraped_data.get("systems", {}).items():
nid = sys_data.get("native_id")
if nid:
native_map[sys_id] = nid
@staticmethod
def native_filename() -> str:
return "System.dat"
@staticmethod
def carries() -> frozenset[str]:
return frozenset({"size", "crc32", "md5", "sha1"})
@staticmethod
def native_sources() -> dict[str, str]:
return {"System.dat": SOURCE_URL}
@classmethod
def writable(cls, fe, require: str = "") -> bool:
"""A clrmamepro rom line is a hash record, and the DAT has no other.
The original never states a rom it cannot hash, so neither do we.
"""
if fe.platform is not None:
return True
return any(fe.hash(h) for h in ("crc32", "md5", "sha1"))
@staticmethod
def _rom_name(fe) -> str:
"""The name the DAT gives a rom.
libretro writes the path for some entries (ep128emu/roms/cpc464.rom)
and the bare name for others whose destination has a directory all
the same (iplromco.dat, which lives under keropi/), so its own
spelling is recorded rather than derived.
"""
declared = fe.native("native_path", "")
return str(declared) if declared else fe.name
def _header(self, originals: dict[str, str], scraped: dict | None) -> list[str]:
"""Reuse the original header verbatim when we have the original."""
original = originals.get(self.native_filename(), "")
if original:
head, sep, _ = original.partition("\ngame (")
if sep:
return head.split("\n")
# Match exact header format of libretro-database/dat/System.dat
version = ""
if scraped_data:
version = scraped_data.get("dat_version", scraped_data.get("version", ""))
lines: list[str] = [
if scraped:
version = scraped.get("dat_version", scraped.get("version", ""))
lines = [
"clrmamepro (",
'\tname "System"',
'\tdescription "System"',
@@ -61,73 +89,76 @@ class Exporter(BaseExporter):
lines.extend(
[
'\tauthor "libretro"',
'\thomepage "https://github.com/libretro/libretro-database/blob/master/dat/System.dat"',
'\turl "https://raw.githubusercontent.com/libretro/libretro-database/master/dat/System.dat"',
'\thomepage "https://github.com/libretro/libretro-database/blob/master'
'/dat/System.dat"',
'\turl "https://raw.githubusercontent.com/libretro/libretro-database'
'/master/dat/System.dat"',
")",
"",
"game (",
'\tname "System"',
'\tcomment "System"',
]
)
return lines
systems = truth_data.get("systems", {})
for sys_id in sorted(systems):
sys_data = systems[sys_id]
files = sys_data.get("files", [])
if not files:
continue
native_name = native_map.get(sys_id, _slug_to_native(sys_id))
lines.append("")
lines.append(f'\tcomment "{native_name}"')
def render(
self,
systems: dict[str, NativeSystem],
report: Report,
originals: dict[str, str],
scraped: dict | None = None,
) -> dict[str, str]:
lines = self._header(originals, scraped)
lines.extend(["game (", '\tname "System"', '\tcomment "System"'])
for system, files in sorted(
self.exportable(systems), key=lambda pair: pair[0].native_id
):
rendered: list[str] = []
for fe in files:
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
if not any(fe.hash(h) for h in ("crc32", "md5", "sha1")):
continue
# Quote names with spaces or special chars (matching original format)
needs_quote = " " in name or "(" in name or ")" in name
name_str = f'"{name}"' if needs_quote else name
rom_parts = [f"name {name_str}"]
size = fe.get("size")
parts = [f"name {_quote(self._rom_name(fe))}"]
size = fe.size()
if size:
rom_parts.append(f"size {size}")
crc = fe.get("crc32", "")
parts.append(f"size {size}")
crc = fe.hash("crc32")
if crc:
rom_parts.append(f"crc {str(crc).upper()}")
md5 = fe.get("md5", "")
if isinstance(md5, list):
md5 = md5[0] if md5 else ""
parts.append(f"crc {crc.upper()}")
md5 = fe.hash("md5")
if md5:
rom_parts.append(f"md5 {md5}")
sha1 = fe.get("sha1", "")
if isinstance(sha1, list):
sha1 = sha1[0] if sha1 else ""
parts.append(f"md5 {md5}")
sha1 = fe.hash("sha1")
if sha1:
rom_parts.append(f"sha1 {sha1}")
parts.append(f"sha1 {sha1}")
rendered.append(f"\trom ( {' '.join(parts)} )")
lines.append(f"\trom ( {' '.join(rom_parts)} )")
if not rendered:
continue
lines.append("")
# libretro's comment is the system name as the DAT spells it,
# "Atari - 400-800". Prettifying it drops the separator.
lines.append(f'\tcomment "{system.native_id}"')
lines.extend(rendered)
lines.append(")")
lines.append("")
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
return {self.native_filename(): "\n".join(lines)}
def validate(self, truth_data: dict, output_path: str) -> list[str]:
content = Path(output_path).read_text(encoding="utf-8")
def validate(
self,
systems: dict[str, NativeSystem],
produced: dict[str, str],
) -> list[str]:
content = produced[self.native_filename()]
parsed = parse_dat(content)
exported_names: set[str] = set()
for rom in parsed:
exported_names.add(rom.name)
exported = {rom.name for rom in parsed}
issues: list[str] = []
for sys_data in truth_data.get("systems", {}).values():
for fe in sys_data.get("files", []):
name = fe.get("name", "")
if name.startswith("_") or self._is_pattern(name):
for system in systems.values():
for fe in system.files:
if not any(fe.hash(h) for h in ("crc32", "md5", "sha1")):
continue
if name not in exported_names:
issues.append(f"missing: {name}")
if self._rom_name(fe) not in exported:
issues.append(f"absent from the DAT: {system.native_id}/{fe.name}")
if not content.rstrip().endswith(")"):
issues.append("the game block is not closed")
return issues
scripts/generate_truth.py
+1 -6
View File
@@ -50,11 +50,6 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
default=None,
help="hardware target filter",
)
parser.add_argument(
"--include-archived",
action="store_true",
help="include archived platforms with --all",
)
parser.add_argument(
"--platforms-dir",
default=DEFAULT_PLATFORMS_DIR,
@@ -87,7 +82,7 @@ def main(argv: list[str] | None = None) -> None:
if args.all:
platforms = list_registered_platforms(
args.platforms_dir,
include_archived=args.include_archived,
include_archived=True,
)
else:
platforms = [args.platform]
scripts/pipeline.py
+38 -21
View File
@@ -190,6 +190,20 @@ def check_consistency(verify_output: str, pack_output: str) -> bool:
return all_ok
class _Skipped:
"""A step that did not run. Truthy, so it never fails the pipeline, and
distinct, so the summary does not report it as work that was done."""
def __bool__(self) -> bool:
return True
def __repr__(self) -> str:
return "SKIPPED"
SKIPPED = _Skipped()
def main():
parser = argparse.ArgumentParser(description="Run the full retrobios pipeline")
parser.add_argument(
@@ -296,7 +310,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 2/8 refresh data directories: SKIPPED (--offline) ---")
results["refresh_data"] = True
results["refresh_data"] = SKIPPED
# Step 2a: Refresh MAME BIOS hashes
if not args.offline:
@@ -308,7 +322,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 2a refresh MAME hashes: SKIPPED (--offline) ---")
results["mame_hashes"] = True
results["mame_hashes"] = SKIPPED
# Step 2a2: Refresh FBNeo BIOS hashes
if not args.offline:
@@ -320,7 +334,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 2a2 refresh FBNeo hashes: SKIPPED (--offline) ---")
results["fbneo_hashes"] = True
results["fbneo_hashes"] = SKIPPED
# Step 2b: Check buildbot system directory (non-blocking)
if args.check_buildbot and not args.offline:
@@ -341,27 +355,23 @@ def main():
"--output-dir",
str(Path(args.output_dir) / "truth"),
]
if args.include_archived:
truth_cmd.append("--include-archived")
if args.target:
truth_cmd.extend(["--target", args.target])
ok, _ = run(truth_cmd, "2c generate truth")
results["generate_truth"] = ok
all_ok = all_ok and ok
else:
results["generate_truth"] = True
results["generate_truth"] = SKIPPED
# Step 2d: Diff truth vs scraped
if args.with_truth or args.with_export:
diff_cmd = [sys.executable, "scripts/diff_truth.py", "--all"]
if args.include_archived:
diff_cmd.append("--include-archived")
diff_cmd.extend(["--truth-dir", str(Path(args.output_dir) / "truth")])
ok, _ = run(diff_cmd, "2d diff truth")
results["diff_truth"] = ok
all_ok = all_ok and ok
else:
results["diff_truth"] = True
results["diff_truth"] = SKIPPED
# Step 2e: Export native formats
if args.with_export:
@@ -374,13 +384,16 @@ def main():
"--truth-dir",
str(Path(args.output_dir) / "truth"),
]
if args.include_archived:
export_cmd.append("--include-archived")
# Seven of the formats carry code, so the export patches the
# platform's own file rather than regenerating it. Offline that
# file has to be in the cache already.
if not args.offline:
export_cmd.append("--fetch")
ok, _ = run(export_cmd, "2e export native")
results["export_native"] = ok
all_ok = all_ok and ok
else:
results["export_native"] = True
results["export_native"] = SKIPPED
# Step 3: Verify
verify_cmd = [sys.executable, "scripts/verify.py", "--all"]
@@ -457,7 +470,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 4/8 generate packs: SKIPPED (--skip-packs) ---")
results["generate_packs"] = True
results["generate_packs"] = SKIPPED
# Step 4b: Generate install manifests
if not args.skip_packs:
@@ -480,7 +493,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 4b/8 generate install manifests: SKIPPED (--skip-packs) ---")
results["generate_manifests"] = True
results["generate_manifests"] = SKIPPED
# Step 4c: Generate target manifests
if not args.skip_packs:
@@ -496,7 +509,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 4c/8 generate target manifests: SKIPPED (--skip-packs) ---")
results["generate_target_manifests"] = True
results["generate_target_manifests"] = SKIPPED
# Step 5: Consistency check
if pack_output and verify_output:
@@ -505,7 +518,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 5/8 consistency check: SKIPPED ---")
results["consistency"] = True
results["consistency"] = SKIPPED
# Step 6: Pack integrity (extract + hash verification)
if not args.skip_packs:
@@ -524,7 +537,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 6/8 pack integrity: SKIPPED (--skip-packs) ---")
results["pack_integrity"] = True
results["pack_integrity"] = SKIPPED
# Step 7: Generate README
if not args.skip_docs:
@@ -543,7 +556,7 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 7/8 generate readme: SKIPPED (--skip-docs) ---")
results["generate_readme"] = True
results["generate_readme"] = SKIPPED
# Step 8: Generate site pages
if not args.skip_docs:
@@ -555,13 +568,17 @@ def main():
all_ok = all_ok and ok
else:
print("\n--- 8/8 generate site: SKIPPED (--skip-docs) ---")
results["generate_site"] = True
results["generate_site"] = SKIPPED
# Summary
total_elapsed = time.monotonic() - total_start
print(f"\n{'=' * 60}")
for step, ok in results.items():
print(f" {step:.<40} {'OK' if ok else 'FAILED'}")
for step, outcome in results.items():
if outcome is SKIPPED:
label = "SKIPPED"
else:
label = "OK" if outcome else "FAILED"
print(f" {step:.<40} {label}")
print(f" {'total':.<40} {total_elapsed:.1f}s")
print(f"{'=' * 60}")
print(f" Pipeline {'COMPLETE' if all_ok else 'FINISHED WITH ERRORS'}")
scripts/scraper/base_scraper.py
+43
View File
@@ -28,6 +28,15 @@ class BiosRequirement:
required: bool = True
zipped_file: str | None = None # If set, md5 is for this ROM inside the ZIP
native_id: str | None = None # Original system name before normalization
sha256: str | None = None
alt_md5: str | None = None
# How the platform itself writes the file reference. Its own spelling
# is not always derivable from ours: libretro writes iplromco.dat bare
# but ep128emu/roms/cpc464.rom with its directory.
native_path: str | None = None
# Fields the platform declares that have no equivalent in our model.
# Kept verbatim so the native file can be written back unchanged.
native: dict[str, object] = field(default_factory=dict)
@dataclass
@@ -58,6 +67,40 @@ class ChangeSet:
MAX_RESPONSE_SIZE = 50 * 1024 * 1024 # 50 MB
def requirement_entry(req: BiosRequirement) -> dict:
"""Serialize a requirement to a platform YAML file entry.
Carries the platform's own system id and any field that has no place in
our model. Without them the transcription is lossy in one direction:
several native systems collapse onto one slug, and the exporter can no
longer tell which of them a file came from.
"""
entry: dict = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
for field_name in ("sha1", "md5", "sha256", "crc32"):
value = getattr(req, field_name, None)
if value:
entry[field_name] = str(value).lower()
if req.size:
entry["size"] = req.size
if req.zipped_file:
entry["zipped_file"] = req.zipped_file
if req.alt_md5:
entry["alt_md5"] = str(req.alt_md5).lower()
if req.native_id:
entry["native_system"] = req.native_id
if req.native_path and req.native_path != req.name:
entry["native_path"] = req.native_path
for key in sorted(req.native):
value = req.native[key]
if value not in (None, "", [], {}):
entry[key] = value
return entry
def _read_limited(resp: object, max_bytes: int = MAX_RESPONSE_SIZE) -> bytes:
"""Read an HTTP response with a size limit to prevent OOM."""
chunks: list[bytes] = []
scripts/scraper/batocera_scraper.py
+6 -13
View File
@@ -19,7 +19,7 @@ from pathlib import Path
from common import yaml_load
from .base_scraper import BaseScraper, BiosRequirement
from .base_scraper import BaseScraper, BiosRequirement, requirement_entry
PLATFORM_NAME = "batocera"
@@ -309,7 +309,8 @@ class Scraper(BaseScraper):
bios_files = sys_data.get("biosFiles", [])
for bios in bios_files:
file_path = bios.get("file", "")
declared_path = bios.get("file", "")
file_path = declared_path
md5 = _resolve_truncated_md5(bios.get("md5", ""), md5_index)
zipped_file = bios.get("zippedFile", "")
@@ -324,9 +325,11 @@ class Scraper(BaseScraper):
system=system_slug,
md5=md5 or None,
destination=file_path,
native_path=declared_path,
required=True,
zipped_file=zipped_file or None,
native_id=sys_key,
native={"native_name": sys_data.get("name", "")},
)
)
@@ -365,17 +368,7 @@ class Scraper(BaseScraper):
sys_entry["name"] = dname
systems[req.system] = sys_entry
entry = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.md5:
entry["md5"] = req.md5
if req.zipped_file:
entry["zipped_file"] = req.zipped_file
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
batocera_version = ""
if _STABLE_TAG != "master":
scripts/scraper/bizhawk_scraper.py
+5 -11
View File
@@ -22,6 +22,7 @@ import re
try:
from .base_scraper import (
requirement_entry,
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
@@ -29,6 +30,7 @@ try:
)
except ImportError:
from base_scraper import (
requirement_entry,
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
@@ -352,6 +354,8 @@ class Scraper(BaseScraper):
sha1=rec["sha1"],
size=rec["size"] if rec["size"] else None,
required=rec.get("status") != "Bad",
destination=rec["name"],
native_id=rec["system"],
)
requirements.append(req)
@@ -366,17 +370,7 @@ class Scraper(BaseScraper):
if req.system not in systems:
systems[req.system] = {"files": []}
entry: dict = {
"name": req.name,
"destination": req.name,
"required": req.required,
}
if req.sha1:
entry["sha1"] = req.sha1.lower()
if req.size:
entry["size"] = req.size
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
version = _STABLE_TAG if _STABLE_TAG != "master" else ""
scripts/scraper/emudeck_scraper.py
+31 -12
View File
@@ -18,9 +18,19 @@ import urllib.error
import urllib.request
try:
from .base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from .base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
except ImportError:
from base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
PLATFORM_NAME = "emudeck"
@@ -54,9 +64,18 @@ HASH_ARRAY_MAP = {
"SaturnBios": "sega-saturn",
}
# Every check checkBIOS.sh defines, and the system it stands for. A function
# missing here is a check whose hashes nothing would carry.
FUNCTION_HASH_MAP = {
"checkPS1BIOS": "sony-playstation",
"checkPS2BIOS": "sony-playstation-2",
"checkSegaCDBios": "sega-mega-cd",
"checkSaturnBios": "sega-saturn",
"checkDreamcastBios": "sega-dreamcast",
"checkDSBios": "nintendo-ds",
"checkCitronBios": "nintendo-switch",
"checkRyujinxBios": "nintendo-switch",
"checkYuzuBios": "nintendo-switch",
}
SYSTEM_SLUG_MAP = {
@@ -171,13 +190,17 @@ _RE_ARRAY = re.compile(
re.MULTILINE,
)
# checkBIOS.sh declares its checks as `checkPS1BIOS(){`, with no `function`
# keyword and no settled casing for BIOS.
_RE_FUNC = re.compile(
r"function\s+(check\w+Bios)\s*\(\)",
r"^[ \t]*(?:function\s+)?(check\w*(?:BIOS|Bios))\s*\(\)\s*\{",
re.MULTILINE,
)
# The hash list is named differently in each check (PSBios, hashes, ...), so
# it is found by shape rather than by name.
_RE_LOCAL_HASHES = re.compile(
r"local\s+hashes=\(\s*((?:[0-9a-fA-F]+\s*)+)\)",
r"(?:local\s+)?\w+=\(\s*((?:[0-9a-fA-F]{32}\s*)+)\)",
re.MULTILINE,
)
@@ -346,6 +369,7 @@ class Scraper(BaseScraper):
system=system,
destination=f.get("destination", f["name"]),
required=True,
native_id=system,
)
)
@@ -357,6 +381,7 @@ class Scraper(BaseScraper):
md5=md5,
destination="",
required=True,
native_id=system,
)
)
@@ -379,6 +404,7 @@ class Scraper(BaseScraper):
system=system,
destination=f.get("destination", f["name"]),
required=True,
native_id=system,
)
)
@@ -398,14 +424,7 @@ class Scraper(BaseScraper):
if req.system not in systems:
systems[req.system] = {"files": []}
entry: dict = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.md5:
entry["md5"] = req.md5
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
version = ""
try:
scripts/scraper/libretro_scraper.py
+8 -16
View File
@@ -11,7 +11,12 @@ from __future__ import annotations
import urllib.error
import urllib.request
from .base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from .base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
from .dat_parser import parse_dat, parse_dat_metadata, validate_dat_format
PLATFORM_NAME = "libretro"
@@ -132,6 +137,7 @@ class Scraper(BaseScraper):
destination=destination,
required=True,
native_id=native_system,
native_path=rom.name,
)
)
@@ -247,21 +253,7 @@ class Scraper(BaseScraper):
system_entry["docs"] = cm["docs"]
systems[req.system] = system_entry
entry = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.sha1:
entry["sha1"] = req.sha1
if req.md5:
entry["md5"] = req.md5
if req.crc32:
entry["crc32"] = req.crc32
if req.size:
entry["size"] = req.size
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
# Systems not in System.dat but needed for RetroArch -added via
# shared groups in _shared.yml. The includes directive is resolved
scripts/scraper/misterfpga_scraper.py
+13 -11
View File
@@ -29,9 +29,19 @@ import zipfile
from datetime import datetime, timezone
try:
from .base_scraper import BaseScraper, BiosRequirement, _read_limited
from .base_scraper import (
BaseScraper,
BiosRequirement,
_read_limited,
requirement_entry,
)
except ImportError:
from base_scraper import BaseScraper, BiosRequirement, _read_limited
from base_scraper import (
BaseScraper,
BiosRequirement,
_read_limited,
requirement_entry,
)
PLATFORM_NAME = "misterfpga"
@@ -193,15 +203,7 @@ class Scraper(BaseScraper):
"docs": f"https://github.com/MiSTer-devel/{repo}" if repo else "",
},
)
file_entry: dict = {
"name": req.name,
"destination": req.destination,
"required": req.required,
"md5": req.md5,
}
if req.size is not None:
file_entry["size"] = req.size
entry["files"].append(file_entry)
entry["files"].append(requirement_entry(req))
for entry in systems.values():
if not entry["docs"]:
scripts/scraper/recalbox_scraper.py
+27 -107
View File
@@ -15,11 +15,9 @@ Recalbox verification logic:
from __future__ import annotations
import sys
from common import parse_untrusted_xml
from .base_scraper import BaseScraper, BiosRequirement
from .base_scraper import BaseScraper, BiosRequirement, requirement_entry
PLATFORM_NAME = "recalbox"
@@ -136,6 +134,7 @@ class Scraper(BaseScraper):
for system_elem in root.findall(".//system"):
platform = system_elem.get("platform", "")
system_slug = SYSTEM_SLUG_MAP.get(platform, platform)
fullname = system_elem.get("fullname", "")
for bios_elem in system_elem.findall("bios"):
paths_str = bios_elem.get("path", "")
@@ -154,11 +153,30 @@ class Scraper(BaseScraper):
md5_list = [m.strip() for m in md5_str.split(",") if m.strip()]
all_md5 = ",".join(md5_list) if md5_list else None
dedup_key = primary_path
dedup_key = (platform, primary_path)
if dedup_key in seen:
continue
seen.add(dedup_key)
native: dict[str, object] = {}
if fullname:
native["native_name"] = fullname
core = bios_elem.get("core", "").strip()
if core:
native["core"] = core
note = bios_elem.get("note", "").strip()
if note:
native["note"] = note
# Recalbox reads a missing attribute as true for both flags,
# so only the explicit value carries information.
hash_match = bios_elem.get("hashMatchMandatory")
if hash_match is not None:
native["hash_match_mandatory"] = hash_match != "false"
if bios_elem.get("mandatory") is not None:
native["mandatory_declared"] = mandatory
if len(paths) > 1:
native["alt_paths"] = paths[1:]
requirements.append(
BiosRequirement(
name=name,
@@ -167,56 +185,12 @@ class Scraper(BaseScraper):
destination=primary_path,
required=mandatory,
native_id=platform,
native=native,
)
)
return requirements
def fetch_full_requirements(self) -> list[dict]:
"""Parse es_bios.xml preserving all Recalbox-specific fields."""
raw = self._fetch_raw()
root = parse_untrusted_xml(raw, "es_bios.xml")
requirements = []
for system_elem in root.findall(".//system"):
platform = system_elem.get("platform", "")
system_name = system_elem.get("name", platform)
system_slug = SYSTEM_SLUG_MAP.get(platform, platform)
for bios_elem in system_elem.findall("bios"):
paths_str = bios_elem.get("path", "")
md5_str = bios_elem.get("md5", "")
core = bios_elem.get("core", "")
mandatory = bios_elem.get("mandatory", "true") != "false"
hash_match_mandatory = (
bios_elem.get("hashMatchMandatory", "true") != "false"
)
note = bios_elem.get("note", "")
paths = [p.strip() for p in paths_str.split("|") if p.strip()]
md5_list = [m.strip() for m in md5_str.split(",") if m.strip()]
if not paths:
continue
name = paths[0].split("/")[-1] if "/" in paths[0] else paths[0]
requirements.append(
{
"name": name,
"system": system_slug,
"system_name": system_name,
"paths": paths,
"md5_list": md5_list,
"core": core,
"mandatory": mandatory,
"hash_match_mandatory": hash_match_mandatory,
"note": note,
}
)
return requirements
def validate_format(self, raw_data: str) -> bool:
"""Validate es_bios.xml format."""
return "<biosList" in raw_data and "<system" in raw_data and "<bios" in raw_data
@@ -225,7 +199,7 @@ class Scraper(BaseScraper):
"""Generate a platform YAML config dict from scraped data."""
requirements = self.fetch_requirements()
systems = {}
systems: dict[str, dict] = {}
for req in requirements:
if req.system not in systems:
sys_entry: dict = {"files": []}
@@ -233,15 +207,7 @@ class Scraper(BaseScraper):
sys_entry["native_id"] = req.native_id
systems[req.system] = sys_entry
entry = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.md5:
entry["md5"] = req.md5
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
version = _STABLE_TAG if _STABLE_TAG != "master" else ""
if not version:
@@ -262,55 +228,9 @@ class Scraper(BaseScraper):
def main():
"""CLI entry point."""
import argparse
import json
from .base_scraper import scraper_cli
parser = argparse.ArgumentParser(description="Scrape Recalbox es_bios.xml")
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--json", action="store_true")
parser.add_argument(
"--full", action="store_true", help="Show full Recalbox-specific fields"
)
parser.add_argument("--output", "-o")
args = parser.parse_args()
scraper = Scraper()
try:
if args.full:
reqs = scraper.fetch_full_requirements()
print(json.dumps(reqs[:5], indent=2))
print(f"\nTotal: {len(reqs)} BIOS entries")
return
reqs = scraper.fetch_requirements()
except (ConnectionError, ValueError) as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
if args.dry_run:
from collections import defaultdict
by_system = defaultdict(list)
for r in reqs:
by_system[r.system].append(r)
for sys_name, files in sorted(by_system.items()):
print(f"\n{sys_name} ({len(files)} files):")
for f in files[:5]:
print(f" {f.name} (md5={f.md5[:12] if f.md5 else 'N/A'}...)")
if len(files) > 5:
print(f" ... +{len(files) - 5} more")
print(f"\nTotal: {len(reqs)} BIOS files across {len(by_system)} systems")
return
if args.json:
config = scraper.generate_platform_yaml()
print(json.dumps(config, indent=2))
return
by_system = {}
for r in reqs:
by_system.setdefault(r.system, []).append(r)
print(f"Scraped {len(reqs)} BIOS files across {len(by_system)} systems")
scraper_cli(Scraper, "Scrape Recalbox es_bios.xml")
if __name__ == "__main__":
scripts/scraper/retrobat_scraper.py
+23 -12
View File
@@ -11,9 +11,19 @@ from __future__ import annotations
import json
try:
from .base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from .base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
except ImportError:
from base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
PLATFORM_NAME = "retrobat"
@@ -73,7 +83,8 @@ class Scraper(BaseScraper):
if not isinstance(bios, dict):
continue
file_path = bios.get("file", "")
declared_path = bios.get("file", "")
file_path = declared_path
md5 = bios.get("md5", "")
if not file_path:
@@ -85,6 +96,11 @@ class Scraper(BaseScraper):
name = file_path.split("/")[-1] if "/" in file_path else file_path
native: dict[str, object] = {}
sys_name = sys_data.get("name", "") if isinstance(sys_data, dict) else ""
if sys_name:
native["native_name"] = sys_name
requirements.append(
BiosRequirement(
name=name,
@@ -92,6 +108,9 @@ class Scraper(BaseScraper):
md5=md5 or None,
destination=file_path,
required=True,
native_id=sys_key,
native_path=declared_path,
native=native,
)
)
@@ -139,15 +158,7 @@ class Scraper(BaseScraper):
sys_entry["name"] = dname
systems[req.system] = sys_entry
entry = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.md5:
entry["md5"] = req.md5
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
version = ""
tag = fetch_github_latest_version(GITHUB_REPO)
scripts/scraper/retrodeck_scraper.py
+15 -10
View File
@@ -36,10 +36,10 @@ import urllib.request
from pathlib import Path
try:
from .base_scraper import BaseScraper, BiosRequirement
from .base_scraper import BaseScraper, BiosRequirement, requirement_entry
except ImportError:
sys.path.insert(0, str(Path(__file__).parent.parent))
from scraper.base_scraper import BaseScraper, BiosRequirement
from scraper.base_scraper import BaseScraper, BiosRequirement, requirement_entry
PLATFORM_NAME = "retrodeck"
COMPONENTS_REPO = "RetroDECK/components"
@@ -385,13 +385,25 @@ class Scraper(BaseScraper):
continue
seen.add(key)
native: dict[str, object] = {"component": comp_key}
description = str(entry.get("description", "")).strip()
if description:
native["description"] = description
if required_raw not in (None, ""):
native["required_label"] = str(required_raw)
sha256 = str(entry.get("sha256", "")).strip().lower()
requirements.append(
BiosRequirement(
name=filename,
system=system,
destination=destination,
md5=md5,
sha256=sha256 or None,
required=required,
native_id=str(raw_system),
native=native,
)
)
@@ -413,14 +425,7 @@ class Scraper(BaseScraper):
systems: dict[str, dict] = {}
for req in reqs:
sys_entry = systems.setdefault(req.system, {"files": []})
file_entry: dict = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.md5:
file_entry["md5"] = req.md5
sys_entry["files"].append(file_entry)
sys_entry["files"].append(requirement_entry(req))
try:
from .base_scraper import fetch_github_latest_version
scripts/scraper/retropie_scraper.py
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env python3
"""Scraper for RetroPie's package list.
Source: RetroPie/RetroPie-Setup -> scriptmodules/
Format: one bash script per package, `rp_module_id` naming it
RetroPie publishes no BIOS list: platforms.cfg carries extensions and full
names, and the files a package needs are named in prose inside its
`rp_module_help`, without hashes. So there is nothing here to transcribe as
requirements, and `fetch_requirements` returns none. What RetroPie does
state precisely is which packages it ships, and that is what this reads.
It matters because the config said `cores: all_libretro`, inherited from
RetroArch: RetroPie claimed every libretro core in existence while shipping
96 of them, and claimed none of the standalone emulators it also packages
(openmsx, pcsx2, xroar, sdltrs, amiberry and the rest). Both halves were
wrong, in opposite directions.
"""
from __future__ import annotations
import io
import re
import tarfile
import urllib.error
import urllib.request
try:
from .base_scraper import BaseScraper, BiosRequirement
except ImportError:
from base_scraper import BaseScraper, BiosRequirement
PLATFORM_NAME = "retropie"
SOURCE_URL = (
"https://codeload.github.com/RetroPie/RetroPie-Setup/tar.gz/refs/heads/master"
)
GITHUB_REPO = "RetroPie/RetroPie-Setup"
MAX_ARCHIVE = 64 * 1024 * 1024
_MODULE_ID = re.compile(r'^rp_module_id="([^"]+)"', re.MULTILINE)
# Sections RetroPie does not build as emulators: setup helpers, themes,
# drivers and the like carry no core.
_PACKAGE_DIRS = ("emulators", "libretrocores", "ports")
class Scraper(BaseScraper):
"""Scraper for the RetroPie package list."""
def __init__(self, url: str = SOURCE_URL):
super().__init__(url=url)
self._modules: list[str] | None = None
def _fetch_archive(self) -> bytes:
request = urllib.request.Request(
self.url, headers={"User-Agent": "retrobios-scraper/1.0"}
)
try:
with urllib.request.urlopen(request, timeout=60) as response:
payload = response.read(MAX_ARCHIVE + 1)
except urllib.error.URLError as exc:
raise ConnectionError(f"Failed to fetch {self.url}: {exc}") from exc
if len(payload) > MAX_ARCHIVE:
raise ValueError(f"{self.url}: response larger than {MAX_ARCHIVE} bytes")
return payload
def module_ids(self, payload: bytes | None = None) -> list[str]:
"""Every package RetroPie ships, by the id its script declares."""
if self._modules is not None:
return self._modules
raw = payload if payload is not None else self._fetch_archive()
found: set[str] = set()
with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as archive:
for member in archive:
if not member.isfile() or not member.name.endswith(".sh"):
continue
relative = member.name.split("/", 1)[-1]
parts = relative.split("/")
if len(parts) != 3 or parts[0] != "scriptmodules":
continue
if parts[1] not in _PACKAGE_DIRS:
continue
handle = archive.extractfile(member)
if handle is None:
continue
text = handle.read().decode("utf-8", errors="replace")
match = _MODULE_ID.search(text)
if match:
found.add(match.group(1))
self._modules = sorted(found)
return self._modules
def fetch_requirements(self) -> list[BiosRequirement]:
"""None: RetroPie names BIOS in prose, with no hash to transcribe.
The prose is read where it can be acted on, by the exporter that
rewrites those sentences.
"""
self.module_ids()
return []
def validate_format(self, raw_data: str) -> bool:
return bool(self.module_ids())
def generate_platform_yaml(self) -> dict:
"""Build the RetroPie platform configuration.
The systems stay inherited from RetroArch: RetroPie installs the
libretro cores and reads the same files, at BIOS/ instead of
system/. Only the core list is its own.
"""
# A libretro package is lr-<core>; a standalone package is named
# after the emulator itself.
cores = sorted({module.removeprefix("lr-") for module in self.module_ids()})
return {
"inherits": "retroarch",
"platform": "RetroPie",
"homepage": "https://retropie.org.uk",
"source": SOURCE_URL,
"base_destination": "BIOS",
"cores": cores,
}
def main() -> None:
try:
from .base_scraper import scraper_cli
except ImportError:
from base_scraper import scraper_cli
scraper_cli(Scraper, "Scrape the RetroPie package list")
if __name__ == "__main__":
main()
scripts/scraper/rocknix_scraper.py
+16 -16
View File
@@ -18,9 +18,19 @@ from __future__ import annotations
import re
try:
from .base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from .base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
except ImportError:
from base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
PLATFORM_NAME = "rocknix"
@@ -136,16 +146,15 @@ class Scraper(BaseScraper):
# Declared paths are relative to the storage root: strip the
# leading bios/ so destinations match base_destination
destination = path[len("bios/"):] if path.startswith("bios/") else path
md5_values = [bios.get("md5", ""), bios.get("altmd5", "")]
md5 = ",".join(m for m in md5_values if m) or None
req = BiosRequirement(
name=destination.rsplit("/", 1)[-1],
system=system_slug,
md5=md5,
md5=bios.get("md5", "") or None,
alt_md5=bios.get("altmd5", "") or None,
destination=destination,
required=True,
native_id=key,
native={"native_name": entry["name"]},
)
if bios.get("zippedFile"):
req.zipped_file = bios["zippedFile"]
@@ -168,16 +177,7 @@ class Scraper(BaseScraper):
"name": names.get(req.native_id, req.system),
},
)
file_entry: dict = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.md5:
file_entry["md5"] = req.md5
if getattr(req, "zipped_file", None):
file_entry["zipped_file"] = req.zipped_file
entry["files"].append(file_entry)
entry["files"].append(requirement_entry(req))
return {
"platform": "ROCKNIX",
scripts/scraper/romm_scraper.py
+14 -18
View File
@@ -26,9 +26,19 @@ import json
import sys
try:
from .base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from .base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
except ImportError:
from base_scraper import BaseScraper, BiosRequirement, fetch_github_latest_version
from base_scraper import (
BaseScraper,
BiosRequirement,
fetch_github_latest_version,
requirement_entry,
)
PLATFORM_NAME = "romm"
@@ -165,6 +175,7 @@ class Scraper(BaseScraper):
size=size,
destination=f"{slug}/{filename}",
required=True,
native_id=slug,
)
)
@@ -183,7 +194,6 @@ class Scraper(BaseScraper):
for key in list(data.keys())[:5]:
if ":" not in key:
return False
_, _entry = key.split(":", 1), data[key]
if not isinstance(data[key], dict):
return False
if "md5" not in data[key] and "sha1" not in data[key]:
@@ -200,21 +210,7 @@ class Scraper(BaseScraper):
if req.system not in systems:
systems[req.system] = {"files": []}
entry: dict = {
"name": req.name,
"destination": req.destination,
"required": req.required,
}
if req.sha1:
entry["sha1"] = req.sha1
if req.md5:
entry["md5"] = req.md5
if req.crc32:
entry["crc32"] = req.crc32
if req.size:
entry["size"] = req.size
systems[req.system]["files"].append(entry)
systems[req.system]["files"].append(requirement_entry(req))
version = _STABLE_TAG if _STABLE_TAG != "master" else ""
scripts/truth.py
+17
View File
@@ -180,9 +180,24 @@ def _merge_file_into_system(
"note",
"aliases",
"contents",
"region",
):
if file_entry.get(field) is not None and existing.get(field) is None:
existing[field] = file_entry[field]
# Search order is a fact of the code, so it travels with the entry.
# The best rank any core gives it is kept and the disagreement is
# recorded beside it, the way slot.py already does: the rank says
# how early to read the file, the conflict says the order cannot
# decide which single file to keep.
theirs = file_entry.get("priority")
if theirs is not None:
ours = existing.get("priority")
if ours is None:
existing["priority"] = theirs
else:
if ours != theirs:
existing["priority_conflict"] = True
existing["priority"] = min(ours, theirs)
return
entry: dict = {"name": file_entry["name"]}
@@ -204,6 +219,8 @@ def _merge_file_into_system(
"max_size",
"aliases",
"contents",
"priority",
"region",
):
val = file_entry.get(field)
if val is not None:
tests/test_audit_regressions.py
+21
View File
@@ -1219,6 +1219,27 @@ class FreshnessGuardMechanics(unittest.TestCase):
self.assertIn("Generated on B", handle.read())
class PipelineReportsWhatItDid(unittest.TestCase):
"""A step that did not run must not be summarised as OK.
--offline skips the network steps and --with-export gates three more.
Reporting them as OK claims work nobody did, and a run whose export
never happened read exactly like one where it had.
"""
def test_a_skipped_step_is_not_reported_as_done(self):
import pipeline
self.assertTrue(bool(pipeline.SKIPPED), "a skip must not fail the run")
self.assertEqual(repr(pipeline.SKIPPED), "SKIPPED")
self.assertIsNot(pipeline.SKIPPED, True)
def test_every_skip_branch_uses_the_sentinel(self):
source = (ROOT / "scripts" / "pipeline.py").read_text()
stale = re.findall(r'results\["(\w+)"\] = True', source)
self.assertEqual(stale, [], f"steps still claiming OK when skipped: {stale}")
if __name__ == "__main__":
unittest.main()
tests/test_e2e.py
+40 -26
View File
@@ -3996,19 +3996,19 @@ class TestE2E(unittest.TestCase):
},
}
out_path = os.path.join(self.root, "System.dat")
exporter = SystemDatExporter()
exporter.export(truth, out_path, scraped_data=scraped)
from exporter.baseline import build_native_model
with open(out_path) as fh:
content = fh.read()
systems, report = build_native_model(truth, scraped)
exporter = SystemDatExporter()
produced = exporter.render(systems, report, {}, scraped)
content = produced["System.dat"]
self.assertIn("Sony - PlayStation", content)
self.assertIn("scph5501.bin", content)
self.assertIn("b056ee5a4d65937e1a3a17e1e78f3258ea49c38e", content)
self.assertIn('name "System"', content)
self.assertIn("71AF80B4", content) # CRC uppercase
issues = exporter.validate(truth, out_path)
issues = exporter.validate(systems, produced)
self.assertEqual(issues, [])
# Discovery finds the systemdat exporter
@@ -4037,12 +4037,11 @@ class TestE2E(unittest.TestCase):
},
}
out_path = os.path.join(self.root, "System_numeric.dat")
SystemDatExporter().export(truth, out_path, scraped_data={"systems": {}})
from exporter.baseline import build_native_model
with open(out_path) as fh:
content = fh.read()
self.assertIn("crc 70295038", content)
systems, report = build_native_model(truth, {"systems": {}})
produced = SystemDatExporter().render(systems, report, {}, {"systems": {}})
self.assertIn("crc 70295038", produced["System.dat"])
# ---------------------------------------------------------------
# Full truth + diff integration test
@@ -4417,15 +4416,28 @@ class TestE2E(unittest.TestCase):
"sony-playstation": {"native_id": "psx", "files": []},
}
}
out = os.path.join(self.root, "batocera-systems")
exp = Exporter()
exp.export(truth, out, scraped_data=scraped)
from exporter.baseline import build_native_model
content = Path(out).read_text(encoding="utf-8")
# The systems dict is one block inside the checker Batocera runs, so
# the exporter is handed the script it patches.
skeleton = (
"#!/usr/bin/env python\n"
"systems = {\n"
"}\n"
"def checkBios(systems, prefix, filterROMs):\n"
" return {}\n"
)
systems, report = build_native_model(truth, scraped)
exp = Exporter()
produced = exp.render(
systems, report, {"batocera-systems": skeleton}, scraped
)
content = produced["batocera-systems"]
self.assertIn('"psx"', content)
self.assertIn("scph5501.bin", content)
self.assertIn("b" * 32, content)
self.assertEqual(exp.validate(truth, out), [])
self.assertIn("def checkBios(", content)
self.assertEqual(exp.validate(systems, produced), [])
def test_180_recalbox_exporter_round_trip(self):
"""Recalbox exporter produces valid es_bios.xml."""
@@ -4453,11 +4465,12 @@ class TestE2E(unittest.TestCase):
"sony-playstation": {"native_id": "psx", "files": []},
}
}
out = os.path.join(self.root, "es_bios.xml")
exp = Exporter()
exp.export(truth, out, scraped_data=scraped)
from exporter.baseline import build_native_model
content = Path(out).read_text(encoding="utf-8")
systems, report = build_native_model(truth, scraped)
exp = Exporter()
produced = exp.render(systems, report, {}, scraped)
content = produced["es_bios.xml"]
self.assertIn("<biosList", content)
self.assertIn('platform="psx"', content)
self.assertIn("fullname=", content)
@@ -4465,7 +4478,7 @@ class TestE2E(unittest.TestCase):
# mandatory="true" is the default, not emitted (matching Recalbox format)
self.assertNotIn('mandatory="false"', content)
self.assertIn('core="libretro/c"', content)
self.assertEqual(exp.validate(truth, out), [])
self.assertEqual(exp.validate(systems, produced), [])
def test_181_retrobat_exporter_round_trip(self):
"""RetroBat exporter produces valid JSON."""
@@ -4495,16 +4508,17 @@ class TestE2E(unittest.TestCase):
"sony-playstation": {"native_id": "psx", "files": []},
}
}
out = os.path.join(self.root, "batocera-systems.json")
exp = Exporter()
exp.export(truth, out, scraped_data=scraped)
from exporter.baseline import build_native_model
data = _json.loads(Path(out).read_text(encoding="utf-8"))
systems, report = build_native_model(truth, scraped)
exp = Exporter()
produced = exp.render(systems, report, {}, scraped)
data = _json.loads(produced["batocera-systems.json"])
self.assertIn("psx", data)
self.assertTrue(
any("scph5501" in bf["file"] for bf in data["psx"]["biosFiles"])
)
self.assertEqual(exp.validate(truth, out), [])
self.assertEqual(exp.validate(systems, produced), [])
def test_182_exporter_discovery(self):
"""All exporters are discovered by the plugin system."""
tests/test_exporters.py
+786 -129
View File
@@ -1,14 +1,22 @@
#!/usr/bin/env python3
"""Native-format exporters.
export_native.py turns the profile data back into each platform's own file:
Recalbox's es_bios.xml, Batocera's systems list, RetroDECK's manifests. The
point is that a platform maintainer can take the output and use it, so the
format has to be theirs, not an approximation of it. None of it was covered.
export_native.py rewrites each platform's own file, corrected: Recalbox's
es_bios.xml, Batocera's systems list, RetroDECK's manifests. The point is
that a maintainer can take the output and use it, so what matters is
whether the platform would still read it, and whether anything it already
declared went missing on the way through.
The fidelity cases read the platforms' real files from the upstream cache
and skip when it has not been populated (python scripts/export_native.py
--all --fetch), because a test that needs the network is not a test.
"""
from __future__ import annotations
import json
import re
import subprocess
import sys
import tempfile
import unittest
@@ -18,8 +26,20 @@ from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO_ROOT / "scripts"))
from common import ( # noqa: E402
list_registered_platforms,
load_platform_config,
parse_untrusted_xml,
)
from exporter import discover_exporters # noqa: E402
from exporter.base_exporter import BaseExporter # noqa: E402
from export_native import pinned_base # noqa: E402
from exporter.baseline import ( # noqa: E402
NativeFile,
build_native_model,
)
UPSTREAM_CACHE = REPO_ROOT / ".cache" / "upstream-native"
TRUTH = {
"systems": {
@@ -29,13 +49,17 @@ TRUTH = {
"name": "scph5501.bin",
"path": "psx/scph5501.bin",
"md5": "490f666e1afb15b7362b406ed1cea246",
"size": 524288,
"required": True,
"_cores": ["duckstation"],
},
{
"name": "scph7001.bin",
"path": "psx/scph7001.bin",
"md5": "1e68c231d0896b7eadcad1d7d8e76129",
"size": 524288,
"required": False,
"_cores": ["duckstation"],
},
# Carries an md5 on purpose: without one it would be dropped
# for being incomplete, and the placeholder guard would never
@@ -44,7 +68,37 @@ TRUTH = {
"name": "<user-selected>.bin",
"path": "psx/PLACEHOLDER-MUST-NOT-SHIP.bin",
"md5": "00000000000000000000000000000000",
"size": 1,
"required": False,
"_cores": ["duckstation"],
},
],
},
},
}
SCRAPED = {
"systems": {
"sony-playstation": {
"native_id": "psx",
"files": [
{
"name": "scph5501.bin",
"destination": "psx/scph5501.bin",
"native_system": "psx",
"native_name": "Sony PlayStation",
"md5": "deadbeefdeadbeefdeadbeefdeadbeef",
"core": "libretro/duckstation",
"required": True,
},
{
"name": "unknown-to-truth.bin",
"destination": "psx/unknown-to-truth.bin",
"native_system": "psx",
"native_name": "Sony PlayStation",
"md5": "11111111111111111111111111111111",
"core": "libretro/duckstation",
"required": True,
},
],
},
@@ -52,161 +106,764 @@ TRUTH = {
}
def model(truth: dict = TRUTH, scraped: dict | None = SCRAPED):
return build_native_model(truth, scraped)
class Reconciliation(unittest.TestCase):
"""The platform's file corrected, not replaced by ours."""
def test_a_file_the_truth_says_nothing_about_is_kept(self):
systems, _ = model()
names = {fe.name for fe in systems["psx"].files}
self.assertIn("unknown-to-truth.bin", names)
def test_the_truth_overrides_a_hash_the_platform_got_wrong(self):
systems, report = model()
entry = next(fe for fe in systems["psx"].files if fe.name == "scph5501.bin")
self.assertEqual(entry.hash("md5"), "490f666e1afb15b7362b406ed1cea246")
self.assertTrue(any("scph5501" in c for c in report.hashes_corrected))
def test_a_file_only_the_truth_knows_is_added(self):
systems, report = model()
names = {fe.name for fe in systems["psx"].files}
self.assertIn("scph7001.bin", names)
self.assertEqual(report.files_added, 1)
def test_a_placeholder_is_never_added(self):
systems, _ = model()
names = {fe.name for fe in systems["psx"].files}
self.assertNotIn("<user-selected>.bin", names)
def test_systems_are_keyed_by_the_name_the_platform_uses(self):
systems, _ = model()
self.assertEqual(sorted(systems), ["psx"])
def test_the_platform_own_grouping_survives_a_collapsed_slug(self):
"""Several native systems can share one of our slugs.
Recalbox files pcengine, pcenginecd and supergrafx under one slug.
Grouping on the slug would publish one of the three and drop the
rest, which is how eleven Recalbox platforms went missing.
"""
scraped = {
"systems": {
"nec-pc-engine": {
"native_id": "pcengine",
"files": [
{
"name": "syscard3.pce",
"destination": "pcengine/syscard3.pce",
"native_system": "pcengine",
"md5": "38179df8f4ac870017db21ebcbf53114",
},
{
"name": "syscard3.pce",
"destination": "supergrafx/syscard3.pce",
"native_system": "supergrafx",
"md5": "38179df8f4ac870017db21ebcbf53114",
},
],
},
},
}
systems, _ = build_native_model({"systems": {}}, scraped)
self.assertEqual(sorted(systems), ["pcengine", "supergrafx"])
def test_an_entry_the_platform_declares_without_a_hash_is_still_written(self):
"""Batocera and RetroBat both ship entries with an empty md5."""
scraped = {
"systems": {
"nintendo-ds": {
"native_id": "nds",
"files": [
{
"name": "firmware.bin",
"destination": "firmware.bin",
"native_system": "nds",
}
],
}
}
}
systems, _ = build_native_model({"systems": {}}, scraped)
entry = systems["nds"].files[0]
self.assertTrue(BaseExporter.writable(entry, require="md5"))
def test_no_format_drops_what_the_platform_already_declares(self):
"""An export poorer than the file it corrects is not usable."""
systems, _ = model()
for name, cls in discover_exporters().items():
for system in systems.values():
for entry in system.files:
if entry.platform is None:
continue
with self.subTest(platform=name, file=entry.name):
self.assertTrue(cls.writable(entry))
def test_an_addition_the_format_cannot_express_is_refused(self):
truth = {
"systems": {
"nintendo-ds": {"files": [{"name": "no-hash.bin", "required": True}]}
}
}
systems, _ = build_native_model(truth, None)
entry = systems["nintendo-ds"].files[0]
self.assertFalse(BaseExporter.writable(entry, require="md5"))
class ExporterRegistry(unittest.TestCase):
def test_every_exporter_declares_a_platform_and_the_contract(self):
exporters = discover_exporters()
self.assertTrue(exporters, "no exporters discovered")
for platform, cls in exporters.items():
with self.subTest(platform=platform):
"""Every registered platform has an exporter, and it declares itself."""
def test_every_exporter_declares_a_platform_and_a_filename(self):
for name, cls in discover_exporters().items():
with self.subTest(platform=name):
self.assertTrue(cls.platform_name())
self.assertTrue(cls.native_filename())
self.assertTrue(issubclass(cls, BaseExporter))
self.assertEqual(cls.platform_name(), platform)
instance = cls()
self.assertTrue(callable(instance.export))
self.assertTrue(callable(instance.validate))
def test_the_registry_covers_the_platforms_it_claims(self):
for expected in ("recalbox", "batocera", "retrodeck", "romm"):
self.assertIn(expected, discover_exporters())
def test_every_registered_platform_can_be_exported(self):
"""A platform with no exporter is a platform we cannot hand back.
class SharedHelpers(unittest.TestCase):
def test_placeholder_names_are_recognised(self):
for name in ("<user-selected>.bin", "rom*.bin", "a<b>.rom"):
self.assertTrue(BaseExporter._is_pattern(name), name)
for name in ("scph5501.bin", "bios_CD_E.bin"):
self.assertFalse(BaseExporter._is_pattern(name), name)
def test_destination_prefers_path_then_destination_then_name(self):
self.assertEqual(
BaseExporter._dest({"path": "a", "destination": "b", "name": "c"}), "a"
Archived included: the export is a correction we offer, not a
release artefact, and RetroPie's users are not fewer for it.
"""
exporters = discover_exporters()
registered = set(
list_registered_platforms(
str(REPO_ROOT / "platforms"), include_archived=True
)
)
self.assertEqual(BaseExporter._dest({"destination": "b", "name": "c"}), "b")
self.assertEqual(BaseExporter._dest({"name": "c"}), "c")
self.assertEqual(BaseExporter._dest({}), "")
self.assertEqual(registered - set(exporters), set())
def test_a_scraped_display_name_wins_over_the_slug(self):
self.assertEqual(
BaseExporter._display_name("sony-playstation", {"name": "PlayStation"}),
"PlayStation",
def test_no_exporter_is_registered_for_an_unregistered_platform(self):
"""A dead exporter is one nobody will notice has stopped working."""
exporters = set(discover_exporters())
registered = set(
list_registered_platforms(
str(REPO_ROOT / "platforms"), include_archived=True
)
)
self.assertEqual(exporters - registered, set())
def test_every_exporter_says_what_its_format_can_state(self):
"""A correction to a field the file has no place for is not delivered."""
for name, cls in discover_exporters().items():
with self.subTest(platform=name):
carried = cls.carries()
self.assertTrue(carried)
self.assertTrue(
carried
<= {"size", "crc32", "md5", "sha1", "sha256", "required", "name"},
f"{name} claims a field that is not one of ours: {carried}",
)
def test_only_a_format_with_the_field_claims_to_carry_it(self):
"""Recalbox writes mandatory; RetroBat's entries are md5 and file."""
exporters = discover_exporters()
self.assertIn("required", exporters["recalbox"].carries())
self.assertNotIn("required", exporters["retrobat"].carries())
self.assertNotIn("required", exporters["retroarch"].carries())
def test_an_exporter_that_patches_code_says_it_needs_the_original(self):
"""Regenerating a script from data alone drops the code around it."""
for name in ("batocera", "rocknix", "emudeck", "retrodeck", "bizhawk",
"misterfpga"):
with self.subTest(platform=name):
self.assertTrue(discover_exporters()[name].needs_original())
class SearchOrder(unittest.TestCase):
"""A list is read in the order the code looks, best first."""
@staticmethod
def _entry(name, priority=None, position=0):
truth = {"name": name, "required": True, "md5": f"{position:032x}"}
if priority is not None:
truth["priority"] = priority
return NativeFile(
name=name, destination=name, native_system="psx", truth=truth
)
def test_acronyms_stay_upper_case_in_a_derived_name(self):
derived = BaseExporter._display_name("nes-and-snes")
self.assertIn("NES", derived)
self.assertIn("SNES", derived)
def test_the_declared_rank_orders_the_names(self):
exporter = discover_exporters()["retropie"]()
candidates = [
self._entry("scph5502.bin", 5, 0),
self._entry("scph5501.bin", 2, 1),
self._entry("scph7001.bin", 10, 2),
]
self.assertEqual(
exporter._in_search_order(candidates),
["scph5501.bin", "scph5502.bin", "scph7001.bin"],
)
def test_without_a_rank_the_order_the_code_walks_is_kept(self):
"""An undeclared order is the profile's own, not the alphabet."""
exporter = discover_exporters()["retropie"]()
candidates = [
self._entry("zzz.bin", None, 0),
self._entry("aaa.bin", None, 1),
]
self.assertEqual(
exporter._in_search_order(candidates), ["zzz.bin", "aaa.bin"]
)
def test_a_ranked_file_is_read_before_an_unranked_one(self):
exporter = discover_exporters()["retropie"]()
candidates = [
self._entry("unranked.bin", None, 0),
self._entry("ranked.bin", 9, 1),
]
self.assertEqual(
exporter._in_search_order(candidates), ["ranked.bin", "unranked.bin"]
)
def test_cores_that_disagree_keep_the_best_rank(self):
"""Nothing is dropped on the rank here, so the best one wins.
slot.py voids the choice when cores disagree because it keeps one
file; ordering keeps them all, and a file that is some core's first
choice belongs early.
"""
merged = {"name": "scph5501.bin", "priority": 2, "priority_conflict": True}
entry = NativeFile(
name="scph5501.bin",
destination="scph5501.bin",
native_system="psx",
truth=merged,
)
self.assertEqual(entry.priority, 2)
class _ExportFixture(unittest.TestCase):
platform = ""
class AdditionOrder(unittest.TestCase):
"""What we add is listed the way the code looks for it."""
def setUp(self):
if not self.platform:
self.skipTest("base fixture")
self._tmp = tempfile.TemporaryDirectory()
self.out = Path(self._tmp.name) / "out"
self.exporter = discover_exporters()[self.platform]()
TRUTH = {
"systems": {
"sony-playstation": {
"files": [
{
"name": "third.bin",
"md5": "c" * 32,
"required": True,
"priority": 10,
},
{
"name": "first.bin",
"md5": "a" * 32,
"required": True,
"priority": 1,
},
{"name": "unranked.bin", "md5": "b" * 32, "required": True},
]
}
}
}
def tearDown(self):
if hasattr(self, "_tmp"):
self._tmp.cleanup()
def test_a_ranked_addition_is_named_before_a_later_one(self):
systems, _ = build_native_model(self.TRUTH, None)
names = [fe.name for fe in systems["sony-playstation"].files]
self.assertEqual(names, ["first.bin", "third.bin", "unranked.bin"])
def _export(self) -> str:
self.exporter.export(TRUTH, str(self.out))
self.assertTrue(self.out.exists(), "exporter wrote nothing")
return self.out.read_text(encoding="utf-8")
def test_what_the_platform_wrote_keeps_its_place(self):
"""Reordering their own entries would be churn, not a correction."""
scraped = {
"systems": {
"sony-playstation": {
"native_id": "psx",
"files": [
{
"name": "theirs-b.bin",
"destination": "theirs-b.bin",
"native_system": "psx",
"md5": "d" * 32,
},
{
"name": "theirs-a.bin",
"destination": "theirs-a.bin",
"native_system": "psx",
"md5": "e" * 32,
},
],
}
}
}
systems, _ = build_native_model(self.TRUTH, scraped)
names = [fe.name for fe in systems["psx"].files]
self.assertEqual(names[:2], ["theirs-b.bin", "theirs-a.bin"])
self.assertEqual(names[2], "first.bin")
class RecalboxExport(_ExportFixture):
platform = "recalbox"
class PinnedRevision(unittest.TestCase):
"""The file we patch is the revision our transcription came from."""
def test_the_tag_the_scraper_pinned_wins_over_the_branch_tip(self):
wanted = {"batocera-systems": "https://example.invalid/master/batocera-systems"}
scraped = {
"source": "https://raw.githubusercontent.com/batocera-linux"
"/batocera.linux/batocera-43.1/package/batocera-systems"
}
self.assertEqual(
pinned_base(wanted, scraped),
"https://raw.githubusercontent.com/batocera-linux/batocera.linux"
"/batocera-43.1/package/",
)
def test_a_blob_page_is_read_as_raw(self):
"""github.com/.../blob/... names a revision but serves HTML."""
wanted = {"System.dat": "https://example.invalid/System.dat"}
scraped = {
"source": "https://github.com/libretro/libretro-database"
"/blob/master/dat/System.dat"
}
self.assertEqual(
pinned_base(wanted, scraped),
"https://raw.githubusercontent.com/libretro/libretro-database"
"/master/dat/",
)
def test_a_source_that_names_no_wanted_file_changes_nothing(self):
wanted = {"component_manifest.json": "https://example.invalid/x.json"}
scraped = {"source": "https://github.com/RetroDECK/components"}
self.assertEqual(pinned_base(wanted, scraped), "")
def test_every_platform_that_records_its_source_pins_the_export(self):
"""A scraped platform states the revision; the export follows it."""
exporters = discover_exporters()
for name in ("batocera", "recalbox", "bizhawk", "romm", "retroarch"):
with self.subTest(platform=name):
config = load_platform_config(name, str(REPO_ROOT / "platforms"))
base = pinned_base(dict(exporters[name].native_sources()), config)
self.assertTrue(base.startswith("https://"), base)
class RecalboxExport(unittest.TestCase):
def _render(self):
systems, report = model()
exporter = discover_exporters()["recalbox"]()
return exporter, systems, exporter.render(systems, report, {}, None)
def test_output_is_well_formed_xml_in_recalbox_shape(self):
root = ET.fromstring(self._export())
_, _, produced = self._render()
root = ET.fromstring(produced["es_bios.xml"])
self.assertEqual(root.tag, "biosList")
systems = root.findall("system")
self.assertTrue(systems, "no <system> emitted")
self.assertTrue(list(root.iter("system")))
def test_each_declared_file_becomes_a_bios_element_with_its_md5(self):
root = ET.fromstring(self._export())
by_path = {
b.get("path"): b for s in root.findall("system") for b in s.findall("bios")
}
self.assertIn("psx/scph5501.bin", by_path)
self.assertEqual(
by_path["psx/scph5501.bin"].get("md5"),
"490f666e1afb15b7362b406ed1cea246",
)
def test_a_placeholder_entry_is_not_exported_as_a_file(self):
"""Recalbox would otherwise look for a file nobody can supply.
The check is on the destination path, not the name: the exporter
writes path=, so asserting on the name passes whatever the filter
does.
"""
exported = self._export()
self.assertNotIn("PLACEHOLDER-MUST-NOT-SHIP", exported)
self.assertNotIn("<user-selected>", exported)
def test_every_element_carries_what_the_schema_requires(self):
"""es_bios.xsd makes path, md5 and core required on every bios."""
_, _, produced = self._render()
root = ET.fromstring(produced["es_bios.xml"])
for element in root.iter("bios"):
for attribute in ("path", "md5", "core"):
self.assertTrue(element.get(attribute), attribute)
def test_mandatory_is_only_stated_when_it_is_false(self):
"""Recalbox treats a missing attribute as mandatory."""
root = ET.fromstring(self._export())
by_path = {
b.get("path"): b for s in root.findall("system") for b in s.findall("bios")
}
_, _, produced = self._render()
root = ET.fromstring(produced["es_bios.xml"])
by_path = {e.get("path"): e for e in root.iter("bios")}
self.assertIsNone(by_path["psx/scph5501.bin"].get("mandatory"))
self.assertEqual(by_path["psx/scph7001.bin"].get("mandatory"), "false")
def test_the_export_round_trips_through_its_own_validator(self):
self.exporter.export(TRUTH, str(self.out))
self.assertEqual(self.exporter.validate(TRUTH, str(self.out)), [])
def test_a_truncated_export_is_reported_by_the_validator(self):
self.exporter.export(TRUTH, str(self.out))
self.out.write_text(
'<?xml version="1.0" encoding="UTF-8"?>\n<biosList></biosList>\n',
encoding="utf-8",
def test_hash_match_mandatory_does_not_follow_mandatory(self):
"""They are separate axes: Recalbox uses all four combinations."""
_, _, produced = self._render()
root = ET.fromstring(produced["es_bios.xml"])
optional = next(
e for e in root.iter("bios") if e.get("mandatory") == "false"
)
self.assertTrue(
self.exporter.validate(TRUTH, str(self.out)),
"a file missing every entry validated clean",
self.assertIsNone(optional.get("hashMatchMandatory"))
def test_the_export_passes_its_own_validator(self):
exporter, systems, produced = self._render()
self.assertEqual(exporter.validate(systems, produced), [])
def test_a_truncated_export_is_reported(self):
exporter, systems, _ = self._render()
empty = {
"es_bios.xml": '<?xml version="1.0"?>\n<biosList></biosList>\n',
}
self.assertTrue(exporter.validate(systems, empty))
class RommExport(unittest.TestCase):
def _render(self):
systems, report = model()
exporter = discover_exporters()["romm"]()
return exporter, systems, exporter.render(systems, report, {}, None)
def test_every_entry_can_actually_verify(self):
"""RomM matches on size and then on one hash; without both, never."""
_, _, produced = self._render()
data = json.loads(produced["known_bios_files.json"])
self.assertTrue(data)
for key, entry in data.items():
with self.subTest(key=key):
self.assertTrue(entry.get("size"))
self.assertTrue(
any(entry.get(h) for h in ("md5", "sha1", "crc"))
)
def test_size_is_written_as_the_fixture_writes_it(self):
_, _, produced = self._render()
data = json.loads(produced["known_bios_files.json"])
self.assertIsInstance(next(iter(data.values()))["size"], str)
def test_keys_are_the_platform_slug_the_file_belongs_to(self):
_, _, produced = self._render()
data = json.loads(produced["known_bios_files.json"])
self.assertTrue(all(k.startswith("psx:") for k in data))
class BatoceraExport(unittest.TestCase):
def _render(self, originals):
systems, report = model()
exporter = discover_exporters()["batocera"]()
return exporter, systems, exporter.render(systems, report, originals, None)
SKELETON = (
"#!/usr/bin/env python\n"
"systems = {\n"
"}\n"
"def checkBios(systems, prefix, filterROMs):\n"
" return {}\n"
)
def test_an_unhashed_entry_is_written_with_an_empty_md5(self):
scraped = {
"systems": {
"nintendo-ds": {
"native_id": "nds",
"files": [
{
"name": "firmware.bin",
"destination": "firmware.bin",
"native_system": "nds",
}
],
}
}
}
systems, report = build_native_model({"systems": {}}, scraped)
exporter = discover_exporters()["batocera"]()
produced = exporter.render(
systems, report, {"batocera-systems": self.SKELETON}, None
)
self.assertIn('"md5": ""', produced["batocera-systems"])
def test_writing_without_the_platform_file_is_refused(self):
"""A systems dict alone is not the script Batocera runs."""
systems, report = model()
exporter = discover_exporters()["batocera"]()
with self.assertRaises(ValueError):
exporter.render(systems, report, {}, None)
def test_the_code_around_the_data_survives(self):
original = (
"#!/usr/bin/env python\n"
"systems = {\n"
' "psx": { "name": "PS1", "biosFiles": [ '
'{ "md5": "deadbeefdeadbeefdeadbeefdeadbeef", '
'"file": "bios/psx/scph5501.bin" } ] },\n'
"}\n"
"def checkBios(systems, prefix, filterROMs):\n"
" return {}\n"
)
_, _, produced = self._render({"batocera-systems": original})
output = produced["batocera-systems"]
self.assertIn("def checkBios(", output)
self.assertIn("#!/usr/bin/env python", output)
def test_a_note_beside_a_file_is_not_lost_when_the_entry_changes(self):
original = (
"systems = {\n"
' "psx": { "name": "PS1", "biosFiles": [\n'
' # ideally the other revision\n'
' { "md5": "deadbeefdeadbeefdeadbeefdeadbeef", '
'"file": "bios/psx/scph5501.bin" } ] },\n'
"}\n"
"def checkBios(systems, prefix, filterROMs):\n"
" return {}\n"
)
_, _, produced = self._render({"batocera-systems": original})
self.assertIn("# ideally the other revision", produced["batocera-systems"])
class UpstreamFidelity(unittest.TestCase):
"""The real files, compared against what we hand back.
Nothing a platform declares today may be missing from the export: an
export that loses systems is one no maintainer can apply.
"""
@classmethod
def setUpClass(cls):
if not UPSTREAM_CACHE.is_dir():
raise unittest.SkipTest(
"no upstream cache; run scripts/export_native.py --all --fetch"
)
def _export(self, platform: str, truth: dict | None = None) -> dict[str, str]:
cache = UPSTREAM_CACHE / platform
if not cache.is_dir():
self.skipTest(f"no cached original for {platform}")
scraped = load_platform_config(platform, str(REPO_ROOT / "platforms"))
systems, report = build_native_model(truth or {"systems": {}}, scraped)
exporter = discover_exporters()[platform]()
originals: dict[str, str] = {}
for path in cache.rglob("*"):
if not path.is_file():
continue
relative = str(path.relative_to(cache))
if relative.endswith((".zip", ".tar.gz")) and hasattr(exporter, "unpack"):
originals.update(exporter.unpack(path.read_bytes()))
else:
originals[relative] = path.read_text(
encoding="utf-8", errors="replace"
)
produced = exporter.render(systems, report, originals, scraped)
self.assertEqual(exporter.validate(systems, produced), [])
return produced, originals
def test_recalbox_keeps_every_system_and_every_path(self):
produced, originals = self._export("recalbox")
before = parse_untrusted_xml(originals["es_bios.xml"], "es_bios.xml")
after = ET.fromstring(produced["es_bios.xml"])
self.assertEqual(
{s.get("platform") for s in before.iter("system")}
- {s.get("platform") for s in after.iter("system")},
set(),
)
self.assertEqual(
{b.get("path") for b in before.iter("bios")}
- {b.get("path") for b in after.iter("bios")},
set(),
)
def test_the_recalbox_export_satisfies_the_schema_recalbox_ships(self):
schema_path = UPSTREAM_CACHE / "recalbox" / "es_bios.xsd"
if not schema_path.exists():
self.skipTest("es_bios.xsd not cached")
required = self._required_attributes(schema_path.read_text())
produced, _ = self._export("recalbox")
root = ET.fromstring(produced["es_bios.xml"])
for tag, attributes in required.items():
for element in root.iter(tag):
for attribute in attributes:
with self.subTest(tag=tag, attribute=attribute):
self.assertTrue(element.get(attribute))
class EveryExporterProducesSomething(unittest.TestCase):
"""Whatever the format, an exporter must write a non-empty file it accepts."""
@staticmethod
def _required_attributes(schema: str) -> dict[str, set[str]]:
"""Read the required attributes straight out of the platform's XSD.
def test_each_exporter_writes_and_validates_its_own_output(self):
for platform, cls in sorted(discover_exporters().items()):
with self.subTest(platform=platform):
with tempfile.TemporaryDirectory() as directory:
out = Path(directory) / "out"
exporter = cls()
try:
exporter.export(TRUTH, str(out))
except (KeyError, TypeError, ValueError) as exc:
self.fail(f"{platform} export raised {exc!r}")
self.assertTrue(out.exists(), f"{platform} wrote nothing")
self.assertTrue(
out.stat().st_size > 0, f"{platform} wrote an empty file"
)
issues = exporter.validate(TRUTH, str(out))
self.assertEqual(
issues, [], f"{platform} rejects its own output: {issues[:2]}"
)
Scoped to each element's own type: a nested declaration belongs to
the child, and folding it into the parent would demand md5 on
<system>.
"""
namespace = "{http://www.w3.org/2001/XMLSchema}"
root = parse_untrusted_xml(schema, "es_bios.xsd")
required: dict[str, set[str]] = {}
def test_no_exporter_emits_a_placeholder_name(self):
for platform, cls in sorted(discover_exporters().items()):
with self.subTest(platform=platform):
with tempfile.TemporaryDirectory() as directory:
out = Path(directory) / "out"
cls().export(TRUTH, str(out))
exported = out.read_text(encoding="utf-8", errors="replace")
self.assertNotIn(
"PLACEHOLDER-MUST-NOT-SHIP",
exported,
f"{platform} exported a placeholder as a real file",
)
self.assertNotIn("<user-selected>", exported)
def own_attributes(node: ET.Element) -> set[str]:
found: set[str] = set()
for child in node:
if child.tag == f"{namespace}element":
continue
if child.tag == f"{namespace}attribute":
if child.get("use") == "required" and child.get("name"):
found.add(child.get("name"))
continue
found |= own_attributes(child)
return found
for element in root.iter(f"{namespace}element"):
name = element.get("name")
if not name:
continue
names = own_attributes(element)
if names:
required[name] = names
return required
def test_batocera_keeps_every_system_and_still_runs(self):
produced, originals = self._export("batocera")
before = set(
re.findall(r'^\s{4}"([^"]+)":\s*\{\s*"name"', originals["batocera-systems"],
re.M)
)
after = set(
re.findall(r'^\s{4}"([^"]+)":\s*\{\s*"name"',
produced["batocera-systems"], re.M)
)
self.assertEqual(before - after, set())
with tempfile.TemporaryDirectory() as tmp:
script = Path(tmp) / "batocera-systems"
script.write_text(produced["batocera-systems"], encoding="utf-8")
result = subprocess.run(
[sys.executable, str(script), "--createReadme"],
capture_output=True,
text=True,
timeout=120,
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertTrue(result.stdout.strip())
def test_batocera_keeps_the_zip_member_checks(self):
"""checkInsideZip verifies a ROM inside the archive, not the archive."""
produced, originals = self._export("batocera")
self.assertEqual(
produced["batocera-systems"].count("zippedFile"),
originals["batocera-systems"].count("zippedFile"),
)
def test_rocknix_still_runs(self):
produced, _ = self._export("rocknix")
with tempfile.TemporaryDirectory() as tmp:
script = Path(tmp) / "rocknix-systems"
script.write_text(produced["rocknix-systems"], encoding="utf-8")
result = subprocess.run(
[sys.executable, str(script), "--createReadme"],
capture_output=True,
text=True,
timeout=120,
)
self.assertEqual(result.returncode, 0, result.stderr)
def test_emudeck_keeps_every_check_and_still_parses(self):
produced, originals = self._export("emudeck")
pattern = re.compile(r"^(\w+)\(\)\{", re.M)
before = set(pattern.findall(originals["checkBIOS.sh"]))
after = set(pattern.findall(produced["checkBIOS.sh"]))
self.assertEqual(before - after, set())
with tempfile.TemporaryDirectory() as tmp:
script = Path(tmp) / "checkBIOS.sh"
script.write_text(produced["checkBIOS.sh"], encoding="utf-8")
result = subprocess.run(
["bash", "-n", str(script)], capture_output=True, text=True, timeout=60
)
self.assertEqual(result.returncode, 0, result.stderr)
def test_retrobat_keeps_every_system_and_every_file(self):
produced, originals = self._export("retrobat")
before = json.loads(originals["batocera-systems.json"])
after = json.loads(produced["batocera-systems.json"])
self.assertEqual(set(before) - set(after), set())
self.assertEqual(
{(s, f["file"]) for s, d in before.items() for f in d["biosFiles"]}
- {(s, f["file"]) for s, d in after.items() for f in d["biosFiles"]},
set(),
)
def test_retroarch_keeps_every_rom_name_the_dat_declares(self):
produced, originals = self._export("retroarch")
pattern = re.compile(r"^\trom \( name (\"[^\"]+\"|\S+)", re.M)
before = {n.strip('"') for n in pattern.findall(originals["System.dat"])}
after = {n.strip('"') for n in pattern.findall(produced["System.dat"])}
self.assertEqual(before - after, set())
def test_the_dat_keeps_the_system_names_libretro_writes(self):
"""The comment is the name the DAT spells, separator included."""
produced, originals = self._export("retroarch")
comments = re.compile(r'^\tcomment "([^"]+)"', re.M)
before = set(comments.findall(originals["System.dat"]))
after = set(comments.findall(produced["System.dat"]))
self.assertEqual(before - after, set())
def test_the_dat_never_states_a_rom_it_cannot_hash(self):
produced, _ = self._export("retroarch")
for line in re.findall(r"^\trom \( (.*) \)$", produced["System.dat"], re.M):
with self.subTest(line=line[:60]):
self.assertTrue(
any(f" {h} " in line for h in ("crc", "md5", "sha1"))
)
def test_misterfpga_keeps_the_download_urls_it_installs_from(self):
produced, _ = self._export("misterfpga")
database = json.loads(produced["bios_db.json"])
self.assertTrue(database.get("db_id"))
for path, entry in database["files"].items():
with self.subTest(path=path):
self.assertTrue(entry.get("url"))
def test_bizhawk_stays_compilable(self):
produced, originals = self._export("bizhawk")
source = produced["FirmwareDatabase.cs"]
self.assertEqual(source.count("{"), source.count("}"))
self.assertEqual(
source.count("FirmwareAndOption("),
originals["FirmwareDatabase.cs"].count("FirmwareAndOption("),
)
# A core RetroPie packages, with a file its help does not yet name.
RETROPIE_TRUTH = {
"systems": {
"magnavox-odyssey2": {
"files": [
{
"name": "c52.bin",
"required": True,
"md5": "f1071cdb0b6b10dde94d3bc8a6146387",
"_cores": ["o2em"],
}
]
}
}
}
def test_retropie_keeps_every_name_its_packages_already_list(self):
"""RetroPie's BIOS list is prose; a name it writes is never removed."""
produced, originals = self._export("retropie", self.RETROPIE_TRUTH)
exporter = discover_exporters()["retropie"]()
help_of = re.compile(r'rp_module_help="((?:[^"\\]|\\.)*)"')
for relative, text in produced.items():
with self.subTest(module=relative):
before = help_of.search(originals[relative])
after = help_of.search(text)
self.assertIsNotNone(after, "the help string is not closed")
self.assertEqual(
exporter._listed(before.group(1))
- exporter._listed(after.group(1)),
set(),
)
self.assertIn("BIOS", after.group(1))
def test_a_patched_scriptmodule_still_parses_as_bash(self):
produced, _ = self._export("retropie", self.RETROPIE_TRUTH)
self.assertTrue(produced)
with tempfile.TemporaryDirectory() as tmp:
for relative, text in produced.items():
script = Path(tmp) / Path(relative).name
script.write_text(text, encoding="utf-8")
result = subprocess.run(
["bash", "-n", str(script)],
capture_output=True,
text=True,
timeout=60,
)
with self.subTest(module=relative):
self.assertEqual(result.returncode, 0, result.stderr)
def test_retropie_never_drafts_a_sentence_nobody_wrote(self):
"""A package with no BIOS list is a docs change, not a correction."""
produced, originals = self._export("retropie", self.RETROPIE_TRUTH)
for relative in produced:
with self.subTest(module=relative):
self.assertIn("BIOS", originals[relative])
def test_retrodeck_writes_a_manifest_per_component(self):
produced, _ = self._export("retrodeck")
self.assertTrue(produced)
for path, content in produced.items():
with self.subTest(path=path):
self.assertTrue(path.endswith("component_manifest.json"))
manifest = json.loads(content)
component = next(iter(manifest.values()))
self.assertTrue(
component.get("name") or component.get("system"),
"the component lost its identity",
)
if __name__ == "__main__":
tests/test_scraper_contract.py
+72
View File
@@ -116,5 +116,77 @@ class TestConnection(unittest.TestCase):
_StubScraper(error=KeyError("bug")).test_connection()
class RetroPiePackageList(unittest.TestCase):
"""RetroPie states its packages, and nothing else states them for it."""
@staticmethod
def _archive(modules: dict[str, str]) -> bytes:
import io
import tarfile
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w:gz") as archive:
for path, body in modules.items():
payload = body.encode()
info = tarfile.TarInfo(f"RetroPie-Setup-master/{path}")
info.size = len(payload)
archive.addfile(info, io.BytesIO(payload))
return buffer.getvalue()
def _scraper(self, modules: dict[str, str]):
from scraper.retropie_scraper import Scraper
scraper = Scraper()
scraper.module_ids(self._archive(modules))
return scraper
def test_a_package_is_read_from_the_id_its_script_declares(self):
scraper = self._scraper(
{
"scriptmodules/libretrocores/lr-mgba.sh": 'rp_module_id="lr-mgba"\n',
"scriptmodules/emulators/openmsx.sh": 'rp_module_id="openmsx"\n',
}
)
self.assertEqual(scraper.module_ids(), ["lr-mgba", "openmsx"])
def test_a_script_outside_the_package_directories_is_not_a_package(self):
"""Setup helpers, themes and drivers carry no core."""
scraper = self._scraper(
{
"scriptmodules/libretrocores/lr-mgba.sh": 'rp_module_id="lr-mgba"\n',
"scriptmodules/supplementary/autostart.sh": (
'rp_module_id="autostart"\n'
),
"scriptmodules/admin/setup.sh": 'rp_module_id="setup"\n',
}
)
self.assertEqual(scraper.module_ids(), ["lr-mgba"])
def test_the_core_list_drops_the_libretro_prefix(self):
scraper = self._scraper(
{
"scriptmodules/libretrocores/lr-mgba.sh": 'rp_module_id="lr-mgba"\n',
"scriptmodules/emulators/openmsx.sh": 'rp_module_id="openmsx"\n',
}
)
config = scraper.generate_platform_yaml()
self.assertEqual(config["cores"], ["mgba", "openmsx"])
self.assertEqual(config["inherits"], "retroarch")
self.assertEqual(config["base_destination"], "BIOS")
def test_no_requirement_is_invented_from_prose(self):
"""RetroPie names BIOS in prose, without a hash to transcribe."""
scraper = self._scraper(
{
"scriptmodules/libretrocores/lr-mgba.sh": (
'rp_module_id="lr-mgba"\n'
'rp_module_help="Copy the required BIOS file gba_bios.bin'
' to $biosdir"\n'
),
}
)
self.assertEqual(scraper.fetch_requirements(), [])
if __name__ == "__main__":
unittest.main()
wiki/adding-a-platform.md
+11 -8
View File
@@ -276,17 +276,20 @@ class Exporter(BaseExporter):
### Round-trip validation
The exporter enables a scrape-export-compare workflow:
```bash
# Scrape upstream
python -m scripts.scraper.myplatform_scraper --output /tmp/scraped.yml
# Export truth data
python scripts/export_native.py --platform myplatform --output /tmp/exported.json
# Compare exported file with upstream
diff /tmp/scraped.yml /tmp/exported.json
python -m scripts.scraper.myplatform_scraper --output platforms/myplatform.yml
python scripts/export_native.py --platform myplatform --fetch --output-dir tmp/native
diff .cache/upstream-native/myplatform/<their file> \
tmp/native/myplatform/<their file>
```
The export must not lose anything the platform already declares. If your
format carries code as well as data - a script, a C# database, a manifest
whose BIOS list sits beside launch configuration - say so with
`needs_original()` and patch their file rather than writing a new one. An
exporter that regenerates such a file from BIOS data alone hands the
maintainer something that no longer runs.
## Step 6: Create a target scraper (optional)
Target scrapers determine which emulator cores are available on each hardware
wiki/adding-a-scraper.md
+16 -7
View File
@@ -403,20 +403,29 @@ manipulation rather than load-dump to maintain human-readable formatting.
### Round-trip testing
If an exporter exists for the platform, validate the scrape-export-compare cycle:
The scrape and the export are two directions of the same transcription, so
run them against each other: scrape upstream, export it back, and diff the
result against the file you started from. Nothing the platform declares may
be missing from the round trip.
```bash
# Scrape upstream -> platform YAML
python -m scripts.scraper.myplatform_scraper --output /tmp/scraped.yml
python -m scripts.scraper.myplatform_scraper --output platforms/myplatform.yml
# Export truth data -> native format
python scripts/export_native.py --platform myplatform --output /tmp/exported.json
# Rewrite the platform's own file, corrected
python scripts/export_native.py --platform myplatform --fetch \
--output-dir tmp/native
# Compare
diff <(python -m scripts.scraper.myplatform_scraper --json | python -m json.tool) \
/tmp/exported.json
# Compare with the file upstream publishes, cached by --fetch
diff .cache/upstream-native/myplatform/<their file> \
tmp/native/myplatform/<their file>
```
Every line in that diff should be a correction you can defend. A line that
merely reshapes their formatting means the scraper lost something the
exporter then had to invent: add it to the requirement instead, through
`native_path`, `native_system` or the `native` dict.
### Common issues
| Symptom | Cause | Fix |
wiki/advanced-usage.md
+23 -5
View File
@@ -324,15 +324,33 @@ The diff reports:
### Export to native formats
Convert truth data to the native format each platform consumes:
Rewrite a platform's own BIOS file, corrected. The output is the file its
maintainers keep, not our data in their syntax: hashes the truth can prove
are applied, entries the truth says nothing about are left alone, and files
the truth knows and the platform lacks are added.
```bash
python scripts/export_native.py --platform batocera # Python dict (batocera-systems)
python scripts/export_native.py --platform recalbox # XML (es_bios.xml)
python scripts/export_native.py --all --output-dir dist/upstream/
python scripts/export_native.py --platform batocera --fetch # batocera-systems
python scripts/export_native.py --platform recalbox --fetch # es_bios.xml
python scripts/export_native.py --all --fetch --output-dir dist/upstream/
```
This allows submitting corrections upstream in the format maintainers expect.
Seven of the formats carry code as well as data - Batocera's and ROCKNIX's
scripts, EmuDeck's shell library, BizHawk's C# database, RetroDECK's
component manifests, MiSTer's BiosDB, RetroPie's scriptmodules. Those are
patched from the platform's own file, which `--fetch` downloads once into
`.cache/upstream-native/`. Without it the export fails rather than
publishing a fragment.
RetroPie is the odd one: it publishes no BIOS list at all, and the only
declaration is the sentence in each package's `rp_module_help`. The export
adds a missing file name to a list a maintainer already wrote, never
removes one, and never drafts a sentence where there was none.
The run reports what it changed, and what it had to leave out: an entry a
format cannot express is named, never written half-formed. `es_bios.xsd`
makes md5 and core required on every element, and RomM compares the file
size before any hash, so an entry missing either could never verify.
## Emulator-Level Verification
wiki/architecture.md
+2 -2
View File
@@ -55,8 +55,8 @@ emulators/*.yml verify.py checks generate_pack.py resolves
from code verification packs per platform
truth.py generates diff_truth.py export_native.py
ground truth from compares truth vs exports to native formats
emulator profiles scraped platform (DAT, XML, JSON, Bash)
ground truth from compares truth vs rewrites each platform's
emulator profiles scraped platform own file, corrected
```
Pipeline runs all steps in sequence: DB, provenance report, data dirs,
wiki/tools.md
+17 -3
View File
@@ -226,13 +226,27 @@ python scripts/diff_truth.py --all # diff all platforms
### export_native.py
Export truth data to native platform formats (System.dat, es_bios.xml, checkBIOS.sh, etc.).
Rewrite each platform's own BIOS file, corrected: System.dat, es_bios.xml,
batocera-systems, checkBIOS.sh, FirmwareDatabase.cs, the RetroDECK manifests,
MiSTer's BiosDB, RetroPie's scriptmodules. Every platform in the registry,
archived included.
```bash
python scripts/export_native.py --platform batocera
python scripts/export_native.py --all --output-dir dist/upstream/
python scripts/export_native.py --platform batocera --fetch
python scripts/export_native.py --all --fetch --output-dir dist/upstream/
```
`--fetch` downloads the platform's own file once into
`.cache/upstream-native/`, at the revision the platform YAML's `source:`
names. Formats that carry code are patched from it rather than
regenerated, and the export fails without it.
A list is written in the order the code looks: `priority:` first (lowest
wins), then the profile's own declaration order. What a format cannot
state is reported rather than written — EmuDeck's arrays are shared
between emulators its file does not name, so a hash is corrected in place
and never added.
MAME and FBNeo entries are `.zip` ROM sets: their meaningful hashes belong
to the files inside the archive, so the exported DAT lists those entries
without a container sha1. Anyone submitting the DAT upstream should mention