kmoody/libretro
1
0 Fork 0
mirror of https://github.com/Abdess/retroarch_system.git synced 2026-11-06 01:53:56 -06:00
Code Issues Packages Projects Releases Wiki Activity

fix: narrow hashless name matches to their maker

Browse Source 
AmberELEC's nec-pc-88 declares DISK.ROM with an empty md5. QUASI88 reads
that name as n88sub.rom, the 2 KB PC-88 disk ROM (libretro.c:1085), but
the name step served the 16 KB MSX disk ROM, the only primary of that
name. The quasi88 alias never reached by_name, which refuses an alias a
primary already carries.

load_platform_config now writes each entry's system as platform_system.
A platform entry with no hash and no owning emulator, whose system id
names a maker, takes no name candidate filed under another maker, in
the name step and in the hash_mismatch step after it. When nothing is
left, by_shadowed_alias, the profile aliases by_name refuses, answers
within the system's maker folder. names_manufacturer and BUCKET_FOLDERS
move to common.py, and generate_site imports them.

Across every registered platform the only resolution that moves is
AmberELEC nec-pc-88 DISK.ROM, from bios/Microsoft/MSX/DISK.ROM to
bios/NEC/PC-88/N88SUB.ROM. Manifests, truth and exports are otherwise
byte-identical.
This commit is contained in:
Abdessamad Derraz committed 2026-11-05 23:17:04 +01:00
1 parent 9a66ac3219
commit d1ebd411ab
11 files changed
+578 -27
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

database.json
+125 -1
View File
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"generated_at": "2026-11-05T18:33:47Z",
"generated_at": "2026-11-05T22:08:47Z",
"total_files": 19944,
"total_size": 16892646538,
"bios_dir": "bios",
@@ -600408,6 +600408,130 @@
"sf2/custom0.sf2": [
"286b2e1fb21cc79851da01666db6c0b0e88f25e3"
]
},
"by_shadowed_alias": {
"DISK.ROM": [
"bb7103a0818850a039c67ff666a31ce49a8d516f",
"d4181c9f046aafc3fb326b381baac809d9e38d16"
],
"Mac OS ROM": [
"e4fc7560b69d062cb2da5b1ffbe11cd1ca03cc37"
],
"N88EXT1.ROM": [
"4ae4d37409ff99411a623da9f6a44192170a854e"
],
"N88EXT2.ROM": [
"e94278682ef9e9bbb82201f72c50382748dcea2a"
],
"N88EXT3.ROM": [
"af452aed79b072c4d17985830b7c5dca64d4b412"
],
"ROM1": [
"e4fc7560b69d062cb2da5b1ffbe11cd1ca03cc37"
],
"basic10.rom": [
"af7eae90c398ec12942a408fa5ac2722e331738c"
],
"basic2.rom": [
"3693c9cb9bf3b41bae1150f7a8264992468fc8c0"
],
"bios.bin": [
"6e89d1227581c76441a53d605f9e324185f1da33"
],
"bios.rom": [
"6e89d1227581c76441a53d605f9e324185f1da33"
],
"bios9821.rom": [
"0877ffb4b4d1c18283468be3579b72ed8c22e3ac"
],
"biosdsi7.bin": [
"d2b17224e156ee5e55b5d8d9f323b1c9b86dd951"
],
"biosdsi9.bin": [
"db61fa39ddbc5f5ed71fb19cda47609ef0201723"
],
"d8000.rom": [
"0877ffb4b4d1c18283468be3579b72ed8c22e3ac"
],
"disk.rom": [
"bb7103a0818850a039c67ff666a31ce49a8d516f",
"d4181c9f046aafc3fb326b381baac809d9e38d16"
],
"epr-17740.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17740a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17741a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17742a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17951a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17952a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17953a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-17954a.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-19730.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-19854.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"epr-23603.ic8": [
"eff0f54c70bce05ff3a289bf30b1027e1c8cd117"
],
"hisaturn.bin": [
"df94c5b4d47eb3cc404d88b33a8fda237eaf4720"
],
"n88.rom": [
"3b31fc68fa7f47b21c1a1cb027b86b9e87afbfff"
],
"n88_0.rom": [
"d239c26ad7ac5efac6e947b0e9549b1534aa970d"
],
"n88_1.rom": [
"8528eef7946edf6501a6ccb1f416b60c64efac7c"
],
"n88_2.rom": [
"b7c8bcea219b77d9cc3ee0efafe343cc307425d1"
],
"n88_3.rom": [
"efce0b51cab9f0da6cf68507757f1245a2867a72"
],
"ps2-0250j-20100415.bin": [
"f9229fe159d0353b9f0632f3fdc66819c9030458"
],
"rom1.bin": [
"aea7a4c0c7ffe1f212f7b9faecfd728862ac6904"
],
"rom2.bin": [
"5d5ab4313a2d0effe7572bb129193b64cab002c1"
],
"rom2_st.bin": [
"5d5ab4313a2d0effe7572bb129193b64cab002c1"
],
"sega1003.bin": [
"df94c5b4d47eb3cc404d88b33a8fda237eaf4720"
],
"sega_100.bin": [
"df94c5b4d47eb3cc404d88b33a8fda237eaf4720"
],
"sega_100a.bin": [
"df94c5b4d47eb3cc404d88b33a8fda237eaf4720"
],
"vsaturn.bin": [
"df94c5b4d47eb3cc404d88b33a8fda237eaf4720"
]
}
}
}
install/amberelec.json
+7 -7
View File
@@ -6,7 +6,7 @@
"platform": "amberelec",
"display_name": "AmberELEC",
"version": "prerelease-20261019_1835",
"generated": "2026-11-05T18:33:47Z",
"generated": "2026-11-05T22:08:47Z",
"base_destination": "",
"detect": [
{
@@ -30,9 +30,9 @@
],
"standalone_copies": [],
"total_files": 4958,
"total_size": 5348102161,
"total_size": 5348087825,
"pack_files": 5152,
"pack_size": 5348202505,
"pack_size": 5348188169,
"total_omitted": 285,
"omitted_files": [
{
@@ -8414,10 +8414,10 @@
},
{
"dest": "bios/quasi88/DISK.ROM",
"sha1": "032cb1c1c75b9a191fa1230978971698d9d2a17f",
"sha256": "31740a761447d7590c304338d389b0ae114546a8b9f79c9ffd5e374f80b7d354",
"size": 16384,
"repo_path": "bios/Microsoft/MSX/DISK.ROM",
"sha1": "bb7103a0818850a039c67ff666a31ce49a8d516f",
"sha256": "9dc63118728a171ce2ed437ad38ef280b8b7bcf4a5a95fedf18946501b010456",
"size": 2048,
"repo_path": "bios/NEC/PC-88/N88SUB.ROM",
"cores": null,
"systems": [
"nec-pc-88"
install/games/fallback/amberelec.json
+1 -1
View File
@@ -1 +1 @@
{"files":[["bios/128p-0.rom",16384,"bios"],["bios/128p-1.rom",16384,"bios"],["bios/256s-0.rom",16384,"bios"],["bios/256s-1.rom",16384,"bios"],["bios/256s-2.rom",16384,"bios"],["bios/256s-3.rom",16384,"bios"],["bios/32X_G_BIOS.BIN",256,"bios"],["bios/32X_M_BIOS.BIN",2048,"bios"],["bios/32X_S_BIOS.BIN",1024,"bios"],["bios/3do_arcade_saot.bin",524288,"bios"],["bios/5200.rom",2048,"bios"],["bios/64DD_IPL.bin",4194304,"bios"],["bios/7800 BIOS (E).rom",16384,"bios"],["bios/7800 BIOS (U).rom",4096,"bios"],["bios/ATARIBAS.ROM",8192,"bios"],["bios/ATARIOSA.ROM",10240,"bios"],["bios/ATARIOSB.ROM",10240,"bios"],["bios/ATARIXL.ROM",16384,"bios"],["bios/BB01R4_OS.ROM",16384,"bios"],["bios/BIOS.col",8192,"bios"],["bios/BLOOD.RFF",3536782,"game_data"],["bios/BS-X.bin",1048576,"bios"],["bios/CD32 Extended-ROM r40.60 (1993)(Commodore)(CD32).rom",524288,"bios"],["bios/CDTV Extended-ROM v1.0 (1991)(Commodore)(CDTV)[!].rom",262144,"bios"],["bios/CM32LN_CONTROL.ROM",65536,"bios"],["bios/CM32L_CONTROL.ROM",65536,"bios"],["bios/CM32L_PCM.ROM",1048576,"bios"],["bios/DOSBOX.SF2",32319396,"bios"],["bios/DUKE3D.GRP",11035779,"game_data"],["bios/Databases/colecodb.xml",164848,"bios"],["bios/Databases/msxcaswavdb.xml",819679,"bios"],["bios/Databases/msxromdb.xml",930986,"bios"],["bios/Databases/msxsysromdb.xml",157542,"bios"],["bios/Databases/segadb.xml",82318,"bios"],["bios/Databases/svidb.xml",261070,"bios"],["bios/ECS.bin",24576,"bios"],["bios/GUI.RFF",7145,"game_data"],["bios/IVOICE.bin",2048,"bios"],["bios/JiffyDOS_1541-II.bin",16384,"bios"],["bios/JiffyDOS_1571_repl310654.bin",32768,"bios"],["bios/JiffyDOS_1581.bin",32768,"bios"],["bios/JiffyDOS_C128.bin",16384,"bios"],["bios/JiffyDOS_C64.bin",8192,"bios"],["bios/JiffyDOS_SX-64.bin",8192,"bios"],["bios/Kickstart v1.2 r33.180 (1986)(Commodore)(A500-A1000-A2000)[!].rom",262144,"bios"],["bios/Kickstart v1.2 rev 33.166 (1986)(Commodore)(A1000).rom",262144,"bios"],["bios/Kickstart v1.3 r34.5 (1987)(Commodore)(A500-A1000-A2000-CDTV)[!].rom",262144,"bios"],["bios/Kickstart v1.3 r34.5 (1987)(Commodore)(A500-A1000-A2000-CDTV)[o].rom",524288,"bios"],["bios/Kickstart v2.04 r37.175 (1991)(Commodore)(A500+)[!].rom",524288,"bios"],["bios/Kickstart v2.05 r37.350 (1992)(Commodore)(A600HD)[!].rom",524288,"bios"],["bios/Kickstart v3.1 r40.60 (1993)(Commodore)(CD32).rom",524288,"bios"],["bios/Kickstart v3.1 r40.68 (1993)(Commodore)(A1200)[!].rom",524288,"bios"],["bios/Kickstart v3.1 r40.68 (1993)(Commodore)(A3000).rom",524288,"bios"],["bios/Kickstart v3.1 r40.68 (1993)(Commodore)(A4000).rom",524288,"bios"],["bios/MT32_CONTROL.ROM",65536,"bios"],["bios/MT32_PCM.ROM",524288,"bios"],["bios/Machines/COL - Bit Corporation Dina/czz50-1.rom",8192,"bios"],["bios/Machines/COL - Bit Corporation Dina/czz50-2.rom",8192,"bios"],["bios/Machines/COL - ColecoVision with Opcode Memory Extension/coleco.rom",8192,"bios"],["bios/Machines/COL - ColecoVision/coleco.rom",8192,"bios"],["bios/Machines/COL - ColecoVision/config.ini",347,"bios"],["bios/Machines/COL - Spectravideo SVI-603 Coleco/SVI603.ROM",8192,"bios"],["bios/Machines/Forte II Games - Pesadelo/pesadelo.rom",65536,"bios"],["bios/Machines/MSX - C-BIOS/cbios.txt",8924,"bios"],["bios/Machines/MSX - C-BIOS/cbios_logo_msx1.rom",16384,"bios"],["bios/Machines/MSX - C-BIOS/cbios_main_msx1.rom",32768,"bios"],["bios/Machines/MSX/MSX.ROM",32768,"bios"],["bios/Machines/MSX/MSX2.ROM",32768,"bios"],["bios/Machines/MSX/MSX2EXT.ROM",16384,"bios"],["bios/Machines/MSX/MSX2P.ROM",32768,"bios"],["bios/Machines/MSX/MSX2PEXT.ROM",16384,"bios"],["bios/Machines/MSX/MSXDOS2.ROM",65536,"bios"],["bios/Machines/MSX/config.ini",408,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios.txt",8924,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios_logo_msx2.rom",16384,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios_main_msx2.rom",32768,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios_sub.rom",16384,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios.txt",8924,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_logo_msx2+.rom",16384,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_main_msx2+.rom",32768,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_music.rom",16384,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_sub.rom",16384,"bios"],["bios/Machines/MSX2+/MSX.ROM",32768,"bios"],["bios/Machines/MSX2+/MSX2.ROM",32768,"bios"],["bios/Machines/MSX2+/MSX2EXT.ROM",16384,"bios"],["bios/Machines/MSX2+/MSX2P.ROM",32768,"bios"],["bios/Machines/MSX2+/MSX2PEXT.ROM",16384,"bios"],["bios/Machines/MSX2+/MSXDOS2.ROM",65536,"bios"],["bios/Machines/MSX2+/config.ini",773,"bios"],["bios/Machines/MSX2/MSX.ROM",32768,"bios"],["bios/Machines/MSX2/MSX2.ROM",32768,"bios"],["bios/Machines/MSX2/MSX2EXT.ROM",16384,"bios"],["bios/Machines/MSX2/MSX2P.ROM",32768,"bios"],["bios/Machines/MSX2/MSX2PEXT.ROM",16384,"bios"],["bios/Machines/MSX2/MSXDOS2.ROM",65536,"bios"],["bios/Machines/MSX2/config.ini",649,"bios"],["bios/Machines/MSXturboR/config.ini",941,"bios"],["bios/Machines/SVI - Spectravideo SVI-318/svi318.rom",32768,"bios"],["bios/Machines/SVI - Spectravideo SVI-328 Line truncated
{"files":[["bios/128p-0.rom",16384,"bios"],["bios/128p-1.rom",16384,"bios"],["bios/256s-0.rom",16384,"bios"],["bios/256s-1.rom",16384,"bios"],["bios/256s-2.rom",16384,"bios"],["bios/256s-3.rom",16384,"bios"],["bios/32X_G_BIOS.BIN",256,"bios"],["bios/32X_M_BIOS.BIN",2048,"bios"],["bios/32X_S_BIOS.BIN",1024,"bios"],["bios/3do_arcade_saot.bin",524288,"bios"],["bios/5200.rom",2048,"bios"],["bios/64DD_IPL.bin",4194304,"bios"],["bios/7800 BIOS (E).rom",16384,"bios"],["bios/7800 BIOS (U).rom",4096,"bios"],["bios/ATARIBAS.ROM",8192,"bios"],["bios/ATARIOSA.ROM",10240,"bios"],["bios/ATARIOSB.ROM",10240,"bios"],["bios/ATARIXL.ROM",16384,"bios"],["bios/BB01R4_OS.ROM",16384,"bios"],["bios/BIOS.col",8192,"bios"],["bios/BLOOD.RFF",3536782,"game_data"],["bios/BS-X.bin",1048576,"bios"],["bios/CD32 Extended-ROM r40.60 (1993)(Commodore)(CD32).rom",524288,"bios"],["bios/CDTV Extended-ROM v1.0 (1991)(Commodore)(CDTV)[!].rom",262144,"bios"],["bios/CM32LN_CONTROL.ROM",65536,"bios"],["bios/CM32L_CONTROL.ROM",65536,"bios"],["bios/CM32L_PCM.ROM",1048576,"bios"],["bios/DOSBOX.SF2",32319396,"bios"],["bios/DUKE3D.GRP",11035779,"game_data"],["bios/Databases/colecodb.xml",164848,"bios"],["bios/Databases/msxcaswavdb.xml",819679,"bios"],["bios/Databases/msxromdb.xml",930986,"bios"],["bios/Databases/msxsysromdb.xml",157542,"bios"],["bios/Databases/segadb.xml",82318,"bios"],["bios/Databases/svidb.xml",261070,"bios"],["bios/ECS.bin",24576,"bios"],["bios/GUI.RFF",7145,"game_data"],["bios/IVOICE.bin",2048,"bios"],["bios/JiffyDOS_1541-II.bin",16384,"bios"],["bios/JiffyDOS_1571_repl310654.bin",32768,"bios"],["bios/JiffyDOS_1581.bin",32768,"bios"],["bios/JiffyDOS_C128.bin",16384,"bios"],["bios/JiffyDOS_C64.bin",8192,"bios"],["bios/JiffyDOS_SX-64.bin",8192,"bios"],["bios/Kickstart v1.2 r33.180 (1986)(Commodore)(A500-A1000-A2000)[!].rom",262144,"bios"],["bios/Kickstart v1.2 rev 33.166 (1986)(Commodore)(A1000).rom",262144,"bios"],["bios/Kickstart v1.3 r34.5 (1987)(Commodore)(A500-A1000-A2000-CDTV)[!].rom",262144,"bios"],["bios/Kickstart v1.3 r34.5 (1987)(Commodore)(A500-A1000-A2000-CDTV)[o].rom",524288,"bios"],["bios/Kickstart v2.04 r37.175 (1991)(Commodore)(A500+)[!].rom",524288,"bios"],["bios/Kickstart v2.05 r37.350 (1992)(Commodore)(A600HD)[!].rom",524288,"bios"],["bios/Kickstart v3.1 r40.60 (1993)(Commodore)(CD32).rom",524288,"bios"],["bios/Kickstart v3.1 r40.68 (1993)(Commodore)(A1200)[!].rom",524288,"bios"],["bios/Kickstart v3.1 r40.68 (1993)(Commodore)(A3000).rom",524288,"bios"],["bios/Kickstart v3.1 r40.68 (1993)(Commodore)(A4000).rom",524288,"bios"],["bios/MT32_CONTROL.ROM",65536,"bios"],["bios/MT32_PCM.ROM",524288,"bios"],["bios/Machines/COL - Bit Corporation Dina/czz50-1.rom",8192,"bios"],["bios/Machines/COL - Bit Corporation Dina/czz50-2.rom",8192,"bios"],["bios/Machines/COL - ColecoVision with Opcode Memory Extension/coleco.rom",8192,"bios"],["bios/Machines/COL - ColecoVision/coleco.rom",8192,"bios"],["bios/Machines/COL - ColecoVision/config.ini",347,"bios"],["bios/Machines/COL - Spectravideo SVI-603 Coleco/SVI603.ROM",8192,"bios"],["bios/Machines/Forte II Games - Pesadelo/pesadelo.rom",65536,"bios"],["bios/Machines/MSX - C-BIOS/cbios.txt",8924,"bios"],["bios/Machines/MSX - C-BIOS/cbios_logo_msx1.rom",16384,"bios"],["bios/Machines/MSX - C-BIOS/cbios_main_msx1.rom",32768,"bios"],["bios/Machines/MSX/MSX.ROM",32768,"bios"],["bios/Machines/MSX/MSX2.ROM",32768,"bios"],["bios/Machines/MSX/MSX2EXT.ROM",16384,"bios"],["bios/Machines/MSX/MSX2P.ROM",32768,"bios"],["bios/Machines/MSX/MSX2PEXT.ROM",16384,"bios"],["bios/Machines/MSX/MSXDOS2.ROM",65536,"bios"],["bios/Machines/MSX/config.ini",408,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios.txt",8924,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios_logo_msx2.rom",16384,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios_main_msx2.rom",32768,"bios"],["bios/Machines/MSX2 - C-BIOS/cbios_sub.rom",16384,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios.txt",8924,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_logo_msx2+.rom",16384,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_main_msx2+.rom",32768,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_music.rom",16384,"bios"],["bios/Machines/MSX2+ - C-BIOS/cbios_sub.rom",16384,"bios"],["bios/Machines/MSX2+/MSX.ROM",32768,"bios"],["bios/Machines/MSX2+/MSX2.ROM",32768,"bios"],["bios/Machines/MSX2+/MSX2EXT.ROM",16384,"bios"],["bios/Machines/MSX2+/MSX2P.ROM",32768,"bios"],["bios/Machines/MSX2+/MSX2PEXT.ROM",16384,"bios"],["bios/Machines/MSX2+/MSXDOS2.ROM",65536,"bios"],["bios/Machines/MSX2+/config.ini",773,"bios"],["bios/Machines/MSX2/MSX.ROM",32768,"bios"],["bios/Machines/MSX2/MSX2.ROM",32768,"bios"],["bios/Machines/MSX2/MSX2EXT.ROM",16384,"bios"],["bios/Machines/MSX2/MSX2P.ROM",32768,"bios"],["bios/Machines/MSX2/MSX2PEXT.ROM",16384,"bios"],["bios/Machines/MSX2/MSXDOS2.ROM",65536,"bios"],["bios/Machines/MSX2/config.ini",649,"bios"],["bios/Machines/MSXturboR/config.ini",941,"bios"],["bios/Machines/SVI - Spectravideo SVI-318/svi318.rom",32768,"bios"],["bios/Machines/SVI - Spectravideo SVI-328 Line truncated
install/games/index.json
+1 -1
View File
@@ -1 +1 @@
{"arcade":{"fbneo":{"file":"arcade/fbneo.json","gzip_size":115955,"sha256":"88961c9a6be804772baa767f6ee3a9162478c30c60fe55c8363d8fe4fbc909da","size":423362},"flycast":{"file":"arcade/flycast.json","gzip_size":1841,"sha256":"957189a6e993fda08e21c90ffc6ad94aed9ca9e6dd9ed8c7724e29db036791cb","size":5441},"mame":{"file":"arcade/mame.json","gzip_size":154527,"sha256":"b12010301293a218ecfac45706d81a26c3ac0c27eebeca7409bc8f6ce0163e9e","size":604648},"mame0258":{"file":"arcade/mame0258.json","gzip_size":141151,"sha256":"7ecff1979ed2ac123ea3ba4825dde6e2c31906415165fabefc9af93fa48a6de4","size":556059},"mame0278":{"file":"arcade/mame0278.json","gzip_size":148595,"sha256":"7881e3516f451ec83e886aaf41ebac9d60f497e1f954f49749d52890d7955cbf","size":582797},"mame2000":{"file":"arcade/mame2000.json","gzip_size":8624,"sha256":"f0d69b0872ecfc2845eeb7304543e5b17c11db82060b5514c206b6962ed08263","size":23316},"mame2003":{"file":"arcade/mame2003.json","gzip_size":17456,"sha256":"3b44e2a92c80a8f104b7781fcc1c776d025ad2cf4986638efa3ea041baa9a56d","size":48929},"mame2003_plus":{"file":"arcade/mame2003_plus.json","gzip_size":21077,"sha256":"2857283556fa49c20d8160ed113465bf0c2e643ed0b8acd2d38266a78be8e25e","size":62373},"mame2010":{"file":"arcade/mame2010.json","gzip_size":30869,"sha256":"50b1b381b4b07463f69f1f4c512dd1c98a78d4d0e9c3aeaf49b7b1b007b42152","size":96528},"mame2015":{"file":"arcade/mame2015.json","gzip_size":86990,"sha256":"afeb8c32f651421f74c6de057b0c0b837acd14c533b5160bb31f4411e0afffec","size":360117},"mame2016":{"file":"arcade/mame2016.json","gzip_size":101249,"sha256":"1f75147977d9d082a38b8f8e47a02b301f3d6e6db97c9b9282f3e7f27f26ee47","size":406190}},"containers":{"7z":{"description":"Per-file CRCs and sizes sit in the header at the end of the archive, LZMA-compressed by default (kEncodedHeader). One LZMA decode of the header, independent of the archive size, gives them. Python's lzma is an optional module.","re_read":true,"signature":{"bytes_hex":"377abcaf271c","offset":0},"source_ref":"ip7z/7zip@b88c6bc0 DOC/7zFormat.txt:171-189, DOC/7zFormat.txt:318-334, CPP/7zip/Archive/7z/7zHandlerOut.cpp:859"},"cd-sector":{"description":"User data of a CD sector. In a 2048-byte image (.iso) sector N starts at N * 2048. In a raw 2352-byte image it starts at N * 2352 plus 16 for Mode 1 and plus 24 for Mode 2 Form 1 (12 sync bytes, a 4-byte header, and for Mode 2 an 8-byte subheader).","re_read":true,"sector_size":{"iso":2048,"raw":2352},"source_ref":"stenzek/duckstation@526b4a12 src/util/iso_reader.cpp:157-171, src/util/cd_image.h:28-34","user_data_offset":{"mode1":16,"mode2":24}},"chd-v5":{"description":"MAME compressed hunks, big-endian. The header holds sizes, codecs and hashes only, never the system or the region. The overall SHA-1 is what MAME software lists record. Neither SHA-1 equals a Redump track hash: chdman hashes 2448-byte frames (subcode included, zero-filled), pads each track to 4 frames and byte-swaps audio, all tracks in one stream.","fields":{"compressors":{"encoding":"fourcc4","length":16,"offset":16},"header_length":{"encoding":"u32be","length":4,"offset":8,"value":124},"hunk_bytes":{"encoding":"u32be","length":4,"offset":56},"logical_bytes":{"encoding":"u64be","length":8,"offset":32},"map_offset":{"encoding":"u64be","length":8,"offset":40},"metadata_offset":{"encoding":"u64be","length":8,"offset":48},"overall_sha1":{"encoding":"hex","length":20,"offset":84},"parent_sha1":{"encoding":"hex","length":20,"offset":104},"raw_sha1":{"encoding":"hex","length":20,"offset":64},"unit_bytes":{"encoding":"u32be","length":4,"offset":60},"version":{"encoding":"u32be","length":4,"offset":12,"value":5}},"magic":{"bytes":"MComprHD","offset":0},"media_tags":{"CHCD":"cd","CHGD":"gd-rom","CHGT":"gd-rom","CHT2":"cd","CHTR":"cd","DVD ":"dvd","GDDD":"hard-disk"},"metadata_entry":{"description":"16-byte entry header, payload follows. Linked list from metadata_offset.","flags":{"encoding":"u8","length":1,"offset":4},"length":{"encoding":"u24be","length":3,"offset":5},"next":{"encoding":"u64be","length":8,"offset":8},"tag":{"encoding":"ascii","length":4,"offset":0}},"re_read":true,"source_ref":"mamedev/mame@f34f0250 src/lib/util/chd.h:148-162, src/lib/util/chd.cpp:2021-2061, src/lib/util/chd.cpp:2822-2831, src/lib/util/chd.h:212-235, src/lib/util/chd.cpp:1709-1749, src/tools/chdman.cpp:443-445, src/tools/chdman.cpp:2195-2203, src/lib/util/cdrom.cpp:2669-2672, src/emu/romload.cpp:1211"},"gamecube-disc":{"description":"Where disc offset 0 sits inside each container Dolphin opens.","formats":{"ciso":{"disc_header_at":32768,"magic":"CISO"},"gcz":{"disc_header_at":null,"magic_u32le":2970337281,"note":"block 0 is zlib unless bit 63 of its pointer is set"},"iso":{"disc_header_at":0,"magic":null},"rvz":{"disc_header_at":88,"magic":"RVZ\u0001","note":"first 0x80 disc bytes copied into header 2"},"wbfs":{"disc_header_at":null,"magic":"WBFS","note":"disc offset 0 at wlba[0] << wbfs_sector_shift"},"wia":{"disc_header_at":88,"magic":"WIA\u0001","note":Line truncated
{"arcade":{"fbneo":{"file":"arcade/fbneo.json","gzip_size":115955,"sha256":"88961c9a6be804772baa767f6ee3a9162478c30c60fe55c8363d8fe4fbc909da","size":423362},"flycast":{"file":"arcade/flycast.json","gzip_size":1841,"sha256":"957189a6e993fda08e21c90ffc6ad94aed9ca9e6dd9ed8c7724e29db036791cb","size":5441},"mame":{"file":"arcade/mame.json","gzip_size":154527,"sha256":"b12010301293a218ecfac45706d81a26c3ac0c27eebeca7409bc8f6ce0163e9e","size":604648},"mame0258":{"file":"arcade/mame0258.json","gzip_size":141151,"sha256":"7ecff1979ed2ac123ea3ba4825dde6e2c31906415165fabefc9af93fa48a6de4","size":556059},"mame0278":{"file":"arcade/mame0278.json","gzip_size":148595,"sha256":"7881e3516f451ec83e886aaf41ebac9d60f497e1f954f49749d52890d7955cbf","size":582797},"mame2000":{"file":"arcade/mame2000.json","gzip_size":8624,"sha256":"f0d69b0872ecfc2845eeb7304543e5b17c11db82060b5514c206b6962ed08263","size":23316},"mame2003":{"file":"arcade/mame2003.json","gzip_size":17456,"sha256":"3b44e2a92c80a8f104b7781fcc1c776d025ad2cf4986638efa3ea041baa9a56d","size":48929},"mame2003_plus":{"file":"arcade/mame2003_plus.json","gzip_size":21077,"sha256":"2857283556fa49c20d8160ed113465bf0c2e643ed0b8acd2d38266a78be8e25e","size":62373},"mame2010":{"file":"arcade/mame2010.json","gzip_size":30869,"sha256":"50b1b381b4b07463f69f1f4c512dd1c98a78d4d0e9c3aeaf49b7b1b007b42152","size":96528},"mame2015":{"file":"arcade/mame2015.json","gzip_size":86990,"sha256":"afeb8c32f651421f74c6de057b0c0b837acd14c533b5160bb31f4411e0afffec","size":360117},"mame2016":{"file":"arcade/mame2016.json","gzip_size":101249,"sha256":"1f75147977d9d082a38b8f8e47a02b301f3d6e6db97c9b9282f3e7f27f26ee47","size":406190}},"containers":{"7z":{"description":"Per-file CRCs and sizes sit in the header at the end of the archive, LZMA-compressed by default (kEncodedHeader). One LZMA decode of the header, independent of the archive size, gives them. Python's lzma is an optional module.","re_read":true,"signature":{"bytes_hex":"377abcaf271c","offset":0},"source_ref":"ip7z/7zip@b88c6bc0 DOC/7zFormat.txt:171-189, DOC/7zFormat.txt:318-334, CPP/7zip/Archive/7z/7zHandlerOut.cpp:859"},"cd-sector":{"description":"User data of a CD sector. In a 2048-byte image (.iso) sector N starts at N * 2048. In a raw 2352-byte image it starts at N * 2352 plus 16 for Mode 1 and plus 24 for Mode 2 Form 1 (12 sync bytes, a 4-byte header, and for Mode 2 an 8-byte subheader).","re_read":true,"sector_size":{"iso":2048,"raw":2352},"source_ref":"stenzek/duckstation@526b4a12 src/util/iso_reader.cpp:157-171, src/util/cd_image.h:28-34","user_data_offset":{"mode1":16,"mode2":24}},"chd-v5":{"description":"MAME compressed hunks, big-endian. The header holds sizes, codecs and hashes only, never the system or the region. The overall SHA-1 is what MAME software lists record. Neither SHA-1 equals a Redump track hash: chdman hashes 2448-byte frames (subcode included, zero-filled), pads each track to 4 frames and byte-swaps audio, all tracks in one stream.","fields":{"compressors":{"encoding":"fourcc4","length":16,"offset":16},"header_length":{"encoding":"u32be","length":4,"offset":8,"value":124},"hunk_bytes":{"encoding":"u32be","length":4,"offset":56},"logical_bytes":{"encoding":"u64be","length":8,"offset":32},"map_offset":{"encoding":"u64be","length":8,"offset":40},"metadata_offset":{"encoding":"u64be","length":8,"offset":48},"overall_sha1":{"encoding":"hex","length":20,"offset":84},"parent_sha1":{"encoding":"hex","length":20,"offset":104},"raw_sha1":{"encoding":"hex","length":20,"offset":64},"unit_bytes":{"encoding":"u32be","length":4,"offset":60},"version":{"encoding":"u32be","length":4,"offset":12,"value":5}},"magic":{"bytes":"MComprHD","offset":0},"media_tags":{"CHCD":"cd","CHGD":"gd-rom","CHGT":"gd-rom","CHT2":"cd","CHTR":"cd","DVD ":"dvd","GDDD":"hard-disk"},"metadata_entry":{"description":"16-byte entry header, payload follows. Linked list from metadata_offset.","flags":{"encoding":"u8","length":1,"offset":4},"length":{"encoding":"u24be","length":3,"offset":5},"next":{"encoding":"u64be","length":8,"offset":8},"tag":{"encoding":"ascii","length":4,"offset":0}},"re_read":true,"source_ref":"mamedev/mame@f34f0250 src/lib/util/chd.h:148-162, src/lib/util/chd.cpp:2021-2061, src/lib/util/chd.cpp:2822-2831, src/lib/util/chd.h:212-235, src/lib/util/chd.cpp:1709-1749, src/tools/chdman.cpp:443-445, src/tools/chdman.cpp:2195-2203, src/lib/util/cdrom.cpp:2669-2672, src/emu/romload.cpp:1211"},"gamecube-disc":{"description":"Where disc offset 0 sits inside each container Dolphin opens.","formats":{"ciso":{"disc_header_at":32768,"magic":"CISO"},"gcz":{"disc_header_at":null,"magic_u32le":2970337281,"note":"block 0 is zlib unless bit 63 of its pointer is set"},"iso":{"disc_header_at":0,"magic":null},"rvz":{"disc_header_at":88,"magic":"RVZ\u0001","note":"first 0x80 disc bytes copied into header 2"},"wbfs":{"disc_header_at":null,"magic":"WBFS","note":"disc offset 0 at wlba[0] << wbfs_sector_shift"},"wia":{"disc_header_at":88,"magic":"WIA\u0001","note":Line truncated
schemas/database.schema.json
+3
View File
@@ -114,6 +114,9 @@
},
"by_path_suffix": {
"$ref": "#/$defs/listIndex"
},
"by_shadowed_alias": {
"$ref": "#/$defs/listIndex"
}
}
}
scripts/common.py
+104
View File
@@ -90,6 +90,71 @@ def _casefold_name_index(by_name: dict) -> dict[str, list[str]]:
return folded
# bios/Other/<emulator> and bios/Arcade/<emulator or board> file a copy by
# who loads it, not by machine.
BUCKET_FOLDERS = frozenset({"Other", "Arcade"})
def names_manufacturer(sys_id: str, manufacturer: str) -> bool:
"""Whether a system id names a maker folder: nec-pc-88 names NEC."""
slug = manufacturer.lower().replace(" ", "-")
return sys_id == slug or sys_id.startswith(f"{slug}-")
def _maker_folder(path: str, bios_dir: str) -> str | None:
"""The bios/<maker>/ folder a collection path is filed under."""
prefix = f"{bios_dir.rstrip('/')}/"
if not path.startswith(prefix):
return None
parts = path[len(prefix):].split("/")
return parts[0] if len(parts) >= 2 else None
# One slot, keyed like the casefold view: the files dict, its size, the folders.
_maker_folders_cache: list[tuple[dict, int, frozenset[str]]] = []
def _maker_folders(db: dict) -> frozenset[str]:
"""Every maker folder the collection files a copy under."""
files_db = db.get("files", {})
if _maker_folders_cache:
source, size, folders = _maker_folders_cache[0]
if source is files_db and size == len(files_db):
return folders
bios_dir = str(db.get("bios_dir") or "bios")
folders = frozenset(
folder
for entry in files_db.values()
if (folder := _maker_folder(str(entry.get("path", "")), bios_dir))
and folder not in BUCKET_FOLDERS
)
_maker_folders_cache[:] = [(files_db, len(files_db), folders)]
return folders
def system_makers(file_entry: dict, db: dict) -> frozenset[str]:
"""The maker folders a platform entry's system names.
Only a platform's own entry is asked: one that names an owning emulator
is placed by that emulator's tree. Empty when the system names no maker
(pc88, arcade), so nothing is narrowed there.
"""
sys_id = str(file_entry.get("platform_system") or "")
if not sys_id or _owner_tokens(file_entry):
return frozenset()
return frozenset(
folder for folder in _maker_folders(db) if names_manufacturer(sys_id, folder)
)
def other_maker(path: str, makers: frozenset[str], db: dict) -> bool:
"""Whether a path is filed under a maker the entry's system does not name."""
if not makers:
return False
folder = _maker_folder(path, str(db.get("bios_dir") or "bios"))
return folder is not None and folder not in BUCKET_FOLDERS and folder not in makers
def name_match_size_ok(file_entry: dict, candidate_size: int | None) -> bool:
"""Whether a candidate found by name may be the file the entry describes.
@@ -357,6 +422,19 @@ def load_platform_config(platform_name: str, platforms_dir: str = "platforms") -
for sys_id, system in config.get("systems", {}).items()
}
# Each entry carries the system that declares it: a hashless name match is
# narrowed to the maker folder that system names (resolve_local_file).
# Fresh dicts, since a YAML anchor can share one entry between systems.
config["systems"] = {
sys_id: {
**system,
"files": [{**f, "platform_system": sys_id} for f in system["files"]],
}
if isinstance(system, dict) and system.get("files")
else system
for sys_id, system in config.get("systems", {}).items()
}
# The registry key, which per-platform profile fields (installed_on,
# not_bundled_on) name: the display name differs (MiSTer FPGA).
config["_key"] = platform_name
@@ -1573,6 +1651,10 @@ def resolve_local_file(
# hash and path steps above run first, so collecting the real bytes makes
# the entry resolve.
unsourceable = bool(file_entry.get("unsourceable")) or own_file_absent
# A platform's own hashless entry whose system names a maker takes no copy
# filed under another maker: AmberELEC's nec-pc-88 DISK.ROM, which QUASI88
# reads as the 2 KB PC-88 disk ROM, got the 16 KB MSX one by name.
makers = frozenset() if has_strong_hash else system_makers(file_entry, db)
if not has_strong_hash and not unsourceable:
candidates = []
@@ -1637,6 +1719,26 @@ def resolve_local_file(
continue
if os.path.exists(path) and path not in candidates:
candidates.append(path)
if makers and candidates:
candidates = [p for p in candidates if not other_maker(p, makers, db)]
if not candidates:
# The name belongs to another maker's file here. An emulator
# of this system may still read it as an alias of one of its
# own files, an alias by_name leaves out because a primary
# already carries the name: QUASI88 reads DISK.ROM as
# n88sub.rom.
shadowed = db.get("indexes", {}).get("by_shadowed_alias", {})
for try_name in own_names:
for match_sha1 in shadowed.get(try_name, []):
path = files_db.get(match_sha1, {}).get("path", "")
if (
_maker_folder(path, str(db.get("bios_dir") or "bios"))
in makers
and _size_ok(match_sha1)
and os.path.exists(path)
and path not in candidates
):
candidates.append(path)
if candidates:
if zipped_file:
candidates = [p for p in candidates if ".zip" in os.path.basename(p)]
@@ -1701,6 +1803,8 @@ def resolve_local_file(
if zipped_file:
candidates = [c for c in candidates if ".zip" in os.path.basename(c[0])]
# Another maker's file is no mismatch of this one, it is another file.
candidates = [c for c in candidates if not other_maker(c[0], makers, db)]
# A size the emulator itself verifies settles a name collision, so a
# candidate that fits is preferred over one that does not: "ROM" answers
# both an Apple IIgs image and a Macintosh one. It ranks rather than
scripts/generate_db.py
+39 -2
View File
@@ -294,6 +294,28 @@ def build_indexes(files: dict, aliases: dict) -> dict:
}
def shadowed_alias_index(
shadowed: list[tuple[str, set[str], list[str]]], by_name: dict
) -> dict[str, list[str]]:
"""Profile aliases that a primary's name keeps out of by_name.
QUASI88 reads DISK.ROM as n88sub.rom, but DISK.ROM is the MSX disk ROM's
own name, so by_name never points it at the PC-88 file. The resolver reads
this index only where the system names a maker and no copy filed under
that maker answers the name. An entry with no hash designates the content
its own name holds alone in by_name.
"""
index: dict[str, set[str]] = {}
for entry_name, matched, taken in shadowed:
held = by_name.get(entry_name, [])
designated = matched or (set(held) if len(held) == 1 else set())
for alias in taken:
fresh = designated - set(by_name.get(alias, []))
if fresh:
index.setdefault(alias, set()).update(fresh)
return {alias: sorted(sha1s) for alias, sha1s in sorted(index.items())}
def load_cache(cache_path: str) -> dict:
"""Load cache file if it exists."""
try:
@@ -429,8 +451,9 @@ def main() -> int:
if preserved:
print(f" Preserved {preserved} large file entries from existing database")
shadowed: list[tuple[str, set[str], list[str]]] = []
try:
platform_aliases = _collect_all_aliases(files, aliases)
platform_aliases = _collect_all_aliases(files, aliases, shadowed)
except UnreadableSource as exc:
for source in exc.sources:
print(f"Error: cannot read {source}", file=sys.stderr)
@@ -446,6 +469,7 @@ def main() -> int:
provenance_counts = annotate_provenance(files, snapshots)
indexes = build_indexes(files, aliases)
indexes["by_shadowed_alias"] = shadowed_alias_index(shadowed, indexes["by_name"])
total_size = sum(entry["size"] for entry in files.values())
database = {
@@ -508,11 +532,17 @@ class UnreadableSource(Exception):
self.sources = sources
def _collect_all_aliases(files: dict, copies: dict | None = None) -> dict:
def _collect_all_aliases(
files: dict,
copies: dict | None = None,
shadowed: list[tuple[str, set[str], list[str]]] | None = None,
) -> dict:
"""Collect alternate filenames from platform YAMLs, core-info, and known aliases.
Registers alternate names so generate_pack can resolve files stored under different names.
``copies`` maps a sha1 to the other files the scan found with that content.
``shadowed`` receives, per profile entry, its name, the contents its hashes
designate and the aliases a primary's name already carries.
"""
md5_to_sha1 = {}
name_to_sha1 = {}
@@ -635,6 +665,13 @@ def _collect_all_aliases(files: dict, copies: dict | None = None) -> dict:
entry_aliases.insert(0, entry_name)
if not entry_aliases:
continue
if shadowed is not None:
taken = [
alias for alias in entry_aliases
if alias != entry_name and alias in named_primaries
]
if taken:
shadowed.append((entry_name, set(matched), taken))
if not matched and len(holders.get(entry_name, ())) == 1:
# A name carried by several contents names none of them:
# quasi88's disk.rom aliases went to whichever
scripts/generate_site.py
+5 -14
View File
@@ -40,6 +40,8 @@ from typing import TypeVar
sys.path.insert(0, os.path.dirname(__file__))
from common import (
BUCKET_FOLDERS,
names_manufacturer,
upstream_profile_index,
build_zip_contents_index,
GAME_DATA_TOPS,
@@ -219,21 +221,10 @@ def _copy_places(db: dict) -> dict[str, set[tuple[str, str, str]]]:
return places
# bios/Other/<emulator> and bios/Arcade/<emulator or board> file a copy by
# who loads it, not by machine. A copy also filed under a machine's maker
# places the system there.
_BUCKET_FOLDERS = frozenset({"Other", "Arcade"})
def _slug(manufacturer: str) -> str:
return manufacturer.lower().replace(" ", "-")
def _names_manufacturer(sys_id: str, manufacturer: str) -> bool:
slug = _slug(manufacturer)
return sys_id == slug or sys_id.startswith(f"{slug}-")
def _copies_by_path(db: dict) -> dict[str, set[tuple[str, str, str]]]:
"""The copy places of each content, keyed by the path the database keeps."""
files = db.get("files", {})
@@ -276,13 +267,13 @@ def _system_folder(
declared = str(fe.get("name", "")).casefold()
named = {
(m, c) for m, c, base in held
if base.casefold() == declared or _names_manufacturer(sys_id, m)
if base.casefold() == declared or names_manufacturer(sys_id, m)
}
for place in named or {(m, c) for m, c, _ in held}:
votes[place] = votes.get(place, 0) + 1
if votes:
own = [place for place in votes if _names_manufacturer(sys_id, place[0])]
makers = [place for place in votes if place[0] not in _BUCKET_FOLDERS]
own = [place for place in votes if names_manufacturer(sys_id, place[0])]
makers = [place for place in votes if place[0] not in BUCKET_FOLDERS]
candidates = own or makers or list(votes)
return max(sorted(candidates), key=lambda place: votes[place])
for fe in files:
tests/test_maker_folder_names.py
+274
View File
@@ -0,0 +1,274 @@
"""A hashless platform entry takes no file filed under another maker.
AmberELEC's nec-pc-88 declares DISK.ROM with an empty md5. QUASI88 reads that
name as n88sub.rom, the 2 KB PC-88 disk ROM (libretro.c:1085), but the name
step served the 16 KB MSX disk ROM, the only primary called DISK.ROM. The
quasi88 alias could not point the name at the PC-88 file, since by_name
refuses an alias a primary already carries. by_shadowed_alias keeps it, and
the resolver reads it only once the system's own maker folder is all that
remains.
"""
from __future__ import annotations
import hashlib
import json
import os
import subprocess
import sys
import tempfile
import unittest
import zipfile
from pathlib import Path
import yaml
REPO_ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO_ROOT / "scripts"))
import common # noqa: E402
import generate_db # noqa: E402
import generate_pack as builder # noqa: E402
import generate_site # noqa: E402
from verify import verify_platform # noqa: E402
MSX_DISK = b"M" * 16384
PC88_DISK = b"N" * 2048
QUASI88 = """\
emulator: QUASI88
type: libretro
systems: [nec-pc-88]
cores: [quasi88]
files:
- name: n88sub.rom
path: quasi88/n88sub.rom
required: false
aliases: [N88SUB.ROM, DISK.ROM]
"""
PLATFORM = {
"platform": "Handheld",
"version": "1",
"verification_mode": "md5",
"base_destination": "",
"cores": ["quasi88"],
"systems": {
"nec-pc-88": {
"files": [
{"name": "DISK.ROM", "destination": "bios/quasi88/DISK.ROM",
"required": True},
],
},
"microsoft-msx": {
"files": [
{"name": "DISK.ROM", "destination": "bios/Machines/DISK.ROM",
"required": True},
],
},
},
}
class _Collection(unittest.TestCase):
def setUp(self):
(REPO_ROOT / "tmp").mkdir(exist_ok=True)
self._tmp = tempfile.TemporaryDirectory(dir=REPO_ROOT / "tmp")
self._cwd = os.getcwd()
os.chdir(self._tmp.name)
for relative, payload in (
("bios/Microsoft/MSX/DISK.ROM", MSX_DISK),
("bios/NEC/PC-88/n88sub.rom", PC88_DISK),
):
Path(relative).parent.mkdir(parents=True, exist_ok=True)
Path(relative).write_bytes(payload)
Path("emulators").mkdir()
Path("platforms").mkdir()
Path("emulators/quasi88.yml").write_text(QUASI88)
Path("platforms/handheld.yml").write_text(yaml.dump(PLATFORM))
Path("platforms/_registry.yml").write_text(
yaml.dump({"platforms": {"handheld": {"status": "active"}}})
)
subprocess.run(
[sys.executable, str(REPO_ROOT / "scripts/generate_db.py"), "--force",
"--bios-dir", "bios", "--output", "database.json"],
check=True, capture_output=True,
)
self.db = common.load_database("database.json")
common._platform_config_cache.clear()
common._emulator_profiles_cache.clear()
self.config = common.load_platform_config("handheld", "platforms")
self.profiles = common.load_emulator_profiles("emulators")
def tearDown(self):
common._platform_config_cache.clear()
common._emulator_profiles_cache.clear()
os.chdir(self._cwd)
self._tmp.cleanup()
def _entry(self, system: str) -> dict:
return self.config["systems"][system]["files"][0]
def _manifest(self) -> dict:
return builder.generate_manifest(
"handheld", "platforms", self.db, "bios", "platforms/_registry.yml",
emulators_dir="emulators", emu_profiles=self.profiles, offline=True,
)
class ShadowedAliasIndex(_Collection):
def test_the_alias_a_primary_carries_is_kept_apart(self):
sha1 = hashlib.sha1(PC88_DISK).hexdigest()
indexes = self.db["indexes"]
self.assertEqual(indexes["by_shadowed_alias"]["DISK.ROM"], [sha1])
self.assertNotIn(sha1, indexes["by_name"]["DISK.ROM"])
def test_an_entry_with_no_single_content_designates_nothing(self):
shadowed = [("a.rom", set(), ["B.ROM"]), ("c.rom", {"1" * 40}, ["B.ROM"])]
by_name = {"a.rom": ["2" * 40, "3" * 40], "B.ROM": ["4" * 40]}
self.assertEqual(
generate_db.shadowed_alias_index(shadowed, by_name), {"B.ROM": ["1" * 40]}
)
class DiskRomGoesToThePc88File(_Collection):
def test_the_entry_knows_its_system(self):
self.assertEqual(self._entry("nec-pc-88")["platform_system"], "nec-pc-88")
def test_resolution(self):
entry = self._entry("nec-pc-88")
path, status = common.resolve_local_file(
entry, self.db, dest_hint=entry["destination"]
)
self.assertEqual((path, status), ("bios/NEC/PC-88/n88sub.rom", "name_exact"))
def test_the_msx_system_keeps_the_msx_file(self):
entry = self._entry("microsoft-msx")
path, _status = common.resolve_local_file(
entry, self.db, dest_hint=entry["destination"]
)
self.assertEqual(path, "bios/Microsoft/MSX/DISK.ROM")
def test_a_declared_hash_still_decides(self):
entry = {**self._entry("nec-pc-88"), "md5": hashlib.md5(MSX_DISK).hexdigest()}
path, status = common.resolve_local_file(entry, self.db)
self.assertEqual((path, status), ("bios/Microsoft/MSX/DISK.ROM", "md5_exact"))
def test_an_emulator_entry_is_not_narrowed(self):
entry = {**self._entry("nec-pc-88"), "source_profile": "msx-emu"}
path, _status = common.resolve_local_file(entry, self.db)
self.assertEqual(path, "bios/Microsoft/MSX/DISK.ROM")
def test_no_pc88_copy_leaves_it_missing_not_mismatched(self):
Path("bios/NEC/PC-88/n88sub.rom").unlink()
entry = self._entry("nec-pc-88")
self.assertEqual(
common.resolve_local_file(entry, self.db, dest_hint=entry["destination"]),
(None, "not_found"),
)
def test_verify(self):
report = verify_platform(self.config, self.db, "emulators", self.profiles)
paths = {d["name"]: set() for d in report["details"]}
for d in report["details"]:
paths[d["name"]].add(d.get("path"))
self.assertEqual(
paths["DISK.ROM"],
{"bios/NEC/PC-88/n88sub.rom", "bios/Microsoft/MSX/DISK.ROM"},
)
def test_pack(self):
zip_path = builder.generate_pack(
"handheld", "platforms", self.db, "bios", "out",
emulators_dir="emulators", emu_profiles=self.profiles, offline=True,
)
with zipfile.ZipFile(zip_path) as zf:
self.assertEqual(zf.read("bios/quasi88/DISK.ROM"), PC88_DISK)
self.assertEqual(zf.read("bios/Machines/DISK.ROM"), MSX_DISK)
def test_the_manifest_changes_only_that_row(self):
manifest = self._manifest()
original = (common.load_platform_config, common.system_makers)
def untagged(*args, **kwargs):
config = original[0](*args, **kwargs)
for system in config["systems"].values():
for f in system.get("files") or []:
f.pop("platform_system", None)
return config
common._platform_config_cache.clear()
builder.load_platform_config = common.load_platform_config = untagged
common.system_makers = lambda *args: frozenset()
try:
before = self._manifest()
finally:
builder.load_platform_config = common.load_platform_config = original[0]
common.system_makers = original[1]
common._platform_config_cache.clear()
rows = {f["dest"]: f for f in manifest["files"]}
old_rows = {f["dest"]: f for f in before["files"]}
self.assertEqual(rows["bios/quasi88/DISK.ROM"]["size"], len(PC88_DISK))
self.assertEqual(old_rows["bios/quasi88/DISK.ROM"]["size"], len(MSX_DISK))
del rows["bios/quasi88/DISK.ROM"], old_rows["bios/quasi88/DISK.ROM"]
self.assertEqual(rows, old_rows)
moved = len(MSX_DISK) - len(PC88_DISK)
for key in ("total_size", "pack_size"):
self.assertEqual(before[key] - manifest[key], moved)
changed = {"files", "total_size", "pack_size", "generated"}
self.assertEqual(
json.dumps({k: v for k, v in manifest.items() if k not in changed}, sort_keys=True),
json.dumps({k: v for k, v in before.items() if k not in changed}, sort_keys=True),
)
class SiteSharesTheMakerTest(unittest.TestCase):
def test_one_copy(self):
self.assertIs(generate_site.names_manufacturer, common.names_manufacturer)
self.assertIs(generate_site.BUCKET_FOLDERS, common.BUCKET_FOLDERS)
class NoHashlessNameMatchCrossesMakers(unittest.TestCase):
"""Every registered platform: a name match never lands on another maker."""
def test_registry(self):
self.addCleanup(os.chdir, os.getcwd())
os.chdir(REPO_ROOT)
db_path = REPO_ROOT / "database.json"
if not db_path.exists():
self.skipTest("database.json not built")
db = common.load_database(str(db_path))
registry = common.load_data_dir_registry("platforms")
folders = {
parts[1]
for entry in db["files"].values()
if len(parts := entry["path"].split("/")) >= 3 and parts[0] == "bios"
} - common.BUCKET_FOLDERS
crossings = []
for platform in common.list_registered_platforms("platforms", include_archived=True):
config = common.load_platform_config(platform, "platforms")
for sys_id, system in config.get("systems", {}).items():
makers = {f for f in folders if common.names_manufacturer(sys_id, f)}
if not makers:
continue
for entry in system.get("files") or []:
if any(entry.get(k) for k in ("sha1", "md5", "sha256", "crc32")):
continue
path, status = common.resolve_local_file(
entry, db, dest_hint=entry.get("destination", ""),
data_dir_registry=registry,
)
parts = (path or "").split("/")
if (
status.startswith("name")
and len(parts) >= 3
and parts[0] == "bios"
and parts[1] not in common.BUCKET_FOLDERS
and parts[1] not in makers
):
crossings.append(f"{platform} {sys_id} {entry.get('name')}: {path}")
self.assertEqual(crossings, [])
if __name__ == "__main__":
unittest.main()
wiki/architecture.md
+8 -1
View File
@@ -99,7 +99,8 @@ Upstream sources Scrapers parse generate_db.py scans
batocera-systems builds database.json
es_bios.xml (recalbox) (SHA1 primary key,
core-info .info files indexes: by_md5, by_name,
FirmwareDatabase.cs by_crc32, by_sha256, by_path_suffix)
FirmwareDatabase.cs by_crc32, by_sha256, by_path_suffix,
by_shadowed_alias)
bios_db.json.zip (MiSTer)
MAME/FBNeo source
@@ -412,6 +413,12 @@ resolution.
before a path or a name, so a path never wins over a hash. The steps, in order,
are listed under [file resolution chain](verification-modes.md#file-resolution-chain).
A platform entry with no hash is never answered by name with a file filed under
another maker than the one its system id names. When that leaves nothing, the
profile aliases a primary's name keeps out of `by_name` (`by_shadowed_alias`)
answer, within that maker's folder: AmberELEC's `nec-pc-88` `DISK.ROM` is QUASI88's
`n88sub.rom`, not the MSX disk ROM.
## Platform inheritance
Platform configs support `inherits:` to share definitions.
wiki/verification-modes.md
+11
View File
@@ -387,6 +387,17 @@ Two kinds of entry skip every answer given by a name alone: steps 6 and 9, the
The content steps still apply to both, so collecting the real bytes, or restoring
the asset, makes the entry resolve.
A platform's own entry with no hash, whose system id names a maker (`nec-pc-88`
names `NEC`), takes no file filed under another maker's folder in steps 6 and 7.
`bios/Other/` and `bios/Arcade/` file copies by emulator and stay open to it.
`load_platform_config` writes the system on each entry as `platform_system`. When
no copy is left, step 6 reads `by_shadowed_alias`, the profile aliases that
`by_name` refuses because a primary already carries the name, and keeps only the
files under the system's maker. AmberELEC's `DISK.ROM` got the 16 KB MSX disk ROM
by name. QUASI88 reads that name as `n88sub.rom`, the 2 KB PC-88 disk ROM, and
that is now the file served. An entry that names its emulator is placed by that
emulator's tree instead.
The `hash_mismatch` status means a file with the right name or path exists but its hash
does not match. This still resolves to a local path (the file is present), but verification
will report it as UNTESTED with a reason string showing the expected vs actual hash prefix.