fix: let a deep path answer for the file it names

README.md

@@ -63,12 +63,12 @@ These are the files an emulator loads from disk instead of carrying inside itsel
 
 Each file is checked the way your platform checks it. Most compare a checksum, the fingerprint of a file's contents, which catches a corrupt or unexpected copy. RetroArch, Lakka and RetroPie only look for the filename, because that is all their code does: the Coverage table says which applies to you. Independently of that, the collection records five fingerprints per file, and wherever an emulator's code states an expected size or hash, that value is read from its source and rechecked here.
 
-- **3 files** the platforms' emulators load are still to be found, and 15 more cannot be sourced at all (per-user keys, user-filled slots, dumps nobody has made); both are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/)
+- **6 files** the platforms' emulators load are still to be found, and 15 more cannot be sourced at all (per-user keys, user-filled slots, dumps nobody has made); both are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/)
 - **12 platforms** supported with platform-specific verification
 - **449 emulators** profiled from source (RetroArch cores + standalone)
 - **470 systems** handled by those emulators (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
-- **10,332 files**, each with its SHA1, MD5, SHA256, CRC32 and Adler-32 fingerprints: 4,984 system files, 2,809 arcade ROM sets, 2,539 game and engine data files
-- **553 of 4,984 system files** matched to dump-preservation catalogs (No-Intro, Redump, TOSEC); arcade sets and engine data fall outside what those catalogs index
+- **10,330 files**, each with its SHA1, MD5, SHA256, CRC32 and Adler-32 fingerprints: 4,983 system files, 2,808 arcade ROM sets, 2,539 game and engine data files
+- **553 of 4,983 system files** matched to dump-preservation catalogs (No-Intro, Redump, TOSEC); arcade sets and engine data fall outside what those catalogs index
 - **12706 MB** total collection size
 
 ## Supported systems
@@ -81,17 +81,17 @@ Full list with per-file details: **[https://abdess.github.io/retrobios/](https:/
 
 | Platform | On its BIOS list | Files its emulators load | Checked by |
 |----------|-----------------:|-------------------------:|------------|
-| Batocera | 353/353 | 1,302/1,316 | MD5 hash |
-| BizHawk | 118/118 | 371/375 | SHA1 hash |
+| Batocera | 353/353 | 1,299/1,316 | MD5 hash |
+| BizHawk | 118/118 | 370/375 | SHA1 hash |
 | EmuDeck | 161/161 | 427/427 | MD5 hash |
-| Lakka | 530/530 | 1,209/1,224 | file presence |
+| Lakka | 530/530 | 1,206/1,224 | file presence |
 | MiSTer FPGA | 72/72 | - | MD5 hash |
-| ROCKNIX | 38/38 | 1,639/1,654 | MD5 hash |
-| Recalbox | 346/346 | 831/843 | MD5 hash |
-| RetroArch | 530/530 | 1,209/1,224 | file presence |
-| RetroBat | 343/343 | 940/953 | MD5 hash |
-| RetroDECK | 2,008/2,008 | 1,253/1,269 | MD5 hash |
-| RetroPie * | 530/530 | 1,209/1,224 | file presence |
+| ROCKNIX | 38/38 | 1,636/1,654 | MD5 hash |
+| Recalbox | 346/346 | 828/843 | MD5 hash |
+| RetroArch | 530/530 | 1,206/1,224 | file presence |
+| RetroBat | 343/343 | 938/953 | MD5 hash |
+| RetroDECK | 2,008/2,008 | 1,250/1,269 | MD5 hash |
+| RetroPie * | 530/530 | 1,206/1,224 | file presence |
 | RomM | 381/381 | 284/293 | MD5 hash |
 
 Each fraction is what the pack has over what is needed, counting required and optional files alike since both ship. The first column is the BIOS list the platform publishes. The second counts files its emulators load that this list never mentions, found by reading their source code, and it is routinely several times larger. A short fraction means files are still missing, and they are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/).
@@ -174,4 +174,4 @@ The scripts and tooling are released under the [MIT License](LICENSE).
 The BIOS and firmware files are not covered by that license: they are third-party system software, preserved and provided for personal backup, archival, and interoperability with emulation software. [NOTICE](NOTICE) sets out their status and how to ask for a file to be removed.
 The reasoning, and where it is weakest, is in the [FAQ](https://abdess.github.io/retrobios/wiki/faq/#is-this-legal).
 
-*Auto-generated on 2026-09-05T05:01:56Z*
+*Auto-generated on 2026-09-05T07:33:35Z*

scripts/common.py

@@ -637,9 +637,33 @@ def resolve_local_file(
         # layout, so the two meet on a tail rather than on the whole string.
         # Longest tail first, and never down to the bare filename: that is the
         # weaker step below, and five files answer to GameIndex.yaml.
+        # Two kinds of ambiguity appear once every tail is indexed. A
+        # shortened tail can be claimed by several emulators shipping the same
+        # asset tree, and it then proves nothing: skip it. The full tail can be
+        # claimed twice as well, one storing it at the root of its own
+        # directory and one nested deeper; there the file whose path carries
+        # the least on top of the destination is the one that IS the
+        # destination.
+        # The descent drops at most one leading segment. A destination and the
+        # tree differ by the wrapper directory at most; beyond that the tail
+        # stops describing this file and starts describing whichever emulator
+        # happens to store the same asset tree.
         hint_parts = dest_hint.split("/")
-        for start in range(len(hint_parts) - 1):
-            for match_sha1 in by_path_suffix.get("/".join(hint_parts[start:]), []):
+        for start in range(min(2, len(hint_parts) - 1)):
+            tail = "/".join(hint_parts[start:])
+            candidates = by_path_suffix.get(tail, [])
+            if start and len(candidates) > 1:
+                continue
+            if len(candidates) > 1:
+                depth = len(tail.split("/"))
+                candidates = sorted(
+                    candidates,
+                    key=lambda h: len(
+                        files_db.get(h, {}).get("path", "").split("/")
+                    )
+                    - depth,
+                )
+            for match_sha1 in candidates:
                 if match_sha1 not in files_db:
                     continue
                 path = files_db[match_sha1]["path"]
@@ -704,15 +728,24 @@ def resolve_local_file(
                 path = entry["path"]
                 if os.path.exists(path) and path not in seen_paths:
                     seen_paths.add(path)
-                    candidates.append((path, entry.get("md5", "")))
+                    candidates.append(
+                        (path, entry.get("md5", ""), entry.get("size"))
+                    )
+
+    if zipped_file:
+        candidates = [c for c in candidates if ".zip" in os.path.basename(c[0])]
+    # A size the emulator itself verifies settles a name collision, so a
+    # candidate that fits is preferred over one that does not: "ROM" answers
+    # both an Apple IIgs image and a Macintosh one. It ranks rather than
+    # excludes, because a present file of the wrong size is reported as
+    # present and untested, and dropping it would report it as absent.
+    sized = [c for c in candidates if name_match_size_ok(file_entry, c[2])]
+    if sized:
+        candidates = sized
 
     if candidates:
-        if zipped_file:
-            candidates = [
-                (p, m) for p, m in candidates if ".zip" in os.path.basename(p)
-            ]
         if md5_set and not (sha1_candidates or sha256_candidates or crc_raw):
-            for path, db_md5 in candidates:
+            for path, db_md5, _size in candidates:
                 if ".zip" in os.path.basename(path):
                     try:
                         composite = md5_composite(path).lower()
@@ -725,20 +758,20 @@ def resolve_local_file(
         # When zipped_file is set, only accept candidates that contain it
         if zipped_file:
             valid = []
-            for path, m in candidates:
+            for path, m, size in candidates:
                 try:
                     with zipfile.ZipFile(path) as zf:
                         inner_names = {n.casefold() for n in zf.namelist()}
                         if zipped_file.casefold() in inner_names:
-                            valid.append((path, m))
+                            valid.append((path, m, size))
                 except (zipfile.BadZipFile, OSError):
                     pass
             if valid:
-                primary = [p for p, _ in valid if "/.variants/" not in p]
+                primary = [c[0] for c in valid if "/.variants/" not in c[0]]
                 return (primary[0] if primary else valid[0][0]), "hash_mismatch"
             # No candidate contains the zipped_file -fall through to step 5
         elif not unsourceable:
-            primary = [p for p, _ in candidates if "/.variants/" not in p]
+            primary = [c[0] for c in candidates if "/.variants/" not in c[0]]
             return (primary[0] if primary else candidates[0][0]), "hash_mismatch"
 
     # 5. zipped_file content match via pre-built index (last resort:

scripts/generate_db.py

@@ -165,17 +165,24 @@ def scan_bios_dir(bios_dir: Path, cache: dict, force: bool) -> tuple[dict, dict,
     return files, aliases, new_cache
 
 
-def _path_suffix(rel_path: str) -> str:
-    """Extract the path suffix after bios/Manufacturer/Console/.
+def _path_suffixes(rel_path: str) -> list[str]:
+    """Every tail of a stored path that can answer a declared destination.
 
-    bios/Nintendo/GameCube/GC/USA/IPL.bin -> GC/USA/IPL.bin
-    bios/Sony/PlayStation/scph5501.bin -> scph5501.bin
+    A destination and the tree meet on a tail, and which tail is unknowable
+    from here: one profile writes GC/USA/IPL.bin, another writes the whole
+    nand/.../00000000.app.romfs. Indexing only the tail after
+    bios/Manufacturer/Console answered the first and never the second, so
+    every same-named file collapsed onto one entry.
+
+    The bare filename is left out on purpose. by_name already holds it, and a
+    name alone is the weakest evidence there is.
+
+    bios/Nintendo/GameCube/GC/USA/IPL.bin -> GC/USA/IPL.bin, USA/IPL.bin
     """
-    parts = rel_path.replace("\\", "/").split("/")
-    # Skip: bios / Manufacturer / Console (3 segments)
-    if len(parts) > 3 and parts[0] == "bios":
-        return "/".join(parts[3:])
-    return parts[-1]
+    parts = [p for p in rel_path.replace("\\", "/").split("/") if p]
+    if parts and parts[0] == "bios":
+        parts = parts[1:]
+    return ["/".join(parts[-depth:]) for depth in range(len(parts), 1, -1)]
 
 
 def build_indexes(files: dict, aliases: dict) -> dict:
@@ -198,12 +205,13 @@ def build_indexes(files: dict, aliases: dict) -> dict:
         by_sha256[entry["sha256"]] = sha1
 
         # Path suffix index for regional variant resolution
-        suffix = _path_suffix(entry["path"])
-        if suffix != name:
-            # Only index when suffix adds info beyond the filename
+        for suffix in _path_suffixes(entry["path"]):
+            if suffix == name:
+                continue
             if suffix not in by_path_suffix:
                 by_path_suffix[suffix] = []
-            by_path_suffix[suffix].append(sha1)
+            if sha1 not in by_path_suffix[suffix]:
+                by_path_suffix[suffix].append(sha1)
 
     # Add alias names to by_name index (aliases have different filenames for same SHA1)
     for sha1, alias_list in aliases.items():
@@ -214,8 +222,9 @@ def build_indexes(files: dict, aliases: dict) -> dict:
             if sha1 not in by_name[name]:
                 by_name[name].append(sha1)
             # Also index alias paths in by_path_suffix
-            suffix = _path_suffix(alias["path"])
-            if suffix != name:
+            for suffix in _path_suffixes(alias["path"]):
+                if suffix == name:
+                    continue
                 if suffix not in by_path_suffix:
                     by_path_suffix[suffix] = []
                 if sha1 not in by_path_suffix[suffix]:

tests/test_path_suffix.py

@@ -0,0 +1,186 @@
+"""Which tails of a stored path can answer a declared destination."""
+
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
+
+from common import resolve_local_file  # noqa: E402
+from generate_db import _path_suffixes  # noqa: E402
+
+
+class TestPathSuffixes(unittest.TestCase):
+    """Every tail down to two segments, never the bare filename."""
+
+    def test_longest_first_and_never_the_bare_name(self):
+        self.assertEqual(
+            _path_suffixes("bios/Nintendo/GameCube/GC/USA/IPL.bin"),
+            [
+                "Nintendo/GameCube/GC/USA/IPL.bin",
+                "GameCube/GC/USA/IPL.bin",
+                "GC/USA/IPL.bin",
+                "USA/IPL.bin",
+            ],
+        )
+
+    def test_a_deep_tree_keeps_every_level(self):
+        # Seven 3DS system archives differ only past the sixth segment; a
+        # single key per file merged all seven onto one.
+        suffixes = _path_suffixes(
+            "bios/Nintendo/3DS/Citra/nand/0000/title/0004009b/00014002/"
+            "content/00000000.app.romfs"
+        )
+        self.assertIn(
+            "nand/0000/title/0004009b/00014002/content/00000000.app.romfs",
+            suffixes,
+        )
+        self.assertEqual(suffixes[0].split("/")[0], "Nintendo")
+
+    def test_a_file_at_the_root_has_no_tail(self):
+        self.assertEqual(_path_suffixes("bios/thing.rom"), [])
+
+    def test_a_path_without_the_bios_prefix_is_still_split(self):
+        self.assertEqual(
+            _path_suffixes("Sony/PlayStation/scph5501.bin"),
+            ["Sony/PlayStation/scph5501.bin", "PlayStation/scph5501.bin"],
+        )
+
+    def test_backslashes_and_empty_segments_are_normalised(self):
+        self.assertEqual(
+            _path_suffixes("bios\\\\Sega\\\\Saturn\\\\sat.bin"),
+            ["Sega/Saturn/sat.bin", "Saturn/sat.bin"],
+        )
+
+
+class TestResolutionAmbiguity(unittest.TestCase):
+    """A tail that could name several files names none of them."""
+
+    @classmethod
+    def setUpClass(cls):
+        cls.root = Path(tempfile.mkdtemp(prefix="suffix-"))
+        cls.previous = os.getcwd()
+        cls.paths = {
+            "a" * 40: "bios/Other/emu_one/resources/shaders/convert.glsl",
+            "b" * 40: "bios/Other/emu_two/pcsx2/resources/shaders/convert.glsl",
+            "c" * 40: "bios/Other/emu_three/assets/sounds/unlock.wav",
+            "d" * 40: "bios/Vendor/Console/GC/JAP/IPL.bin",
+        }
+        for sha1, rel in cls.paths.items():
+            target = cls.root / rel
+            target.parent.mkdir(parents=True, exist_ok=True)
+            target.write_bytes(sha1[:4].encode())
+        files, by_name, by_suffix = {}, {}, {}
+        for sha1, rel in cls.paths.items():
+            name = rel.rsplit("/", 1)[-1]
+            files[sha1] = {"path": rel, "name": name, "md5": "", "size": 4,
+                           "crc32": "", "sha1": sha1}
+            by_name.setdefault(name, []).append(sha1)
+            for suffix in _path_suffixes(rel):
+                if suffix != name:
+                    by_suffix.setdefault(suffix, []).append(sha1)
+        cls.db = {
+            "files": files,
+            "indexes": {"by_md5": {}, "by_name": by_name, "by_crc32": {},
+                        "by_path_suffix": by_suffix},
+        }
+        os.chdir(cls.root)
+
+    @classmethod
+    def tearDownClass(cls):
+        os.chdir(cls.previous)
+        shutil.rmtree(cls.root, ignore_errors=True)
+
+    def _resolve(self, name: str, hint: str):
+        return resolve_local_file({"name": name}, self.db, dest_hint=hint)
+
+    def test_a_full_tail_claimed_twice_goes_to_the_closest_owner(self):
+        # emu_one stores it right at its destination; emu_two nests it one
+        # directory deeper. The one that IS the destination wins.
+        path, status = self._resolve(
+            "convert.glsl", "resources/shaders/convert.glsl"
+        )
+        self.assertEqual(status, "path_exact")
+        self.assertIn("emu_one", path)
+
+    def test_a_tail_two_segments_short_is_not_followed(self):
+        # "sounds/achievements/unlock.wav" would reach emu_three, which is a
+        # different emulator's tree, so the name step answers instead.
+        path, status = self._resolve(
+            "unlock.wav", "pcsx2/resources/sounds/unlock.wav"
+        )
+        self.assertEqual(status, "name_exact")
+        self.assertIn("emu_three", path)
+
+    def test_one_segment_of_descent_is_allowed(self):
+        path, status = self._resolve("IPL.bin", "system/GC/JAP/IPL.bin")
+        self.assertEqual(status, "path_exact")
+        self.assertIn("JAP", path)
+
+    def test_an_exact_tail_still_wins(self):
+        path, status = self._resolve("IPL.bin", "GC/JAP/IPL.bin")
+        self.assertEqual(status, "path_exact")
+        self.assertIn("JAP", path)
+
+
+class TestSizeGateOnTheNameStep(unittest.TestCase):
+    """A name answer must still be a plausible size for the entry."""
+
+    @classmethod
+    def setUpClass(cls):
+        cls.root = Path(tempfile.mkdtemp(prefix="sizegate-"))
+        cls.previous = os.getcwd()
+        small = cls.root / "bios/Apple/IIGS/ROM"
+        large = cls.root / "bios/Apple/Macintosh/ROM"
+        for path, payload in ((small, b"x" * 8), (large, b"y" * 64)):
+            path.parent.mkdir(parents=True, exist_ok=True)
+            path.write_bytes(payload)
+        cls.db = {
+            "files": {
+                "a" * 40: {"path": "bios/Apple/IIGS/ROM", "name": "ROM",
+                           "md5": "", "size": 8, "crc32": "", "sha1": "a" * 40},
+                "b" * 40: {"path": "bios/Apple/Macintosh/ROM", "name": "ROM",
+                           "md5": "", "size": 64, "crc32": "", "sha1": "b" * 40},
+            },
+            "indexes": {
+                "by_md5": {}, "by_crc32": {}, "by_path_suffix": {},
+                "by_name": {"ROM": ["a" * 40, "b" * 40]},
+            },
+        }
+        os.chdir(cls.root)
+
+    @classmethod
+    def tearDownClass(cls):
+        os.chdir(cls.previous)
+        shutil.rmtree(cls.root, ignore_errors=True)
+
+    def test_a_candidate_outside_the_declared_range_is_refused(self):
+        entry = {"name": "ROM", "min_size": 32, "max_size": 128,
+                 "validation": ["size"]}
+        path, _ = resolve_local_file(entry, self.db)
+        self.assertIn("Macintosh", path)
+
+    def test_a_present_file_of_the_wrong_size_is_still_returned(self):
+        # Reported as present and untested, never as absent: the user has a
+        # file under that name and needs to be told it is the wrong one.
+        entry = {"name": "ROM", "size": 999, "validation": ["size"]}
+        path, status = resolve_local_file(entry, self.db)
+        self.assertIsNotNone(path)
+        self.assertEqual(status, "hash_mismatch")
+
+    def test_a_size_without_validation_rejects_nothing(self):
+        # Documented behaviour: a size is informative until the emulator
+        # itself checks it.
+        entry = {"name": "ROM", "size": 999}
+        path, _ = resolve_local_file(entry, self.db)
+        self.assertIsNotNone(path)
+
+
+if __name__ == "__main__":
+    unittest.main()