feat: place RomM firmware by its own template

RomM 5.4 reads each platform's firmware where filesystem.structure.firmware
in config.yml says, bios/{platform} by default, and suggests {platform}/bios
for a {platform}/roms library (rommapp/romm 5.4.0,
backend/config/config_manager.py:58,101-109,205-225,564-568,1105-1124).
The installer only wrote bios/<platform>/. It now reads the template from
config.yml beside the library, takes --romm-firmware when that file is
elsewhere, refuses a platform-first library with no template, and detects
RomM by /romm/config/config.yml too.

install.ps1

@@ -11,7 +11,7 @@
 
     $ErrorActionPreference = "Stop"
     $defaultInstallUrl = "https://raw.githubusercontent.com/Abdess/retrobios/main/install.py"
-    $defaultInstallSha256 = "5ed37d4dabc5a6b28a9093f38a25506b20ec2f49c3587692efd695b017d1caf4"
+    $defaultInstallSha256 = "ddd92362d6b82d9d1797e1325d45898b4ed265c504b861d1b645d44c33d05fdb"
     $maximumInstallerBytes = 2MB
     $installerExit = 0
     $installer = if ($PSScriptRoot) { Join-Path $PSScriptRoot "install.py" } else { $null }

install.py

@@ -1100,6 +1100,9 @@ _PATH_RULES: tuple[tuple[str, str, str, str | None], ...] = (
     ("linux", "emudeck", "$HOME/Emulation/bios", None),
     ("linux", "retrodeck", "$HOME/.var/app/net.retrodeck.retrodeck", "$HOME/retrodeck"),
     ("linux", "romm", "/romm/library/bios", None),
+    # A library laid out as {platform}/bios has no library/bios: RomM's
+    # config folder is the evidence (config_manager.py:264-265).
+    ("linux", "romm", "/romm/config/config.yml", "/romm/library/bios"),
     ("linux", "retropie", "$HOME/RetroPie/BIOS", None),
     ("windows", "bizhawk", "%USERPROFILE%\\BizHawk\\Firmware", None),
     ("linux", "misterfpga", "/media/fat/MiSTer", "/media/fat/games"),
@@ -4127,6 +4130,111 @@ def _print_emulators(plat: str, manifest: dict, files: list[dict], omitted: list
         print(f"  {name:<{width}}  {'' if profile == name else 'same as ' + profile}".rstrip())
 
 
+# RomM 5.4 places each platform's firmware by `filesystem.structure.firmware`
+# in config.yml, "bios/{platform}" by default, relative to the library
+# (rommapp/romm 5.4.0 68f71259, backend/config/config_manager.py:58,101-109,
+# 205-225,564-568). The library and config.yml both sit under the RomM base
+# folder (config/__init__.py:46-48, config_manager.py:264-265). A library
+# laid out as {platform}/roms makes RomM stop until config.yml declares its
+# templates, and RomM suggests firmware: "{platform}/bios" there
+# (config_manager.py:1105-1124).
+
+
+def parse_romm_firmware(template: str) -> tuple[tuple[str, ...], tuple[str, ...]]:
+    """The literal folders before and after {platform}, as RomM parses them."""
+    sections = [section for section in template.strip().split("/") if section]
+    if not sections:
+        raise ValueError("the RomM firmware template is empty")
+
+    def macro(section: str) -> str | None:
+        if section.startswith("{") and section.endswith("}"):
+            return section[1:-1].strip()
+        return None
+
+    if any(section in (".", "..") for section in sections):
+        raise ValueError(f"the RomM firmware template {template!r} climbs out of the library")
+    platform_at = [i for i, section in enumerate(sections) if macro(section) == "platform"]
+    if len(platform_at) != 1:
+        raise ValueError(f"the RomM firmware template {template!r} needs one '{{platform}}'")
+    before, after = sections[:platform_at[0]], sections[platform_at[0] + 1:]
+    if any(macro(section) is not None for section in before + after):
+        raise ValueError(f"the RomM firmware template {template!r} takes literal folder "
+                         "names around '{platform}'")
+    return tuple(before), tuple(after)
+
+
+def _romm_configured_firmware(config: Path) -> str | None:
+    """filesystem.structure.firmware from RomM's config.yml, read without a
+    YAML library: only that nested scalar is needed."""
+    try:
+        lines = config.read_text(encoding="utf-8", errors="replace").splitlines()
+    except OSError:
+        return None
+    path: list[tuple[int, str]] = []
+    for line in lines:
+        stripped = line.split(" #", 1)[0].rstrip()
+        if not stripped.strip() or stripped.lstrip().startswith(("#", "-")):
+            continue
+        indent = len(stripped) - len(stripped.lstrip())
+        key, sep, value = stripped.strip().partition(":")
+        if not sep:
+            continue
+        while path and path[-1][0] >= indent:
+            path.pop()
+        keys = [name for _indent, name in path] + [key.strip().strip("'\"")]
+        if keys == ["filesystem", "structure", "firmware"] and value.strip():
+            return value.strip().strip("'\"")
+        path.append((indent, key.strip().strip("'\"")))
+    return None
+
+
+def romm_firmware_layout(
+    bios_path: Path, template: str | None
+) -> tuple[tuple[str, ...], tuple[str, ...]] | None:
+    """Where RomM reads each platform's firmware, relative to its library.
+
+    bios_path is the folder the default template names, so its parent is the
+    library. The template comes from --romm-firmware, else from config.yml
+    beside the library. None keeps the pack's own bios/<platform> layout.
+    """
+    library = bios_path.parent
+    if template is None:
+        template = _romm_configured_firmware(library.parent / "config" / "config.yml")
+    if template is not None:
+        return parse_romm_firmware(template)
+    platform_first = not (library / "roms").is_dir() and any(
+        path.is_dir() for path in library.glob("*/roms")
+    )
+    if platform_first:
+        raise ValueError(
+            f"{library} keeps its roms inside each platform folder, and RomM then "
+            "reads firmware where filesystem.structure.firmware in its config.yml "
+            "says, which was not found here. RomM suggests \"{platform}/bios\" for "
+            "this layout: run again with --romm-firmware \"{platform}/bios\", or "
+            "with the value config.yml holds."
+        )
+    return None
+
+
+def romm_relayout(
+    files: list[dict], bios_path: Path,
+    before: tuple[str, ...], after: tuple[str, ...],
+) -> tuple[list[dict], Path]:
+    """The manifest's bios/<platform>/<file> entries under RomM's template,
+    with the library as the root they are relative to."""
+    if (before, after) == (("bios",), ()):
+        return files, bios_path
+    moved = []
+    for entry in files:
+        slug, _, rest = str(entry["dest"]).partition("/")
+        if not rest:
+            # Firmware of no platform: RomM lists none, the default place stays.
+            moved.append({**entry, "dest": f"bios/{slug}"})
+            continue
+        moved.append({**entry, "dest": "/".join((*before, slug, *after, rest))})
+    return moved, bios_path.parent
+
+
 def _older_list_advice() -> str:
     """How to reach a file list that records what an option needs.
 
@@ -4338,6 +4446,13 @@ def main() -> None:
         action="store_true",
         help="list the cores whose extra files a platform installs and exit",
     )
+    parser.add_argument(
+        "--romm-firmware",
+        metavar="TEMPLATE",
+        help="RomM: where each platform's firmware goes in the library, as "
+             "filesystem.structure.firmware in its config.yml (default: read "
+             "from config.yml, else bios/{platform})",
+    )
     parser.add_argument(
         "--list-emulators",
         action="store_true",
@@ -4595,6 +4710,18 @@ def _install(args: argparse.Namespace, real_stdout: TextIO) -> None:
                           "--platform to install it alone.", file=sys.stderr)
                 sys.exit(1)
 
+    if args.romm_firmware is not None:
+        # Applied or refused, never ignored.
+        if not any(name == "romm" for name, _path in platforms):
+            print("Error: --romm-firmware applies to RomM, which this run does not "
+                  "install for.", file=sys.stderr)
+            sys.exit(1)
+        try:
+            parse_romm_firmware(args.romm_firmware)
+        except ValueError as exc:
+            print(f"Error: {exc}", file=sys.stderr)
+            sys.exit(1)
+
     # Every check --installed-cores makes before anything is written: a
     # refusal for the second platform must not follow an install of the first.
     cores_found: dict[tuple[str, Path], list[str]] = {}
@@ -4785,6 +4912,20 @@ def _install(args: argparse.Namespace, real_stdout: TextIO) -> None:
             total_omitted += len(omitted_files)
             omitted_seen.extend(omitted_files)
 
+        if plat_name == "romm":
+            try:
+                layout = romm_firmware_layout(bios_path, args.romm_firmware)
+            except ValueError as exc:
+                print(f"Error: {exc}", file=sys.stderr)
+                sys.exit(1)
+            if layout is not None:
+                files, library = romm_relayout(files, bios_path, *layout)
+                if library != bios_path:
+                    installed_paths[installed_paths.index(bios_path)] = library
+                    print(f"  Firmware laid out as {'/'.join(layout[0] + ('{platform}',) + layout[1])} "
+                          f"under {library}")
+                    bios_path = library
+
         print("\nChecking existing files...")
         to_download, up_to_date, mismatched = check_local(
             files, bios_path, jobs=args.jobs, term=term

install.sh

@@ -18,7 +18,7 @@ esac
 TEMP_INSTALLER=""
 TEMP_DIRECTORY=""
 DEFAULT_INSTALL_URL="https://raw.githubusercontent.com/Abdess/retrobios/main/install.py"
-DEFAULT_INSTALL_SHA256="5ed37d4dabc5a6b28a9093f38a25506b20ec2f49c3587692efd695b017d1caf4"
+DEFAULT_INSTALL_SHA256="ddd92362d6b82d9d1797e1325d45898b4ed265c504b861d1b645d44c33d05fdb"
 MAX_INSTALLER_BYTES=2097152
 
 cleanup() {

platforms/_registry.yml

@@ -511,6 +511,14 @@ platforms:
       - os: linux
         method: path_exists
         path: /romm/library/bios
+      # RomM 5.4 places firmware by filesystem.structure.firmware in this
+      # file, relative to /romm/library (rommapp/romm 5.4.0,
+      # backend/config/config_manager.py:58,101-109,264-265,564-568): a
+      # {platform}/bios library has no library/bios to find.
+      - os: linux
+        method: path_exists
+        path: /romm/config/config.yml
+        bios_path: /romm/library/bios
     cores:
     - atari800
     - clk

tests/test_install.py

@@ -3991,5 +3991,100 @@ class BatoceraAndKnulliByWhatTheirBuildsWrite(unittest.TestCase):
         self.assertIn("derivative_board(plat_name)", block)
 
 
+class RomMFirmwareFollowsItsTemplate(unittest.TestCase):
+    """RomM 5.4 reads firmware where filesystem.structure.firmware says, and
+    a {platform}/roms library is told to declare "{platform}/bios". The
+    installer only wrote bios/<platform>/, where such a library lists none."""
+
+    CONFIG = (
+        "# RomM\n"
+        "exclude:\n"
+        "  platforms: []\n"
+        "filesystem:\n"
+        "  structure:\n"
+        "    default: \"{platform}/roms/{game}\"\n"
+        "    firmware: \"{platform}/bios\"  # as RomM suggests\n"
+        "system:\n"
+        "  firmware: elsewhere\n"
+    )
+
+    def _romm(self, config: str | None) -> Path:
+        base = Path(_tempdir(self))
+        library = base / "library"
+        (library / "psx" / "roms").mkdir(parents=True)
+        if config is not None:
+            (base / "config").mkdir()
+            (base / "config" / "config.yml").write_text(config)
+        return library / "bios"
+
+    def test_templates_parse_as_romm_parses_them(self):
+        self.assertEqual(install.parse_romm_firmware("bios/{platform}"), (("bios",), ()))
+        self.assertEqual(install.parse_romm_firmware("{platform}/bios"), ((), ("bios",)))
+        self.assertEqual(install.parse_romm_firmware("/a/{platform}/b/c/"),
+                         (("a",), ("b", "c")))
+        for bad in ("", "bios", "{platform}/{game}", "{region}/{platform}",
+                    "../{platform}", "{platform}/{platform}"):
+            with self.subTest(template=bad), self.assertRaises(ValueError):
+                install.parse_romm_firmware(bad)
+
+    def test_the_template_is_read_from_config_yml(self):
+        bios = self._romm(self.CONFIG)
+        layout = install.romm_firmware_layout(bios, None)
+        self.assertEqual(layout, ((), ("bios",)))
+        files = [{"dest": "psx/scph5501.bin"}, {"dest": "3do/panafz1.bin"}]
+        moved, root = install.romm_relayout(files, bios, *layout)
+        self.assertEqual(root, bios.parent)
+        self.assertEqual([f["dest"] for f in moved],
+                         ["psx/bios/scph5501.bin", "3do/bios/panafz1.bin"])
+
+    def test_the_default_keeps_the_pack_layout(self):
+        bios = self._romm("filesystem:\n  structure:\n    firmware: bios/{platform}\n")
+        files = [{"dest": "psx/scph5501.bin"}]
+        self.assertEqual(install.romm_relayout(files, bios, *install.romm_firmware_layout(
+            bios, None)), (files, bios))
+
+    def test_a_platform_first_library_without_a_template_is_refused(self):
+        bios = self._romm(None)
+        with self.assertRaises(ValueError) as raised:
+            install.romm_firmware_layout(bios, None)
+        self.assertIn('--romm-firmware "{platform}/bios"', str(raised.exception))
+        self.assertEqual(install.romm_firmware_layout(bios, "{platform}/bios"),
+                         ((), ("bios",)))
+
+    def test_the_option_is_refused_where_romm_is_not_installed(self):
+        err = io.StringIO()
+        with unittest.mock.patch.object(install, "detect_os", return_value="linux"), \
+                unittest.mock.patch("sys.argv", [
+                    "install.py", "--platform", "batocera", "--dest", _tempdir(self),
+                    "--romm-firmware", "{platform}/bios", "--no-input"]), \
+                contextlib.redirect_stdout(io.StringIO()), \
+                contextlib.redirect_stderr(err), self.assertRaises(SystemExit) as caught:
+            install.main()
+        self.assertEqual(caught.exception.code, 1)
+        self.assertIn("--romm-firmware applies to RomM", err.getvalue())
+
+
+class RomMFirmwareInstallsUnderItsTemplate(unittest.TestCase, _LoopbackRun):
+    """The whole run, over loopback: config.yml beside the library decides."""
+
+    def setUp(self):
+        self.env = self.serve(self)
+
+    def test_lynx_lands_in_its_platform_folder(self):
+        base = self.home / "romm"
+        (base / "library" / "lynx" / "roms").mkdir(parents=True)
+        (base / "config").mkdir()
+        (base / "config" / "config.yml").write_text(
+            "filesystem:\n  structure:\n    default: \"{platform}/roms/{game}\"\n"
+            "    firmware: \"{platform}/bios\"\n")
+        proc = _install_run(self.env, "--platform", "romm", "--dest",
+                            str(base / "library" / "bios"), "--system", "atari-lynx",
+                            "--no-input")
+        self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr)
+        self.assertTrue((base / "library" / "lynx" / "bios" / "lynxboot.img").is_file())
+        self.assertFalse((base / "library" / "bios").exists())
+        self.assertIn("{platform}/bios", proc.stdout)
+
+
 if __name__ == "__main__":
     unittest.main()

wiki/installer.md

@@ -235,6 +235,7 @@ what the options name, everything by default.
 | `--list-systems` | Print a platform's systems with the files and size each holds |
 | `--list-cores` | Print the cores whose extra files a platform installs |
 | `--list-emulators` | Print every name `--emulator` accepts, each buildbot name beside the emulator it stands for |
+| `--romm-firmware TEMPLATE` | RomM: where each platform's firmware goes in the library, as `filesystem.structure.firmware` in RomM's `config.yml` says (`{platform}/bios` for a library laid out as `{platform}/roms`). Read from `config.yml` beside the library when it is there. See [RomM firmware layout](#romm-firmware-layout) |
 | `--no-input` | Never ask a question, even at a terminal |
 | `--check` | Report and exit without writing |
 | `--dry-run` | Show what would be downloaded, by system and with sizes, without writing anything |
@@ -306,6 +307,7 @@ not already found that way.
 | `~/Emulation/bios` | EmuDeck | the same |
 | `~/.var/app/net.retrodeck.retrodeck` | RetroDECK | `~/retrodeck` |
 | `/romm/library/bios` | RomM | the same |
+| `/romm/config/config.yml` | RomM | `/romm/library/bios`, laid out by its firmware template |
 | `~/RetroPie/BIOS` | RetroPie | the same |
 | `%USERPROFILE%\BizHawk\Firmware` | BizHawk | the same |
 | `EmuHawk.exe` or `EmuHawkMono.sh` in a `BizHawk*` folder of the home folder or one level down (`~/Games/BizHawk-2.11.1`), the newest | BizHawk | `Firmware` beside it |
@@ -415,6 +417,23 @@ replaced by the file rather than written through: a write through the link
 would land outside the folder the user opted into. The summary counts each
 copy as copied, skipped (folder absent) or failed, and names every failure.
 
+## RomM firmware layout
+
+RomM 5.4 places each platform's firmware where `filesystem.structure.firmware`
+in its `config.yml` says, relative to the library: `bios/{platform}` by
+default, the layout of the pack. A library laid out as `{platform}/roms` makes
+RomM stop until `config.yml` declares its templates, and RomM suggests
+`firmware: "{platform}/bios"` for it.
+
+The installer reads that value from `config.yml` in the `config` folder beside
+the library, `/romm/config/config.yml` inside the container, and places each
+file accordingly: with `{platform}/bios`, `psx/scph5501.bin` goes to
+`library/psx/bios/scph5501.bin`. `--dest` still names the library's `bios`
+folder, whose parent is the library. Run from the host of a container, where
+`config.yml` sits elsewhere, `--romm-firmware` gives the same value. A
+library laid out as `{platform}/roms` with no template found is refused
+rather than filled in the wrong place, and the message gives the command.
+
 ## Android
 
 Android reports itself as Linux, so the runtime is named from its own