mirror of
https://github.com/Abdess/retroarch_system.git
synced 2026-11-11 12:33:53 -06:00
feat: place files outside the bios tree
A platform that binds part of a standalone layout leaves the entries no binding covers in a folder outside the BIOS tree: they are no longer packed at the BIOS root, and the manifest names them (root: emulator, outside_bios_folder). A launcher that executes a profile's file binds the program folder: in the BIOS tree, below the roms directory a registry entry names (root: roms, installed into the folder holding the BIOS folder), or nowhere a pack reaches. A moved entry resolves by its own path as well as by its destination. uae4all gains kick.rom and rom.key in its working directory and loses a not_bundled_on that named Recalbox, which runs uae4all-rpi. lexaloffle declares that its programs read beside their executable, lists the Windows build RetroBat runs and the pico8_dyn ROCKNIX runs on x86_64.
This commit is contained in:
1 parent
b5ee4ab09e
commit
65e1db9fe1
19 files changed
+802
-141
No files matched your search
@@ -24,12 +24,19 @@ cores:
|
||||
systems:
|
||||
- pico8
|
||||
- voxatron
|
||||
standalone_directories:
|
||||
- "executable directory"
|
||||
|
||||
notes: |
|
||||
Commercial fantasy consoles by Lexaloffle Games LLP, sold as ZIP archives
|
||||
per platform. Nothing in either program checks a hash of its own files:
|
||||
the hashes below are those of the files in the official 0.2.7 archives.
|
||||
|
||||
Each program finds its own folder through /proc/self/exe and reads its
|
||||
.dat beside itself. The paths below are the layout of the archives, whose
|
||||
top folders are pico-8/ and voxatron/: a launcher that runs one of these
|
||||
files from a folder places the program folder there.
|
||||
|
||||
PICO-8 0.2.7 ships five archives. pico-8_0.2.7_amd64.zip holds pico8
|
||||
(x86_64, SDL2 linked in) and pico8_dyn (x86_64, system libSDL2).
|
||||
pico-8_0.2.7_raspi.zip holds pico8 (armhf, SDL2 linked in, needs the
|
||||
@@ -67,7 +74,7 @@ notes: |
|
||||
component_manifest.json lists pico8, pico8.dat and pico8_dyn as required.
|
||||
- RetroBat Pico8.Generator.cs runs pico8.exe from the folder its config
|
||||
names pico8 and returns nothing when pico8.exe is absent. That folder is
|
||||
an emulator install, outside the BIOS tree, so no entry below targets it.
|
||||
an emulator install, outside the BIOS tree.
|
||||
- ArkOS (dArkOS pico8/pico8.sh) prefers roms/pico-8/pico8_64, then
|
||||
roms/pico-8/pico8_dyn, then extracts pico8_64 and pico8.dat from a
|
||||
*raspi.zip in that folder, and starts Fake08 when the binary or
|
||||
@@ -137,13 +144,13 @@ files:
|
||||
path: pico-8/pico8_dyn
|
||||
system: pico8
|
||||
required: false
|
||||
unread_on: [amberelec, emuelec, knulli, recalbox, retrobat, rocknix]
|
||||
unread_on: [amberelec, emuelec, knulli, recalbox, retrobat]
|
||||
size: 1805928
|
||||
md5: 724435a9e6f3ef8a55c648fc88651826
|
||||
sha1: 9b2e573250c70382b4071709a85c8d56c0371d59
|
||||
description: "PICO-8 0.2.7 x86_64 executable, system libSDL2"
|
||||
note: "The pico8_dyn of pico-8_0.2.7_amd64.zip, the same program linked against the system SDL2. RetroDECK lists it as required in its manifest, but its launcher executes pico8 only. Batocera never runs it."
|
||||
source_ref: "pico-8 0.2.7 amd64 pico8_dyn, NEEDED libSDL2-2.0.so.0, RetroDECK pico-8/component_manifest.json:26-32, RetroDECK pico-8/component_launcher.sh:3"
|
||||
note: "The pico8_dyn of pico-8_0.2.7_amd64.zip, the same program linked against the system SDL2. RetroDECK lists it as required in its manifest, but its launcher executes pico8 only. Batocera never runs it. ROCKNIX runs it from roms/pico-8/ on x86_64, its HW_ARCH other than aarch64."
|
||||
source_ref: "pico-8 0.2.7 amd64 pico8_dyn, NEEDED libSDL2-2.0.so.0, RetroDECK pico-8/component_manifest.json:26-32, RetroDECK pico-8/component_launcher.sh:3, rocknix projects/ROCKNIX/packages/emulators/standalone/pico-8/scripts/start_pico8.sh:10-17, 48"
|
||||
|
||||
- name: pico8_dyn
|
||||
path: pico-8/pico8_dyn
|
||||
@@ -173,14 +180,37 @@ files:
|
||||
path: pico-8/pico8.dat
|
||||
system: pico8
|
||||
required: true
|
||||
unread_on: [retrobat]
|
||||
size: 11243101
|
||||
md5: 10b397305093c80109096319a538ba56
|
||||
sha1: 0f92093752bddb12210c7db6d93af3ba2413cfce
|
||||
description: "PICO-8 0.2.7 data file (boot sequence, themes, demo carts)"
|
||||
note: "PICO-8 opens it from its own folder and quits with \"couldn't load <folder>/pico8.dat\" when it is absent (the amd64 pico8, run without it). Paired with the binary's own version, identical in pico-8_0.2.7_amd64.zip and pico-8_0.2.7_raspi.zip. AmberELEC and ArkOS check for it before starting PICO-8, and KNULLI's installer stops when the archive lacks it."
|
||||
note: "PICO-8 opens it from its own folder and quits with \"couldn't load <folder>/pico8.dat\" when it is absent (the amd64 pico8, run without it). Paired with the binary's own version, identical in pico-8_0.2.7_amd64.zip, pico-8_0.2.7_raspi.zip and pico-8_0.2.7_windows.zip. AmberELEC and ArkOS check for it before starting PICO-8, and KNULLI's installer stops when the archive lacks it."
|
||||
source_ref: "pico-8 0.2.7 amd64 pico8, string pico8.dat at 0x231bc0, pico-8 0.2.7 raspi pico8_64, string pico8.dat at 0x110138, amberelec packages/amberelec/sources/pico-8.sh:51-52, dArkOS pico8/pico8.sh:136-142, knulli board/fsoverlay/usr/bin/knulli-install-pico8:11, 48-51, 69"
|
||||
|
||||
- name: pico8.exe
|
||||
path: pico-8/pico8.exe
|
||||
system: pico8
|
||||
required: true
|
||||
unread_on: [amberelec, batocera, emuelec, knulli, recalbox, retrodeck, rocknix]
|
||||
size: 2171788
|
||||
md5: 55b1b9db521ffd0cea0995174bd12a2b
|
||||
sha1: 1d1b504d3283d750869a93ed9befa1e49928d68e
|
||||
description: "PICO-8 0.2.7 Windows executable"
|
||||
note: "The pico8.exe of pico-8_0.2.7_windows.zip. RetroBat runs it from its pico8 emulator folder and starts nothing when it is absent."
|
||||
source_ref: "retrobat emulatorLauncher/Generators/Pico8.Generator.cs:14-18, 56-57"
|
||||
|
||||
- name: SDL2.dll
|
||||
path: pico-8/SDL2.dll
|
||||
system: pico8
|
||||
required: true
|
||||
unread_on: [amberelec, batocera, emuelec, knulli, recalbox, retrodeck, rocknix]
|
||||
size: 1088000
|
||||
md5: 098c661f98cd7fd85a82ebc34db5be15
|
||||
sha1: 9174251921d675c204e683cdea65a8c906fcba28
|
||||
description: "SDL2 library of the PICO-8 0.2.7 Windows build"
|
||||
note: "pico8.exe imports SDL2.dll, which Windows loads from the executable's folder. Shipped beside it in pico-8_0.2.7_windows.zip."
|
||||
source_ref: "pico-8 0.2.7 windows pico8.exe, import table"
|
||||
|
||||
- name: config.txt
|
||||
unsourceable: "written by the emulator on exit, no canonical file"
|
||||
system: pico8
|
||||
|
||||
+15
-2
@@ -88,7 +88,6 @@ files:
|
||||
- name: "aros-amiga-m68k-rom.bin"
|
||||
path: "kickstarts/aros-amiga-m68k-rom.bin"
|
||||
bundled: true
|
||||
not_bundled_on: [recalbox]
|
||||
system: commodore-amiga
|
||||
description: "AROS Kickstart replacement, main ROM (build of 2014-01-10)"
|
||||
required: false
|
||||
@@ -102,7 +101,6 @@ files:
|
||||
- name: "aros-amiga-m68k-ext.bin"
|
||||
path: "kickstarts/aros-amiga-m68k-ext.bin"
|
||||
bundled: true
|
||||
not_bundled_on: [recalbox]
|
||||
system: commodore-amiga
|
||||
description: "AROS Kickstart replacement, extended ROM (build of 2014-01-10)"
|
||||
required: false
|
||||
@@ -119,3 +117,18 @@ files:
|
||||
description: "Cloanto key for AMIROMTYPE1 encrypted Kickstarts"
|
||||
required: false
|
||||
source_ref: "upstream src/main.cpp:106-111, source src/memory.cpp:732-776"
|
||||
|
||||
- name: "kick.rom"
|
||||
path: "kick.rom"
|
||||
system: commodore-amiga
|
||||
description: "Custom Kickstart, the file the custom choice opens until the user picks another"
|
||||
required: false
|
||||
source_ref: "source src/gp2x/menu/menu_config.cpp:139, source src/gp2x/menu/menu_config.cpp:1243-1246, source src/menu_guichan/menu_guichan.cpp:91-99, source src/menu_guichan/menu_guichan.cpp:1004-1006, source src/main.cpp:100-104"
|
||||
note: "Read from the working directory. default_prefs also falls back to it when the selected Kickstart is missing from kickstarts/."
|
||||
|
||||
- name: "rom.key"
|
||||
path: "rom.key"
|
||||
system: commodore-amiga
|
||||
description: "Cloanto key, read from the working directory when kickstarts/rom.key is absent"
|
||||
required: false
|
||||
source_ref: "source src/main.cpp:106-111, source src/memory.cpp:732-776"
|
||||
+1
-1
@@ -16,7 +16,7 @@
|
||||
# and Invoke-WebRequest throws on a cut transfer before the hash is read:
|
||||
# the site's build rewrites this line there (scripts/sitebootstrap.py).
|
||||
$mismatchAdvice = "The download was most likely cut short: try again. Within minutes of an update of this project the two files can come from caches of different ages. Waiting five minutes settles that."
|
||||
$defaultInstallSha256 = "c2265e2ff8fae9b485f195aa47c4a487f16a287ab6ad2f8f8a133bfede96c983"
|
||||
$defaultInstallSha256 = "3f44e2c6757db61c78a1df6781a0bbb78294a66cda5752617fc2d3b501847bf0"
|
||||
$maximumInstallerBytes = 2MB
|
||||
$installerExit = 0
|
||||
$installer = if ($PSScriptRoot) { Join-Path $PSScriptRoot "install.py" } else { $null }
|
||||
|
||||
+78
-10
@@ -141,6 +141,8 @@ OMISSION_REASONS = {
|
||||
"no_save_directory": "no save directory",
|
||||
"no_core_folder": "no core folder",
|
||||
"no_save_system": "no system runs the core",
|
||||
"outside_bios_folder": "read outside the BIOS folder",
|
||||
"no_roms_directory": "no roms folder",
|
||||
}
|
||||
# What a file is when it is not firmware, as the manifest's category field
|
||||
# says, and as the installer words it. Firmware carries no category.
|
||||
@@ -3590,6 +3592,8 @@ class SavePlace(NamedTuple):
|
||||
# BIOS folder -> where the files read below its frontend's save directory go,
|
||||
# set once the save directory is known (save_directory_for).
|
||||
_SAVE_DIRS: dict[Path, SavePlace] = {}
|
||||
# BIOS folder -> the platform's roms folder, set by place_roms_entries.
|
||||
_ROMS_DIRS: dict[Path, Path] = {}
|
||||
|
||||
|
||||
def _entry_place(root: Path, entry: dict) -> tuple[Path, object]:
|
||||
@@ -3600,6 +3604,11 @@ def _entry_place(root: Path, entry: dict) -> tuple[Path, object]:
|
||||
core's own folder where the frontend sorts saves by core. Any other is
|
||||
relative to the BIOS folder.
|
||||
"""
|
||||
if entry.get("root") == "roms":
|
||||
roms = _ROMS_DIRS.get(root)
|
||||
if roms is None:
|
||||
raise ValueError(f"no roms folder known for {entry.get('dest')}")
|
||||
return roms, entry.get("dest")
|
||||
if entry.get("root") != "save":
|
||||
return root, entry.get("dest")
|
||||
place = _SAVE_DIRS.get(root)
|
||||
@@ -3625,17 +3634,24 @@ def _entry_path(root: Path, entry: dict) -> Path:
|
||||
|
||||
def _entry_key(entry: dict) -> str:
|
||||
"""What tells one entry of the list from another: its dest, below the
|
||||
BIOS folder or below the save directory. KNULLI lists its own
|
||||
fbneo/hiscore.dat in the BIOS tree and FB Alpha 2012's below the save
|
||||
directory, two files under one dest. NUL never appears in a dest."""
|
||||
BIOS folder, below the save directory or in an emulator's own folder.
|
||||
KNULLI lists its own fbneo/hiscore.dat in the BIOS tree and FB Alpha
|
||||
2012's below the save directory, two files under one dest. NUL never
|
||||
appears in a dest."""
|
||||
dest = str(entry.get("dest"))
|
||||
return f"save\0{dest}" if entry.get("root") == "save" else dest
|
||||
|
||||
|
||||
def _check_root(entry: dict, dest: str) -> None:
|
||||
"""root is absent or save, and core_folder one folder name of a save entry."""
|
||||
root = entry.get("root")
|
||||
if root is not None and root != "save":
|
||||
if root == "emulator":
|
||||
# A path in one emulator's own folder: two emulators read two files.
|
||||
return f"emulator\0{','.join(entry.get('cores') or [])}\0{dest}"
|
||||
return f"{root}\0{dest}" if root in ("save", "roms") else dest
|
||||
|
||||
|
||||
def _check_root(entry: dict, dest: str, omitted: bool = False) -> None:
|
||||
"""root is absent or save, emulator only on a file left out (a path in
|
||||
the emulator's own folder, outside the BIOS tree), and core_folder one
|
||||
folder name of a save entry."""
|
||||
root = entry.get("root")
|
||||
if root not in (None, "save", "roms") and not (omitted and root == "emulator"):
|
||||
raise ValueError(f"invalid root for {dest}")
|
||||
folder = entry.get("core_folder")
|
||||
if folder is None:
|
||||
@@ -3646,6 +3662,25 @@ def _check_root(entry: dict, dest: str) -> None:
|
||||
raise ValueError(f"core_folder must be one folder name for {dest}")
|
||||
|
||||
|
||||
def _check_roms_directory(data: dict) -> None:
|
||||
"""The manifest's roms_directory, which every installable root: roms entry needs."""
|
||||
layout = data.get("roms_directory")
|
||||
installable = [
|
||||
entry for field in ("files", "case_variants")
|
||||
for entry in data.get(field) or [] if isinstance(entry, dict)
|
||||
]
|
||||
if layout is None:
|
||||
if any(entry.get("root") == "roms" for entry in installable):
|
||||
raise ValueError("entries below the roms folder and no roms_directory")
|
||||
return
|
||||
if (
|
||||
not isinstance(layout, dict)
|
||||
or set(layout) != {"bios_folder"}
|
||||
or len(_safe_relative_path(layout["bios_folder"], "roms_directory.bios_folder").parts) != 1
|
||||
):
|
||||
raise ValueError("invalid roms_directory")
|
||||
|
||||
|
||||
def _check_save_directory(data: dict) -> None:
|
||||
"""The manifest's save_directory, which every root: save entry needs."""
|
||||
layout = data.get("save_directory")
|
||||
@@ -3949,6 +3984,7 @@ def _validate_manifest(data: object, plat: str) -> dict:
|
||||
for index, entry in enumerate(data_dirs):
|
||||
_check_data_directory(entry, f"data_directories[{index}]")
|
||||
_check_save_directory(data)
|
||||
_check_roms_directory(data)
|
||||
|
||||
declared_total_files = data.get("total_files")
|
||||
if declared_total_files is not None and declared_total_files != len(files):
|
||||
@@ -3974,7 +4010,7 @@ def _validate_manifest(data: object, plat: str) -> dict:
|
||||
raise ValueError(f"duplicate or conflicting omitted destination: {dest}")
|
||||
seen_omitted.add(key)
|
||||
_check_omission(entry, dest)
|
||||
_check_root(entry, dest)
|
||||
_check_root(entry, dest, omitted=True)
|
||||
claims = entry.get("alternatives", [])
|
||||
if not isinstance(claims, list) or len(claims) > MAX_SELECTION_TAGS:
|
||||
raise ValueError(f"invalid omitted alternatives for {dest}")
|
||||
@@ -5214,6 +5250,36 @@ def place_save_entries(
|
||||
return kept, unplaced
|
||||
|
||||
|
||||
def place_roms_entries(
|
||||
files: list[dict], bios_path: Path, manifest: dict, display: str, say: bool = True,
|
||||
) -> tuple[list[dict], list[dict]]:
|
||||
"""Find the roms folder for the entries a launcher reads below it.
|
||||
|
||||
The platform keeps its BIOS folder in its roms folder under the name the
|
||||
list gives (ROCKNIX's /storage/roms/bios): a BIOS folder of that name
|
||||
puts the roms folder above it. Any other is a folder this run cannot
|
||||
relate to the roms, and the entries are left out with the reason.
|
||||
"""
|
||||
below = [entry for entry in files if entry.get("root") == "roms"]
|
||||
if not below:
|
||||
return files, []
|
||||
layout = manifest.get("roms_directory") or {}
|
||||
if layout and bios_path.name == layout.get("bios_folder"):
|
||||
_ROMS_DIRS[bios_path] = bios_path.parent
|
||||
return files, []
|
||||
if say:
|
||||
print(f" {_plural(len(below), 'file')} read from {display}'s roms folder left out: "
|
||||
f"{bios_path} is not its {layout.get('bios_folder') or 'BIOS'} folder.")
|
||||
kept = [entry for entry in files if entry.get("root") != "roms"]
|
||||
unplaced = [
|
||||
{"dest": entry["dest"], "name": PurePosixPath(entry["dest"]).name,
|
||||
"system": "", "systems": list(entry.get("systems") or []), "required": False,
|
||||
"reason": "no_roms_directory", "cores": entry.get("cores"), "root": "roms"}
|
||||
for entry in below
|
||||
]
|
||||
return kept, unplaced
|
||||
|
||||
|
||||
# Per BIOS folder, where its save files go in words, or why that is not
|
||||
# known: read by the lines that count them once a choice is made.
|
||||
_SAVE_WHERE: dict[Path, str] = {}
|
||||
@@ -15827,6 +15893,8 @@ def _install(args: argparse.Namespace, real_stdout: TextIO) -> None:
|
||||
display = display_name(plat_name, manifest)
|
||||
files, unplaced = place_save_entries(
|
||||
files, bios_path, manifest, args.save_dir, display, say=False)
|
||||
files, beyond = place_roms_entries(files, bios_path, manifest, display, say=not listing)
|
||||
unplaced = [*unplaced, *beyond]
|
||||
if not manifest.get("save_directory"):
|
||||
# Its own list leaves these files out: naming the folder with
|
||||
# --save-dir installs none of them, and was advised in a loop.
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@ DEFAULT_INSTALL_URL="https://raw.githubusercontent.com/Abdess/retrobios/main/ins
|
||||
# curl or wget stop on a cut transfer before the hash is read: the site's
|
||||
# build rewrites this line there (scripts/sitebootstrap.py).
|
||||
MISMATCH_ADVICE="The download was most likely cut short: try again. Within minutes of an update of this project the two files can come from caches of different ages. Waiting five minutes settles that."
|
||||
DEFAULT_INSTALL_SHA256="c2265e2ff8fae9b485f195aa47c4a487f16a287ab6ad2f8f8a133bfede96c983"
|
||||
DEFAULT_INSTALL_SHA256="3f44e2c6757db61c78a1df6781a0bbb78294a66cda5752617fc2d3b501847bf0"
|
||||
MAX_INSTALLER_BYTES=2097152
|
||||
|
||||
cleanup() {
|
||||
|
||||
@@ -27,8 +27,8 @@
|
||||
{
|
||||
"dest": "roms/ports/retrobios/install.py",
|
||||
"repo_path": "install.py",
|
||||
"size": 1130265,
|
||||
"sha256": "c2265e2ff8fae9b485f195aa47c4a487f16a287ab6ad2f8f8a133bfede96c983",
|
||||
"size": 1133449,
|
||||
"sha256": "3f44e2c6757db61c78a1df6781a0bbb78294a66cda5752617fc2d3b501847bf0",
|
||||
"executable": false
|
||||
},
|
||||
{
|
||||
@@ -71,8 +71,8 @@
|
||||
"muos": {
|
||||
"archive": {
|
||||
"dest": "ARCHIVE/RetroBIOS.muxapp",
|
||||
"size": 1183890,
|
||||
"sha256": "0d0b07197b3dd080ee41d67fe055881ab7c549f769861ada39e58ed8dda04e5a"
|
||||
"size": 1187074,
|
||||
"sha256": "f98b3e8613e5572430097d4a0133d914986d17c47999020459a8555695a9a427"
|
||||
},
|
||||
"members": [
|
||||
{
|
||||
@@ -92,8 +92,8 @@
|
||||
{
|
||||
"name": "RetroBIOS/install.py",
|
||||
"repo_path": "install.py",
|
||||
"size": 1130265,
|
||||
"sha256": "c2265e2ff8fae9b485f195aa47c4a487f16a287ab6ad2f8f8a133bfede96c983",
|
||||
"size": 1133449,
|
||||
"sha256": "3f44e2c6757db61c78a1df6781a0bbb78294a66cda5752617fc2d3b501847bf0",
|
||||
"executable": false
|
||||
},
|
||||
{
|
||||
@@ -166,8 +166,8 @@
|
||||
{
|
||||
"dest": ".config/modules/retrobios/install.py",
|
||||
"repo_path": "install.py",
|
||||
"size": 1130265,
|
||||
"sha256": "c2265e2ff8fae9b485f195aa47c4a487f16a287ab6ad2f8f8a133bfede96c983",
|
||||
"size": 1133449,
|
||||
"sha256": "3f44e2c6757db61c78a1df6781a0bbb78294a66cda5752617fc2d3b501847bf0",
|
||||
"executable": false
|
||||
},
|
||||
{
|
||||
|
||||
@@ -843,6 +843,13 @@ platforms:
|
||||
# sources/RK3566/retroarch.cfg:664-665 at 20261001) and setsettings.sh
|
||||
# leaves it: saves sit beside each game, no single place.
|
||||
save_directory: null
|
||||
# The BIOS folder sits in the roms directory (/storage/roms/bios, the
|
||||
# detect entry below), where start_pico8.sh also runs PICO-8 from
|
||||
# /storage/roms/pico-8. A launcher path below path is placed there, and
|
||||
# the installer finds that directory as the folder holding bios_folder.
|
||||
roms_directory:
|
||||
path: /storage/roms
|
||||
bios_folder: bios
|
||||
schedule: weekly
|
||||
target_scraper: rocknix_targets
|
||||
target_source: https://github.com/ROCKNIX/distribution/blob/next/projects/ROCKNIX/packages/virtual/emulators/package.mk
|
||||
|
||||
@@ -179,7 +179,7 @@
|
||||
"dest": {
|
||||
"type": "string",
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$)).+$",
|
||||
"description": "Where the file would go, below base_destination, or below the save directory when root is save."
|
||||
"description": "Where the file would go, below base_destination, below the save directory when root is save, below the roms folder when root is roms, or in the emulator's own folder when root is emulator."
|
||||
},
|
||||
"name": {
|
||||
"type": "string",
|
||||
@@ -204,7 +204,9 @@
|
||||
"below_platform_folder",
|
||||
"no_save_directory",
|
||||
"no_core_folder",
|
||||
"no_save_system"
|
||||
"no_save_system",
|
||||
"outside_bios_folder",
|
||||
"no_roms_directory"
|
||||
],
|
||||
"description": "Why the file is left out, as a code. enumDescriptions gives the meaning of each code, in the order of enum.",
|
||||
"enumDescriptions": [
|
||||
@@ -216,7 +218,9 @@
|
||||
"The platform lists each system folder flat (firmware_folder_flat) and the file would sit in a subfolder it does not read.",
|
||||
"The core reads the file below the save directory and the platform has no save directory a pack can fill.",
|
||||
"The frontend sorts saves by core and the core's profile does not name its folder (core_folder).",
|
||||
"The launcher gives each system its own save folder and no system of the platform runs the core."
|
||||
"The launcher gives each system its own save folder and no system of the platform runs the core.",
|
||||
"The platform runs the emulator from a folder outside the BIOS tree and links none of the BIOS tree where the emulator reads this file (root: emulator).",
|
||||
"The platform's launcher runs the file from its roms folder (root: roms) and the run cannot tell where that folder is: the BIOS folder is not the one the platform keeps in it."
|
||||
]
|
||||
},
|
||||
"category": {
|
||||
@@ -361,8 +365,12 @@
|
||||
}
|
||||
},
|
||||
"root": {
|
||||
"description": "save: dest is relative to the save directory the frontend hands the core (RETRO_ENVIRONMENT_GET_SAVE_DIRECTORY), not to the BIOS directory. Absent for a file of the BIOS tree.",
|
||||
"const": "save"
|
||||
"description": "save: dest is relative to the save directory the frontend hands the core (RETRO_ENVIRONMENT_GET_SAVE_DIRECTORY), not to the BIOS directory. emulator: dest is the path the emulator reads in its own folder, which the platform keeps outside the BIOS tree (reason outside_bios_folder). roms: dest is relative to the platform's roms folder (roms_directory). Absent for a file of the BIOS tree.",
|
||||
"enum": [
|
||||
"save",
|
||||
"emulator",
|
||||
"roms"
|
||||
]
|
||||
},
|
||||
"core_folder": {
|
||||
"description": "The core's library_name, the folder a frontend sorting saves by core (RetroArch's sort_savefiles_enable) inserts between its save directory and dest. Only with root: save. Absent on a save entry when the core reads the same place sorted or not.",
|
||||
@@ -515,6 +523,21 @@
|
||||
"const": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"roms_directory": {
|
||||
"description": "Where the platform keeps its roms folder, present when an entry carries root: roms. bios_folder is the name of the BIOS folder inside it: a BIOS folder of that name has the roms folder as its parent, any other leaves root: roms entries out (no_roms_directory).",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"bios_folder"
|
||||
],
|
||||
"properties": {
|
||||
"bios_folder": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^[^/\\\\]+$"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
@@ -533,7 +556,7 @@
|
||||
"dest": {
|
||||
"type": "string",
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$)).+$",
|
||||
"description": "Path below base_destination, or below the save directory when root is save."
|
||||
"description": "Path below base_destination, below the save directory when root is save, or below the roms folder when root is roms."
|
||||
},
|
||||
"sha1": {
|
||||
"type": "string",
|
||||
@@ -1016,8 +1039,11 @@
|
||||
}
|
||||
},
|
||||
"root": {
|
||||
"description": "save: dest is relative to the save directory the frontend hands the core (RETRO_ENVIRONMENT_GET_SAVE_DIRECTORY), not to the BIOS directory. Absent for a file of the BIOS tree.",
|
||||
"const": "save"
|
||||
"description": "save: dest is relative to the save directory the frontend hands the core (RETRO_ENVIRONMENT_GET_SAVE_DIRECTORY), not to the BIOS directory. roms: dest is relative to the platform's roms folder, which roms_directory locates (a file its launcher runs from there, ROCKNIX's roms/pico-8). Absent for a file of the BIOS tree.",
|
||||
"enum": [
|
||||
"save",
|
||||
"roms"
|
||||
]
|
||||
},
|
||||
"core_folder": {
|
||||
"description": "The core's library_name, the folder a frontend sorting saves by core (RetroArch's sort_savefiles_enable) inserts between its save directory and dest. Only with root: save. Absent on a save entry when the core reads the same place sorted or not.",
|
||||
|
||||
+88
-14
@@ -1482,6 +1482,29 @@ def resolve_local_file(
|
||||
system_dir = file_entry.get("system_dir")
|
||||
if system_dir and dest_hint.startswith(f"{system_dir}/"):
|
||||
dest_hint = dest_hint[len(system_dir) + 1:]
|
||||
# A launcher that moves the layout (EmuDeck links azahar's sysdata to
|
||||
# bios/azahar/keys) changes where the file goes, not the tail the
|
||||
# collection indexes it under. Both are asked, and the stronger proof
|
||||
# wins: a content or path match over a name, the destination on a tie.
|
||||
layout = file_entry.get("layout_path")
|
||||
if layout and layout != dest_hint:
|
||||
plain = {k: v for k, v in file_entry.items() if k != "layout_path"}
|
||||
answers = [
|
||||
resolve_local_file(
|
||||
plain, db, zip_contents, dest_hint=hint, _depth=_depth,
|
||||
data_dir_registry=data_dir_registry,
|
||||
)
|
||||
for hint in (dest_hint, str(layout))
|
||||
]
|
||||
for found in answers:
|
||||
if found[0] is not None and (
|
||||
resolution_is_hash_exact(found[1]) or found[1] == "path_exact"
|
||||
):
|
||||
return found
|
||||
for found in answers:
|
||||
if found[0] is not None and found[1] not in ("not_found", "hash_mismatch"):
|
||||
return found
|
||||
return next((found for found in answers if found[0] is not None), answers[0])
|
||||
# A name written with its directory is the destination when no path is
|
||||
# given: gam4980/E.BIN resolved by its basename took another model's OS
|
||||
# in verify --emulator and the emulator pack, while the platform pack,
|
||||
@@ -2624,6 +2647,9 @@ HANDED_PREFIX = "handed:"
|
||||
# Marks, in the same set, a libretro core and the system directory the
|
||||
# platform hands it (``core_system_dirs``), as JSON.
|
||||
CORE_DIR_PREFIX = "coredir:"
|
||||
# Marks, in the same set, a file the platform's launcher executes
|
||||
# (launchers.executed_files), as JSON.
|
||||
EXECUTED_PREFIX = "exec:"
|
||||
|
||||
|
||||
def core_system_dirs(
|
||||
@@ -2689,14 +2715,19 @@ def standalone_set(config: dict) -> set[str]:
|
||||
for core in entry.get("cores") or []
|
||||
}
|
||||
if names - {n for n in names if n.startswith(CORE_DIR_PREFIX)}:
|
||||
from launchers import handed_directories # noqa: PLC0415
|
||||
from launchers import executed_files, handed_directories # noqa: PLC0415
|
||||
|
||||
names |= {
|
||||
HANDED_PREFIX + json.dumps(
|
||||
[h.emulator, h.folder, h.directory, sorted(h.handles), h.source_ref]
|
||||
[h.emulator, h.folder, h.directory, sorted(h.handles), h.source_ref,
|
||||
h.roms_folder]
|
||||
)
|
||||
for h in handed_directories(config)
|
||||
}
|
||||
names |= {
|
||||
EXECUTED_PREFIX + json.dumps([run.emulator, run.name, run.place, run.source_ref])
|
||||
for run in executed_files(config)
|
||||
}
|
||||
return names
|
||||
|
||||
|
||||
@@ -2724,11 +2755,30 @@ def _handed(standalone_cores: Iterable[str]) -> list:
|
||||
for tag in standalone_cores:
|
||||
if not tag.startswith(HANDED_PREFIX):
|
||||
continue
|
||||
emulator, folder, directory, handles, ref = json.loads(tag[len(HANDED_PREFIX):])
|
||||
found.append(HandedDirectory(emulator, folder, directory, frozenset(handles), ref))
|
||||
emulator, folder, directory, handles, ref, roms = json.loads(tag[len(HANDED_PREFIX):])
|
||||
found.append(HandedDirectory(emulator, folder, directory, frozenset(handles), ref, roms))
|
||||
return found
|
||||
|
||||
|
||||
def _executed(standalone_cores: Iterable[str]) -> list:
|
||||
"""The files a standalone set's launchers execute, as launchers read them."""
|
||||
from launchers import ExecutedFile # noqa: PLC0415
|
||||
|
||||
found = []
|
||||
for tag in standalone_cores:
|
||||
if tag.startswith(EXECUTED_PREFIX):
|
||||
emulator, name, place, ref = json.loads(tag[len(EXECUTED_PREFIX):])
|
||||
found.append(ExecutedFile(emulator, name, tuple(place) if place else None, ref))
|
||||
return found
|
||||
|
||||
|
||||
# Why a standalone entry has no destination on a platform that relocates its
|
||||
# layout: the manifest's omission reason, named as the installer words it.
|
||||
OUTSIDE_BIOS_FOLDER = "outside_bios_folder"
|
||||
# The entry root of a file read below the platform's roms directory.
|
||||
ROMS_ROOT = "roms"
|
||||
|
||||
|
||||
def standalone_layout_path(entry: dict, profile: dict) -> str | None:
|
||||
"""The path of an entry relative to the directories its standalone build declares.
|
||||
|
||||
@@ -2744,13 +2794,16 @@ def standalone_layout_path(entry: dict, profile: dict) -> str | None:
|
||||
return entry.get("path") or None
|
||||
|
||||
|
||||
def standalone_destination(entry: dict, profile: dict, bindings: dict[str, str]) -> str | None:
|
||||
"""Where a platform puts a standalone entry, relative to the BIOS root.
|
||||
def standalone_destination(
|
||||
entry: dict, profile: dict, bindings: dict[str, tuple[str, str]]
|
||||
) -> tuple[str, str] | None:
|
||||
"""Where a platform puts a standalone entry: (tree, path).
|
||||
|
||||
``bindings`` are the folders the platform's launchers give the layout
|
||||
``bindings`` are the places the platform's launchers give the layout
|
||||
(launchers.standalone_bindings): RetroPie and ROCKNIX give Flycast bios/dc,
|
||||
EmuDeck bios/flycast/bios, RetroPie gives UAE4All2's kickstarts/ bios/amiga.
|
||||
None when no binding covers the entry: it keeps the profile's layout.
|
||||
EmuDeck bios/flycast/bios, RetroPie gives UAE4All2's kickstarts/ bios/amiga,
|
||||
ROCKNIX runs PICO-8 from roms/pico-8. The tree is launchers.BIOS_TREE or
|
||||
launchers.ROMS_TREE. None when no binding covers the entry.
|
||||
"""
|
||||
layout = standalone_layout_path(entry, profile)
|
||||
if layout is None or not bindings:
|
||||
@@ -2762,15 +2815,21 @@ def standalone_destination(entry: dict, profile: dict, bindings: dict[str, str])
|
||||
|
||||
def standalone_bindings_for(
|
||||
emu_name: str, profile: dict, standalone_cores: Iterable[str]
|
||||
) -> dict[str, str]:
|
||||
"""The folders a platform gives this emulator's standalone layout, by subfolder."""
|
||||
) -> dict[str, tuple[str, str]]:
|
||||
"""The places a platform gives this emulator's standalone layout, by subfolder."""
|
||||
declared = profile.get("standalone_directories") or []
|
||||
if not declared:
|
||||
return {}
|
||||
from launchers import standalone_bindings # noqa: PLC0415
|
||||
|
||||
names = {emu_name, *(str(c) for c in profile.get("cores") or [])}
|
||||
return standalone_bindings(names, declared, _handed(standalone_cores))
|
||||
layout = [
|
||||
path for f in profile.get("files") or []
|
||||
if isinstance(f, dict) and (path := standalone_layout_path(f, profile))
|
||||
]
|
||||
return standalone_bindings(
|
||||
names, declared, _handed(standalone_cores), _executed(standalone_cores), layout
|
||||
)
|
||||
|
||||
|
||||
def platform_layouts(
|
||||
@@ -2794,10 +2853,25 @@ def platform_layouts(
|
||||
# The standalone view carries the platform's destination, so every
|
||||
# reader of standalone_path (builder, manifest, verify, slots) places
|
||||
# the file in the folder the platform hands the emulator.
|
||||
# An entry of the layout no binding covers is read in a folder the
|
||||
# platform keeps outside the BIOS tree: UAE4All2's kick.rom in
|
||||
# RetroPie's $md_inst. It is marked, never left at the BIOS root
|
||||
# where the emulator does not read it.
|
||||
files = []
|
||||
for f in profile.get("files") or []:
|
||||
placed = standalone_destination(f, profile, bindings) if isinstance(f, dict) else None
|
||||
files.append({**f, "standalone_path": placed} if placed else f)
|
||||
if not isinstance(f, dict) or standalone_layout_path(f, profile) is None:
|
||||
files.append(f)
|
||||
continue
|
||||
placed = standalone_destination(f, profile, bindings)
|
||||
if placed is None or placed[0] == "outside":
|
||||
files.append({**f, "unplaced": OUTSIDE_BIOS_FOLDER})
|
||||
continue
|
||||
tree, path = placed
|
||||
moved = {**f, "standalone_path": path,
|
||||
"layout_path": standalone_layout_path(f, profile)}
|
||||
# A file the launcher runs from the roms directory (ROCKNIX's
|
||||
# roms/pico-8) is relative to it, and says so.
|
||||
files.append({**moved, "root": ROMS_ROOT} if tree else moved)
|
||||
standalone = {**profile, "files": files}
|
||||
dual = {c.removeprefix(DUAL_MODE_PREFIX) for c in standalone_cores
|
||||
if c.startswith(DUAL_MODE_PREFIX)}
|
||||
|
||||
@@ -856,7 +856,7 @@ def _cross_reference_profile(
|
||||
# archive.
|
||||
# A platform declares files of its BIOS tree: none answers for one
|
||||
# the core reads below the save directory.
|
||||
in_platform = not save_root and declaration_settles(
|
||||
in_platform = not save_root and f.get("root") != "roms" and declaration_settles(
|
||||
f, fname, platform_names, effective_path, index.get("folded_slots", frozenset())
|
||||
)
|
||||
|
||||
@@ -941,6 +941,10 @@ def _cross_reference_profile(
|
||||
}
|
||||
if save_root:
|
||||
entry["root"] = "save"
|
||||
elif f.get("unplaced"):
|
||||
entry["unplaced"] = f["unplaced"]
|
||||
elif f.get("root") == "roms":
|
||||
entry["root"] = "roms"
|
||||
elif index.get("launcher_candidates") is not None:
|
||||
# What verify hands the launcher layer for the same entry.
|
||||
probe = {
|
||||
|
||||
+48
-11
@@ -38,6 +38,7 @@ from pathlib import Path, PurePosixPath
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import resolve_platform_cores as _platform_cores
|
||||
from common import (
|
||||
ROMS_ROOT,
|
||||
installed_data_refs,
|
||||
platform_arg,
|
||||
require_directory,
|
||||
@@ -1302,6 +1303,10 @@ def generate_pack(
|
||||
continue
|
||||
if not dest:
|
||||
continue
|
||||
if fe.get("root") == ROMS_ROOT:
|
||||
# Read below the roms directory: the pack is extracted into
|
||||
# the BIOS folder and cannot reach it, the installer places it.
|
||||
continue
|
||||
# Core extras: _collect_emulator_extras already adjusted
|
||||
# destinations for slug-based platforms. Apply the effective
|
||||
# prefix (base_dest, or inferred from YAML when base_dest is
|
||||
@@ -4256,7 +4261,8 @@ def _attach_claims(
|
||||
})
|
||||
continue
|
||||
claim_path, _ = validated_choice(
|
||||
claim, claim_path, db, validation_index or {}, bios_dir, None, dest, owners,
|
||||
claim, claim_path, db, validation_index or {}, bios_dir, None,
|
||||
claim.get("layout_path") or dest, owners,
|
||||
)
|
||||
copy, _size = _manifest_entry(
|
||||
claim_path, manifest_dest, [claim_emu], db, repo_root, {}
|
||||
@@ -4447,9 +4453,14 @@ def _manifest_core_entries(
|
||||
# extras prefix when it is an internal layout dir (RetroDECK bios/).
|
||||
# A file read below the save directory is written relative to the
|
||||
# directory the core is handed, and says so (save_entry_fields).
|
||||
manifest_dest = (
|
||||
save_manifest_dest(fe) if save_root else _manifest_dest_of(full_dest, base_dest)
|
||||
)
|
||||
roms_root = fe.get("root") == ROMS_ROOT
|
||||
if save_root:
|
||||
manifest_dest = save_manifest_dest(fe)
|
||||
elif roms_root:
|
||||
# Relative to the roms directory, which the entry names.
|
||||
manifest_dest = dest
|
||||
else:
|
||||
manifest_dest = _manifest_dest_of(full_dest, base_dest)
|
||||
|
||||
entry, file_size = _manifest_entry(
|
||||
local_path, manifest_dest, [source_emu] if source_emu else [],
|
||||
@@ -4458,6 +4469,8 @@ def _manifest_core_entries(
|
||||
)
|
||||
if entry is not None and save_root:
|
||||
entry.update(save_entry_fields(fe))
|
||||
elif entry is not None and roms_root:
|
||||
entry["root"] = ROMS_ROOT
|
||||
if entry is None:
|
||||
if not _claim_platform_omission(
|
||||
omitted_by_destination.get(full_dest), source_emu
|
||||
@@ -4624,10 +4637,18 @@ def _record_unplaceable(unplaceable: list[dict], record_omission: Callable[...,
|
||||
"""
|
||||
for u in unplaceable:
|
||||
where = u.get("path") or u.get("name", "")
|
||||
if u.get("unplaced"):
|
||||
key = f"emulator:{u.get('profile') or u.get('emulator', '')}:{where}"
|
||||
elif u.get("root") == ROMS_ROOT:
|
||||
key = f"roms:{where}"
|
||||
elif u.get("root") == "save":
|
||||
key = f"save:{save_manifest_dest(u)}"
|
||||
else:
|
||||
key = where
|
||||
record_omission(
|
||||
# Keyed apart from the BIOS tree's paths, which a file read below
|
||||
# the save directory can repeat.
|
||||
f"save:{save_manifest_dest(u)}" if u.get("root") == "save" else where,
|
||||
# the save directory or in the emulator's folder can repeat.
|
||||
key,
|
||||
u, u.get("system") or "",
|
||||
# The profile key, the name a target's core list and the
|
||||
# installer's narrowing match: the display name matched none.
|
||||
@@ -4805,12 +4826,22 @@ def generate_manifest(
|
||||
if cores is None else extra_region_key(file_entry, declared_root(config))
|
||||
)
|
||||
save_root = file_entry.get("root") == "save"
|
||||
# Read in the emulator's own folder, outside the BIOS tree: dest is
|
||||
# the path the emulator reads, relative to that folder.
|
||||
outside = bool(file_entry.get("unplaced"))
|
||||
if outside:
|
||||
dest_fields: dict = {"dest": sanitize_pack_path(file_entry.get("path") or ""),
|
||||
"root": "emulator"}
|
||||
elif file_entry.get("root") == ROMS_ROOT:
|
||||
dest_fields = {"dest": sanitize_pack_path(
|
||||
file_entry.get("destination") or file_entry.get("path") or ""
|
||||
), "root": ROMS_ROOT}
|
||||
elif save_root:
|
||||
dest_fields = {"dest": save_manifest_dest(file_entry), **save_entry_fields(file_entry)}
|
||||
else:
|
||||
dest_fields = {"dest": manifest_destination(full_destination)}
|
||||
omitted_by_destination[full_destination] = {
|
||||
"dest": (
|
||||
save_manifest_dest(file_entry) if save_root
|
||||
else manifest_destination(full_destination)
|
||||
),
|
||||
**(save_entry_fields(file_entry) if save_root else {}),
|
||||
**dest_fields,
|
||||
"name": str(file_entry.get("name") or ""),
|
||||
"system": system_id,
|
||||
"required": bool(file_entry.get("required", True)),
|
||||
@@ -5164,6 +5195,12 @@ def generate_manifest(
|
||||
}
|
||||
if data_directories:
|
||||
result["data_directories"] = data_directories
|
||||
roms = config.get("roms_directory") or {}
|
||||
if any(entry.get("root") == ROMS_ROOT
|
||||
for entry in [*manifest_files, *result["omitted_files"]]) and roms:
|
||||
# Where the installer finds the roms directory: the folder holding
|
||||
# the BIOS folder, which the platform names bios_folder.
|
||||
result["roms_directory"] = {"bios_folder": str(roms.get("bios_folder") or "")}
|
||||
saved = [entry for entry in manifest_files if entry.get("root") == "save"]
|
||||
if saved:
|
||||
layout = save_layout(config)
|
||||
|
||||
+138
-24
@@ -216,6 +216,18 @@ HANDED_KINDS = ("links", "passes_path")
|
||||
# that directory in standalone_directories: the call its code reads it with.
|
||||
WORKING_DIRECTORY_PASSED = "working directory of "
|
||||
WORKING_DIRECTORY = "getcwd"
|
||||
# The handle a profile declares when its program reads its files beside its
|
||||
# own executable: a launcher record executing one of the profile's files
|
||||
# places the folder holding that file where the launcher runs it from.
|
||||
EXECUTABLE_DIRECTORY = "executable directory"
|
||||
|
||||
# The trees a placed folder belongs to: the BIOS tree every destination is
|
||||
# relative to, and the roms directory of a platform whose registry names it.
|
||||
BIOS_TREE = ""
|
||||
ROMS_TREE = "roms"
|
||||
# A folder of neither tree, which no pack and no installer can fill.
|
||||
OUTSIDE_TREE = "outside"
|
||||
Place = tuple[str, str]
|
||||
|
||||
_HOME = re.compile(r"^(?:\$\{HOME\}|\$HOME|\$home|~)(?=/|$)")
|
||||
# Flatpak sets XDG_DATA_HOME to ~/.var/app/<id>/data inside the sandbox.
|
||||
@@ -264,6 +276,15 @@ class HandedDirectory:
|
||||
directory: str
|
||||
handles: frozenset[str]
|
||||
source_ref: str
|
||||
roms_folder: str | None = None
|
||||
|
||||
def place(self) -> Place | None:
|
||||
"""The folder as a place, None when it lies outside both roots."""
|
||||
if self.folder is not None:
|
||||
return (BIOS_TREE, self.folder)
|
||||
if self.roms_folder is not None:
|
||||
return (ROMS_TREE, self.roms_folder)
|
||||
return None
|
||||
|
||||
|
||||
def handed_directories(config: dict) -> list[HandedDirectory]:
|
||||
@@ -295,37 +316,110 @@ def handed_directories(config: dict) -> list[HandedDirectory]:
|
||||
for token in passes_to.split()
|
||||
if (handle := normalize_handle(token)) and handle != directory
|
||||
)
|
||||
folder = _folder_in_bios(raw, base)
|
||||
for emulator in (e.strip() for e in str(record.get("emulator", "")).split(",")):
|
||||
found.append(HandedDirectory(
|
||||
emulator=emulator,
|
||||
folder=_folder_in_bios(raw, base),
|
||||
folder=folder,
|
||||
directory=directory,
|
||||
handles=handles,
|
||||
source_ref=str(record.get("source_ref", "")),
|
||||
roms_folder=None if folder is not None else folder_in_roms(raw, config),
|
||||
))
|
||||
return found
|
||||
|
||||
|
||||
def folder_in_roms(directory: str, config: dict) -> str | None:
|
||||
"""A directory relative to the platform's roms directory, None outside it.
|
||||
|
||||
Only a platform whose registry names the roms directory has one
|
||||
(``roms_directory.path``): ROCKNIX's /storage/roms, which holds its BIOS
|
||||
folder and the folders its launchers run programs from.
|
||||
"""
|
||||
roms = str((config.get("roms_directory") or {}).get("path") or "").rstrip("/")
|
||||
directory = directory.strip().rstrip("/")
|
||||
if not roms or not directory.startswith(roms + "/"):
|
||||
return None
|
||||
rest = directory[len(roms) + 1:]
|
||||
return None if re.search(r"[\s<>$:;]", rest) else rest
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecutedFile:
|
||||
"""A file a platform's launcher executes, and where it lies.
|
||||
|
||||
``place`` is (root, path): the BIOS tree or the roms directory, None
|
||||
when the file lies outside both (RetroBat's emulator folders).
|
||||
"""
|
||||
|
||||
emulator: str
|
||||
name: str
|
||||
place: Place | None
|
||||
source_ref: str
|
||||
|
||||
|
||||
def executed_files(config: dict) -> list[ExecutedFile]:
|
||||
"""Every file a platform's launchers execute, from records passing it to exec."""
|
||||
base = str(config.get("base_destination") or "bios")
|
||||
found: list[ExecutedFile] = []
|
||||
for record in config.get("launchers") or []:
|
||||
if record.get("kind") not in ("fixed_name", "names_searched"):
|
||||
continue
|
||||
if str(record.get("passes_to") or "").strip() != "exec":
|
||||
continue
|
||||
raw = str(record.get("search_dir") or "").strip()
|
||||
directories = _directories(record, base)
|
||||
roms = folder_in_roms(raw, config) if raw and not directories else None
|
||||
# A program file needs no extension (pico8, vox): the names are
|
||||
# taken as written, placeholders left out.
|
||||
for name in (str(n).strip() for n in record.get("names") or []):
|
||||
if not name or re.search(r"[<>{}]", name):
|
||||
continue
|
||||
if directories:
|
||||
place: Place | None = (BIOS_TREE, sanitize_pack_path(
|
||||
f"{directories[0]}/{name}" if directories[0] else name
|
||||
))
|
||||
elif roms is not None:
|
||||
place = (ROMS_TREE, sanitize_pack_path(f"{roms}/{name}" if roms else name))
|
||||
else:
|
||||
place = None
|
||||
for emulator in (e.strip() for e in str(record.get("emulator", "")).split(",")):
|
||||
found.append(ExecutedFile(
|
||||
emulator, name.rsplit("/", 1)[-1], place, str(record.get("source_ref", ""))
|
||||
))
|
||||
return found
|
||||
|
||||
|
||||
def standalone_bindings(
|
||||
names: Iterable[str], declared: Iterable[str], handed: Iterable[HandedDirectory]
|
||||
) -> dict[str, str]:
|
||||
"""The folders of the BIOS tree a platform gives a standalone build's layout.
|
||||
names: Iterable[str],
|
||||
declared: Iterable[str],
|
||||
handed: Iterable[HandedDirectory],
|
||||
executed: Iterable[ExecutedFile] = (),
|
||||
layout: Iterable[str] = (),
|
||||
) -> dict[str, Place]:
|
||||
"""The folders a platform gives a standalone build's layout.
|
||||
|
||||
``declared`` are the directories the build resolves its paths against,
|
||||
as its profile names them (``standalone_directories``). The result maps
|
||||
a subfolder of that layout ("" for the whole of it) to the folder of the
|
||||
BIOS tree the emulator reads it from.
|
||||
a subfolder of that layout ("" for the whole of it) to the place, a
|
||||
folder of the BIOS tree or of the roms directory, the emulator reads it
|
||||
from.
|
||||
|
||||
A record handing a folder of the tree under a declared directory gives
|
||||
it directly: RetroPie points Flycast's ~/.local/share/flycast at bios/dc.
|
||||
A record handing a directory outside the tree is followed through the
|
||||
links that put a folder of the tree at it or below it: ROCKNIX's patched
|
||||
Flycast uses /storage/roms/dreamcast/data, which its start script links
|
||||
to bios/dc, and RetroPie starts UAE4All2 in $md_inst, whose kickstarts
|
||||
folder it links to bios/amiga.
|
||||
A record handing a folder under a declared directory gives it directly:
|
||||
RetroPie points Flycast's ~/.local/share/flycast at bios/dc. A record
|
||||
handing a directory outside the trees is followed through the links that
|
||||
put a folder at it or below it: ROCKNIX's patched Flycast uses
|
||||
/storage/roms/dreamcast/data, which its start script links to bios/dc,
|
||||
and RetroPie starts UAE4All2 in $md_inst, whose kickstarts folder it
|
||||
links to bios/amiga.
|
||||
|
||||
A subfolder two records give two folders is left out: its files keep
|
||||
the profile's layout, relative to the BIOS root.
|
||||
A profile whose program reads its files beside its executable declares
|
||||
EXECUTABLE_DIRECTORY, and ``layout`` holds its paths: a launcher that
|
||||
executes one of them gives the folder holding it. Batocera runs
|
||||
pico-8/pico8 from its BIOS folder, ROCKNIX pico8_64 from
|
||||
/storage/roms/pico-8.
|
||||
|
||||
A subfolder two records give two places is left out.
|
||||
"""
|
||||
names = set(names)
|
||||
wanted = {normalize_handle(d) for d in declared}
|
||||
@@ -340,7 +434,7 @@ def standalone_bindings(
|
||||
return handle[len(directory) + 1:]
|
||||
return None
|
||||
|
||||
found: dict[str, set[str]] = {}
|
||||
found: dict[str, set[Place]] = {}
|
||||
for record in records:
|
||||
subs = {
|
||||
sub for handle in record.handles for directory in wanted
|
||||
@@ -348,21 +442,41 @@ def standalone_bindings(
|
||||
}
|
||||
for sub in subs:
|
||||
if record.folder is not None:
|
||||
found.setdefault(sub, set()).add(record.folder)
|
||||
found.setdefault(sub, set()).add((BIOS_TREE, record.folder))
|
||||
continue
|
||||
# A link into the BIOS tree comes first: the pack fills that.
|
||||
linked = False
|
||||
for link in records:
|
||||
if link.folder is None:
|
||||
target = link.place()
|
||||
if target is None:
|
||||
continue
|
||||
for handle in link.handles:
|
||||
rest = below(handle, record.directory)
|
||||
if rest is not None:
|
||||
key = "/".join(p for p in (sub, rest) if p)
|
||||
found.setdefault(key, set()).add(link.folder)
|
||||
return {sub: next(iter(folders)) for sub, folders in found.items() if len(folders) == 1}
|
||||
found.setdefault(key, set()).add(target)
|
||||
linked = True
|
||||
if not linked and record.roms_folder is not None:
|
||||
found.setdefault(sub, set()).add((ROMS_TREE, record.roms_folder))
|
||||
if EXECUTABLE_DIRECTORY in wanted:
|
||||
paths = list(layout)
|
||||
for run in executed:
|
||||
if run.emulator not in names:
|
||||
continue
|
||||
# A program run from a folder outside both trees (RetroBat's
|
||||
# emulators/pico8) binds its folder to no place a pack fills.
|
||||
root, path = run.place or (OUTSIDE_TREE, run.name)
|
||||
folder = path.rsplit("/", 1)[0] if "/" in path else ""
|
||||
for entry_path in paths:
|
||||
if entry_path.rsplit("/", 1)[-1] != run.name:
|
||||
continue
|
||||
sub = entry_path.rsplit("/", 1)[0] if "/" in entry_path else ""
|
||||
found.setdefault(sub, set()).add((root, folder))
|
||||
return {sub: next(iter(places)) for sub, places in found.items() if len(places) == 1}
|
||||
|
||||
|
||||
def bound_path(path: str, bindings: dict[str, str]) -> str | None:
|
||||
"""Where the BIOS tree holds a layout path, None when no binding covers it.
|
||||
def bound_path(path: str, bindings: dict[str, Place]) -> Place | None:
|
||||
"""Where a platform holds a layout path, None when no binding covers it.
|
||||
|
||||
The longest subfolder that contains the path wins: a link of
|
||||
kickstarts/ places kickstarts/kick13.rom even when the whole layout is
|
||||
@@ -372,8 +486,8 @@ def bound_path(path: str, bindings: dict[str, str]) -> str | None:
|
||||
if sub and not path.startswith(sub + "/"):
|
||||
continue
|
||||
rest = path[len(sub) + 1:] if sub else path
|
||||
folder = bindings[sub]
|
||||
return f"{folder}/{rest}" if folder else rest
|
||||
root, folder = bindings[sub]
|
||||
return (root, f"{folder}/{rest}" if folder else rest)
|
||||
return None
|
||||
|
||||
|
||||
|
||||
+48
-6
@@ -17,7 +17,7 @@ from common import resolve_local_file
|
||||
from common import resolve_platform_cores
|
||||
from common import core_system_dirs, platform_layouts, system_dir_path
|
||||
from common import standalone_set as platform_standalone_set
|
||||
from common import sanitize_pack_path
|
||||
from common import ROMS_ROOT, sanitize_pack_path
|
||||
from common import size_fits
|
||||
from validation import deliverable, read_from_save_dir, read_from_system_dir
|
||||
from packpaths import libretro_systems, save_destination, save_layout
|
||||
@@ -43,11 +43,24 @@ def extra_full_destination(fe: dict, prefix: str) -> str:
|
||||
"""
|
||||
if fe.get("root") == "save":
|
||||
return str(fe.get("pack_path") or "")
|
||||
if fe.get("root") == ROMS_ROOT:
|
||||
return roms_destination(sanitize_pack_path(fe.get("destination", fe.get("name", ""))))
|
||||
return prefixed_destination(
|
||||
sanitize_pack_path(fe.get("destination", fe.get("name", ""))), prefix
|
||||
)
|
||||
|
||||
|
||||
# Keys a destination below the roms directory apart from the BIOS tree's
|
||||
# paths. Never written into a pack: the pack is extracted into the BIOS
|
||||
# folder, and the installer places these files (``root: roms``).
|
||||
ROMS_KEY = "roms:"
|
||||
|
||||
|
||||
def roms_destination(dest: str) -> str:
|
||||
"""The key of a destination below the roms directory."""
|
||||
return f"{ROMS_KEY}{dest}"
|
||||
|
||||
|
||||
def prefixed_destination(dest: str, prefix: str) -> str:
|
||||
"""A core extra's destination from the pack root.
|
||||
|
||||
@@ -309,9 +322,11 @@ class ExtraResolver:
|
||||
return asset, "release_asset", ""
|
||||
if status in ("not_found", "external", "user_provided") or not local_path:
|
||||
return None, status, ""
|
||||
# The cores reading a moved layout's file are found under the
|
||||
# emulator's own path, not the folder the platform put it in.
|
||||
local_path, disagreement = validated_choice(
|
||||
fe, local_path, self.db, self.validation_index, self.bios_dir, None,
|
||||
dest, self.validation_owners,
|
||||
fe.get("layout_path") or dest, self.validation_owners,
|
||||
)
|
||||
# The platform never reads the bytes, the core does: a copy it turns
|
||||
# away by its own size check is not its file, on any platform.
|
||||
@@ -462,9 +477,10 @@ def unplaceable_extras(
|
||||
placement = ExtraPlacement(config, "", profiles)
|
||||
unplaced: dict[tuple[str, str, str], str] = {}
|
||||
for u in undeclared:
|
||||
if not u.get("in_repo"):
|
||||
if not u.get("in_repo") and not u.get("unplaced"):
|
||||
continue
|
||||
if not placement.slug_based and u.get("root") != "save":
|
||||
if (not placement.slug_based and u.get("root") not in ("save", ROMS_ROOT)
|
||||
and not u.get("unplaced")):
|
||||
continue
|
||||
dest = _extra_dest(u)
|
||||
if not placement.places(u, dest):
|
||||
@@ -550,7 +566,8 @@ def _agnostic_scan_extras(
|
||||
continue
|
||||
# The scan walks the BIOS tree: a file read below the save
|
||||
# directory is placed by the gap report's pass.
|
||||
if read_from_save_dir(f, standalone):
|
||||
if (read_from_save_dir(f, standalone) or f.get("unplaced")
|
||||
or f.get("root") == ROMS_ROOT):
|
||||
continue
|
||||
# An agnostic BIOS mode says the BIOS filename is free, not that
|
||||
# every file the emulator loads is. A free filename still has a
|
||||
@@ -1001,6 +1018,9 @@ class ExtraPlacement:
|
||||
self.native_systems = native_slug_systems(config)
|
||||
self.flat = bool(config.get("firmware_folder_flat"))
|
||||
self.save = save_layout(config)
|
||||
# The roms directory the registry names, for a file a launcher runs
|
||||
# from it (ROCKNIX's roms/pico-8).
|
||||
self.roms = bool((config.get("roms_directory") or {}).get("path"))
|
||||
self.config = config
|
||||
self.norm_map = (
|
||||
{_norm_system_id(sid): sid for sid in self.platform_systems}
|
||||
@@ -1019,6 +1039,10 @@ class ExtraPlacement:
|
||||
directory and goes below the platform's save folder, which a
|
||||
platform keeping saves beside the content does not have.
|
||||
"""
|
||||
if entry.get("unplaced"):
|
||||
return None
|
||||
if entry.get("root") == ROMS_ROOT:
|
||||
return (dest, roms_destination(dest)) if self.roms else None
|
||||
if entry.get("root") == "save":
|
||||
placed = self.save_places(entry, dest)
|
||||
return placed[0] if placed else None
|
||||
@@ -1039,6 +1063,8 @@ class ExtraPlacement:
|
||||
platform that hands each system a save folder of its own goes once
|
||||
below each system that runs the core.
|
||||
"""
|
||||
if entry.get("unplaced"):
|
||||
return []
|
||||
if entry.get("root") == "save":
|
||||
return self.save_places(entry, dest)
|
||||
placed = self.place(entry, dest)
|
||||
@@ -1075,6 +1101,10 @@ class ExtraPlacement:
|
||||
|
||||
def unplaced_reason(self, entry: dict, dest: str) -> str:
|
||||
"""Why ``place`` returned None for this entry."""
|
||||
if entry.get("unplaced"):
|
||||
return str(entry["unplaced"])
|
||||
if entry.get("root") == ROMS_ROOT:
|
||||
return "no_roms_directory"
|
||||
if entry.get("root") == "save":
|
||||
if self.save is None:
|
||||
return "no_save_directory"
|
||||
@@ -1196,6 +1226,12 @@ def _collect_emulator_extras(
|
||||
# user's machine. The mode a file is bundled for was settled
|
||||
# upstream, an entry scoped to another build never gets here.
|
||||
continue
|
||||
if u.get("unplaced"):
|
||||
# Read in a folder outside the BIOS tree: named, held or not,
|
||||
# never written where the emulator does not read it.
|
||||
if unplaceable is not None:
|
||||
unplaceable.append({**u, "unplaceable": u["unplaced"]})
|
||||
continue
|
||||
if not u["in_repo"] and absent is None:
|
||||
continue
|
||||
# For archive entries, use the archive name for resolution
|
||||
@@ -1291,7 +1327,9 @@ def _collect_emulator_extras(
|
||||
# Only duplicate files already covered at another destination.
|
||||
# An unsourceable entry is a file no collection holds: the name
|
||||
# covered elsewhere belongs to another program's file.
|
||||
if fname not in covered_names or f.get("unsourceable"):
|
||||
# Nor is one placed outside the BIOS tree by the layout.
|
||||
if (fname not in covered_names or f.get("unsourceable") or f.get("unplaced")
|
||||
or f.get("root") == ROMS_ROOT):
|
||||
continue
|
||||
# Installed with the emulator or by the platform's package,
|
||||
# as the first pass already decided for the same entry.
|
||||
@@ -1530,8 +1568,12 @@ def _report_extra(u: dict, name: str, dest: str) -> dict:
|
||||
extra["root"] = "save"
|
||||
if u.get("core_folder") is not None:
|
||||
extra["core_folder"] = u["core_folder"]
|
||||
elif u.get("root") == ROMS_ROOT:
|
||||
extra["root"] = ROMS_ROOT
|
||||
if u.get("system_dir"):
|
||||
extra["system_dir"] = u["system_dir"]
|
||||
if u.get("layout_path"):
|
||||
extra["layout_path"] = u["layout_path"]
|
||||
# The archive layout the core opens ranks same-named copies by what
|
||||
# they hold. Without it the first copy indexed won.
|
||||
if u.get("contents"):
|
||||
|
||||
+2
-1
@@ -228,7 +228,7 @@ def profile_claims(
|
||||
# outside the BIOS tree, where no platform declares a file.
|
||||
if outside_gap_scope(entry, is_standalone) or read_from_save_dir(
|
||||
entry, is_standalone
|
||||
):
|
||||
) or entry.get("unplaced"):
|
||||
continue
|
||||
# A member of a romset is not a file at the BIOS root: the archive
|
||||
# holding it occupies the destination (_archive_claims). Claiming
|
||||
@@ -306,6 +306,7 @@ def _archive_claims(
|
||||
if isinstance(entry, dict)
|
||||
and not outside_gap_scope(entry, is_standalone)
|
||||
and not read_from_save_dir(entry, is_standalone)
|
||||
and not entry.get("unplaced")
|
||||
]
|
||||
prefix = profile.get("archive_prefix", "")
|
||||
claims: list[Claim] = []
|
||||
|
||||
+9
-7
@@ -427,13 +427,15 @@ def _has_exploitable_data(entry: dict) -> bool:
|
||||
|
||||
|
||||
def _system_dir_files(profile: dict, standalone: bool) -> list[dict]:
|
||||
"""Entries of the build the platform runs, read from the system directory."""
|
||||
return list(
|
||||
filter(
|
||||
read_from_system_dir,
|
||||
filter_files_by_mode(profile.get("files", []), standalone=standalone),
|
||||
)
|
||||
)
|
||||
"""Entries of the build the platform runs, read from the system directory.
|
||||
|
||||
An entry the platform's launchers leave outside the BIOS tree
|
||||
(``unplaced``) is not: the platform has no file of its own there.
|
||||
"""
|
||||
return [
|
||||
f for f in filter_files_by_mode(profile.get("files", []), standalone=standalone)
|
||||
if read_from_system_dir(f) and not f.get("unplaced")
|
||||
]
|
||||
|
||||
|
||||
def _archives_not_members(files: list[dict]) -> list[dict]:
|
||||
|
||||
+23
-3
@@ -592,7 +592,10 @@ def find_undeclared_files(
|
||||
# Below the save directory each core is handed a folder
|
||||
# of its own: two cores reading hiscore.dat there read
|
||||
# two files.
|
||||
("save", core_save_folder(profile)) if save_root else None,
|
||||
("save", core_save_folder(profile)) if save_root
|
||||
else ("roms",) if f.get("root") == "roms"
|
||||
# In the emulator's own folder: no other core reads it.
|
||||
else ("emulator", emu_name) if f.get("unplaced") else None,
|
||||
)
|
||||
if not fname or unread_on(f, config.get("_key", "")):
|
||||
continue
|
||||
@@ -600,9 +603,11 @@ def find_undeclared_files(
|
||||
duplicate = seen_key in seen_files
|
||||
if duplicate and (first is None or first["profile"] == emu_name):
|
||||
continue
|
||||
# Nor does it answer for one read below the roms directory.
|
||||
verdict = _candidate_verdict(
|
||||
f, fname, is_standalone, include_all,
|
||||
{} if save_root else declared_names, dest, folded_slots,
|
||||
{} if save_root or f.get("root") == "roms" else declared_names,
|
||||
dest, folded_slots,
|
||||
)
|
||||
if verdict == "settled":
|
||||
seen_files.add(seen_key)
|
||||
@@ -759,7 +764,14 @@ def find_undeclared_files(
|
||||
members = with_archive_members(f, profile.get("files", [])).get("members")
|
||||
if members:
|
||||
entry["members"] = members
|
||||
if launcher_layer and not save_root:
|
||||
if f.get("layout_path"):
|
||||
entry["layout_path"] = f["layout_path"]
|
||||
if f.get("unplaced"):
|
||||
entry["unplaced"] = f["unplaced"]
|
||||
elif f.get("root") == "roms":
|
||||
# Read below the roms directory (ROCKNIX's roms/pico-8).
|
||||
entry["root"] = "roms"
|
||||
elif launcher_layer and not save_root:
|
||||
launcher_candidates.append((emu_name, is_standalone, f, entry))
|
||||
if duplicate:
|
||||
first.setdefault("other_claims", []).append(entry)
|
||||
@@ -867,6 +879,8 @@ def find_exclusion_notes(
|
||||
|
||||
|
||||
_UNPLACED_REASONS = {
|
||||
"outside_bios_folder": "read in a folder the platform keeps outside the BIOS tree",
|
||||
"no_roms_directory": "read below a roms folder the platform's registry does not name",
|
||||
"no_platform_slug": "no platform slug for its system",
|
||||
"below_platform_folder": "below the folder the platform lists",
|
||||
"no_save_directory": "read below a save directory the platform keeps in no fixed place",
|
||||
@@ -890,6 +904,12 @@ def _mark_unplaced(config: dict, undeclared: list[dict], profiles: dict) -> None
|
||||
reason = unplaced.get(
|
||||
(u.get("emulator", ""), u.get("name", ""), u.get("path") or "")
|
||||
)
|
||||
if not reason and u.get("root") == "roms":
|
||||
# The pack is extracted into the BIOS folder: the installer
|
||||
# writes this file into the roms folder, the pack cannot.
|
||||
u["in_pack"] = False
|
||||
u["omitted"] = "read below the roms folder, which only the installer reaches"
|
||||
continue
|
||||
if reason:
|
||||
u["in_pack"] = False
|
||||
u["omitted"] = _UNPLACED_REASONS[reason]
|
||||
|
||||
@@ -56,20 +56,21 @@ class ALinkMovesTheDestination(unittest.TestCase):
|
||||
self.addCleanup(shutil.rmtree, self.root, True)
|
||||
bios = self.root / "bios" / "Commodore" / "Amiga"
|
||||
bios.mkdir(parents=True)
|
||||
data = b"KICKSTART 1.3"
|
||||
(bios / "kick13.rom").write_bytes(data)
|
||||
sha1 = hashlib.sha1(data).hexdigest()
|
||||
md5 = hashlib.md5(data).hexdigest()
|
||||
files, by_md5, by_name = {}, {}, {}
|
||||
for name, data in (("kick13.rom", b"KICKSTART 1.3"), ("kick.rom", b"CUSTOM KICKSTART")):
|
||||
(bios / name).write_bytes(data)
|
||||
sha1 = hashlib.sha1(data).hexdigest()
|
||||
md5 = hashlib.md5(data).hexdigest()
|
||||
files[sha1] = {"path": str(bios / name), "name": name, "sha1": sha1,
|
||||
"md5": md5, "crc32": "", "size": len(data)}
|
||||
by_md5[md5] = sha1
|
||||
by_name[name] = [sha1]
|
||||
self.db = {
|
||||
"files": {sha1: {
|
||||
"path": str(bios / "kick13.rom"), "name": "kick13.rom", "sha1": sha1,
|
||||
"md5": md5, "crc32": "", "size": len(data),
|
||||
}},
|
||||
"indexes": {
|
||||
"by_md5": {md5: sha1}, "by_name": {"kick13.rom": [sha1]},
|
||||
"by_crc32": {}, "by_path_suffix": {},
|
||||
},
|
||||
"files": files,
|
||||
"indexes": {"by_md5": by_md5, "by_name": by_name, "by_crc32": {},
|
||||
"by_path_suffix": {}},
|
||||
}
|
||||
kick13 = by_name["kick13.rom"][0]
|
||||
self.platforms = self.root / "platforms"
|
||||
self.emulators = self.root / "emulators"
|
||||
self.platforms.mkdir()
|
||||
@@ -88,11 +89,16 @@ class ALinkMovesTheDestination(unittest.TestCase):
|
||||
"emulator": "Kick", "type": "standalone", "cores": ["kick"],
|
||||
"systems": ["commodore-amiga"],
|
||||
"standalone_directories": ["getcwd"],
|
||||
"files": [{"name": "kick13.rom", "path": "kickstarts/kick13.rom",
|
||||
"sha1": sha1, "required": False}],
|
||||
"files": [
|
||||
{"name": "kick13.rom", "path": "kickstarts/kick13.rom",
|
||||
"sha1": kick13, "required": False},
|
||||
# Read in the working directory itself, which the platform
|
||||
# links nowhere in the BIOS tree.
|
||||
{"name": "kick.rom", "path": "kick.rom", "required": False},
|
||||
],
|
||||
}))
|
||||
|
||||
def pack_names(self, platform: str) -> list[str]:
|
||||
def pack_names(self, platform: str, name: str = "kick13.rom") -> list[str]:
|
||||
with redirect_stdout(io.StringIO()):
|
||||
zip_path = generate_pack(
|
||||
platform, str(self.platforms), self.db, str(self.root / "bios"),
|
||||
@@ -100,21 +106,27 @@ class ALinkMovesTheDestination(unittest.TestCase):
|
||||
offline=True,
|
||||
)
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
return [n for n in zf.namelist() if n.endswith("kick13.rom")]
|
||||
return [n for n in zf.namelist() if n.rsplit("/", 1)[-1] == name]
|
||||
|
||||
def manifest_dests(self, platform: str) -> list[str]:
|
||||
def manifest(self, platform: str) -> dict:
|
||||
with redirect_stdout(io.StringIO()):
|
||||
manifest = generate_manifest(
|
||||
return generate_manifest(
|
||||
platform, str(self.platforms), self.db, str(self.root / "bios"),
|
||||
str(self.platforms / "_registry.yml"),
|
||||
emulators_dir=str(self.emulators), offline=True,
|
||||
)
|
||||
return [f["dest"] for f in manifest["files"] if f["dest"].endswith("kick13.rom")]
|
||||
|
||||
def verify_paths(self, platform: str) -> list[str]:
|
||||
def manifest_dests(self, platform: str, name: str = "kick13.rom") -> list[str]:
|
||||
return [f["dest"] for f in self.manifest(platform)["files"]
|
||||
if f["dest"].rsplit("/", 1)[-1] == name]
|
||||
|
||||
def verify_entries(self, platform: str, name: str = "kick13.rom") -> list[dict]:
|
||||
config = load_platform_config(platform, str(self.platforms))
|
||||
result = verify_platform(config, self.db, str(self.emulators))
|
||||
return [u["path"] for u in result["undeclared_files"] if u["name"] == "kick13.rom"]
|
||||
return [u for u in result["undeclared_files"] if u["name"] == name]
|
||||
|
||||
def verify_paths(self, platform: str) -> list[str]:
|
||||
return [u["path"] for u in self.verify_entries(platform)]
|
||||
|
||||
def test_the_link_moves_it_in_pack_manifest_and_verify(self) -> None:
|
||||
self.assertEqual(self.pack_names("linkplat"), ["amiga/kick13.rom"])
|
||||
@@ -126,6 +138,59 @@ class ALinkMovesTheDestination(unittest.TestCase):
|
||||
self.assertEqual(self.manifest_dests("plainplat"), ["kickstarts/kick13.rom"])
|
||||
self.assertEqual(self.verify_paths("plainplat"), ["kickstarts/kick13.rom"])
|
||||
|
||||
def test_a_file_outside_every_binding_is_named_never_placed(self) -> None:
|
||||
self.assertEqual(self.pack_names("linkplat", "kick.rom"), [])
|
||||
manifest = self.manifest("linkplat")
|
||||
self.assertEqual(self.manifest_dests("linkplat", "kick.rom"), [])
|
||||
omitted = [o for o in manifest["omitted_files"] if o["name"] == "kick.rom"]
|
||||
self.assertEqual(len(omitted), 1)
|
||||
self.assertEqual(
|
||||
(omitted[0]["dest"], omitted[0]["root"], omitted[0]["reason"], omitted[0]["cores"]),
|
||||
("kick.rom", "emulator", "outside_bios_folder", ["kick"]),
|
||||
)
|
||||
(entry,) = self.verify_entries("linkplat", "kick.rom")
|
||||
self.assertFalse(entry["in_pack"])
|
||||
self.assertIn("outside the BIOS tree", entry["omitted"])
|
||||
|
||||
def test_a_root_file_of_another_core_is_not_claimed_by_it(self) -> None:
|
||||
# Another core reads kick.rom at the BIOS root: kick's own kick.rom,
|
||||
# in its working directory, is a different place.
|
||||
for name in ("linkplat", "plainplat"):
|
||||
path = self.platforms / f"{name}.yml"
|
||||
data = yaml.safe_load(path.read_text())
|
||||
data["cores"] = ["kick", "other"]
|
||||
path.write_text(yaml.safe_dump(data))
|
||||
(self.emulators / "other.yml").write_text(yaml.safe_dump({
|
||||
"emulator": "Other", "type": "libretro", "cores": ["other"],
|
||||
"systems": ["commodore-amiga"],
|
||||
"files": [{"name": "kick.rom", "path": "kick.rom", "required": False}],
|
||||
}))
|
||||
_emulator_profiles_cache.clear()
|
||||
manifest = self.manifest("linkplat")
|
||||
(held,) = [f for f in manifest["files"] if f["dest"] == "kick.rom"]
|
||||
self.assertEqual(held["cores"], ["other"])
|
||||
omitted = [o for o in manifest["omitted_files"] if o["name"] == "kick.rom"]
|
||||
self.assertEqual([(o["root"], o["cores"]) for o in omitted], [("emulator", ["kick"])])
|
||||
|
||||
def test_the_installer_accepts_the_omission(self) -> None:
|
||||
import install
|
||||
|
||||
manifest = self.manifest("linkplat")
|
||||
omitted = next(o for o in manifest["omitted_files"] if o["name"] == "kick.rom")
|
||||
install._check_omission(omitted, omitted["dest"])
|
||||
# Two emulators reading kick.rom in their own folders read two files.
|
||||
self.assertNotEqual(
|
||||
install._entry_key(omitted), install._entry_key({**omitted, "cores": ["other"]})
|
||||
)
|
||||
self.assertNotEqual(install._entry_key(omitted), install._entry_key({"dest": "kick.rom"}))
|
||||
self.assertEqual(install.OMISSION_REASONS[omitted["reason"]], "read outside the BIOS folder")
|
||||
with self.assertRaises(ValueError):
|
||||
install._check_root({"root": "emulator"}, "kick.rom")
|
||||
install._check_root(omitted, "kick.rom", omitted=True)
|
||||
|
||||
def test_without_relocation_the_root_file_stays_at_the_root(self) -> None:
|
||||
self.assertEqual(self.pack_names("plainplat", "kick.rom"), ["kick.rom"])
|
||||
|
||||
def test_the_link_alone_binds_nothing(self) -> None:
|
||||
# Without the working directory the link's $md_inst is not the
|
||||
# directory the profile's paths are relative to.
|
||||
@@ -145,9 +210,158 @@ class UAE4AllOnRetroPie(unittest.TestCase):
|
||||
(view, standalone), = platform_layouts("uae4all", profile, standalone_set(config))
|
||||
self.assertTrue(standalone)
|
||||
for entry in view["files"]:
|
||||
self.assertEqual(
|
||||
entry["standalone_path"], "amiga/" + entry["path"].removeprefix("kickstarts/")
|
||||
if entry["path"].startswith("kickstarts/"):
|
||||
self.assertEqual(
|
||||
entry["standalone_path"], "amiga/" + entry["path"].removeprefix("kickstarts/")
|
||||
)
|
||||
else:
|
||||
# kick.rom and rom.key in the working directory, $md_inst
|
||||
self.assertEqual(entry.get("unplaced"), "outside_bios_folder", entry["path"])
|
||||
|
||||
|
||||
|
||||
PROGRAM = b"PROGRAM BINARY"
|
||||
PROGRAM_DATA = b"PROGRAM DATA"
|
||||
|
||||
|
||||
class AProgramRunFromItsFolder(unittest.TestCase):
|
||||
"""A program that reads its files beside its executable.
|
||||
|
||||
The profile gives the layout of the program's archive (prog/prog,
|
||||
prog/prog.dat) and declares the executable directory. A platform that
|
||||
runs prog/prog from its BIOS folder keeps that layout, one that runs it
|
||||
from roms/prog places the program folder below its roms folder, and one
|
||||
that runs it from an emulator folder names its files as not placeable.
|
||||
"""
|
||||
|
||||
def setUp(self) -> None:
|
||||
_emulator_profiles_cache.clear()
|
||||
(REPO_ROOT / "tmp").mkdir(exist_ok=True)
|
||||
self.root = Path(tempfile.mkdtemp(dir=REPO_ROOT / "tmp"))
|
||||
self.addCleanup(shutil.rmtree, self.root, True)
|
||||
bios = self.root / "bios" / "Maker" / "Prog"
|
||||
bios.mkdir(parents=True)
|
||||
files, by_md5, by_name = {}, {}, {}
|
||||
for name, data in (("prog", PROGRAM), ("prog.dat", PROGRAM_DATA)):
|
||||
(bios / name).write_bytes(data)
|
||||
sha1 = hashlib.sha1(data).hexdigest()
|
||||
md5 = hashlib.md5(data).hexdigest()
|
||||
files[sha1] = {"path": str(bios / name), "name": name, "sha1": sha1,
|
||||
"md5": md5, "crc32": "", "size": len(data)}
|
||||
by_md5[md5] = sha1
|
||||
by_name[name] = [sha1]
|
||||
self.db = {"files": files, "indexes": {
|
||||
"by_md5": by_md5, "by_name": by_name, "by_crc32": {}, "by_path_suffix": {}}}
|
||||
self.platforms = self.root / "platforms"
|
||||
self.emulators = self.root / "emulators"
|
||||
self.platforms.mkdir()
|
||||
self.emulators.mkdir()
|
||||
run = {"emulator": "prog", "build": "standalone", "kind": "fixed_name",
|
||||
"passes_to": "exec", "source_ref": "start_prog.sh:9"}
|
||||
launchers = {
|
||||
"biosplat": [{**run, "names": ["prog/prog"], "search_dir": "/userdata/bios"}],
|
||||
"romsplat": [{**run, "names": ["prog"], "search_dir": "/storage/roms/prog"}],
|
||||
"exeplat": [{**run, "names": ["prog"], "search_dir": "<emulator folder>"}],
|
||||
"plainplat": [],
|
||||
}
|
||||
for name, records in launchers.items():
|
||||
(self.platforms / f"{name}.yml").write_text(yaml.safe_dump({
|
||||
"platform": name, "version": "1", "verification_mode": "existence",
|
||||
"base_destination": "bios", "cores": ["prog"], "standalone_cores": ["prog"],
|
||||
"launchers": records,
|
||||
"systems": {"prog-system": {"files": []}},
|
||||
}))
|
||||
(self.platforms / "_registry.yml").write_text(yaml.safe_dump({"platforms": {
|
||||
**{name: {"config": f"{name}.yml"} for name in launchers},
|
||||
"romsplat": {"config": "romsplat.yml",
|
||||
"roms_directory": {"path": "/storage/roms", "bios_folder": "bios"}},
|
||||
}}))
|
||||
(self.emulators / "prog.yml").write_text(yaml.safe_dump({
|
||||
"emulator": "Prog", "type": "standalone", "cores": ["prog"],
|
||||
"systems": ["prog-system"],
|
||||
"standalone_directories": ["executable directory"],
|
||||
"files": [
|
||||
{"name": "prog", "path": "prog/prog", "sha1": by_name["prog"][0],
|
||||
"required": True},
|
||||
{"name": "prog.dat", "path": "prog/prog.dat",
|
||||
"sha1": by_name["prog.dat"][0], "required": True},
|
||||
],
|
||||
}))
|
||||
|
||||
def pack(self, platform: str) -> list[str]:
|
||||
with redirect_stdout(io.StringIO()):
|
||||
zip_path = generate_pack(
|
||||
platform, str(self.platforms), self.db, str(self.root / "bios"),
|
||||
str(self.root / "out" / platform), emulators_dir=str(self.emulators),
|
||||
offline=True,
|
||||
)
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
return sorted(n for n in zf.namelist() if n.split("/")[-1].startswith("prog"))
|
||||
|
||||
def manifest(self, platform: str) -> dict:
|
||||
with redirect_stdout(io.StringIO()):
|
||||
return generate_manifest(
|
||||
platform, str(self.platforms), self.db, str(self.root / "bios"),
|
||||
str(self.platforms / "_registry.yml"),
|
||||
emulators_dir=str(self.emulators), offline=True,
|
||||
)
|
||||
|
||||
def verify(self, platform: str) -> list[dict]:
|
||||
config = load_platform_config(platform, str(self.platforms))
|
||||
result = verify_platform(config, self.db, str(self.emulators))
|
||||
return [u for u in result["undeclared_files"] if u["name"].startswith("prog")]
|
||||
|
||||
def test_run_from_the_bios_folder_keeps_the_layout(self) -> None:
|
||||
for platform in ("biosplat", "plainplat"):
|
||||
with self.subTest(platform=platform):
|
||||
self.assertEqual(self.pack(platform), ["prog/prog", "prog/prog.dat"])
|
||||
self.assertEqual(
|
||||
sorted(f["dest"] for f in self.manifest(platform)["files"]),
|
||||
["prog/prog", "prog/prog.dat"],
|
||||
)
|
||||
|
||||
def test_run_from_the_roms_folder_is_installed_there(self) -> None:
|
||||
self.assertEqual(self.pack("romsplat"), [])
|
||||
manifest = self.manifest("romsplat")
|
||||
self.assertEqual(
|
||||
sorted((f["dest"], f.get("root")) for f in manifest["files"]),
|
||||
[("prog/prog", "roms"), ("prog/prog.dat", "roms")],
|
||||
)
|
||||
self.assertEqual(manifest["roms_directory"], {"bios_folder": "bios"})
|
||||
entries = self.verify("romsplat")
|
||||
self.assertEqual(sorted(u["path"] for u in entries), ["prog/prog", "prog/prog.dat"])
|
||||
self.assertTrue(all(u["in_pack"] is False for u in entries))
|
||||
|
||||
def test_run_from_an_emulator_folder_is_named(self) -> None:
|
||||
self.assertEqual(self.pack("exeplat"), [])
|
||||
manifest = self.manifest("exeplat")
|
||||
self.assertEqual([f["dest"] for f in manifest["files"]], [])
|
||||
self.assertEqual(
|
||||
sorted((o["dest"], o["root"], o["reason"]) for o in manifest["omitted_files"]),
|
||||
[("prog/prog", "emulator", "outside_bios_folder"),
|
||||
("prog/prog.dat", "emulator", "outside_bios_folder")],
|
||||
)
|
||||
|
||||
def test_the_installer_finds_the_roms_folder_above_the_bios_folder(self) -> None:
|
||||
import install
|
||||
|
||||
manifest = self.manifest("romsplat")
|
||||
install._check_roms_directory(manifest)
|
||||
files = list(manifest["files"])
|
||||
card = self.root / "card" / "roms" / "bios"
|
||||
kept, left = install.place_roms_entries(files, card, manifest, "romsplat", say=False)
|
||||
self.assertEqual((len(kept), left), (2, []))
|
||||
self.assertEqual(
|
||||
install._entry_path(card, kept[0]),
|
||||
(card.parent / kept[0]["dest"]).resolve()
|
||||
if install._entry_path(card, kept[0]).is_absolute() else card.parent / kept[0]["dest"],
|
||||
)
|
||||
other = self.root / "elsewhere"
|
||||
kept, left = install.place_roms_entries(files, other, manifest, "romsplat", say=False)
|
||||
self.assertEqual(kept, [])
|
||||
self.assertEqual({o["reason"] for o in left}, {"no_roms_directory"})
|
||||
with self.assertRaises(ValueError):
|
||||
install._check_roms_directory({"files": files})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -106,9 +106,10 @@ class HandedDirectories(unittest.TestCase):
|
||||
|
||||
class Root(unittest.TestCase):
|
||||
def root(self, *launchers: dict, base: str = "bios") -> str | None:
|
||||
return self.bindings(*launchers, base=base).get("")
|
||||
found = self.bindings(*launchers, base=base).get("")
|
||||
return found[1] if found else None
|
||||
|
||||
def bindings(self, *launchers: dict, base: str = "bios") -> dict[str, str]:
|
||||
def bindings(self, *launchers: dict, base: str = "bios") -> dict[str, tuple[str, str]]:
|
||||
return standalone_bindings(
|
||||
{"emu"}, PROFILE["standalone_directories"], handed_directories(config(*launchers, base=base))
|
||||
)
|
||||
@@ -144,8 +145,8 @@ class Root(unittest.TestCase):
|
||||
found = self.bindings(
|
||||
link("emu", "$bios_path/Triforce", "$XDG_DATA_HOME/emu/Triforce is a symlink to it"),
|
||||
)
|
||||
self.assertEqual(found, {"Triforce": "Triforce"})
|
||||
self.assertEqual(bound_path("Triforce/segaboot.gcm", found), "Triforce/segaboot.gcm")
|
||||
self.assertEqual(found, {"Triforce": ("", "Triforce")})
|
||||
self.assertEqual(bound_path("Triforce/segaboot.gcm", found), ("", "Triforce/segaboot.gcm"))
|
||||
self.assertIsNone(bound_path("boot.bin", found))
|
||||
|
||||
def test_two_folders_give_nothing(self) -> None:
|
||||
@@ -201,10 +202,10 @@ class Layout(unittest.TestCase):
|
||||
|
||||
def test_destination_function(self) -> None:
|
||||
entry = PROFILE["files"][0]
|
||||
self.assertEqual(standalone_destination(entry, PROFILE, {"": "dc"}), "dc/boot.bin")
|
||||
self.assertEqual(standalone_destination(entry, PROFILE, {"": ("", "dc")}), ("", "dc/boot.bin"))
|
||||
self.assertIsNone(standalone_destination(entry, PROFILE, {}))
|
||||
# The entry both builds share keeps path in a dual-build profile.
|
||||
self.assertIsNone(standalone_destination(PROFILE["files"][2], PROFILE, {"": "dc"}))
|
||||
self.assertIsNone(standalone_destination(PROFILE["files"][2], PROFILE, {"": ("", "dc")}))
|
||||
|
||||
|
||||
class FlycastOnEachPlatform(unittest.TestCase):
|
||||
@@ -228,7 +229,7 @@ class FlycastOnEachPlatform(unittest.TestCase):
|
||||
cfg = load_platform_config(platform, str(REPO_ROOT / "platforms"))
|
||||
self.assertEqual(
|
||||
standalone_bindings_for("flycast", self.profile, standalone_set(cfg)),
|
||||
{"": folder},
|
||||
{"": ("", folder)},
|
||||
)
|
||||
|
||||
def test_no_standalone_file_lands_at_the_bios_root(self) -> None:
|
||||
|
||||
@@ -208,14 +208,22 @@ resolved. Three strategies exist:
|
||||
- **Omitted**: fallback to system ID intersection, `built_by` profiles excluded.
|
||||
|
||||
Two lists say how each resolved emulator is laid out. `standalone_cores` names the
|
||||
ones the platform launches as standalone apps: their files go to `standalone_path`,
|
||||
under the folder the platform hands the emulator when its `launchers:` say so. A
|
||||
`links` or `passes_path` record of a whole directory that names one of the profile's
|
||||
`standalone_directories` gives that folder (`launchers.standalone_root`): RetroPie
|
||||
links Flycast's `~/.local/share/flycast` to `BIOS/dc`, EmuDeck the Flatpak's data
|
||||
directory to `bios/flycast/bios`, and ROCKNIX's patched data directory is followed
|
||||
through the link its start script makes. Two records giving two folders leave the
|
||||
profile's layout.
|
||||
ones the platform launches as standalone apps: their files go to `standalone_path`
|
||||
(`path` for a profile with no libretro build), under the folder the platform hands
|
||||
the emulator when its `launchers:` say so. A `links` or `passes_path` record of a
|
||||
directory at or below one of the profile's `standalone_directories` binds that
|
||||
subfolder of the layout to a folder (`launchers.standalone_bindings`): RetroPie
|
||||
links Flycast's `~/.local/share/flycast` to `BIOS/dc`, EmuDeck yuzu's `keys` to
|
||||
`bios/yuzu/keys`, and ROCKNIX's patched data directory is followed through the link
|
||||
its start script makes. A launch script that enters a folder hands the `getcwd`
|
||||
directory: RetroPie starts UAE4All2 in `$md_inst`, whose `kickstarts` it links to
|
||||
`BIOS/amiga`. A profile declaring `executable directory` is bound where a record
|
||||
executes one of its files: Batocera runs `pico-8/pico8` from `bios`, ROCKNIX
|
||||
`pico8_64` from `/storage/roms/pico-8`, below the `roms_directory` its registry entry
|
||||
names, which the manifest writes as `root: roms`. Two records giving two folders
|
||||
leave the profile's layout. Once a platform binds part of a layout, an entry no
|
||||
binding covers is read in a folder outside the BIOS tree: it is never packed, and
|
||||
the manifest names it (`root: emulator`, `outside_bios_folder`).
|
||||
`dual_mode_cores` names those it launches both ways (RetroPie packages `lr-hatari`
|
||||
and `hatari`): both layouts are packed. A profile without a standalone build keeps
|
||||
its libretro layout whatever the platform calls it.
|
||||
|
||||
Reference in new issue
Block a user