chore: merge removal of the archived status

This commit is contained in:
Abdessamad Derraz committed 2026-11-06 13:10:29 +01:00
commit 1a29b802ce
93 files changed
+281 -452

No files matched your search

+3 -5
View File
@@ -69,13 +69,11 @@ The Files column counts what a pack holds once extracted, the figure a file mana
| RetroArch | 8,504 | 6.0 GB | `system/` | [Download](https://abdess.github.io/retrobios/get/retroarch/) |
| RetroBat | 12,399 | 7.9 GB | `bios/` | [Download](https://abdess.github.io/retrobios/get/retrobat/) |
| RetroDECK | - | - | `~/retrodeck/` | [Use the command](https://abdess.github.io/retrobios/get/retrodeck/) |
| RetroPie * | - | - | `~/RetroPie/BIOS/` | [Use the command](https://abdess.github.io/retrobios/get/retropie/) |
| RetroPie | - | - | `~/RetroPie/BIOS/` | [Use the command](https://abdess.github.io/retrobios/get/retropie/) |
| RomM | 1,087 | 1.5 GB | `library/bios/` | [Download](https://abdess.github.io/retrobios/get/romm/) |
ZIPs are made for the setups most people download. The others read "Use the command": the installer above installs every file for them and checks each one, and `--dry-run` shows what it would install ([why no ZIP](https://abdess.github.io/retrobios/wiki/faq/#why-is-there-no-zip-for-my-setup)).
\* Archived: the installer serves it, and checks that cover every setup include it only on request.
## What's included
BIOS, firmware, and system files for consoles from Atari to PlayStation 3.
@@ -117,7 +115,7 @@ Full list with per-file details: **[https://abdess.github.io/retrobios/](https:/
| RetroArch | 526/526 | 7,919/7,986 | name only |
| RetroBat | 348/348 | 5,135/9,659 | MD5 |
| RetroDECK | 2,007/2,008 | 6,706/6,993 | MD5 |
| RetroPie * | 532/532 | 8,118/12,772 | name only |
| RetroPie | 532/532 | 8,118/12,772 | name only |
| RomM | 383/383 | 83/875 | size, then any hash |
Each fraction is what the pack has over what is needed, counting required and optional files alike since both ship. The first column is the BIOS list the platform publishes. The second counts files its emulators load that this list never mentions, found by reading their source code, and it is routinely several times larger. A short fraction means files are still missing, and they are named in the [gap analysis](https://abdess.github.io/retrobios/gaps/).
@@ -208,4 +206,4 @@ The scripts and tooling are released under the [MIT License](LICENSE).
The BIOS and firmware files are not covered by that license: they are third-party system software, preserved and provided for personal backup, archival, and interoperability with emulation software. [NOTICE](NOTICE) sets out their status and how to ask for a file to be removed.
The reasoning, and where it is weakest, is in [the project's legal position](https://abdess.github.io/retrobios/wiki/legal/).
*Auto-generated on 2026-11-06T09:30:57Z*
*Auto-generated on 2026-11-06T10:21:06Z*
+1 -1
View File
@@ -4,7 +4,7 @@ How platform YAML files work and where subdirectory requirements come from.
## Files
- `_registry.yml` -- platform metadata (name, status, display order)
- `_registry.yml` -- platform metadata (name, release pack, display order)
- `_shared.yml` -- shared file groups with canonical destinations
- `<platform>.yml` -- per-platform BIOS declarations
- Inheritance: `lakka.yml` inherits `retroarch`, `retropie.yml` inherits `retroarch`
-2
View File
@@ -650,8 +650,6 @@ guides:
- >-
RetroPie also installs standalone emulators (openMSX, PCSX2, XRoar, Amiberry and
others) whose files only the RetroPie file list carries.
- >-
RetroPie is archived here: checks of every setup include it only on request.
pack_manual: |
Option B: Via network share
--------------------------
-18
View File
@@ -184,7 +184,6 @@ _standalone_copies: &standalone_copies
platforms:
retroarch:
config: retroarch.yml
status: active
release_pack: true
device_group: computer
extract_to: system/
@@ -503,7 +502,6 @@ platforms:
standalone_copies: *standalone_copies
batocera:
config: batocera.yml
status: active
release_pack: true
device_group: gaming-os
extract_to: /userdata/bios/
@@ -541,7 +539,6 @@ platforms:
standalone_copies: *standalone_copies
recalbox:
config: recalbox.yml
status: active
device_group: gaming-os
extract_to: /recalbox/share/bios/
logo: https://raw.githubusercontent.com/homarr-labs/dashboard-icons/main/svg/recalbox.svg
@@ -579,7 +576,6 @@ platforms:
standalone_copies: *standalone_copies
retrobat:
config: retrobat.yml
status: active
release_pack: true
device_group: computer
extract_to: bios/
@@ -627,7 +623,6 @@ platforms:
standalone_copies: *standalone_copies
emudeck:
config: emudeck.yml
status: active
release_pack: true
device_group: computer
extract_to: ~/Emulation/bios/
@@ -688,7 +683,6 @@ platforms:
standalone_copies: *standalone_copies
lakka:
config: lakka.yml
status: active
device_group: gaming-os
extract_to: /storage/system/
logo: https://raw.githubusercontent.com/libretro/retroarch-assets/master/src/xmb/flatui/lakka.svg
@@ -719,7 +713,6 @@ platforms:
bios_path: /storage/system
retrodeck:
config: retrodeck.yml
status: active
device_group: computer
extract_to: ~/retrodeck/
logo: https://raw.githubusercontent.com/RetroDECK/RetroDECK/main/res/icon.svg
@@ -769,7 +762,6 @@ platforms:
standalone_copies: *standalone_copies
romm:
config: romm.yml
status: active
release_pack: true
device_group: server
extract_to: library/bios/
@@ -838,7 +830,6 @@ platforms:
- xemu
rocknix:
config: rocknix.yml
status: active
device_group: handheld
extract_to: /storage/roms/bios/
logo: https://avatars.githubusercontent.com/u/163176886
@@ -867,7 +858,6 @@ platforms:
# 20230203: the scraper reads the commit of the newest prerelease.
amberelec:
config: amberelec.yml
status: active
device_group: handheld
extract_to: /storage/roms/
logo: https://avatars.githubusercontent.com/u/102820909
@@ -909,7 +899,6 @@ platforms:
# device the release publishes, its cores read by emuelec_targets_scraper.
emuelec:
config: emuelec.yml
status: active
device_group: handheld
extract_to: /storage/roms/
logo: https://avatars.githubusercontent.com/u/57574835
@@ -942,7 +931,6 @@ platforms:
# BR2_PACKAGE_BATOCERA_TARGET_*), read by knulli_targets_scraper.
knulli:
config: knulli.yml
status: active
device_group: handheld
extract_to: /userdata/bios/
logo: https://avatars.githubusercontent.com/u/166268572
@@ -982,7 +970,6 @@ platforms:
derives_from: Batocera
retropie:
config: retropie.yml
status: archived
device_group: gaming-os
extract_to: ~/RetroPie/BIOS/
logo: https://avatars.githubusercontent.com/u/11378204
@@ -1010,7 +997,6 @@ platforms:
path: $HOME/RetroPie/BIOS
bizhawk:
config: bizhawk.yml
status: active
device_group: computer
extract_to: Firmware/
# EmuHawk's application icon (<ApplicationIcon> in BizHawk.Client.EmuHawk.csproj).
@@ -1047,7 +1033,6 @@ platforms:
bios_subdir: Firmware
misterfpga:
config: misterfpga.yml
status: active
release_pack: true
device_group: fpga
extract_to: /media/fat/games/
@@ -1074,7 +1059,6 @@ platforms:
bios_path: /media/fat/games
minui:
config: minui.yml
status: active
device_group: handheld
# The card root: minarch hands each core SDCARD_PATH/Bios/<TAG>
# (workspace/all/minarch/minarch.c:2943).
@@ -1103,7 +1087,6 @@ platforms:
first_line_prefix: MinUI-
nextui:
config: nextui.yml
status: active
device_group: handheld
# minarch hands each core SDCARD_PATH/Bios/<TAG>
# (workspace/all/minarch/ma_core.c:76).
@@ -1131,7 +1114,6 @@ platforms:
first_line_prefix: NextUI-
myminui:
config: myminui.yml
status: active
device_group: handheld
# minarch hands each core SDCARD_PATH/Bios/<TAG>
# (workspace/all/minarch/minarch.c:3532).
+2 -6
View File
@@ -2,7 +2,7 @@
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://abdess.github.io/retrobios/api/v1/schemas/registry.schema.json",
"title": "RetroBIOS platform registry",
"description": "platforms/_registry.yml: each registered platform, its status, and whether a release publishes a ZIP pack for it. Install detection and the other per-platform fields are read by their own tools.",
"description": "platforms/_registry.yml: each registered platform and whether a release publishes a ZIP pack for it. Install detection and the other per-platform fields are read by their own tools.",
"type": "object",
"required": ["platforms"],
"properties": {
@@ -12,13 +12,9 @@
"propertyNames": { "pattern": "^[a-z0-9_]+$" },
"additionalProperties": {
"type": "object",
"required": ["config", "status"],
"required": ["config"],
"properties": {
"config": { "type": "string", "pattern": "\\.yml$" },
"status": {
"description": "active: scraped and released. archived: the configuration is kept, no scheduled scrape.",
"enum": ["active", "archived"]
},
"release_pack": {
"description": "true when a release publishes the platform's full ZIP pack. Absent or false: the installer serves the platform, its manifest is written all the same.",
"type": "boolean"
-1
View File
@@ -533,7 +533,6 @@ def main() -> None:
if args.all:
platforms = list_registered_platforms(
args.platforms_dir,
include_archived=True,
)
elif args.platform:
require_platform(args.platform, args.platforms_dir)
+2 -2
View File
@@ -134,7 +134,7 @@ def release_pack_labels(root: Path = REPO_ROOT) -> dict[str, str]:
"""Platform -> the platform part of the release pack that serves it.
Only the platforms the registry marks release_pack have a pack, grouped
as the release builds them (pipeline.py --include-archived). install.py
as the release builds them (pipeline.py). install.py
refuses a pack whose name does not spell the platform, so a platform
outside these groups has no pack to copy, whatever its file list.
"""
@@ -146,7 +146,7 @@ def release_pack_labels(root: Path = REPO_ROOT) -> dict[str, str]:
from generate_pack import _name_part, _platform_pack_stem # noqa: PLC0415
platforms_dir = str(root / "platforms")
pool = list_release_pack_platforms(platforms_dir, include_archived=True)
pool = list_release_pack_platforms(platforms_dir)
labels: dict[str, str] = {}
for group, representative in group_identical_platforms(pool, platforms_dir):
stem = _platform_pack_stem(group, representative, platforms_dir)
+1 -1
View File
@@ -51,7 +51,7 @@ def bundled_only(db: dict, profiles: dict, platforms_dir: str, install_dir: Path
"""Collected files a bundled entry resolves to and nothing else needs."""
consumers = FileConsumers(db)
needed: set[str] = set(shipped_by_manifests(install_dir))
for name in list_registered_platforms(platforms_dir, include_archived=True):
for name in list_registered_platforms(platforms_dir):
config = load_platform_config(name, platforms_dir)
for system in config.get("systems", {}).values():
for entry in system.get("files", []):
+2 -2
View File
@@ -308,7 +308,7 @@ def _scrapable_platforms(platforms_dir: Path) -> list[tuple[str, str, Path]]:
with (platforms_dir / "_registry.yml").open(encoding="utf-8") as fh:
registry = (yaml_load(fh) or {}).get("platforms") or {}
rows = []
for name in list_registered_platforms(str(platforms_dir), include_archived=True):
for name in list_registered_platforms(str(platforms_dir)):
path = platforms_dir / f"{name}.yml"
if not path.is_file():
continue
@@ -513,7 +513,7 @@ def check_native(
except export_native.SourcesIndexUnreadable as exc:
return [Finding("native", index.name, ERROR, detail=str(exc))]
findings: list[Finding] = []
for name in list_registered_platforms(str(platforms_dir), include_archived=True):
for name in list_registered_platforms(str(platforms_dir)):
exporter_class = exporters.get(name)
if not exporter_class:
continue
+1 -1
View File
@@ -239,7 +239,7 @@ def main() -> int:
print(f"ERROR: {SUMS}: {exc}", file=sys.stderr)
return 2
released = set(list_release_pack_platforms(args.platforms_dir, include_archived=True))
released = set(list_release_pack_platforms(args.platforms_dir))
findings = compare_release(record, sums, assets, tag, local_files, released,
installer_files(args.install_dir))
if args.json:
+12 -24
View File
@@ -507,10 +507,7 @@ def load_platform_registry(platforms_dir: str = "platforms") -> dict:
return (yaml_load(f) or {}).get("platforms", {})
def list_registered_platforms(
platforms_dir: str = "platforms",
include_archived: bool = False,
) -> list[str]:
def list_registered_platforms(platforms_dir: str = "platforms") -> list[str]:
"""List platforms registered in _registry.yml.
Only registered platforms generate packs and appear in CI.
@@ -527,9 +524,6 @@ def list_registered_platforms(
registry = yaml_load(f) or {}
platforms = []
for name, meta in sorted(registry.get("platforms", {}).items()):
status = meta.get("status", "active")
if status == "archived" and not include_archived:
continue
config_path = os.path.join(platforms_dir, meta.get("config", f"{name}.yml"))
if not os.path.exists(config_path):
raise FileNotFoundError(
@@ -539,10 +533,7 @@ def list_registered_platforms(
return platforms
def list_release_pack_platforms(
platforms_dir: str = "platforms",
include_archived: bool = False,
) -> list[str]:
def list_release_pack_platforms(platforms_dir: str = "platforms") -> list[str]:
"""Registered platforms a release publishes a ZIP pack for.
`release_pack: true` in _registry.yml is the whole list: the build of
@@ -553,7 +544,7 @@ def list_release_pack_platforms(
registry = load_platform_registry(platforms_dir)
return [
name
for name in list_registered_platforms(platforms_dir, include_archived)
for name in list_registered_platforms(platforms_dir)
if registry.get(name, {}).get("release_pack") is True
]
@@ -641,7 +632,7 @@ def platform_arg(value: str) -> str:
key = re.sub(r"[^a-z0-9]", "", value.lower())
if yaml is None or not os.path.isdir("platforms"):
return value
known = list_registered_platforms("platforms", include_archived=True)
known = list_registered_platforms("platforms")
for name in known:
if re.sub(r"[^a-z0-9]", "", name.lower()) == key:
return name
@@ -665,7 +656,7 @@ def require_platform(name: str, platforms_dir: str = "platforms") -> None:
return
import difflib # noqa: PLC0415
known = list_registered_platforms(platforms_dir, include_archived=True)
known = list_registered_platforms(platforms_dir)
close = difflib.get_close_matches(name, known, n=1)
hint = f" Did you mean '{close[0]}'?" if close else ""
raise SystemExit(
@@ -3460,19 +3451,16 @@ def list_system_ids(emulators_dir: str) -> None:
def print_platform_listing(platforms_dir: str) -> None:
"""Every registered platform, one per line, marked when archived or
served by the installer only.
"""Every registered platform, one per line, marked when the installer
alone serves it.
Every platform takes --platform. --all covers the active ones, and the
pack builder's --all the release packs. Printed by --list-platforms in
Every platform takes --platform. --all covers every one, and the pack
builder's --all the release packs. Printed by --list-platforms in
verify.py and generate_pack.py, the name install.py gives it.
"""
active = set(list_registered_platforms(platforms_dir))
released = set(list_release_pack_platforms(platforms_dir, include_archived=True))
for name in list_registered_platforms(platforms_dir, include_archived=True):
marks = [m for m, on in (("archived", name not in active),
("installer only", name not in released)) if on]
print(f"{name} ({', '.join(marks)})" if marks else name)
released = set(list_release_pack_platforms(platforms_dir))
for name in list_registered_platforms(platforms_dir):
print(name if name in released else f"{name} (installer only)")
def list_platform_system_ids(platform_name: str, platforms_dir: str) -> None:
+4 -4
View File
@@ -83,7 +83,7 @@ def load_platform_files(
declared = {}
platform_data_dirs = {}
for platform_name in platforms or list_registered_platforms(
platforms_dir, include_archived=True
platforms_dir
):
config = load_platform_config(platform_name, platforms_dir)
for sys_id, system in config.get("systems", {}).items():
@@ -114,7 +114,7 @@ def declared_names(
[
load_platform_config(platform_name, platforms_dir)
for platform_name in platforms or list_registered_platforms(
platforms_dir, include_archived=True
platforms_dir
)
],
db,
@@ -133,7 +133,7 @@ def folded_declared_slots(
slots: set[str] = set()
for platform_name in platforms or list_registered_platforms(
platforms_dir, include_archived=True
platforms_dir
):
config = load_platform_config(platform_name, platforms_dir)
slots |= case_folded_slots(config, platform_declarations(config, db))
@@ -1257,7 +1257,7 @@ def main() -> None:
standalone_cores = merged_standalone_set(
load_platform_config(name, args.platforms_dir)
for name in list_registered_platforms(args.platforms_dir, include_archived=True)
for name in list_registered_platforms(args.platforms_dir)
)
if not profiles:
+1 -1
View File
@@ -215,7 +215,7 @@ def main() -> None:
if args.all:
platforms = list_registered_platforms(
args.platforms_dir, include_archived=True
args.platforms_dir
)
else:
require_platform(args.platform, args.platforms_dir)
-1
View File
@@ -1009,7 +1009,6 @@ def main() -> None:
if args.all:
platforms = list_registered_platforms(
args.platforms_dir,
include_archived=True,
)
else:
require_platform(args.platform, args.platforms_dir)
+1 -1
View File
@@ -623,7 +623,7 @@ def _collect_all_aliases(
platforms_dir = Path("platforms")
if platforms_dir.is_dir():
for platform_name in list_registered_platforms(
str(platforms_dir), include_archived=True
str(platforms_dir)
):
config_file = platforms_dir / f"{platform_name}.yml"
try:
+29 -57
View File
@@ -1779,7 +1779,7 @@ def generate_emulator_pack(
else:
report_missing(archive_name, archive_dest)
# Pack individual files (skip archived ones). The entries of one
# Pack individual files (skip archive members). The entries of one
# profile on one destination are the releases the code accepts
# there (raze's seven SW.GRP): the first one the core accepts
# fills the slot, as verify --emulator credits it, and the slot
@@ -2605,10 +2605,7 @@ def _resolve_systems(args: argparse.Namespace, parser: argparse.ArgumentParser)
config = None
systems = sorted({
sid
for name in list_registered_platforms(
args.platforms_dir,
include_archived=args.include_archived or args.manifest,
)
for name in list_registered_platforms(args.platforms_dir)
for sid in load_platform_config(name, args.platforms_dir).get("systems", {})
})
listing = "--platform <name> --list-systems"
@@ -2933,9 +2930,7 @@ def _run_verify_packs(args: argparse.Namespace) -> None:
elif args.all:
# The packs --all builds: a platform without release_pack has no pack
# to check, and its absence from the directory is no failure.
platforms = list_release_pack_platforms(
args.platforms_dir, include_archived=args.include_archived
)
platforms = list_release_pack_platforms(args.platforms_dir)
else:
raise ValueError("--verify-packs requires --platform or --all")
@@ -3334,15 +3329,6 @@ def _refuse_refresh_data(args: argparse.Namespace, parser: argparse.ArgumentPars
parser.error(f"{name} is incompatible with {flag}")
def _refuse_include_archived(
args: argparse.Namespace, parser: argparse.ArgumentParser
) -> None:
"""Only --all chooses among registered platforms. Every other mode names
its own platform, emulator, system or hash."""
if args.include_archived and not args.all:
parser.error("--include-archived requires --all")
def _refuse_unapplied_flags(args: argparse.Namespace, parser: argparse.ArgumentParser) -> None:
"""Refuse every flag the requested mode would not apply.
@@ -3350,7 +3336,6 @@ def _refuse_unapplied_flags(args: argparse.Namespace, parser: argparse.ArgumentP
about an artifact the caller did not name. Run before any quick-exit
mode, since --verify-packs and --manifest-targets return early.
"""
_refuse_include_archived(args, parser)
# Parsed before the quick-exit modes: --verify-packs returns early and
# still needs the region priority list to narrow its expectation.
args.regions = []
@@ -3392,8 +3377,7 @@ QUICK_MODE_FLAGS: dict[str, set[str]] = {
# Checks packs already on disk against the platform's own list, narrowed
# by region and target and by nothing else.
"--verify-packs": {
"--platform", "--all", "--target", "--region", "--include-archived",
"--no-game-data",
"--platform", "--all", "--target", "--region", "--no-game-data",
},
# One manifest per hardware target, from the target files themselves.
"--manifest-targets": set(),
@@ -3430,7 +3414,6 @@ def _given_flags(args: argparse.Namespace) -> set[str]:
"--list-emulators": args.list_emulators,
"--list-systems": args.list_systems,
"--list-targets": args.list_targets,
"--include-archived": args.include_archived,
"--refresh-data": args.refresh_data,
"--no-refresh-data": args.no_refresh_data,
}
@@ -3486,9 +3469,6 @@ def _main(held: contextlib.ExitStack) -> None:
parser.add_argument(
"--list-systems", action="store_true", help="List available systems"
)
parser.add_argument(
"--include-archived", action="store_true", help="Include archived platforms"
)
parser.add_argument(
"--platforms-dir", default=DEFAULT_PLATFORMS_DIR,
help="platform YAML directory (default: %(default)s)",
@@ -3760,16 +3740,12 @@ def _main(held: contextlib.ExitStack) -> None:
# Platform mode
if args.all and args.manifest:
# Install manifests cover every registered platform: the installer
# serves each one, with or without a release pack. "Archived" keeps a
# platform out of the runs that cover every platform, not out of the
# installer: it has not lost its users.
platforms = list_registered_platforms(args.platforms_dir, include_archived=True)
# serves each one, with or without a release pack.
platforms = list_registered_platforms(args.platforms_dir)
elif args.all:
# A release publishes the packs the registry marks release_pack and
# nothing else: a pack built for another platform is one nobody uploads.
platforms = list_release_pack_platforms(
args.platforms_dir, include_archived=args.include_archived
)
platforms = list_release_pack_platforms(args.platforms_dir)
if not platforms:
parser.error("no registered platform has release_pack: true")
elif args.platform:
@@ -5291,31 +5267,27 @@ def _expected_pack_names(
for p in platforms
}
# Groups depend on which platforms a run built: the release packs of
# --all, alone or with --include-archived. Both compositions are names a
# pack can carry.
for include_archived in (False, True):
pool = list_release_pack_platforms(
platforms_dir, include_archived=include_archived
)
cache = None
if target_name:
try:
cache, pool = build_target_cores_cache(
pool, target_name, platforms_dir, is_all=True
)
except ValueError:
# No platform has the target: no pack was built under it.
continue
for group, representative in group_identical_platforms(
pool, platforms_dir, cache
):
if len(group) < 2:
continue
stem = _platform_pack_stem(group, representative, platforms_dir)
for tags in narrow_tags(representative):
for p in group:
if p in names:
names[p].add(f"{stem}{tags}_BIOS_Pack.zip")
# --all.
pool = list_release_pack_platforms(platforms_dir)
cache = None
if target_name:
try:
cache, pool = build_target_cores_cache(
pool, target_name, platforms_dir, is_all=True
)
except ValueError:
# No platform has the target: no pack was built under it.
return names
for group, representative in group_identical_platforms(
pool, platforms_dir, cache
):
if len(group) < 2:
continue
stem = _platform_pack_stem(group, representative, platforms_dir)
for tags in narrow_tags(representative):
for p in group:
if p in names:
names[p].add(f"{stem}{tags}_BIOS_Pack.zip")
return names
@@ -5442,7 +5414,7 @@ def verify_and_finalize_packs(
# By the exact name this run's reduction gives a platform's pack, never by
# substring: a _Custom or regional pack left by another run was judged
# against this run's expectation and failed it.
registered = list_registered_platforms(platforms_dir, include_archived=True)
registered = list_registered_platforms(platforms_dir)
expected = _expected_pack_names(
registered, platforms_dir, regions, target_name, no_game_data=no_game_data
)
+4 -21
View File
@@ -379,7 +379,6 @@ def _paragraphs(texts: list[str]) -> list[str]:
def download_table(
coverages: dict,
archived: set[str],
extract_paths: dict[str, str],
record: dict,
) -> list[str]:
@@ -389,12 +388,9 @@ def download_table(
"|----------|------:|---------------:|-----------|----------|",
]
for name, cov in sorted(coverages.items(), key=lambda x: x[1]["platform"]):
display = cov["platform"]
if name in archived:
display = f"{display} *"
files, size = release_totals(name, record)
lines.append(
f"| {display} | {f'{files:,}' if files else '-'} |"
f"| {cov['platform']} | {f'{files:,}' if files else '-'} |"
f" {format_size(size) if size else '-'} |"
f" {extract_paths.get(cov['platform'], '')} |"
f" {download_links(name, record, 'Download', site=SITE_URL)} |"
@@ -530,7 +526,7 @@ def generate_readme(db: dict, platforms_dir: str, db_path: str = "database.json"
total_size = db.get("total_size", 0)
ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
platform_names = list_registered_platforms(platforms_dir, include_archived=True)
platform_names = list_registered_platforms(platforms_dir)
from common import load_data_dir_registry
@@ -632,29 +628,16 @@ def generate_readme(db: dict, platforms_dir: str, db_path: str = "database.json"
# a pack whose entries already carry their own root (RetroDECK) extracts
# one level above it.
extract_paths = _extract_cells(platforms_dir)
archived = {
name
for name, entry in load_platform_registry(platforms_dir).items()
if entry.get("status") == "archived"
}
lines.extend(
download_table(
coverages, archived, extract_paths,
coverages, extract_paths,
release_record.load_advertised(platforms_dir=platforms_dir),
)
)
lines.extend(["", no_pack_line()])
lines.extend(_paragraphs(extract_notes(platforms_dir, released_only=True)))
if archived:
lines.extend(
[
"",
"\\* Archived: the installer serves it, and checks that cover every"
" setup include it only on request.",
]
)
lines.extend(
[
@@ -753,7 +736,7 @@ def generate_readme(db: dict, platforms_dir: str, db_path: str = "database.json"
)
for name, cov in sorted(coverages.items(), key=lambda x: x[1]["platform"]):
display = f"{cov['platform']} *" if name in archived else cov["platform"]
display = cov["platform"]
checked = check_label(cov["mode"], cov["config"].get("hash_rule"))
core_total = (
cov["core_present"] + cov["core_missing"] + cov["core_unsourceable"]
+9 -22
View File
@@ -1073,7 +1073,7 @@ def _release_pack_platforms() -> frozenset[str]:
@functools.lru_cache(maxsize=None)
def _release_packs_in(platforms_dir: str) -> frozenset[str]:
return frozenset(list_release_pack_platforms(platforms_dir, include_archived=True))
return frozenset(list_release_pack_platforms(platforms_dir))
# Per-entry fields a folder check needs beyond the hashes: an md5 is a ZIP
@@ -3393,28 +3393,23 @@ SETUP_TABLE_ANCHOR = "every-setup-at-a-glance"
SETUP_TABLE_LINK = f"{SETUP_TABLE_PAGE}#{SETUP_TABLE_ANCHOR}"
def setup_table(coverages: dict, registry: dict | None = None) -> list[str]:
def setup_table(coverages: dict) -> list[str]:
"""Every setup side by side: the files on its own list, the files in its
ZIP, and how it checks them. Written under the Reference level, on the
page that explains each way of checking."""
registry = registry or {}
lines = [
f"## Every setup at a glance {{ #{SETUP_TABLE_ANCHOR} }}",
"",
"| Setup | Files on its own list | Files in the ZIP | How it checks |",
"|-------|------:|------:|---------------|",
]
archived = False
for name, cov in sorted(coverages.items(), key=lambda item: item[1]["platform"].casefold()):
status = registry.get(name, {}).get("status")
archived |= status == "archived"
listed = f"{cov['total']:,}" if cov["total"] else "No list of its own"
files, _size = release_totals(name, release_record.load_advertised())
zipped = (f"{files:,}" if files
else "No ZIP yet" if has_release_pack(name) else "No ZIP")
lines.append(
f"| [{cov['platform']}](../platforms/{name}.md)"
f"{' (archived)' if status == 'archived' else ''} | {listed} | {zipped} | "
f"| [{cov['platform']}](../platforms/{name}.md) | {listed} | {zipped} | "
f"{check_label(cov['mode'], cov['config'].get('hash_rule'))} |"
)
lines += [
@@ -3424,16 +3419,13 @@ def setup_table(coverages: dict, registry: dict | None = None) -> list[str]:
]
lines += ["No ZIP: the setup is served by the command alone. "
f"[Why?](../{NO_ZIP_FAQ})", ""]
if archived:
lines += ["An archived setup keeps its page and the command. Checks that cover "
"every setup include it only on request.", ""]
return lines
def with_setup_table(page: str, coverages: dict, registry: dict | None = None) -> str:
def with_setup_table(page: str, coverages: dict) -> str:
"""The comparison table placed before the Next steps of its page, or at
its end."""
table = "\n".join(setup_table(coverages, registry))
table = "\n".join(setup_table(coverages))
marker = "\n## Next steps"
if marker in page:
head, tail = page.split(marker, 1)
@@ -4607,11 +4599,6 @@ def generate_platform_page(
"",
]
if registry.get(name, {}).get("status") == "archived":
lines.extend(markup.aside("warning", "Archived platform", [
f"{display}'s list is kept as last read. Its pack is still built.",
]))
summary = GUIDES[name].summary if name in GUIDES else ""
if summary:
lines += [summary + (f" [{display} website]({homepage})." if homepage else ""), ""]
@@ -12471,7 +12458,7 @@ def _generate(args: argparse.Namespace) -> None:
registry = (yaml_load(f) or {}).get("platforms", {})
platform_names = list_registered_platforms(
args.platforms_dir, include_archived=True
args.platforms_dir
)
from common import load_data_dir_registry
@@ -12767,7 +12754,7 @@ def _generate(args: argparse.Namespace) -> None:
table_page = docs / SETUP_TABLE_PAGE
if table_page.is_file():
write_if_changed(str(table_page), with_setup_table(
table_page.read_text(encoding="utf-8"), coverages, registry))
table_page.read_text(encoding="utf-8"), coverages))
# data-model.md is generated (contains live DB stats)
write_if_changed(
str(wiki_dest / "data-model.md"), generate_wiki_data_model(db, profiles)
@@ -12776,8 +12763,8 @@ def _generate(args: argparse.Namespace) -> None:
print("Generating which-pack page...")
write_if_changed(str(docs / "which-pack.md"), generate_which_pack())
# One permanent download address per registered platform, archived ones
# included: the README's table links each of them.
# One permanent download address per registered platform: the README's
# table links each of them.
print("Generating download pages...")
get_dir = docs / release_record.GET_DIR
get_dir.mkdir(parents=True, exist_ok=True)
-1
View File
@@ -91,7 +91,6 @@ def main(argv: list[str] | None = None) -> None:
if args.all:
platforms = list_registered_platforms(
args.platforms_dir,
include_archived=True,
)
else:
require_platform(args.platform, args.platforms_dir)
+7 -18
View File
@@ -1,13 +1,8 @@
#!/usr/bin/env python3
"""List the registered platforms.
Respects the `status` field in _registry.yml:
- active: listed by default
- archived: listed with --include-archived only
"""List the platforms registered in _registry.yml.
Usage:
python scripts/list_platforms.py # active platforms
python scripts/list_platforms.py --include-archived # archived ones too
python scripts/list_platforms.py
python scripts/list_platforms.py >> "$GITHUB_OUTPUT"
"""
@@ -24,22 +19,16 @@ from common import list_registered_platforms
PLATFORMS_DIR = "platforms"
def list_platforms(include_archived: bool = False) -> list[str]:
"""List platform config files, filtering by status from _registry.yml."""
return list_registered_platforms(PLATFORMS_DIR, include_archived=include_archived)
def list_platforms() -> list[str]:
"""Every registered platform."""
return list_registered_platforms(PLATFORMS_DIR)
def main() -> None:
parser = argparse.ArgumentParser(allow_abbrev=False, description="List available platforms")
# The name verify, generate_pack, slots and pipeline use. --all is the
# older spelling, still read so existing commands keep working.
parser.add_argument(
"--include-archived", "--all", dest="include_archived", action="store_true",
help="include archived platforms",
)
args = parser.parse_args()
parser.parse_args()
platforms = list_platforms(include_archived=args.include_archived)
platforms = list_platforms()
if not platforms:
print("No platform configs found", file=sys.stderr)
+1 -1
View File
@@ -43,7 +43,7 @@ def platform_cards() -> dict[str, tuple[str, str]]:
return {
f"platforms/{name}": (load_platform_config(name, str(platforms))["platform"],
PLATFORM_TAGLINE)
for name in list_registered_platforms(str(platforms), include_archived=True)
for name in list_registered_platforms(str(platforms))
}
+12 -29
View File
@@ -34,8 +34,7 @@ Steps, numbered as the run prints them:
come from .cache/large/ only.
Usage:
python scripts/pipeline.py # active platforms
python scripts/pipeline.py --include-archived # all platforms
python scripts/pipeline.py # every registered platform
python scripts/pipeline.py --skip-packs # skip steps 4, 4b, 4c, 5 and 6
python scripts/pipeline.py --skip-docs # skip steps 7, 8 and 8b
python scripts/pipeline.py --offline # no network, see above
@@ -326,13 +325,13 @@ def _match_key(label: str) -> set[str]:
} - {""}
def release_pack_keys(platforms_dir: str = "platforms", include_archived: bool = False) -> set[str]:
def release_pack_keys(platforms_dir: str = "platforms") -> set[str]:
"""Comparable identities of the platforms --all builds a pack for: their
registry ids and display names, normalized as _match_key does."""
from common import list_release_pack_platforms, load_platform_config # noqa: PLC0415
keys: set[str] = set()
for name in list_release_pack_platforms(platforms_dir, include_archived):
for name in list_release_pack_platforms(platforms_dir):
keys |= _match_key(name)
keys |= _match_key(str(load_platform_config(name, platforms_dir).get("platform", name)))
return keys
@@ -714,8 +713,7 @@ def narrowed_variants(
def check_narrowed_packs(
emulators_dir: str = "emulators", platforms_dir: str = "platforms",
db_path: str = "database.json", include_archived: bool = False,
full: bool = False,
db_path: str = "database.json", full: bool = False,
) -> bool:
"""Build each narrowed platform pack in memory and compare it with verify.
@@ -753,7 +751,7 @@ def check_narrowed_packs(
profiles = load_emulator_profiles(emulators_dir)
registry = load_data_dir_registry(platforms_dir)
groups = group_identical_platforms(
list_registered_platforms(platforms_dir, include_archived=include_archived),
list_registered_platforms(platforms_dir),
platforms_dir,
)
was_offline = packresolve._OFFLINE
@@ -837,7 +835,7 @@ def split_differences(
def check_split_packs(
emulators_dir: str = "emulators", platforms_dir: str = "platforms",
db_path: str = "database.json", include_archived: bool = False,
db_path: str = "database.json",
) -> bool:
"""Build each platform's --split parts in memory and compare them with
the full pack: the parts must add up to it, path for path and byte for
@@ -863,7 +861,7 @@ def check_split_packs(
profiles = load_emulator_profiles(emulators_dir)
registry = load_data_dir_registry(platforms_dir)
groups = group_identical_platforms(
list_registered_platforms(platforms_dir, include_archived=include_archived),
list_registered_platforms(platforms_dir),
platforms_dir,
)
was_offline = packresolve._OFFLINE
@@ -917,8 +915,6 @@ def integrity_command(args: argparse.Namespace) -> list[str]:
sys.executable, "scripts/generate_pack.py", "--all", "--verify-packs",
"--output-dir", args.output_dir,
]
if args.include_archived:
cmd.append("--include-archived")
# Without the target a targeted run reported every system the target
# removed as missing from a pack that was exactly right. Without
# --all-variants the five narrowed packs were never opened, and an empty
@@ -1073,9 +1069,6 @@ def _truth_steps(args: argparse.Namespace, results: dict) -> bool:
def main() -> None:
parser = argparse.ArgumentParser(allow_abbrev=False, description="Run the full retrobios pipeline")
parser.add_argument(
"--include-archived", action="store_true", help="Include archived platforms"
)
parser.add_argument(
"--skip-packs",
action="store_true",
@@ -1311,8 +1304,6 @@ def main() -> None:
sys.executable, "scripts/verify.py", "--all", "--strict",
"--destinations", destinations_file,
]
if args.include_archived:
verify_cmd.append("--include-archived")
if args.target:
verify_cmd.extend(["--target", args.target])
if args.offline:
@@ -1338,11 +1329,9 @@ def main() -> None:
# upstream declaration no build can repair, so this reports and never gates.
# A crash is not a report: without --strict slots.py exits 0 on every
# conflict, so a non-zero code means it never finished reading.
# The same platforms and cores verify judged: without these flags the
# arbitration of an --include-archived run never looked at RetroPie.
# The same cores verify judged: without the target the arbitration of a
# targeted run judged cores the target does not ship.
slots_cmd = [sys.executable, "scripts/slots.py"]
if args.include_archived:
slots_cmd.append("--include-archived")
if args.target:
slots_cmd.extend(["--target", args.target])
ok, _ = run(slots_cmd, "3b slot arbitration")
@@ -1381,8 +1370,6 @@ def main() -> None:
"--output-dir",
args.output_dir,
]
if args.include_archived:
pack_cmd.append("--include-archived")
if args.offline:
pack_cmd.append("--offline")
else:
@@ -1412,8 +1399,6 @@ def main() -> None:
"--output-dir",
"install",
]
if args.include_archived:
manifest_cmd.append("--include-archived")
if args.offline:
manifest_cmd.append("--offline")
else:
@@ -1479,15 +1464,13 @@ def main() -> None:
elif pack_output and verify_output:
ok = check_consistency(
verify_output, pack_output,
release_pack_keys(include_archived=args.include_archived),
release_pack_keys(),
)
# Counts can agree while the paths differ, both ways at once.
ok = check_pack_destinations(destinations_file, pack_output) and ok
ok = check_emulator_packs(full=args.full_narrowings) and ok
ok = check_narrowed_packs(
include_archived=args.include_archived, full=args.full_narrowings,
) and ok
ok = check_split_packs(include_archived=args.include_archived) and ok
ok = check_narrowed_packs(full=args.full_narrowings) and ok
ok = check_split_packs() and ok
print(
f"--- {step_title(STEP_LABELS['consistency'])}, counts and paths: "
f"{'OK' if ok else 'FAILED'} ---"
+2 -5
View File
@@ -3532,17 +3532,14 @@ def _platform_cores(emulators_dir: str, platforms_dir: str) -> tuple[tuple[str,
(platform, frozenset(resolve_platform_cores(
load_platform_config(platform, platforms_dir), profiles
)))
for platform in list_registered_platforms(platforms_dir, include_archived=True)
for platform in list_registered_platforms(platforms_dir)
)
def shipping_platforms(
name: str, emulators_dir: str, platforms_dir: str = "platforms",
) -> list[str]:
"""The registered platforms whose cores resolve to this profile.
Archived platforms count: RetroPie still builds the packages it lists.
"""
"""The registered platforms whose cores resolve to this profile."""
return [
platform
for platform, cores in _platform_cores(
+2 -2
View File
@@ -68,10 +68,10 @@ def _extract_folders(record: dict, platforms_dir: str) -> dict[str, str]:
def without_pack(platforms_dir: str) -> list[str]:
"""Display names of the registered setups no release publishes a ZIP for."""
released = set(list_release_pack_platforms(platforms_dir, include_archived=True))
released = set(list_release_pack_platforms(platforms_dir))
names = [
str(load_platform_config(key, platforms_dir).get("platform", key))
for key in list_registered_platforms(platforms_dir, include_archived=True)
for key in list_registered_platforms(platforms_dir)
if key not in released
]
return sorted(names, key=str.casefold)
+2 -2
View File
@@ -58,7 +58,7 @@ def release_owners(platforms_dir: str = "platforms") -> dict[str, list[str]]:
"""
import generate_pack
platforms = list_release_pack_platforms(platforms_dir, include_archived=True)
platforms = list_release_pack_platforms(platforms_dir)
owners: dict[str, list[str]] = {}
for platform, names in generate_pack._expected_pack_names(
platforms, platforms_dir, None, None
@@ -150,7 +150,7 @@ def advertised(record: dict, platforms_dir: str = "platforms") -> dict:
"""
if not record:
return {}
released = set(list_release_pack_platforms(platforms_dir, include_archived=True))
released = set(list_release_pack_platforms(platforms_dir))
packs = {}
for name, pack in record.get("packs", {}).items():
serves = [p for p in pack.get("platforms", ()) if p in released]
+1 -1
View File
@@ -111,7 +111,7 @@ class SystemResolver:
use[str(sid)] = use.get(str(sid), 0) + 1
ids = set(use)
native: dict[str, dict[str, set[str]]] = {}
for platform in list_registered_platforms(platforms_dir, include_archived=True):
for platform in list_registered_platforms(platforms_dir):
config = load_platform_config(platform, platforms_dir)
ids.update(config.get("systems", {}))
names: dict[str, set[str]] = {}
+1 -1
View File
@@ -293,7 +293,7 @@ def platform_targets(platforms_dir: str) -> dict[str, set[str]]:
their MD5 is not an archive hash and is skipped.
"""
targets: dict[str, set[str]] = defaultdict(set)
for name in list_registered_platforms(platforms_dir, include_archived=True):
for name in list_registered_platforms(platforms_dir):
config = load_platform_config(name, platforms_dir)
for system in config.get("systems", {}).values():
for entry in system.get("files", []) or []:
+1 -1
View File
@@ -533,7 +533,7 @@ def _write_release(args: argparse.Namespace) -> None:
def _platform_archives() -> list[tuple[str, str]]:
"""(file name, system) for every file a registered platform declares."""
stated: list[tuple[str, str]] = []
for name in list_registered_platforms(str(_PLATFORMS_DIR), include_archived=True):
for name in list_registered_platforms(str(_PLATFORMS_DIR)):
config = load_platform_config(name, str(_PLATFORMS_DIR))
for system, data in (config.get("systems") or {}).items():
for entry in (data or {}).get("files") or []:
+1 -1
View File
@@ -69,7 +69,7 @@ def relevant_archives(emulators_dir: str, platforms_dir: str) -> set[str]:
archive = entry.get("archive", "")
if archive:
names.add(archive)
for platform_name in list_registered_platforms(platforms_dir, include_archived=True):
for platform_name in list_registered_platforms(platforms_dir):
config = load_platform_config(platform_name, platforms_dir)
for system in config.get("systems", {}).values():
for entry in system.get("files", []) or []:
+3 -3
View File
@@ -328,8 +328,8 @@ def _registry_in(platforms_dir: str) -> dict:
def setups() -> tuple[str, ...]:
"""The registered setups, archived ones included: each has its own page,
pack and file list, so every count of setups on the site is this one."""
"""The registered setups: each has its own page, pack and file list, so
every count of setups on the site is this one."""
return _setups_in(str(siteinputs.current().platforms))
@@ -337,7 +337,7 @@ def setups() -> tuple[str, ...]:
def _setups_in(platforms_dir: str) -> tuple[str, ...]:
from common import list_registered_platforms
return tuple(sorted(list_registered_platforms(platforms_dir, include_archived=True)))
return tuple(sorted(list_registered_platforms(platforms_dir)))
def setups_named(named: list[str]) -> str:
+5 -6
View File
@@ -288,9 +288,9 @@ def _handheld_guide(data: object, registry: dict, guides: dict, where: str) -> H
def load_guides(path: Path = GUIDES_FILE, registry: dict | None = None) -> dict:
"""The guides file, checked against the registry.
Every registered platform, archived ones included, has a guide and a
device group the file describes. A mistake raises GuideError naming the
key, so the site is never built from half a guide.
Every registered platform has a guide and a device group the file
describes. A mistake raises GuideError naming the key, so the site is
never built from half a guide.
"""
yaml = require_yaml()
if registry is None:
@@ -413,9 +413,8 @@ def _plain_folders(name: str) -> str:
def setup_count() -> int:
"""Setups the site covers: every registered platform, archived ones
included, since each keeps its page, its pack and its command. The one
count every page states."""
"""Setups the site covers: every registered platform, since each keeps
its page, its pack and its command. The one count every page states."""
return len(GUIDES)
+1 -1
View File
@@ -1436,7 +1436,7 @@ def system_search_aliases(sys_id: str) -> list[str]:
platforms_dir = str(siteinputs.current().platforms)
index: dict[str, set[str]] = {}
for platform in list_registered_platforms(platforms_dir, include_archived=True):
for platform in list_registered_platforms(platforms_dir):
config = load_platform_config(platform, platforms_dir)
names = platform_system_names(config, list(config.get("systems", {})))
for owner, spelled in names.items():
+1 -9
View File
@@ -999,10 +999,6 @@ def main() -> int:
help="emulator profile directory (default: %(default)s)",
)
parser.add_argument("--json", action="store_true", help="JSON output")
parser.add_argument(
"--include-archived", action="store_true",
help="with all platforms, the archived ones too (RetroPie)",
)
parser.add_argument(
"--target", "-t", help="judge only the cores a hardware target builds",
)
@@ -1028,14 +1024,10 @@ def main() -> int:
profiles = load_emulator_profiles(args.emulators_dir)
if args.platform:
require_platform(args.platform, args.platforms_dir)
if args.include_archived and args.platform:
parser.error("--include-archived widens the list of all platforms; --platform names one")
names = (
[args.platform]
if args.platform
else list_registered_platforms(
args.platforms_dir, include_archived=args.include_archived
)
else list_registered_platforms(args.platforms_dir)
)
target_cores: dict[str, set[str] | None] = {}
if args.target:
+1 -1
View File
@@ -193,7 +193,7 @@ def load_platform_hashes(platforms_dir: str, db: dict | None = None) -> dict:
# and a Recalbox md5 field split into the revisions it lists. The raw
# file kept "a,b,c" as one value, so a PR adding the seventh accepted
# falcon.img was told its hash differs.
for name in list_registered_platforms(platforms_dir, include_archived=True):
for name in list_registered_platforms(platforms_dir):
try:
config = load_platform_config(name, platforms_dir)
except (OSError, yaml.YAMLError) as exc:
+4 -6
View File
@@ -416,8 +416,8 @@ def _registry_contract() -> list[str]:
"""The registry against its schema, and a release that still ships a pack.
`release_pack` is the only list of the ZIPs a release publishes: a
registry where no active platform carries it would cut a release with
no pack at all, and every download link would lead to the command.
registry where no platform carries it would cut a release with no pack
at all, and every download link would lead to the command.
"""
registry_path = ROOT / "platforms" / "_registry.yml"
if not registry_path.is_file():
@@ -428,12 +428,10 @@ def _registry_contract() -> list[str]:
out = _errors(_validator("registry.schema.json"), registry, label)
platforms = registry.get("platforms") or {}
if isinstance(platforms, dict) and not any(
isinstance(entry, dict)
and entry.get("release_pack") is True
and entry.get("status", "active") == "active"
isinstance(entry, dict) and entry.get("release_pack") is True
for entry in platforms.values()
):
out.append(f"{label}: no active platform has release_pack: true")
out.append(f"{label}: no platform has release_pack: true")
return out
+2 -2
View File
@@ -410,9 +410,9 @@ def _check_get_pages(
if not (platforms_dir / "_registry.yml").is_file():
return [f"no platform registry in {platforms_dir}"]
record = release_record.load_advertised(record_path, str(platforms_dir))
released = set(list_release_pack_platforms(str(platforms_dir), include_archived=True))
released = set(list_release_pack_platforms(str(platforms_dir)))
issues: list[str] = []
for platform in list_registered_platforms(str(platforms_dir), include_archived=True):
for platform in list_registered_platforms(str(platforms_dir)):
rel = f"{release_record.GET_DIR}/{platform}/index.html"
page = pages.get((site / rel).resolve())
if page is None or page.redirect:
+2 -11
View File
@@ -18,7 +18,6 @@ Cross-references emulator profiles to detect undeclared files used by available
Usage:
python scripts/verify.py --all
python scripts/verify.py --platform batocera
python scripts/verify.py --all --include-archived
python scripts/verify.py --all --json
"""
@@ -2366,7 +2365,7 @@ def verify_emulator(
emu_results: dict[str, list[tuple[dict, bool, bool]]] = {}
for file_entry in files:
# Skip archived files (verified as archive units above)
# Skip archive members (verified as archive units above)
if file_entry.get("archive"):
continue
@@ -2754,7 +2753,6 @@ def _refuse_listing_narrowings(
("--region", getattr(args, "region", None)),
("--target", getattr(args, "target", None)),
("--standalone", args.standalone),
("--include-archived", args.include_archived),
("--json", args.json),
("--strict", args.strict),
("--verbose", args.verbose),
@@ -2774,7 +2772,6 @@ def _refuse_mode_flags(args: argparse.Namespace, parser: argparse.ArgumentParser
"--standalone requires --emulator or --system"),
(args.standalone and args.platform,
"--standalone is incompatible with --platform"),
(args.include_archived and not args.all, "--include-archived requires --all"),
(args.target and not (args.platform or args.all),
"--target requires --platform or --all"),
(args.target and (args.emulator or (args.system and not args.platform)),
@@ -2840,10 +2837,6 @@ def _main(held: contextlib.ExitStack) -> None:
parser.add_argument(
"--list-platforms", action="store_true", help="List available platforms"
)
parser.add_argument(
"--include-archived", action="store_true",
help="With --all, also verify the archived platforms (RetroPie)",
)
parser.add_argument("--target", "-t", help="Hardware target (e.g., switch, rpi4)")
parser.add_argument(
"--region", help="Region priority list, best first (e.g. us,eu,jp)"
@@ -2981,9 +2974,7 @@ def _main(held: contextlib.ExitStack) -> None:
# Platform mode (existing)
if args.all:
platforms = list_registered_platforms(
args.platforms_dir, include_archived=args.include_archived
)
platforms = list_registered_platforms(args.platforms_dir)
elif args.platform:
platforms = [args.platform]
else:
+1 -1
View File
@@ -114,7 +114,7 @@ class HeldArchivesMatchTheirRecipes(unittest.TestCase):
pinned: set[str] = set()
zipped: dict[str, set[tuple[str, str]]] = defaultdict(set)
platforms_dir = str(REPO_ROOT / "platforms")
for platform in list_registered_platforms(platforms_dir, include_archived=True):
for platform in list_registered_platforms(platforms_dir):
config = load_platform_config(platform, platforms_dir)
for system in config.get("systems", {}).values():
for entry in system.get("files", []) or []:
+8 -11
View File
@@ -594,7 +594,7 @@ class EverySharedGroupServesAPlatform(unittest.TestCase):
groups = set((yaml.safe_load(handle) or {}).get("shared_groups") or {})
served = {
entry.get("shared_group")
for name in list_registered_platforms(platforms_dir, include_archived=True)
for name in list_registered_platforms(platforms_dir)
for system in load_platform_config(name, platforms_dir).get("systems", {}).values()
for entry in system.get("files", [])
}
@@ -977,8 +977,7 @@ class PipelineRegressions(unittest.TestCase):
import argparse # noqa: PLC0415
def command(**flags) -> list[str]:
base = {"output_dir": "dist", "include_archived": False,
"target": None, "all_variants": False}
base = {"output_dir": "dist", "target": None, "all_variants": False}
return pipeline.integrity_command(argparse.Namespace(**{**base, **flags}))
self.assertIn("--all-variants", command(all_variants=True))
@@ -4258,7 +4257,6 @@ class EveryToolRefusesAFlagItWouldDrop(unittest.TestCase):
["--list-systems", "--region", "us"],
["--emulator", "handy", "--target", "switch"],
["--platform", "retroarch", "--standalone"],
["--platform", "retroarch", "--include-archived"],
],
"scripts/pipeline.py": [
["--offline", "--skip-packs", "--source", "truth"],
@@ -4452,7 +4450,7 @@ class TheIssueFormListsEveryPlatform(unittest.TestCase):
item["attributes"]["options"] for item in form["body"]
if item.get("id") == "platform"
)
for name in list_registered_platforms(str(ROOT / "platforms"), include_archived=True):
for name in list_registered_platforms(str(ROOT / "platforms")):
display = load_platform_config(name, str(ROOT / "platforms"))["platform"]
self.assertIn(display, options)
@@ -5090,7 +5088,7 @@ class BundledFilesAreNotToSource(unittest.TestCase):
sys.path.insert(0, str(ROOT / "scripts"))
from common import list_registered_platforms, load_emulator_profiles
known = set(list_registered_platforms(str(ROOT / "platforms"), include_archived=True))
known = set(list_registered_platforms(str(ROOT / "platforms")))
unknown = sorted({
(name, platform)
for name, profile in load_emulator_profiles(str(ROOT / "emulators")).items()
@@ -6001,7 +5999,7 @@ class AHashScanningCoreContradictsNoPlatform(unittest.TestCase):
db = json.loads((root / "database.json").read_text(encoding="utf-8"))
claimed = [
(platform, conflict.destination)
for platform in list_registered_platforms(str(root / "platforms"), include_archived=True)
for platform in list_registered_platforms(str(root / "platforms"))
for conflict in slots.scan_platform(platform, profiles, db, str(root / "platforms"))
if any(claim.emulator == "ares" for claim in conflict.profile_claims)
]
@@ -6616,9 +6614,9 @@ class TheReleaseBuildsEveryServedPackInOneRun(unittest.TestCase):
"""The release ran the pipeline, then built and checked RetroPie by hand:
the consistency check between verify and the pack never covered it."""
def test_the_pipeline_step_includes_archived_platforms(self):
def test_the_pipeline_step_covers_every_platform(self):
page = (ROOT / "wiki" / "release-process.md").read_text(encoding="utf-8")
self.assertIn("python scripts/pipeline.py --include-archived", page)
self.assertIn("python scripts/pipeline.py\n", page)
self.assertNotIn("generate_pack.py --platform retropie", page)
@@ -7187,8 +7185,7 @@ class AnUnknownSystemIsRefusedBeforeAnyWork(unittest.TestCase):
parser = argparse.ArgumentParser()
args = argparse.Namespace(
system="sony-playstation", platform="retroarch", all=False,
platforms_dir=str(ROOT / "platforms"), include_archived=False,
manifest=False,
platforms_dir=str(ROOT / "platforms"), manifest=False,
)
generate_pack._resolve_systems(args, parser)
self.assertEqual(args.system_ids, ["sony-playstation"])
+1 -1
View File
@@ -45,7 +45,7 @@ class DeclaredFilesResolveWithoutDataCaches(unittest.TestCase):
db = load_database("database.json")
zip_contents = build_zip_contents_index(db)
offenders = []
for platform in list_registered_platforms(str(platforms_dir), include_archived=True):
for platform in list_registered_platforms(str(platforms_dir)):
config = load_platform_config(platform, str(platforms_dir))
for sys_id, system in config.get("systems", {}).items():
for entry in system.get("files", []) or []:
-1
View File
@@ -82,7 +82,6 @@ class ThePackReadmeNamesTheSpellingsItLeavesOut(unittest.TestCase):
Path("platforms/p.yml").write_text(yaml.dump(self.config))
Path("platforms/_registry.yml").write_text(yaml.dump({"platforms": {"p": {
"status": "active",
"install": {"detect": [{"os": os_name} for os_name in systems]},
}}}))
common._platform_config_cache.clear()
+1 -1
View File
@@ -108,7 +108,7 @@ class EveryReaderHasTheTreeOrAReason(unittest.TestCase):
registry = common.load_data_dir_registry(platforms_dir)
profiles = common.load_emulator_profiles(str(REPO_ROOT / "emulators"))
unexplained = []
for platform in common.list_registered_platforms(platforms_dir, include_archived=True):
for platform in common.list_registered_platforms(platforms_dir):
config = common.load_platform_config(platform, platforms_dir)
handed = common.standalone_set(config)
for core in sorted(common.resolve_platform_cores(config, profiles)):
+1 -1
View File
@@ -203,7 +203,7 @@ class AFileNobodyHoldsIsAnOmission(unittest.TestCase):
"systems": {"diablo": {"files": []}},
}))
(root / "platforms" / "_registry.yml").write_text(yaml.safe_dump({
"platforms": {"fakeplat": {"config": "fakeplat.yml", "status": "active"}},
"platforms": {"fakeplat": {"config": "fakeplat.yml"}},
}))
(root / "emulators" / "devil.yml").write_text(yaml.safe_dump({
"emulator": "Devil", "type": "libretro", "cores": ["devil"],
+1 -1
View File
@@ -82,7 +82,7 @@ class _Collection(unittest.TestCase):
}
Path("platforms/exist.yml").write_text(yaml.dump(self.config))
Path("platforms/_registry.yml").write_text(
yaml.dump({"platforms": {"exist": {"status": "active"}}})
yaml.dump({"platforms": {"exist": {"config": "exist.yml"}}})
)
Path("emulators/sized.yml").write_text(SIZED)
Path("emulators/unsized.yml").write_text(UNSIZED)
+2 -2
View File
@@ -286,7 +286,7 @@ class TheSmsBiosKeepsItsNames(unittest.TestCase):
os.chdir(tmp)
try:
Path("platforms").mkdir()
Path("platforms/_registry.yml").write_text("platforms:\n p:\n status: active\n")
Path("platforms/_registry.yml").write_text("platforms:\n p:\n config: p.yml\n")
Path("platforms/p.yml").write_text(
"systems:\n sms:\n files:\n"
' - name: "bios_U.sms"\n sha1: "v13"\n'
@@ -306,7 +306,7 @@ class TheSmsBiosKeepsItsNames(unittest.TestCase):
os.chdir(tmp)
try:
Path("platforms").mkdir()
Path("platforms/_registry.yml").write_text("platforms:\n p:\n status: active\n")
Path("platforms/_registry.yml").write_text("platforms:\n p:\n config: p.yml\n")
Path("platforms/p.yml").write_text(
"systems:\n sms:\n files:\n"
' - name: "bios_U.sms"\n sha1: "v13"\n'
+1 -1
View File
@@ -87,7 +87,7 @@ class DirectoryFixture(unittest.TestCase):
(self.platforms / "demo.yml").write_text(yaml.dump(PLATFORM))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"demo": {"status": "active"}}})
yaml.dump({"platforms": {"demo": {"config": "demo.yml"}}})
)
common._emulator_profiles_cache.clear()
+6 -7
View File
@@ -1954,13 +1954,13 @@ class APlatformListIsDescribedNotJudged(unittest.TestCase):
self.assertIsNone(pattern.search(page.read_text(encoding="utf-8")))
class ArchivedIsWhatTheToolsDo(unittest.TestCase):
"""The README and the setup pages said an archived platform is no longer
scraped on a schedule, and adding-a-platform said `active` feeds a CI
matrix and the scrapers. No workflow calls list_platforms.py and no tool
reads `schedule:`: what `archived` changes is which full runs include it."""
class NoTextClaimsAScheduledScrape(unittest.TestCase):
"""The README and the setup pages said a platform is scraped on a
schedule, and adding-a-platform said the registry feeds a CI matrix and
the scrapers. No workflow calls list_platforms.py and no tool reads
`schedule:`."""
def test_no_text_ties_the_status_to_automation(self):
def test_no_text_ties_the_registry_to_automation(self):
import generate_pack # noqa: PLC0415
import generate_readme # noqa: PLC0415
import generate_site # noqa: PLC0415
@@ -1979,7 +1979,6 @@ class ArchivedIsWhatTheToolsDo(unittest.TestCase):
for claim in ("scraped on a schedule", "read again on a schedule",
"CI matrix", "without the automation"):
self.assertNotIn(claim, text)
self.assertIn("`--include-archived`", texts["adding-a-platform.md"])
for workflow in (ROOT / ".github" / "workflows").glob("*.yml"):
self.assertNotIn("list_platforms", workflow.read_text(encoding="utf-8"))
+4 -4
View File
@@ -176,7 +176,7 @@ class DownloadLinksKeepTheirAddress(unittest.TestCase):
"bizhawk": {"platform": "BizHawk"},
"romm": {"platform": "RomM"},
}
rows = download_table(coverages, set(), {}, SPLIT_RECORD)[2:]
rows = download_table(coverages, {}, SPLIT_RECORD)[2:]
self.assertEqual(len(rows), 3)
for row in rows:
self.assertNotIn("/releases/", row)
@@ -191,8 +191,8 @@ class DownloadLinksKeepTheirAddress(unittest.TestCase):
readme = _read("README.md")
table = _section(readme, "## Download BIOS packs")
self.assertNotIn("/releases/download/", table)
released = set(list_release_pack_platforms("platforms", include_archived=True))
for key in list_registered_platforms("platforms", include_archived=True):
released = set(list_release_pack_platforms("platforms"))
for key in list_registered_platforms("platforms"):
with self.subTest(platform=key):
self.assertIn(f"]({SITE_URL}get/{key}/)", table)
row = next(line for line in table.splitlines()
@@ -216,7 +216,7 @@ class DownloadLinksKeepTheirAddress(unittest.TestCase):
record = release_record.load_advertised()
page = _which_pack()
for key in list_registered_platforms("platforms", include_archived=True):
for key in list_registered_platforms("platforms"):
if release_record.pack_for(key, record) is not None:
continue
label = load_platform_config(key, "platforms").get("platform", key)
+4 -12
View File
@@ -2486,7 +2486,7 @@ class TestE2E(unittest.TestCase):
},
}
registry = {"platforms": {"splitplat": {"status": "active"}}}
registry = {"platforms": {"splitplat": {"config": "splitplat.yml"}}}
with open(os.path.join(plat_dir, "_registry.yml"), "w") as f:
yaml.dump(registry, f)
plat_cfg = {
@@ -2708,7 +2708,7 @@ class TestE2E(unittest.TestCase):
},
}
registry = {"platforms": {"testplat": {"status": "active"}}}
registry = {"platforms": {"testplat": {"config": "testplat.yml"}}}
with open(os.path.join(plat_dir, "_registry.yml"), "w") as f:
yaml.dump(registry, f)
plat_cfg = {
@@ -4474,7 +4474,6 @@ class TestE2E(unittest.TestCase):
"platforms": {
"testmerge": {
"config": "testmerge.yml",
"status": "active",
"cores": ["core_a", "core_b"],
}
}
@@ -5729,8 +5728,8 @@ struct BurnDriver BurnDrvneogeo = {
os.makedirs(plat_dir)
registry = {
"platforms": {
"alpha": {"status": "active"},
"beta": {"status": "active"},
"alpha": {"config": "alpha.yml"},
"beta": {"config": "beta.yml"},
}
}
with open(os.path.join(plat_dir, "_registry.yml"), "w") as f:
@@ -6215,13 +6214,6 @@ struct BurnDriver BurnDrvneogeo = {
(["--platform", "retroarch", "--offline"], ["--no-refresh-data"], "refuse"),
(["--platform", "retroarch", "--refresh-data"], ["--no-refresh-data"],
"refuse"),
# Only --all chooses among platforms, as verify.py already says.
(["--platform", "retroarch"], ["--include-archived"], "refuse"),
(["--emulator", "handy"], ["--include-archived"], "refuse"),
(["--system", "atari-lynx"], ["--include-archived"], "refuse"),
(["--from-md5", "d8f1"], ["--include-archived"], "refuse"),
(["--platform", "retroarch", "--verify-packs"], ["--include-archived"],
"refuse"),
# Documented as adding core files, read by nothing.
(["--emulator", "handy"], ["--include-extras"], "refuse"),
(["--system", "atari-lynx"], ["--include-extras"], "refuse"),
+4 -4
View File
@@ -593,13 +593,13 @@ class ExporterRegistry(unittest.TestCase):
def test_every_registered_platform_can_be_exported(self):
"""A platform with no exporter is a platform we cannot hand back.
Archived included: the export is a correction we offer, not a
release artefact, and RetroPie's users are not fewer for it.
The export is a correction we offer, not a release artefact: a
platform without a release pack is exported all the same.
"""
exporters = discover_exporters()
registered = set(
list_registered_platforms(
str(REPO_ROOT / "platforms"), include_archived=True
str(REPO_ROOT / "platforms")
)
)
self.assertEqual(registered - set(exporters) - _without_native_list(), set())
@@ -641,7 +641,7 @@ class ExporterRegistry(unittest.TestCase):
exporters = set(discover_exporters())
registered = set(
list_registered_platforms(
str(REPO_ROOT / "platforms"), include_archived=True
str(REPO_ROOT / "platforms")
)
)
self.assertEqual(exporters - registered, set())
+3 -3
View File
@@ -3,8 +3,8 @@
Both kept a hand table keyed by display name. RomM's row said
`bios/{platform_slug}/` while the pack already carries one folder per slug
at its root, so following it nested every file one level too deep. The
README's RetroDECK sentence sat inside the archived-platforms block and
would have vanished the day RetroPie became active again.
README's RetroDECK sentence sat inside a block about RetroPie and would
have vanished with it.
"""
from __future__ import annotations
@@ -34,7 +34,7 @@ class ExtractTargets(unittest.TestCase):
templated = [d for d, folder in extract_targets(PLATFORMS) if "{" in folder]
self.assertEqual(templated, [])
def test_a_pack_with_its_own_root_is_explained_whatever_is_archived(self):
def test_a_pack_with_its_own_root_is_explained(self):
rooted = sorted(
load_platform_config(key, PLATFORMS).get("platform", key)
for key in load_platform_registry(PLATFORMS)
+1 -1
View File
@@ -43,7 +43,7 @@ class FrontendDigestDecides(unittest.TestCase):
(root / "emulators").mkdir()
self.emulators = root / "emulators"
(self.platforms / "_registry.yml").write_text(
yaml.safe_dump({"platforms": {"digestplat": {"status": "active"}}})
yaml.safe_dump({"platforms": {"digestplat": {"config": "digestplat.yml"}}})
)
(self.platforms / "digestplat.yml").write_text(yaml.safe_dump({
"platform": "DigestPlat",
+2 -2
View File
@@ -2243,7 +2243,7 @@ class TestAvailablePlatforms(unittest.TestCase):
self.assertEqual(set(install.AVAILABLE_PLATFORMS), manifests)
def test_every_registered_platform_is_installable(self):
"""Archived platforms still ship packs, so they still need a manifest."""
"""The installer serves every registered platform from its manifest."""
import sys as _sys
_sys.path.insert(0, str(REPO_ROOT / "scripts"))
@@ -2251,7 +2251,7 @@ class TestAvailablePlatforms(unittest.TestCase):
registered = set(
list_registered_platforms(
str(REPO_ROOT / "platforms"), include_archived=True
str(REPO_ROOT / "platforms")
)
)
missing = registered - set(install.AVAILABLE_PLATFORMS)
+3 -3
View File
@@ -1026,7 +1026,7 @@ class AHeldFileTheInstallerCannotFetch(unittest.TestCase):
"systems": {"sys": {"files": []}},
}))
Path("platforms/_registry.yml").write_text(
json.dumps({"platforms": {"p": {"status": "active"}}})
json.dumps({"platforms": {"p": {"config": "p.yml"}}})
)
Path("emulators").mkdir()
Path("emulators/core.yml").write_text(json.dumps({
@@ -1102,7 +1102,7 @@ class AStandaloneLayoutBelowTheBiosFolder(unittest.TestCase):
"standalone_cores": ["dsa"], "systems": {"sys": {"files": []}},
}))
Path("platforms/_registry.yml").write_text(
json.dumps({"platforms": {"p": {"status": "active"}}})
json.dumps({"platforms": {"p": {"config": "p.yml"}}})
)
Path("emulators").mkdir()
Path("emulators/dsa.yml").write_text(json.dumps({
@@ -1189,7 +1189,7 @@ class AnExtraBesideItsDeclarationBelowTheBiosFolder(unittest.TestCase):
]}},
}))
Path("platforms/_registry.yml").write_text(
json.dumps({"platforms": {"p": {"status": "active"}}})
json.dumps({"platforms": {"p": {"config": "p.yml"}}})
)
Path("emulators").mkdir()
Path("emulators/dsa.yml").write_text(json.dumps({
+2 -2
View File
@@ -87,7 +87,7 @@ class _Collection(unittest.TestCase):
Path("emulators/quasi88.yml").write_text(QUASI88)
Path("platforms/handheld.yml").write_text(yaml.dump(PLATFORM))
Path("platforms/_registry.yml").write_text(
yaml.dump({"platforms": {"handheld": {"status": "active"}}})
yaml.dump({"platforms": {"handheld": {"config": "handheld.yml"}}})
)
subprocess.run(
[sys.executable, str(REPO_ROOT / "scripts/generate_db.py"), "--force",
@@ -245,7 +245,7 @@ class NoHashlessNameMatchCrossesMakers(unittest.TestCase):
if len(parts := entry["path"].split("/")) >= 3 and parts[0] == "bios"
} - common.BUCKET_FOLDERS
crossings = []
for platform in common.list_registered_platforms("platforms", include_archived=True):
for platform in common.list_registered_platforms("platforms"):
config = common.load_platform_config(platform, "platforms")
for sys_id, system in config.get("systems", {}).items():
makers = {f for f in folders if common.names_manufacturer(sys_id, f)}
+1 -1
View File
@@ -92,7 +92,7 @@ class SharedGroupsAddNoSecondSpelling(unittest.TestCase):
}
platforms_dir = str(REPO_ROOT / "platforms")
respelled: list[str] = []
for name in list_registered_platforms(platforms_dir, include_archived=True):
for name in list_registered_platforms(platforms_dir):
config = load_platform_config(name, platforms_dir)
own: dict[str, str] = {}
grouped: list[tuple[str, str]] = []
+1 -1
View File
@@ -101,7 +101,7 @@ class _Fixture(unittest.TestCase):
}
(self.platforms / "handheld.yml").write_text(yaml.dump(platform))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"handheld": {"status": "active"}}})
yaml.dump({"platforms": {"handheld": {"config": "handheld.yml"}}})
)
common._platform_config_cache.clear()
common._emulator_profiles_cache.clear()
+1 -1
View File
@@ -176,7 +176,7 @@ class NarrowedPacksAgainstTheirOwnList(unittest.TestCase):
]}}}
Path("platforms/p.yml").write_text(yaml.safe_dump(config), encoding="utf-8")
Path("platforms/_registry.yml").write_text(
yaml.safe_dump({"platforms": {"p": {"status": "active"}}}), encoding="utf-8"
yaml.safe_dump({"platforms": {"p": {"config": "p.yml"}}}), encoding="utf-8"
)
Path("emulators").mkdir()
Path("emulators/c.yml").write_text(yaml.safe_dump({
+2 -3
View File
@@ -102,7 +102,7 @@ class PackCountFixture(unittest.TestCase):
self.registry = {"demo-data": {"local_cache": str(self.data)}}
(self.platforms / "demo.yml").write_text(yaml.dump(PLATFORM))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"demo": {"status": "active"}}})
yaml.dump({"platforms": {"demo": {"config": "demo.yml"}}})
)
(self.emulators / "demo.yml").write_text(PROFILE)
common._emulator_profiles_cache.clear()
@@ -326,7 +326,6 @@ class TheTableDescribesTheRelease(unittest.TestCase):
def test_the_download_table_gives_the_file_count_of_each_pack(self):
rows = generate_readme.download_table(
{"retroarch": {"platform": "RetroArch"}, "vita3k": {"platform": "Vita3K"}},
set(),
{"RetroArch": "`system/`"},
RECORD,
)
@@ -506,7 +505,7 @@ class RefusedFilesAreNotPresent(unittest.TestCase):
}
Path("platforms/hashy.yml").write_text(yaml.dump(config))
Path("platforms/_registry.yml").write_text(
yaml.dump({"platforms": {"hashy": {"status": "active"}}})
yaml.dump({"platforms": {"hashy": {"config": "hashy.yml"}}})
)
common._platform_config_cache.clear()
self.addCleanup(common._platform_config_cache.clear)
+1 -1
View File
@@ -45,7 +45,7 @@ def _registered_platforms() -> list[str]:
sys.path.insert(0, os.path.join(REPO_ROOT, "scripts"))
from common import list_registered_platforms
return list_registered_platforms(PLATFORMS_DIR, include_archived=True)
return list_registered_platforms(PLATFORMS_DIR)
def _profiles_newer_than(path: str, platform_name: str) -> list[str]:
+2 -2
View File
@@ -64,7 +64,7 @@ class ConflictingDeclarations(unittest.TestCase):
}
(self.platforms / "conflict.yml").write_text(yaml.dump(platform))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"conflict": {"status": "active"}}})
yaml.dump({"platforms": {"conflict": {"config": "conflict.yml"}}})
)
common._platform_config_cache.clear()
common._emulator_profiles_cache.clear()
@@ -133,7 +133,7 @@ class AnExtraDoesNotPassAHashCheck(unittest.TestCase):
}
(self.platforms / "hashed.yml").write_text(yaml.dump(platform))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"hashed": {"status": "active"}}})
yaml.dump({"platforms": {"hashed": {"config": "hashed.yml"}}})
)
profile = {
"emulator": "FX", "type": "libretro", "cores": ["fxcore"],
+4 -4
View File
@@ -35,9 +35,9 @@ class AChildLeavesItsParentAlone(unittest.TestCase):
"systems": {"sys-a": {"files": [{"name": "child.bin"}]},
"sys-new": {"files": [{"name": "new.bin"}]}}}},
"_registry.yml": {"platforms": {
"parent": {"status": "active", "scraper": "parent", "hash_type": "md5",
"cores": "all_libretro"},
"child": {"status": "archived", "scraper": "child"},
"parent": {"device_group": "computer", "scraper": "parent",
"hash_type": "md5", "cores": "all_libretro"},
"child": {"device_group": "gaming-os", "scraper": "child"},
}},
}
for name, data in files.items():
@@ -59,7 +59,7 @@ class AChildLeavesItsParentAlone(unittest.TestCase):
def test_the_child_reads_its_own_registry_entry_first(self):
self.load("parent")
child = self.load("child")
self.assertEqual((child["status"], child["scraper"]), ("archived", "child"))
self.assertEqual((child["device_group"], child["scraper"]), ("gaming-os", "child"))
# What it does not say comes from its parent's entry.
self.assertEqual((child["hash_type"], child["cores"]), ("md5", "all_libretro"))
self.assertEqual(
+40 -20
View File
@@ -61,22 +61,17 @@ def _which_pack() -> str:
class TheFieldIsTheList(unittest.TestCase):
def test_the_registry_marks_the_most_downloaded_setups(self):
self.assertEqual(
set(common.list_release_pack_platforms(PLATFORMS, include_archived=True)), RELEASED
set(common.list_release_pack_platforms(PLATFORMS)), RELEASED
)
def test_only_marked_platforms_are_listed_and_archived_ones_on_request(self):
def test_only_marked_platforms_are_listed(self):
with tempfile.TemporaryDirectory(dir=TMP_ROOT) as tmp:
platforms_dir = _write_registry(Path(tmp), {
"big": {"status": "active", "release_pack": True},
"small": {"status": "active"},
"off": {"status": "active", "release_pack": False},
"old": {"status": "archived", "release_pack": True},
"big": {"release_pack": True},
"small": {},
"off": {"release_pack": False},
})
self.assertEqual(common.list_release_pack_platforms(platforms_dir), ["big"])
self.assertEqual(
common.list_release_pack_platforms(platforms_dir, include_archived=True),
["big", "old"],
)
self.assertEqual(
common.list_registered_platforms(platforms_dir), ["big", "off", "small"]
)
@@ -95,6 +90,36 @@ class TheFieldIsTheList(unittest.TestCase):
self.assertTrue(list(validator.iter_errors(registry)))
class NoPlatformIsSetApart(unittest.TestCase):
"""A registry `status: archived` kept RetroPie out of every full run
unless --include-archived was typed, after release_pack had become the
only thing that sets platforms apart. A platform is registered or not."""
def test_the_registry_carries_no_status(self):
registry = yaml.safe_load((REPO_ROOT / "platforms" / "_registry.yml").read_text())
flagged = sorted(
name for name, entry in registry["platforms"].items() if "status" in (entry or {})
)
self.assertEqual(flagged, [])
def test_the_schema_declares_no_status(self):
text = (REPO_ROOT / "schemas" / "registry.schema.json").read_text(encoding="utf-8")
for word in ('"status"', "archived"):
with self.subTest(word=word):
self.assertNotIn(word, text)
def test_no_script_knows_the_word(self):
found = sorted(
str(path.relative_to(REPO_ROOT))
for path in (REPO_ROOT / "scripts").rglob("*.py")
if any(
word in path.read_text(encoding="utf-8").lower()
for word in ("archived", "include_archived", "include-archived")
)
)
self.assertEqual(found, [])
class TheBuildReadsTheField(unittest.TestCase):
def _groups(self, *argv: str) -> list[str]:
import generate_pack
@@ -116,15 +141,10 @@ class TheBuildReadsTheField(unittest.TestCase):
def test_all_builds_the_release_packs_only(self):
self.assertEqual(set(self._groups("--all", "--offline")), RELEASED)
def test_archived_platforms_without_the_field_stay_out(self):
self.assertEqual(
set(self._groups("--all", "--include-archived", "--offline")), RELEASED
)
def test_manifests_cover_every_platform(self):
self.assertEqual(
set(self._groups("--all", "--manifest", "--offline")),
set(common.list_registered_platforms(PLATFORMS, include_archived=True)),
set(common.list_registered_platforms(PLATFORMS)),
)
def test_the_release_owners_are_the_marked_platforms(self):
@@ -177,7 +197,7 @@ class APlatformWithoutAPackFailsNoCheck(unittest.TestCase):
def test_the_pipeline_passes_the_release_platforms(self):
source = (REPO_ROOT / "scripts" / "pipeline.py").read_text(encoding="utf-8")
self.assertIn("release_pack_keys(include_archived=args.include_archived)", source)
self.assertIn("release_pack_keys()", source)
def test_verify_packs_sweeps_the_release_packs_only(self):
import subprocess
@@ -314,7 +334,7 @@ class EveryPlatformHasAPermanentPage(unittest.TestCase):
record = release_record.load_advertised()
with contextlib.chdir(REPO_ROOT):
for name in common.list_registered_platforms(PLATFORMS, include_archived=True):
for name in common.list_registered_platforms(PLATFORMS):
page = generate_site.generate_get_page(name)
pack = release_record.pack_for(name, record)
with self.subTest(platform=name):
@@ -340,7 +360,7 @@ class EveryPlatformHasAPermanentPage(unittest.TestCase):
docs = REPO_ROOT / "website" / "src" / "content" / "docs" / release_record.GET_DIR
if not docs.is_dir():
self.skipTest("the site is not generated")
for name in common.list_registered_platforms(PLATFORMS, include_archived=True):
for name in common.list_registered_platforms(PLATFORMS):
with self.subTest(platform=name):
self.assertTrue(any(docs.glob(f"{name}.md*")))
@@ -368,7 +388,7 @@ class TheSiteCheckResolvesEveryGetPage(unittest.TestCase):
record_path.write_text(json.dumps(record), encoding="utf-8")
pages = {
f"get/{name}/index.html": "install.sh"
for name in common.list_registered_platforms(PLATFORMS, include_archived=True)
for name in common.list_registered_platforms(PLATFORMS)
}
pages["get/romm/index.html"] = f'<a href="{download}v1/RomM_BIOS_Pack.zip">x</a>'
pages["get/knulli/index.html"] = f'<a href="{download}v1/K_BIOS_Pack.zip">x</a>'
+1 -1
View File
@@ -69,7 +69,7 @@ class RequiredTwins(unittest.TestCase):
}
(self.platforms / "twins.yml").write_text(yaml.dump(platform))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"twins": {"status": "active"}}})
yaml.dump({"platforms": {"twins": {"config": "twins.yml"}}})
)
common._platform_config_cache.clear()
common._emulator_profiles_cache.clear()
+2 -2
View File
@@ -198,7 +198,7 @@ class ReportsOverAFixture(unittest.TestCase):
platforms.mkdir()
pinned = hashlib.md5(self.pool.build(self.recipe)).hexdigest()
(platforms / "_registry.yml").write_text(
"platforms:\n demo:\n config: demo.yml\n status: active\n",
"platforms:\n demo:\n config: demo.yml\n",
encoding="utf-8",
)
(platforms / "demo.yml").write_text(
@@ -228,7 +228,7 @@ class ReportsOverAFixture(unittest.TestCase):
platforms = self.root / "platforms"
platforms.mkdir()
(platforms / "_registry.yml").write_text(
"platforms:\n demo:\n config: demo.yml\n status: active\n",
"platforms:\n demo:\n config: demo.yml\n",
encoding="utf-8",
)
(platforms / "demo.yml").write_text(
+1 -1
View File
@@ -932,7 +932,7 @@ class PickerFollowsTheRegistry(unittest.TestCase):
def test_picker_offers_only_registry_packs(self):
offered = _picker_packs()
self.assertTrue(offered)
released = set(list_release_pack_platforms(str(REPO_ROOT / "platforms"), include_archived=True))
released = set(list_release_pack_platforms(str(REPO_ROOT / "platforms")))
self.assertLessEqual(offered, released)
def test_picker_setup_and_get_pages_agree(self):
+1 -1
View File
@@ -30,7 +30,7 @@ class SiteCoreCounts(unittest.TestCase):
profiles = load_emulator_profiles(str(REPO_ROOT / "emulators"))
unique = {k: v for k, v in profiles.items() if v.get("type") not in ("alias", "test")}
coverages = {}
for name in list_registered_platforms(platforms, include_archived=True):
for name in list_registered_platforms(platforms):
config = load_platform_config(name, platforms)
coverages[name] = {"platform": config.get("platform", name), "config": config}
moved: dict[str, str] = {}
+1 -1
View File
@@ -584,7 +584,7 @@ class YourSetupIndexInPlainWords(unittest.TestCase):
def test_insertion_keeps_next_steps_last(self):
page = "# T\n\nBody.\n\n## Next steps\n\n- [A](a.md)\n"
out = generate_site.with_setup_table(page, {}, {})
out = generate_site.with_setup_table(page, {})
self.assertTrue(out.rstrip().endswith("- [A](a.md)"))
self.assertIn("## Every setup at a glance", out)
+1 -1
View File
@@ -31,7 +31,7 @@ FENCED = re.compile(r"^\s*```.*?^\s*```", re.S | re.M)
def _registered() -> list[str]:
return list_registered_platforms(str(REPO_ROOT / "platforms"), include_archived=True)
return list_registered_platforms(str(REPO_ROOT / "platforms"))
def _which_pack() -> str:
+3 -3
View File
@@ -52,7 +52,7 @@ def _nav() -> list:
registry = load_platform_registry(str(platforms))
coverages = {
key: {"platform": load_platform_config(key, str(platforms))["platform"]}
for key in list_registered_platforms(str(platforms), include_archived=True)
for key in list_registered_platforms(str(platforms))
}
manufacturers = {"Sony": {}, "Sega": {}, "ScummVM": {}}
profiles = {
@@ -312,7 +312,7 @@ class Redirects(unittest.TestCase):
class PlatformShareCards(unittest.TestCase):
def test_every_platform_page_shares_its_own_card(self):
for key in list_registered_platforms(str(REPO_ROOT / "platforms"), include_archived=True):
for key in list_registered_platforms(str(REPO_ROOT / "platforms")):
with self.subTest(platform=key):
card = REPO_ROOT / "docs_assets" / "og" / "platforms" / f"{key}.png"
data = card.read_bytes()
@@ -437,7 +437,7 @@ def _front(text: str) -> str:
def _moved_platform_pages() -> list[Path]:
platforms = set(list_registered_platforms(str(REPO_ROOT / "platforms"), include_archived=True))
platforms = set(list_registered_platforms(str(REPO_ROOT / "platforms")))
folder = DOCS / "platforms"
if not folder.is_dir():
return []
+1 -1
View File
@@ -245,7 +245,7 @@ class VisibleWordsStayUnderTheirCeiling(unittest.TestCase):
def test_platform_pages(self):
from common import list_registered_platforms
mains = set(list_registered_platforms(str(REPO_ROOT / "platforms"), include_archived=True))
mains = set(list_registered_platforms(str(REPO_ROOT / "platforms")))
for page in sorted(_SITE.glob("*/index.html")):
name = page.parent.name
text = page.read_text(encoding="utf-8")
+4 -4
View File
@@ -103,7 +103,7 @@ class ReaderMatchedByHash(unittest.TestCase):
def test_every_cited_entry_accepts_the_md5_or_says_by_name(self):
by_name = 0
for platform in list_registered_platforms(PLATFORMS, include_archived=True):
for platform in list_registered_platforms(PLATFORMS):
config = load_platform_config(platform, PLATFORMS)
cats = EntryCategories(config, self.profiles)
for sys_id, system in (config.get("systems") or {}).items():
@@ -276,7 +276,7 @@ class PlatformReferenceInWords(unittest.TestCase):
page = generate_site.generate_platform_page("romm", cov, {})
self.assertIn(">size, then any hash</span>", page)
self.assertNotIn(">MD5</span>", page)
table = "\n".join(generate_site.setup_table({"romm": cov}, {}))
table = "\n".join(generate_site.setup_table({"romm": cov}))
self.assertIn("| size, then any hash |", table)
def test_systems_by_name_with_the_id_in_small_text(self):
@@ -609,7 +609,7 @@ class TheContractsValidate(unittest.TestCase):
schema = json.loads((REPO_ROOT / "schemas" / "platform.schema.json").read_text())
validator = Draft202012Validator(schema)
platforms = str(REPO_ROOT / "platforms")
for name in list_registered_platforms(platforms, include_archived=True):
for name in list_registered_platforms(platforms):
config = load_platform_config(name, platforms)
contract, registry = generate_site._split_contract(config)
with self.subTest(platform=name):
@@ -1264,7 +1264,7 @@ SCHEMA = json.loads((REPO_ROOT / "schemas" / "install-manifest.schema.json").rea
def _released() -> set[str]:
return set(list_release_pack_platforms(str(REPO_ROOT / "platforms"), include_archived=True))
return set(list_release_pack_platforms(str(REPO_ROOT / "platforms")))
class OneAnswerOnReleasePacks(unittest.TestCase):
+1 -1
View File
@@ -338,7 +338,7 @@ PLATFORMS = str(REPO_ROOT / "platforms")
def _sidebar_with_get_pages() -> tuple[dict, list[str]]:
nav = _nav()
names = list_registered_platforms(PLATFORMS, include_archived=True)
names = list_registered_platforms(PLATFORMS)
get_pages = ["get/index.md", *(f"get/{name}.md" for name in names)]
return sitestarlight.sidebar(nav, _present(nav) | set(get_pages), ANCHORS), get_pages
+1 -1
View File
@@ -288,7 +288,7 @@ class EverySystemLinkLandsOnItsSection(unittest.TestCase):
siterender.set_console_sections(gs._group_by_manufacturer(db), siterender.program_names(profiles))
platforms = REPO_ROOT / "platforms"
ids = {str(sid) for profile in profiles.values() for sid in profile.get("systems") or []}
for name in list_registered_platforms(str(platforms), include_archived=True):
for name in list_registered_platforms(str(platforms)):
ids |= {str(sid) for sid in load_platform_config(name, str(platforms)).get("systems") or {}}
cls.ids = sorted(ids)
cls.siterender = siterender
+1 -1
View File
@@ -54,7 +54,7 @@ class SystemNames(unittest.TestCase):
platforms_dir = str(REPO_ROOT / "platforms")
cls.platform_ids = {
sys_id
for name in list_registered_platforms(platforms_dir, include_archived=True)
for name in list_registered_platforms(platforms_dir)
for sys_id in load_platform_config(name, platforms_dir).get("systems", {})
}
cls.profile_ids = _profile_system_ids(cls.profiles)
+1 -1
View File
@@ -120,7 +120,7 @@ class SystemLinks(unittest.TestCase):
cls.sections, _installs = arcade_board_sections(cls.manufacturers, {})
platforms = [
load_platform_config(name, "platforms")
for name in list_registered_platforms("platforms", include_archived=True)
for name in list_registered_platforms("platforms")
]
cls.ids = siterender.system_ids(profiles, platforms)
+1 -1
View File
@@ -248,7 +248,7 @@ class SetupCount(unittest.TestCase):
from common import list_registered_platforms
self.assertEqual(len(sitediagrams.setups()),
len(list_registered_platforms(str(REPO_ROOT / "platforms"), include_archived=True)))
len(list_registered_platforms(str(REPO_ROOT / "platforms"))))
def test_every_rendered_count_agrees(self):
"""Each "RetroArch, Batocera, EmuDeck and N more" names three setups."""
+3 -16
View File
@@ -935,28 +935,15 @@ class AnArchiveIsClaimedByItsMembers(unittest.TestCase):
class SlotsJudgeWhatVerifyJudged(unittest.TestCase):
"""pipeline --include-archived verified RetroPie and built its pack, but
step 3b ran slots.py without the flag (it had none) and never arbitrated
RetroPie's 31 contested destinations; --target was dropped the same way."""
"""Step 3b ran slots.py without the run's --target and arbitrated cores
the target does not ship."""
def test_the_pipeline_passes_both_flags(self):
def test_the_pipeline_passes_the_target(self):
source = (Path(__file__).resolve().parents[1] / "scripts" / "pipeline.py").read_text()
step = source[source.index('slots_cmd = [sys.executable, "scripts/slots.py"]'):]
step = step[:step.index("3b slot arbitration")]
self.assertIn('slots_cmd.append("--include-archived")', step)
self.assertIn('slots_cmd.extend(["--target", args.target])', step)
def test_slots_accepts_them(self):
import subprocess
root = Path(__file__).resolve().parents[1]
result = subprocess.run(
[sys.executable, "scripts/slots.py", "--include-archived", "--platform", "retropie"],
cwd=root, capture_output=True, text=True, timeout=600,
)
self.assertEqual(result.returncode, 2, result.stderr)
self.assertIn("--include-archived", result.stderr)
class ARecordIsFoundByItsPathOnce(unittest.TestCase):
"""_accepted_by_all walked all 19 000 records for every path asked."""
+2 -2
View File
@@ -48,7 +48,7 @@ class SplitPartsCoverTheExtras(unittest.TestCase):
},
}
(root / "platforms" / "_registry.yml").write_text(
yaml.dump({"platforms": {"plat": {"status": "active"}}})
yaml.dump({"platforms": {"plat": {"config": "plat.yml"}}})
)
(root / "platforms" / "plat.yml").write_text(yaml.dump({
"platform": "Plat", "verification_mode": "existence",
@@ -108,7 +108,7 @@ class SplitPartsHoldWhatTheFullPackHolds(unittest.TestCase):
},
}
(root / "platforms" / "_registry.yml").write_text(
yaml.dump({"platforms": {"plat": {"status": "active"}}})
yaml.dump({"platforms": {"plat": {"config": "plat.yml"}}})
)
(root / "platforms" / "plat.yml").write_text(yaml.dump(platform))
for name, profile in profiles.items():
+1 -1
View File
@@ -54,7 +54,7 @@ class SplitPartNames(unittest.TestCase):
},
}
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"longplat": {"status": "active"}}})
yaml.dump({"platforms": {"longplat": {"config": "longplat.yml"}}})
)
self.sha1 = h["sha1"]
+12 -16
View File
@@ -606,22 +606,18 @@ class HelpTextsWrapBetweenWords(unittest.TestCase):
self.assertIn(line, proc.stdout.splitlines())
class ListPlatformsTakesTheSharedFlag(unittest.TestCase):
"""verify, generate_pack, slots and pipeline say --include-archived."""
class PlatformListsNameEveryPlatform(unittest.TestCase):
def test_list_platforms_prints_every_registered_platform(self):
env = {k: v for k, v in os.environ.items() if k != "GITHUB_OUTPUT"}
proc = subprocess.run(
[sys.executable, "scripts/list_platforms.py"],
capture_output=True, text=True, cwd=REPO_ROOT, check=False,
env=env, timeout=60,
)
self.assertEqual(proc.returncode, 0, proc.stderr)
self.assertIn("retropie", proc.stdout)
def test_both_spellings(self):
for flag in ("--include-archived", "--all"):
with self.subTest(flag=flag):
env = {k: v for k, v in os.environ.items() if k != "GITHUB_OUTPUT"}
proc = subprocess.run(
[sys.executable, "scripts/list_platforms.py", flag],
capture_output=True, text=True, cwd=REPO_ROOT, check=False,
env=env, timeout=60,
)
self.assertEqual(proc.returncode, 0, proc.stderr)
self.assertIn("retropie", proc.stdout)
def test_the_pack_list_marks_archived_platforms(self):
def test_the_pack_list_marks_installer_only_platforms(self):
"""generate_pack --list printed retropie like a platform with a
release pack."""
proc = subprocess.run(
@@ -630,7 +626,7 @@ class ListPlatformsTakesTheSharedFlag(unittest.TestCase):
)
self.assertEqual(proc.returncode, 0, proc.stderr)
lines = proc.stdout.splitlines()
self.assertIn("retropie (archived, installer only)", lines)
self.assertIn("retropie (installer only)", lines)
self.assertIn("knulli (installer only)", lines)
self.assertIn("retroarch", lines)
+1 -1
View File
@@ -47,7 +47,7 @@ class TwinDeclarationStatus(unittest.TestCase):
}}
self.db = {"files": files, "indexes": generate_db.build_indexes(files, {})}
Path("platforms/_registry.yml").write_text(
yaml.dump({"platforms": {"twin": {"status": "active"}}})
yaml.dump({"platforms": {"twin": {"config": "twin.yml"}}})
)
self.addCleanup(common._platform_config_cache.clear)
+2 -2
View File
@@ -72,7 +72,7 @@ class SlugPlatform(unittest.TestCase):
}
(self.platforms / "slugs.yml").write_text(yaml.dump(self.config))
(self.platforms / "_registry.yml").write_text(
yaml.dump({"platforms": {"slugs": {"status": "active"}}})
yaml.dump({"platforms": {"slugs": {"config": "slugs.yml"}}})
)
(self.emulators / "dosbox.yml").write_text(PROFILE)
common._platform_config_cache.clear()
@@ -190,7 +190,7 @@ class FlatFirmwareFolder(unittest.TestCase):
},
}
Path("platforms/flat.yml").write_text(yaml.dump(config))
entry = {"status": "active"}
entry = {"config": "flat.yml"}
if flat:
entry["firmware_folder_flat"] = True
Path("platforms/_registry.yml").write_text(
+3 -3
View File
@@ -352,7 +352,7 @@ class PlacementBySystem(_Fixture):
platforms = self.tmp / "platforms"
platforms.mkdir()
(platforms / "_registry.yml").write_text(
"platforms:\n p:\n config: p.yml\n status: active\n", encoding="utf-8")
"platforms:\n p:\n config: p.yml\n", encoding="utf-8")
(platforms / "p.yml").write_text(
"platform: P\nsystems:\n arcade:\n files:\n"
" - {name: a.zip, sha1: aaaa}\n - {name: b.zip, sha1: bbbb}\n",
@@ -434,7 +434,7 @@ class PlatformHashLoading(_Fixture):
platforms = self.tmp / "platforms"
platforms.mkdir()
(platforms / "_registry.yml").write_text(
"platforms:\n demo:\n status: active\n"
"platforms:\n demo:\n config: demo.yml\n"
)
(platforms / "demo.yml").write_text(
"platform: Demo\n"
@@ -461,7 +461,7 @@ class ThePlatformsOwnCheckDecides(_Fixture):
platforms = self.tmp / "platforms"
platforms.mkdir()
(platforms / "_registry.yml").write_text(
f"platforms:\n p:\n config: p.yml\n status: active\n{registry}",
f"platforms:\n p:\n config: p.yml\n{registry}",
encoding="utf-8")
(platforms / "p.yml").write_text(f"platform: P\nsystems:\n s:\n files:\n{systems}",
encoding="utf-8")
+2 -5
View File
@@ -33,16 +33,13 @@ class VerifyRefusesWhatAModeDoesNotRead(unittest.TestCase):
"""The mode x flag matrix generate_pack has, for verify.
--list-emulators --standalone printed the full list, --list-systems --json
printed columns, --platform --include-archived covered nothing more: each
with rc 0, each letting the caller believe the flag applied.
printed columns: each with rc 0, each letting the caller believe the flag
applied.
"""
MATRIX = (
(("--list-emulators",), "--standalone"),
(("--list-systems",), "--json"),
(("--list-targets", "--platform", "retroarch"), "--include-archived"),
(("--platform", "retroarch"), "--include-archived"),
(("--emulator", "handy"), "--include-archived"),
(("--platform", "retroarch"), "--standalone"),
# Game data is categorized against a platform's list and its cores.
(("--emulator", "handy"), "--no-game-data"),
+3 -4
View File
@@ -47,7 +47,6 @@ class VerifyPackChoice(unittest.TestCase):
"db": str(self.db),
"platform": "retroarch",
"all": False,
"include_archived": False,
"emulators_dir": "emulators",
"platforms_dir": "platforms",
"output_dir": str(self.out),
@@ -160,13 +159,13 @@ class VerifyPackChoice(unittest.TestCase):
def test_the_sweep_checks_release_packs_and_a_named_platform_its_own(self):
"""--all checks the packs a release publishes (release_pack): the
RetroPie pack, archived and served by the installer, is left to a
check that names it."""
RetroPie pack, served by the installer, is left to a check that names
it."""
stem = self.gp._platform_pack_stem(["retropie"], "retropie", "platforms")
self._touch(f"{stem}_BIOS_Pack.zip")
self._touch(f"{self.stem}_BIOS_Pack.zip")
checked = self._run(
self._args(platform=None, all=True, include_archived=True)
self._args(platform=None, all=True)
)
self.assertIn(("retroarch", f"{self.stem}_BIOS_Pack.zip"), checked)
self.assertNotIn(("retropie", f"{stem}_BIOS_Pack.zip"), checked)
+1 -1
View File
@@ -61,7 +61,7 @@ class ReleaseAssetNotCached(unittest.TestCase):
}
Path("platforms/big.yml").write_text(yaml.dump(self.config))
Path("platforms/_registry.yml").write_text(
yaml.dump({"platforms": {"big": {"status": "active"}}})
yaml.dump({"platforms": {"big": {"config": "big.yml"}}})
)
common._platform_config_cache.clear()
common._emulator_profiles_cache.clear()
+1 -1
View File
@@ -185,7 +185,7 @@ class DocumentedSteps(unittest.TestCase):
self.assertEqual(sorted(f for f in fields if f"`{f}`" not in page), [])
def test_every_option_says_what_it_does(self):
"""--include-archived and --all-variants, and 94 others, showed in
"""--all-variants, and 95 others, showed in
--help with no text: their constraints surfaced only as errors."""
sources = [*SCRIPTS.rglob("*.py"), REPO_ROOT / "install.py"]
silent = []
+1 -1
View File
@@ -40,7 +40,7 @@ class ZippedMemberIsChecked(unittest.TestCase):
"by_path_suffix": {}},
}
(root / "platforms" / "_registry.yml").write_text(
yaml.dump({"platforms": {"plat": {"status": "active"}}})
yaml.dump({"platforms": {"plat": {"config": "plat.yml"}}})
)
(root / "platforms" / "plat.yml").write_text(yaml.dump({
"platform": "Plat", "verification_mode": "md5",
+5 -8
View File
@@ -163,7 +163,6 @@ written in the registry, where the next scrape leaves it in place.
platforms:
myplatform:
config: myplatform.yml # platform YAML filename in platforms/
status: active # active or archived
release_pack: true # optional: a release publishes its ZIP
device_group: handheld # computer, gaming-os, handheld, fpga or server
scraper: myplatform # matches PLATFORM_NAME, null if hand-maintained
@@ -754,13 +753,11 @@ This executes in sequence, numbered as the run prints them (`STEP_LABELS` in
`--offline` skips steps 2 to 2b2.
A new platform is `status: active`. `status: archived` keeps the platform
out of the runs that cover every platform (`verify.py --all`,
`generate_pack.py --all`, `slots.py`, `pipeline.py`, `list_platforms.py`)
unless `--include-archived` is passed. The installer manifests, the site, the
native export and `check_freshness.py` cover archived platforms as well. No
tool scrapes a platform on its own, archived or not: the `schedule:` field is
the cadence a maintainer keeps by hand.
Every registered platform is covered by the runs that cover every platform
(`verify.py --all`, `slots.py`, `pipeline.py`, `list_platforms.py`), the
installer manifests, the site, the native export and `check_freshness.py`.
No tool scrapes a platform on its own: the `schedule:` field is the cadence
a maintainer keeps by hand.
A new platform has no release ZIP: `release_pack: true` is set for the setups
most people download, and the installer serves every other one from its
+1 -1
View File
@@ -25,7 +25,7 @@ bios/ BIOS and firmware files, organized by Manufacturer/Cons
emulators/ one YAML profile per core/engine
platforms/ one YAML config per platform (scraped from upstream)
_shared.yml shared file groups across platforms
_registry.yml platform metadata (logos, scrapers, status, install config)
_registry.yml platform metadata (logos, scrapers, release packs, install config)
_guides.yml the guide text of each platform's page
_data_dirs.yml data directory definitions (Dolphin Sys, PPSSPP...)
targets/ hardware target configs + _overrides.yml
+2 -2
View File
@@ -237,8 +237,8 @@ pipeline runs here, the archives are checked here, and `gh` uploads them.
# 1. Full pipeline, online, so data directories and MAME/FBNeo hashes are
# fresh. It builds the packs of the platforms marked release_pack (see
# "Which platforms get a pack" below) and the install manifests of every
# platform, RetroPie included with --include-archived.
python scripts/pipeline.py --include-archived
# platform.
python scripts/pipeline.py
# 2. A release file must be under 2 GiB. A larger pack becomes parts that are
# each a ZIP of whole files (Pack.part1of2.zip): any tool opens one, and
+3 -6
View File
@@ -20,7 +20,6 @@ python scripts/pipeline.py --offline --skip-docs # skip readme + site generati
python scripts/pipeline.py --offline --target switch # filter by hardware target
python scripts/pipeline.py --offline --with-truth # include truth generation + diff
python scripts/pipeline.py --offline --with-export # include native format export
python scripts/pipeline.py --offline --include-archived # include archived platforms
python scripts/pipeline.py --check-buildbot # online run + buildbot freshness check
```
@@ -72,7 +71,6 @@ Pipeline flags:
| `--offline` | skip every step that needs the network (2, 2a, 2a2, 2b, 2b2), and pack only the release assets already in `.cache/large/` |
| `--skip-packs` | skip steps 4, 4b, 4c, 5 and 6. Steps 4d and 4e still run, and README and site are still rewritten unless `--skip-docs` is given too |
| `--skip-docs` | skip steps 7, 8 and 8b |
| `--include-archived` | include archived platforms in verify, packs, truth and export |
| `--target TARGET` | filter verify, packs and truth by hardware target |
| `--source {platform,truth,full}` | pack file source, passed through to `generate_pack.py`. Refused with `--skip-packs` |
| `--all-variants` | build the 6 source x required combinations. Refused with `--skip-packs` and with `--source` |
@@ -340,8 +338,7 @@ python scripts/diff_truth.py --all # diff all platforms
Write each platform's own BIOS file, updated from the emulators' code: System.dat, es_bios.xml,
batocera-systems, checkBIOS.sh, FirmwareDatabase.cs, the RetroDECK manifests,
MiSTer's BiosDB, RetroPie's scriptmodules. Every platform in the registry,
archived included.
MiSTer's BiosDB, RetroPie's scriptmodules. Every platform in the registry.
```bash
python scripts/export_native.py --platform batocera --fetch
@@ -1353,7 +1350,7 @@ before a release (wiki/release-process.md). Its lines stay in
| `retired_files.py` | The record of files that left the collection, `retired.json`, which `generate_db.py` appends to and the site's lookup reads. `--backfill` rebuilds it from the history of `database.json` since content addresses were published (`--output` names another file) |
| `dedup.py` | Deduplicate `bios/` (`--dry-run`, `--bios-dir`): byte-identical copies are deleted and one kept. Identical MAME sets under other names stay one file, named in `_mame_clones.json`. Trees addressed by path are left alone (`NODEDUP_DIRS`), under a folder of that name at any depth: BBK, Bungie, C-Dogs SDL, Ikemen GO, mame, PinMAME, RPG Maker, ScummVM, Softdisk, TheXTech, splitwolf, Theme Hospital, c64-emu, opentyrian, tzdata |
| `identity.py` | One content filed under two identities: groups `bios/` by content and reports a group that crosses manufacturers outside `KNOWN_SHARED` (`--strict` exits 1, pipeline step 3a) |
| `slots.py` | Destinations the platform and a core both claim, and how the pack settles each (`--platform`, `--include-archived`, `--target`, `--json`, `--strict`, and pipeline step 3b passes the run's own `--include-archived` and `--target`) |
| `slots.py` | Destinations the platform and a core both claim, and how the pack settles each (`--platform`, `--target`, `--json`, `--strict`, and pipeline step 3b passes the run's own `--target`) |
| `region_audit.py` | Confront every profile `region:` with the regions No-Intro names for the same dump (`--json`, `--db`, `--emulators-dir`). An opinion, never an authority: the code decides, and a disagreement is a line to read in the emulator's source |
| `validate_pr.py` | Validate BIOS files in pull requests, post markdown report |
| `validate_schemas.py` | Validate the data contracts: schemas and semantic invariants. `--source-only` checks `emulators/` and `platforms/` alone, which is what PR validation runs |
@@ -1361,7 +1358,7 @@ before a release (wiki/release-process.md). Its lines stay in
| `generate_installer_doc.py` | Rewrite the generated blocks of `wiki/installer.md` from `install.py`: the tables (options, events, exit codes, keys per route, card markers), the line menu, and the run samples `install.py` prints in a sandbox against the committed `install/*.json`, so a new list rewrites the page (`--check` exits 1 when a block is stale, `--page FILE` writes another copy of the page). Pipeline step 7 |
| `pin_installer.py` | Write the SHA-256 of `install.py` into the pins of `install.sh` and `install.ps1`, which refuse any other installer (`--check` writes nothing and exits 1 when a pin is stale). `check_freshness.py` reports a stale pin and `refresh_stale.py` runs this |
| `auto_fetch.py` | Fetch missing BIOS files from known sources (4-step pipeline). A download is accepted only when every cryptographic hash the entry declares (sha256, sha1, md5) matches. crc32 and size find a file the database already holds, never accept a download. An entry with no cryptographic hash is reported and not searched |
| `list_platforms.py` | List active platforms (`--include-archived` adds the archived ones) |
| `list_platforms.py` | List the registered platforms as JSON |
| `download.py` | Download a pack from GitHub releases, every part checked (Python, stdlib only) |
| `download.sh` | Same, as a shell one-liner (`curl` + `unzip`) |
| `split_pack.py` | Cut a pack over the release asset limit into parts that are each a ZIP of whole files, each with a marker naming every part, read back before the pack is removed |