chore: merge origin/main

This commit is contained in:
Abdessamad Derraz committed 2026-11-06 21:18:18 +01:00
commit 399b5c33c1
19 files changed
+689 -8

No files matched your search

+3 -1
View File
@@ -36,6 +36,8 @@ curl -fsSL https://abdess.github.io/retrobios/i | sh -s -- --platform retroarch
&([scriptblock]::Create((irm https://abdess.github.io/retrobios/i.ps1))) --check
```
![Recording of the one-line command in a terminal. The installer finds an ArkOS SD card with 6 games, plans 43 BIOS files (15.1 MB) for them, installs them and shows the summary.](docs_assets/demo/install.gif)
The script opens a full-screen installer, driven with the arrow keys and Enter, or with a controller on a handheld. It finds your setup and its BIOS folder, downloads only the missing files and checks each one. It can also copy the BIOS files from your old SD card.
Every option, and what the installer detects and writes, is on the [installer page](https://abdess.github.io/retrobios/wiki/installer/#passing-options).
@@ -206,4 +208,4 @@ The scripts and tooling are released under the [MIT License](LICENSE).
The BIOS and firmware files are not covered by that license: they are third-party system software, preserved and provided for personal backup, archival, and interoperability with emulation software. [NOTICE](NOTICE) sets out their status and how to ask for a file to be removed.
The reasoning, and where it is weakest, is in [the project's legal position](https://abdess.github.io/retrobios/wiki/legal/).
*Auto-generated on 2026-11-06T10:21:06Z*
*Auto-generated on 2026-11-06T19:12:52Z*
+9
View File
@@ -0,0 +1,9 @@
{
"width": 800,
"height": 460,
"setup": "ArkOS",
"games": 6,
"systems": 6,
"files": 43,
"size": "15.1 MB"
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 247 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

+46
View File
@@ -0,0 +1,46 @@
# The installer demo of the README and the site. Run scripts/make_demo.py,
# which builds the card, serves the repository on loopback, starts VHS on
# this file from the repository root and encodes what it writes.
Output tmp/demo/recorded.mp4
Set Shell "bash"
Set FontFamily "DejaVu Sans Mono"
Set FontSize 15
Set LineHeight 1.15
Set Width 800
Set Height 460
Set Padding 16
Set Framerate 25
Set TypingSpeed 35ms
Set CursorBlink false
Set Theme { "name": "RetroBIOS", "background": "#12131c", "foreground": "#e6e6f0", "cursor": "#aebcff", "selection": "#3a3d52", "black": "#12131c", "red": "#ef9a9a", "green": "#a5d6a7", "yellow": "#ffcc80", "blue": "#90caf9", "magenta": "#aebcff", "cyan": "#aebcff", "white": "#cfd8dc", "brightBlack": "#7c82a3", "brightRed": "#ef9a9a", "brightGreen": "#a5d6a7", "brightYellow": "#ffcc80", "brightBlue": "#90caf9", "brightMagenta": "#c9d2ff", "brightCyan": "#c9d2ff", "brightWhite": "#e6e6f0" }
# The card mounted at /media/EASYROMS, an empty HOME, the prompt "$ ".
Hide
Type 'exec "$RETROBIOS_DEMO_SESSION"'
Enter
Wait+Screen@20s /\$/
Sleep 500ms
Show
Sleep 700ms
Type "curl -fsSL https://abdess.github.io/retrobios/i | sh -s -- --for-games"
Sleep 600ms
Screenshot tmp/demo/poster.png
Sleep 600ms
Enter
# The first screen: the card found.
Wait+Screen@30s /ArkOS SD card \(EASYROMS\)/
Sleep 2500ms
Enter
# The plan: only the files the games on the card need.
Wait+Screen@60s /For the games on this card: yes/
Sleep 4500ms
Enter
# The summary.
Wait+Screen@60s /RetroBIOS: Done/
Sleep 4500ms
Binary file not shown.
+1 -1
View File
@@ -16,7 +16,7 @@
# and Invoke-WebRequest throws on a cut transfer before the hash is read:
# the site's build rewrites this line there (scripts/sitebootstrap.py).
$mismatchAdvice = "The download was most likely cut short: try again. Within minutes of an update of this project the two files can come from caches of different ages. Waiting five minutes settles that."
$defaultInstallSha256 = "3a78bf2e5a910d0f5c21bc2dc490aa600fc3273c56ae01e9ce5061d9b3578f07"
$defaultInstallSha256 = "b17220355845692a087bbc29cdb4a1008f68b1423d1126f34f00fa0259b53184"
$maximumInstallerBytes = 2MB
$installerExit = 0
$installer = if ($PSScriptRoot) { Join-Path $PSScriptRoot "install.py" } else { $null }
+6 -2
View File
@@ -9831,11 +9831,15 @@ def games_root(setup: dict, bios_path: Path) -> Path | None:
def _may_hold_games(bios_path: Path) -> bool:
"""Whether a rom folder could sit near this BIOS folder, said without the
network: a folder named roms in it or up to two levels above it, or a
path that runs through one (roms, games)."""
network: a folder named roms in it or up to two levels above it, a path
that runs through one (roms, games), or a handheld's games part, whose
console folders sit beside bios (ArkOS's EASYROMS plugged into a
computer was never offered its games)."""
holders = [bios_path, *list(bios_path.absolute().parents)[:2]]
if any(part.casefold() in ("roms", "games") for part in bios_path.absolute().parts[-3:]):
return True
if games_part(bios_path.parent) is not None:
return True
return any(entry.name.casefold() == "roms" and _entry_is_dir(entry)
for holder in holders for entry in _list_dir(holder))
+1 -1
View File
@@ -23,7 +23,7 @@ DEFAULT_INSTALL_URL="https://raw.githubusercontent.com/Abdess/retrobios/main/ins
# curl or wget stop on a cut transfer before the hash is read: the site's
# build rewrites this line there (scripts/sitebootstrap.py).
MISMATCH_ADVICE="The download was most likely cut short: try again. Within minutes of an update of this project the two files can come from caches of different ages. Waiting five minutes settles that."
DEFAULT_INSTALL_SHA256="3a78bf2e5a910d0f5c21bc2dc490aa600fc3273c56ae01e9ce5061d9b3578f07"
DEFAULT_INSTALL_SHA256="b17220355845692a087bbc29cdb4a1008f68b1423d1126f34f00fa0259b53184"
MAX_INSTALLER_BYTES=2097152
cleanup() {
+18
View File
@@ -443,6 +443,9 @@ def format_size(size: int) -> str:
SITE_URL = "https://abdess.github.io/retrobios/"
# The installer demo (scripts/make_demo.py): the recording and what it shows.
DEMO_DIR = Path(__file__).resolve().parent.parent / "docs_assets" / "demo"
DEMO_GIF = "docs_assets/demo/install.gif"
REPO_PATH = "../.."
REPO = "Abdess/retrobios"
# GitHub's "Add file > Upload files" page for bios/: a browser upload by
@@ -473,6 +476,19 @@ def clone_line(db: dict) -> str:
)
def demo_facts() -> dict:
"""What the installer demo shows, as make_demo.py measured it."""
return json.loads((DEMO_DIR / "demo.json").read_text(encoding="utf-8"))
def demo_description(facts: dict | None = None) -> str:
"""The demo in words, for a reader who cannot see it."""
facts = facts or demo_facts()
return (f"Recording of the one-line command in a terminal. The installer finds an "
f"{facts['setup']} SD card with {facts['games']} games, plans {facts['files']} "
f"BIOS files ({facts['size']}) for them, installs them and shows the summary.")
def _existing_contributor_block(readme_path: str = "README.md") -> list[str]:
"""The contributors already published, read back from the file.
@@ -619,6 +635,8 @@ def generate_readme(db: dict, platforms_dir: str, db_path: str = "database.json"
install_command("powershell", "--check"),
"```",
"",
f"![{demo_description()}]({DEMO_GIF})",
"",
"The script opens a full-screen installer, driven with the arrow keys and Enter, or with a"
" controller on a handheld. It finds your setup and its BIOS folder, downloads only the missing"
" files and checks each one. It can also copy the BIOS files from your old SD card.",
+23 -1
View File
@@ -82,6 +82,8 @@ from generate_readme import (
NO_PACK,
asset_links,
contributing_text,
demo_description,
demo_facts,
download_links,
extract_notes,
extract_targets,
@@ -516,6 +518,7 @@ def generate_home(
"**Run one command.** Some setups have a ZIP.",
"**Start a game.**",
]),
*install_demo(),
"## Pick your device",
"",
*setup_picker(registry, display),
@@ -4155,6 +4158,20 @@ def first_screen_words() -> list[str]:
f"and **{installer.TUI_FOLDER_TITLE}**), then offers **{installer.TUI_INSTALL_ROW}**."]
# The demo files the site serves (docs_assets/demo/, scripts/make_demo.py):
# the GIF is the README's, the tape and demo.json are inputs.
DEMO_ASSETS = ("install.webm", "install.mp4", "install.png")
def install_demo() -> list[str]:
"""The installer demo: a video that loads once it is on screen, its
description for screen readers."""
facts = demo_facts()
text = html.escape(demo_description(facts), quote=True)
return [f'<InstallDemo description="{text}" width="{facts["width"]}" '
f'height="{facts["height"]}" />', ""]
def check_pass_line() -> str:
"""The pass line of "How to check": the words `--check` prints only when
no file is missing or wrong (install.py EVERYTHING_IN_PLACE), read from
@@ -7910,7 +7927,7 @@ def generate_emulators_index(profiles: dict) -> str:
def _snippet_noun(names: Iterable[str], firmware: int, fetched: int) -> str:
"""What an emulator's files are called in its snippet. A Switch
emulator reads keys, and "eden keys" is what readers search for; Ship
emulator reads keys, and "eden keys" is what readers search for. Ship
of Harkinian reads game ROMs, which are no BIOS."""
names = list(names)
keys = sum("key" in name.casefold() for name in names)
@@ -11957,6 +11974,7 @@ def generate_which_pack() -> str:
"",
*first_screen_words(),
"",
*install_demo(),
*trust_box(""),
"**Check your folder later.** Add `--check`. It changes nothing.",
"",
@@ -12316,6 +12334,10 @@ def _generate(args: argparse.Namespace) -> None:
source = Path("docs_assets") / folder
if source.is_dir():
shutil.copytree(source, public / "assets" / dest, dirs_exist_ok=True)
demo_dest = public / "assets" / "demo"
demo_dest.mkdir(parents=True, exist_ok=True)
for name in DEMO_ASSETS:
shutil.copy2(Path("docs_assets") / "demo" / name, demo_dest / name)
assets_src = Path(".github") / "assets"
for name, dest_name in [("logo.png", "logo.png"), ("favicon.png", "favicon.png")]:
src = assets_src / name
+295
View File
@@ -0,0 +1,295 @@
#!/usr/bin/env python3
"""Record the installer demo shown in the README and on the site, once.
The recording is the full-screen installer run from the one-line command on
an ArkOS SD card that holds six games: it finds the card, plans only the BIOS
files those games need, installs them and shows the summary. Everything it
reads is local and the same on every machine:
- The card is built under tmp/demo/ (bios/, the folders ArkOS writes on its
games partition, one game per console) and mounted at /media/EASYROMS in a
mount namespace of this user (unshare), on a tmpfs of a card's size. /media
and /mnt are emptied there, so no drive of this machine is shown.
- The repository is served on 127.0.0.1 and named with RETROBIOS_BASE_URL:
the file lists, install/games/ and the BIOS files are this checkout's.
- The one-line command runs this checkout's install.sh and install.py. The
shell of the recording answers `curl -fsSL <site>/i` with a line that
starts the local install.sh, which skips the download of the installer and
its SHA-256 check and runs the rest of the bootstrap as the site's copy does.
- HOME is an empty folder and the prompt is "$ ".
docs_assets/demo/install.tape drives VHS (github.com/charmbracelet/vhs, with
ttyd, ffmpeg and a Chromium on PATH). This script encodes what it records:
docs_assets/demo/install.gif the README
docs_assets/demo/install.webm the site, with install.mp4 as fallback
docs_assets/demo/install.png the first frame shown, the video's poster
docs_assets/demo/demo.json what the recording shows, for the alt text
The site build copies the video files and never runs this script. Run it
again when a screen of the installer or the file list of its games changes:
python scripts/make_demo.py
"""
from __future__ import annotations
import argparse
import functools
import hashlib
import http.server
import importlib.util
import json
import os
import re
import shutil
import subprocess
import sys
import threading
from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
OUT = ROOT / "docs_assets" / "demo"
TAPE = OUT / "install.tape"
WORK = ROOT / "tmp" / "demo"
# What the tape writes, read back here (paths relative to ROOT, the tape's
# working directory).
RECORDED = WORK / "recorded.mp4"
POSTER_SHOT = WORK / "poster.png"
MOUNT = "/media/EASYROMS"
CARD_SIZE = "29g"
SETUP = "ArkOS"
# The card: the folders ArkOS's default tree writes on EASYROMS (install.py
# _GAMES_PART_RULES reads launchimages/), and one game per console folder,
# named as the folder's extensions in install/games/setups/arkos.json allow.
CARD_FOLDERS = ("bios", "bgmusic", "launchimages", "themes/es-theme-nes-box")
GAMES = {
"psx": "Sky Courier (USA).iso",
"segacd": "Night Shift (USA).iso",
"megadrive": "Old Harbour (World).md",
"mastersystem": "Tide Runner (Europe).sms",
"gamegear": "Pocket Quest (World).gg",
"pcengine": "Star Lancer (Japan).pce",
}
GAME_BYTES = 256 * 1024
# A PlayStation disc carries Sony's licence text in sector 4, the text the
# cores read the region from (install/games/systems/sony-playstation.json).
PSX_LICENCE = b" Licensed by Sony Computer Entertainment Amer ica "
# Size targets of the outputs, in bytes: the README and two site pages.
LIMITS = {"install.gif": 1_500_000, "install.webm": 1_000_000,
"install.mp4": 1_000_000, "install.png": 150_000}
GIF_FPS = 10
VIDEO_FPS = 25
def game_bytes(name: str) -> bytes:
"""Bytes of a game no catalog names, the same on every run."""
seed = hashlib.sha256(name.encode("utf-8")).digest()
blocks, counter = [], 0
while len(blocks) * 32 < GAME_BYTES:
blocks.append(hashlib.sha256(seed + counter.to_bytes(4, "big")).digest())
counter += 1
return b"".join(blocks)[:GAME_BYTES]
def build_card(card: Path) -> None:
if card.exists():
shutil.rmtree(card)
for folder in CARD_FOLDERS:
(card / folder).mkdir(parents=True)
(card / "launchimages" / "loading.jpg").write_bytes(b"\xff\xd8\xff\xe0" + bytes(60) + b"\xff\xd9")
(card / "themes" / "es-theme-nes-box" / "theme.xml").write_text(
"<theme>\n</theme>\n", encoding="utf-8")
for folder, name in GAMES.items():
data = bytearray(game_bytes(name))
if folder == "psx":
data[4 * 2048:4 * 2048 + len(PSX_LICENCE)] = PSX_LICENCE
(card / folder).mkdir()
(card / folder / name).write_bytes(bytes(data))
class QuietHandler(http.server.SimpleHTTPRequestHandler):
def log_message(self, format: str, *args: object) -> None: # noqa: A002
pass
@contextmanager
def served_repository() -> Iterator[str]:
"""The repository on a loopback port, the base URL install.py accepts
without HTTPS."""
handler = functools.partial(QuietHandler, directory=str(ROOT))
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
yield f"http://127.0.0.1:{server.server_address[1]}"
finally:
server.shutdown()
server.server_close()
def write_session(work: Path, names: list[str]) -> Path:
"""The program the recording's shell becomes: the card mounted in a
namespace of its own, then a clean interactive bash, or the command
given as arguments, with only the variables names lists."""
rc = work / "bashrc"
rc.write_text(
"PS1='$ '\n"
"unset HISTFILE PROMPT_COMMAND\n"
"curl() {\n"
' if [ "$*" = "-fsSL https://abdess.github.io/retrobios/i" ]; then\n'
' printf \'exec %s "$@"\\n\' "$RETROBIOS_DEMO_BOOTSTRAP"\n'
" else\n"
' command curl "$@"\n'
" fi\n"
"}\n"
"clear\n",
encoding="utf-8",
)
session = work / "session.sh"
session.write_text(
"#!/bin/sh\n"
"# Written by scripts/make_demo.py.\n"
"set -eu\n"
"inner='set -eu\n"
"mount -t tmpfs -o mode=0755 demo /media\n"
"mount -t tmpfs -o mode=0755 demo /mnt\n"
f"mkdir {MOUNT}\n"
f"mount -t tmpfs -o size={CARD_SIZE},mode=0755 EASYROMS {MOUNT}\n"
f'cp -a "$RETROBIOS_DEMO_CARD/." {MOUNT}/\n'
'cd "$HOME"\n'
'exec unshare --user --map-user="$RETROBIOS_DEMO_UID" '
'--map-group="$RETROBIOS_DEMO_GID" env -i '
+ " ".join(f'{name}="${name}"' for name in names) + ' "$@"\'\n'
'if [ "$#" -eq 0 ]; then\n'
f' set -- bash --noprofile --rcfile "{rc}" -i\n'
"fi\n"
'exec unshare --user --mount --map-root-user /bin/sh -c "$inner" sh "$@"\n',
encoding="utf-8",
)
session.chmod(0o755)
return session
def session_env(base_url: str, work: Path, card: Path) -> dict[str, str]:
home = work / "home"
home.mkdir(parents=True, exist_ok=True)
return {
"HOME": str(home),
"PATH": "/usr/local/bin:/usr/bin:/bin",
"LANG": "C.UTF-8",
"TERM": "xterm-256color",
"TMPDIR": str(work / "scratch"),
"RETROBIOS_OS": "linux",
"RETROBIOS_BASE_URL": base_url,
# What the downloaded bootstrap sets: commands given back are the
# one-line command, not a path to a temporary copy.
"RETROBIOS_BOOTSTRAP": "sh",
"RETROBIOS_DEMO_BOOTSTRAP": str(ROOT / "install.sh"),
"RETROBIOS_DEMO_CARD": str(card),
"RETROBIOS_DEMO_UID": str(os.getuid()),
"RETROBIOS_DEMO_GID": str(os.getgid()),
}
def format_size(size: int) -> str:
"""The size as the installer writes it."""
spec = importlib.util.spec_from_file_location("retrobios_install", ROOT / "install.py")
if spec is None or spec.loader is None:
raise RuntimeError("install.py cannot be loaded")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return str(module.format_size(size))
def measure(session: Path, env: dict[str, str]) -> dict:
"""What the recording shows, read from the same run without the screen:
the games read on the card and the files planned for them."""
command = [str(session), sys.executable, str(ROOT / "install.py"), "--platform", "retroarch",
"--dest", f"{MOUNT}/bios", "--for-games", "--dry-run", "--events"]
result = subprocess.run(command, env=env, capture_output=True, text=True, check=False)
if result.returncode != 0:
raise RuntimeError(f"the dry run failed ({result.returncode}):\n{result.stderr}")
events = [json.loads(line) for line in result.stdout.splitlines() if line.startswith("{")]
games = next(e for e in events if e["event"] == "games")
check = next(e for e in events if e["event"] == "check")
size = {key.lower(): int(value) for key, value in
re.findall(r"^Set (Width|Height) (\d+)$", TAPE.read_text(encoding="utf-8"), re.M)}
return {
"width": size["width"],
"height": size["height"],
"setup": SETUP,
"games": games["games"],
"systems": len(games["found"]),
"files": check["missing"],
"size": format_size(check["bytes_needed"]),
}
def ffmpeg(*args: str) -> None:
subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", *args], check=True)
def encode(recorded: Path, poster: Path, out: Path, scratch: Path) -> None:
palette = scratch / "palette.png"
ffmpeg("-i", str(recorded), "-vf", f"fps={GIF_FPS},palettegen=max_colors=48:stats_mode=full",
str(palette))
ffmpeg("-i", str(recorded), "-i", str(palette), "-lavfi",
f"fps={GIF_FPS}[v];[v][1:v]paletteuse=dither=none:diff_mode=rectangle",
"-loop", "0", str(out / "install.gif"))
ffmpeg("-i", str(recorded), "-an", "-r", str(VIDEO_FPS), "-c:v", "libvpx-vp9",
"-b:v", "0", "-crf", "42", "-row-mt", "1", "-deadline", "good",
"-pix_fmt", "yuv420p", str(out / "install.webm"))
ffmpeg("-i", str(recorded), "-an", "-r", str(VIDEO_FPS), "-c:v", "libx264",
"-preset", "veryslow", "-crf", "30", "-tune", "animation", "-pix_fmt", "yuv420p",
"-movflags", "+faststart", str(out / "install.mp4"))
ffmpeg("-i", str(poster), "-pred", "mixed", "-compression_level", "100",
str(out / "install.png"))
def check_sizes(out: Path) -> list[str]:
return [f"{name}: {(out / name).stat().st_size:,} bytes, over {limit:,}"
for name, limit in LIMITS.items() if (out / name).stat().st_size > limit]
def main() -> None:
parser = argparse.ArgumentParser(allow_abbrev=False,
description="Record the installer demo (VHS).")
parser.parse_args()
missing = [tool for tool in ("vhs", "ttyd", "ffmpeg", "unshare") if shutil.which(tool) is None]
if missing:
sys.exit(f"Missing on PATH: {', '.join(missing)} (VHS: github.com/charmbracelet/vhs)")
if WORK.exists():
shutil.rmtree(WORK)
(WORK / "scratch").mkdir(parents=True)
try:
card = WORK / "card"
build_card(card)
with served_repository() as base_url:
env = session_env(base_url, WORK, card)
session = write_session(WORK, sorted(env))
facts = measure(session, env)
vhs_env = {**os.environ, **env, "PATH": os.environ.get("PATH", ""),
"HOME": os.environ.get("HOME", ""),
"RETROBIOS_DEMO_SESSION": str(session)}
subprocess.run(["vhs", str(TAPE.relative_to(ROOT))], cwd=ROOT, env=vhs_env,
check=True)
encode(RECORDED, POSTER_SHOT, OUT, WORK / "scratch")
(OUT / "demo.json").write_text(json.dumps(facts, indent=2) + "\n", encoding="utf-8")
for name in (*LIMITS, "demo.json"):
print(f"{(OUT / name).relative_to(ROOT)} {(OUT / name).stat().st_size:,} bytes")
over = check_sizes(OUT)
if over:
sys.exit("Over the size targets:\n " + "\n ".join(over))
finally:
shutil.rmtree(WORK, ignore_errors=True)
if __name__ == "__main__":
from common import run_cli
run_cli(main)
+2 -1
View File
@@ -29,6 +29,7 @@ COMPONENTS = {
"TabItem": "@astrojs/starlight/components",
"FilePicker": "components/FilePicker.astro",
"MeasurementsExplorer": "components/MeasurementsExplorer.astro",
"InstallDemo": "components/InstallDemo.astro",
}
# Pages of file cards: level 3 headings by the hundred, so the table of
@@ -330,7 +331,7 @@ OG_DIR = Path(__file__).resolve().parent.parent / "docs_assets" / "og"
# An emulator page's card is that of the maker of the first system it runs,
# by page path: "emulators/mgba.mdx" -> "nintendo". One card per emulator
# would be some 900 images in the repository; the maker's names the
# would be some 900 images in the repository, and the maker's names the
# consoles. Filled by generate_site.record_emulator_page.
EMULATOR_CARDS: dict[str, str] = {}
+3
View File
@@ -12923,6 +12923,9 @@ class DriveIsACardByItsFiles(unittest.TestCase):
self.assertEqual(install.games_setups(index, "retroarch", arkos / "bios"), ["arkos"])
self.assertEqual(install.games_setups(index, "retroarch", self.root / "bios"),
["retroarch"])
# Its games are offered without --for-games: EASYROMS keeps no
# roms folder, its console folders sit beside bios.
self.assertTrue(install._may_hold_games(arkos / "bios"))
def test_a_card_beside_a_data_drive_comes_first(self):
"""The ArkOS card and the data drive mounted,
+165
View File
@@ -0,0 +1,165 @@
"""The installer demo of the README and the site (scripts/make_demo.py).
The home page and Get your files score 98 to 100 on a phone: the video must
cost nothing before it is on screen, keep still under reduced motion, and say
in words what it shows. The files are committed, so their sizes are held
here, and the words come from what the recording measured.
"""
from __future__ import annotations
import tests # noqa: F401 offline, scratch under tmp/, the suite's own cache
import json
import re
import struct
import sys
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / "scripts"))
import generate_readme # noqa: E402
import make_demo # noqa: E402
import sitestarlight # noqa: E402
DEMO = ROOT / "docs_assets" / "demo"
COMPONENT = ROOT / "website" / "src" / "components" / "InstallDemo.astro"
DIST = ROOT / "website" / "dist"
def _facts() -> dict:
return json.loads((DEMO / "demo.json").read_text(encoding="utf-8"))
class TheFilesAreThereAndSmall(unittest.TestCase):
def test_each_output_is_under_its_target(self):
for name, limit in make_demo.LIMITS.items():
with self.subTest(name=name):
self.assertLessEqual((DEMO / name).stat().st_size, limit)
def test_the_gif_and_the_poster_have_the_recorded_size(self):
facts = _facts()
head = (DEMO / "install.gif").read_bytes()[:10]
self.assertEqual(head[:6], b"GIF89a")
self.assertEqual(struct.unpack("<HH", head[6:10]), (facts["width"], facts["height"]))
png = (DEMO / "install.png").read_bytes()[:24]
self.assertEqual(png[:8], b"\x89PNG\r\n\x1a\n")
self.assertEqual(struct.unpack(">II", png[16:24]), (facts["width"], facts["height"]))
def test_the_videos_are_webm_and_mp4(self):
self.assertEqual((DEMO / "install.webm").read_bytes()[:4], b"\x1a\x45\xdf\xa3")
self.assertEqual((DEMO / "install.mp4").read_bytes()[4:8], b"ftyp")
class TheTapeAndTheDriverAgree(unittest.TestCase):
def setUp(self):
self.tape = make_demo.TAPE.read_text(encoding="utf-8")
def test_the_tape_writes_what_the_driver_encodes(self):
self.assertIn(f"Output {make_demo.RECORDED.relative_to(ROOT).as_posix()}\n", self.tape)
self.assertIn(f"Screenshot {make_demo.POSTER_SHOT.relative_to(ROOT).as_posix()}\n",
self.tape)
def test_the_size_in_demo_json_is_the_tapes(self):
facts = _facts()
self.assertIn(f"Set Width {facts['width']}\n", self.tape)
self.assertIn(f"Set Height {facts['height']}\n", self.tape)
def test_the_command_typed_is_the_one_the_pages_give(self):
from siteguides import install_command # noqa: PLC0415
self.assertIn(f'Type "{install_command("sh", "--for-games")}"\n', self.tape)
def test_the_shell_answers_only_the_site_address(self):
typed = re.search(r'Type "curl -fsSL (\S+) \|', self.tape)
self.assertIsNotNone(typed)
driver = Path(make_demo.__file__).read_text(encoding="utf-8")
self.assertIn(f'"$*" = "-fsSL {typed.group(1)}"', driver)
class TheReadmeShowsTheGif(unittest.TestCase):
def test_the_gif_sits_under_the_commands_with_its_description(self):
readme = (ROOT / "README.md").read_text(encoding="utf-8")
line = f"![{generate_readme.demo_description()}]({generate_readme.DEMO_GIF})"
self.assertIn(line, readme)
self.assertLess(readme.index("## Quick install"), readme.index(line))
self.assertLess(readme.index(line), readme.index("## Download BIOS packs"))
def test_the_description_names_what_the_recording_measured(self):
facts = _facts()
words = generate_readme.demo_description(facts)
for value in (facts["setup"], f"{facts['games']} games", f"{facts['files']} BIOS files",
facts["size"]):
self.assertIn(str(value), words)
class TheVideoLoadsOnlyOnScreen(unittest.TestCase):
def setUp(self):
self.source = COMPONENT.read_text(encoding="utf-8")
self.markup = self.source.split("<script>", 1)[0]
def test_nothing_is_fetched_from_the_html(self):
self.assertIn('preload="none"', self.markup)
self.assertNotRegex(self.markup, r"\s(src|poster)=")
self.assertEqual(len(re.findall(r"<source data-src=", self.markup)), 2)
self.assertIn("data-poster=", self.markup)
self.assertIn("IntersectionObserver", self.source)
def test_it_is_muted_looped_and_inline(self):
for attribute in ("muted", "loop", "playsinline"):
self.assertRegex(self.markup, rf"\n\t\t{attribute}\n")
def test_webm_comes_before_mp4(self):
self.assertLess(self.markup.index("install.webm"), self.markup.index("install.mp4"))
def test_reduced_motion_keeps_the_poster_with_a_button(self):
self.assertIn("matchMedia('(prefers-reduced-motion: reduce)')", self.source)
self.assertIn("let wanted = !still();", self.source)
self.assertIn('<button type="button" class="rb-demo-toggle" hidden>Play</button>',
self.markup)
def test_screen_readers_get_the_description(self):
self.assertIn('aria-describedby="rb-demo-text"', self.markup)
self.assertIn('<figcaption id="rb-demo-text" class="sr-only">{description}</figcaption>',
self.markup)
def test_the_component_is_registered(self):
self.assertEqual(sitestarlight.COMPONENTS["InstallDemo"], "components/InstallDemo.astro")
class TheTwoPagesShowIt(unittest.TestCase):
def test_home_and_get_your_files(self):
import generate_site # noqa: PLC0415
tag = generate_site.install_demo()[0]
self.assertIn(f'description="{generate_readme.demo_description()}"', tag)
source = Path(generate_site.__file__).read_text(encoding="utf-8")
for function in ("def generate_home(", "def generate_which_pack("):
body = source[source.index(function):]
body = body[:body.index("\ndef ", 1)]
with self.subTest(function=function):
self.assertIn("*install_demo(),", body)
def test_the_site_serves_the_video_not_the_gif(self):
import generate_site # noqa: PLC0415
self.assertEqual(generate_site.DEMO_ASSETS, ("install.webm", "install.mp4", "install.png"))
def test_the_built_pages_fetch_nothing_of_it(self):
if not (DIST / "index.html").is_file():
self.skipTest("no built site")
for page in ("index.html", "which-pack/index.html"):
html = (DIST / page).read_text(encoding="utf-8")
figure = re.search(r'<figure class="rb-demo[^"]*".*?</figure>', html, re.S)
with self.subTest(page=page):
self.assertIsNotNone(figure)
self.assertIn('preload="none"', figure.group(0))
self.assertNotRegex(figure.group(0), r"\s(src|poster)=")
for name in ("install.webm", "install.mp4", "install.png"):
self.assertTrue((DIST / "assets" / "demo" / name).is_file())
if __name__ == "__main__":
unittest.main()
+115
View File
@@ -0,0 +1,115 @@
---
// The installer demo, recorded by scripts/make_demo.py and served from
// public/assets/demo/. Nothing of it loads before it is on screen: the
// poster and the sources are set by the script below once the video is in
// view (preload="none", no src in the HTML). With reduced motion or Save-Data
// the poster stays, and the button plays it. The button pauses it otherwise.
interface Props {
description: string;
width: string;
height: string;
}
const { description, width, height } = Astro.props;
const base = import.meta.env.BASE_URL.replace(/\/?$/, '/');
const asset = (name: string) => `${base}assets/demo/${name}`;
---
<figure class="rb-demo" data-demo data-pagefind-ignore>
<video
width={width}
height={height}
muted
loop
playsinline
preload="none"
disablepictureinpicture
data-poster={asset('install.png')}
aria-describedby="rb-demo-text"
>
<source data-src={asset('install.webm')} type="video/webm" />
<source data-src={asset('install.mp4')} type="video/mp4" />
</video>
<button type="button" class="rb-demo-toggle" hidden>Play</button>
<figcaption id="rb-demo-text" class="sr-only">{description}</figcaption>
</figure>
<script>
const still = () =>
matchMedia('(prefers-reduced-motion: reduce)').matches ||
Boolean((navigator as Navigator & { connection?: { saveData?: boolean } }).connection?.saveData);
for (const figure of document.querySelectorAll<HTMLElement>('[data-demo]')) {
const video = figure.querySelector('video');
const toggle = figure.querySelector<HTMLButtonElement>('.rb-demo-toggle');
if (!video || !toggle) continue;
let wanted = !still();
let visible = false;
let loaded = false;
const sync = () => {
if (wanted && visible) {
if (!loaded) {
loaded = true;
for (const source of video.querySelectorAll('source')) source.src = source.dataset.src ?? '';
video.load();
}
video.play().catch(() => {
wanted = false;
toggle.textContent = 'Play';
});
} else {
video.pause();
}
toggle.textContent = wanted ? 'Pause' : 'Play';
};
toggle.hidden = false;
toggle.addEventListener('click', () => {
wanted = !wanted;
sync();
});
new IntersectionObserver(
(entries) => {
visible = entries.some((entry) => entry.isIntersecting);
if (visible && !video.poster) video.poster = video.dataset.poster ?? '';
sync();
},
{ threshold: 0.25 },
).observe(video);
}
</script>
<style>
.rb-demo {
position: relative;
margin-inline: 0;
}
.rb-demo video {
display: block;
width: 100%;
height: auto;
background: #12131c;
border: 1px solid var(--rb-border);
border-radius: 0.5rem;
}
.rb-demo-toggle {
position: absolute;
inset-inline-end: 0.5rem;
inset-block-end: 0.5rem;
min-height: 2.75rem;
padding: 0.4rem 1rem;
font: inherit;
font-size: var(--sl-text-sm);
color: var(--sl-color-white);
background: var(--rb-surface);
border: 1px solid var(--rb-border-strong);
border-radius: 0.4rem;
cursor: pointer;
}
.rb-demo-toggle:hover {
background: var(--sl-color-accent-low);
}
.rb-demo-toggle:focus-visible {
outline: 2px solid var(--sl-color-accent);
outline-offset: 2px;
}
</style>
+1 -1
View File
@@ -69,7 +69,7 @@ Every module in `tests/`, by the area it guards.
| Pack probe | the cores a platform runs for a system, staging grown to a fixed point, runs mapped to pack and manifest entries, the line and its schema, the probe lock per run, that no page shows its lines. One class builds a real pack narrowed to PlayStation and lays it out both ways, from the archive and with `install.py --dest`, and skips without the collection | `test_pack_probe` |
| Automation runner, disk cleanup, asset sync and bit-flip repair | fake jobs, git repositories with a bare origin and session worktrees, a committed file with one bit flipped, canned CI runs in place of `gh` | `test_autorun` |
| Truth and native export | synthetic + cached originals | `test_export_counts`, `test_export_native_layer`, `test_export_report`, `test_exporters`, `test_truth_data_dir_readers`, `test_truth_diff`, `test_truth_layout` |
| Scrapers, upstream and provenance | inline sources, synthetic | `test_batocera_md5_list`, `test_check_freshness`, `test_cited_lines`, `test_contributing_text`, `test_coverage_cache`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_dedup`, `test_doc_pages`, `test_doc_surfaces`, `test_elec_scrapers`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_launchers`, `test_launchers_emudeck`, `test_launchers_recalbox`, `test_launchers_retrobat`, `test_launchers_retrodeck`, `test_launchers_retropie`, `test_launchers_rocknix`, `test_mame_devices`, `test_mame_parser`, `test_mame_ref_audit`, `test_optional_dependencies`, `test_pin_build`, `test_profile_sync`, `test_prose_style`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_retired_files`, `test_romset_recipes`, `test_schema_semantic_gate`, `test_scraper_contract`, `test_site_agents`, `test_site_arcade_boards`, `test_site_availability`, `test_site_bootstrap`, `test_site_built_gate`, `test_site_components`, `test_site_console_folders`, `test_site_content_data`, `test_site_copy`, `test_site_cores`, `test_site_css`, `test_site_density_cards`, `test_site_emulator_page`, `test_site_emulators`, `test_site_entry`, `test_site_entry_tables`, `test_site_exports`, `test_site_funnel`, `test_site_images`, `test_site_inputs`, `test_site_insights`, `test_site_integration`, `test_site_nav`, `test_site_picker`, `test_site_platform_density`, `test_site_platform_hashes`, `test_site_privacy`, `test_site_reference`, `test_site_render`, `test_site_search`, `test_site_system_links`, `test_site_system_names`, `test_site_system_sections`, `test_site_systems`, `test_site_validation`, `test_site_weight`, `test_site_wiki`, `test_site_write`, `test_source_ref_targets`, `test_upstream`, `test_upstream_index`, `test_validate_pr`, `test_validate_pr_flow`, `test_validate_pr_hashes`, `test_version_basis`, `test_workflow_paths` |
| Scrapers, upstream and provenance | inline sources, synthetic | `test_batocera_md5_list`, `test_check_freshness`, `test_cited_lines`, `test_contributing_text`, `test_coverage_cache`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_dedup`, `test_doc_pages`, `test_doc_surfaces`, `test_elec_scrapers`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_launchers`, `test_launchers_emudeck`, `test_launchers_recalbox`, `test_launchers_retrobat`, `test_launchers_retrodeck`, `test_launchers_retropie`, `test_launchers_rocknix`, `test_mame_devices`, `test_mame_parser`, `test_mame_ref_audit`, `test_optional_dependencies`, `test_pin_build`, `test_profile_sync`, `test_prose_style`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_retired_files`, `test_romset_recipes`, `test_schema_semantic_gate`, `test_scraper_contract`, `test_site_agents`, `test_site_arcade_boards`, `test_site_availability`, `test_site_bootstrap`, `test_site_built_gate`, `test_site_components`, `test_site_console_folders`, `test_site_content_data`, `test_site_copy`, `test_site_cores`, `test_site_css`, `test_site_demo`, `test_site_density_cards`, `test_site_emulator_page`, `test_site_emulators`, `test_site_entry`, `test_site_entry_tables`, `test_site_exports`, `test_site_funnel`, `test_site_images`, `test_site_inputs`, `test_site_insights`, `test_site_integration`, `test_site_nav`, `test_site_picker`, `test_site_platform_density`, `test_site_platform_hashes`, `test_site_privacy`, `test_site_reference`, `test_site_render`, `test_site_search`, `test_site_system_links`, `test_site_system_names`, `test_site_system_sections`, `test_site_systems`, `test_site_validation`, `test_site_weight`, `test_site_wiki`, `test_site_write`, `test_source_ref_targets`, `test_upstream`, `test_upstream_index`, `test_validate_pr`, `test_validate_pr_flow`, `test_validate_pr_hashes`, `test_version_basis`, `test_workflow_paths` |
| Games on the card | small DATs, listxml and setup sources, a fake fetcher, fixture install lists, and the committed data against its schemas | `test_arcade_parents`, `test_game_catalogs`, `test_game_headers`, `test_game_regions`, `test_games_index`, `test_romfolders_card`, `test_romfolders_es` |
## Test architecture
+1
View File
@@ -1373,6 +1373,7 @@ before a release (wiki/release-process.md). Its lines stay in
| `generate_site.py` | Generate the site (this documentation) into the Starlight project in `website/` |
| `sitesources.py` | Record which cited source paths each pinned upstream tree holds, for the site's source links (`--fetch` reads a tree no cache holds) |
| `site_insights.py` | What readers look for and do not find, from totals only: searches that show the site but are rarely clicked, file names searched and whether the collection holds them, site searches that found nothing, broken addresses, "Did this page help?" answers, GitHub traffic and release downloads (`--days`, `--skip gsc,goatcounter,github`, `--output`). Reads Search Console, GoatCounter and GitHub with credentials kept outside the repository. [What the site counts](privacy.md) |
| `make_demo.py` | Record the installer demo of the README and the site into `docs_assets/demo/`: an ArkOS card with six games built under `tmp/demo/` and mounted in a mount namespace of the user, this checkout served on loopback, `docs_assets/demo/install.tape` run by VHS (needs `vhs`, `ttyd`, `ffmpeg`, `unshare`), then the GIF, WebM, MP4 and poster encoded and held to their size targets. `demo.json` records what the recording shows, for the alt text. Never run by the build |
| `validate_site.py` | Validate rendered metadata, heading order, image alternatives, JSON-LD, local resources, links and fragments, and the page size and search index budgets |
| `romset_recipes.py` | Identify which emulator version an arcade archive matches, rebuild a pinned archive from ROMs already held, record a recipe for every archive the collection holds and build any of them from the members it holds (see [Archive recipes](#archive-recipes)) |
| `archive_build.py` | The archive recipes behind `romset_recipes.py --capture`, `--build` and `--build-all`: capture, member lookup by sha1, verified build |