Commit Graph
1789 Commits
Author SHA1 Message Date
Abdessamad Derraz dd64434122 chore: merge main 2026-10-27 22:05:47 +01:00
Abdessamad Derraz e90fb85050 fix: bind the interpreter into the probe sandbox
The sandbox bound only /usr, /lib and /bin. On the CI runner setup-python puts Python under /opt/hostedtoolcache, so the worker could not start and every core_probe and campaign run crashed (7 tests, plus their replay without jsonschema). A venv, pyenv or uv do the same. The interpreter's prefix and every link on the way to it are now bound read-only, each at its own path. Reproduced with uv's Python 3.13 outside /usr, 92 tests pass under it and under the system Python.
2026-10-27 22:05:34 +01:00
Abdessamad Derraz 06ca83bdca fix: one short hash length across the site
Merging the card and platform branches left two SHORT_HASH definitions, 8 and 12. The later one silently won, so the platform rows showed 12 characters while their tests expected 8. One constant now, 12, the card anchor's length. The copy-button selector skips buttons that carry their own data-copy, and the tests follow the merged markup.
2026-10-27 22:02:41 +01:00
Abdessamad Derraz 917ab7c94b chore: merge platform file rows 2026-10-27 21:46:44 +01:00
Abdessamad Derraz 793b16cf67 chore: merge folded cards and one install command 2026-10-27 21:45:48 +01:00
Abdessamad Derraz e7d33caa3f feat: platform file rows that open full width
Each file is a details whose summary is the row: name, folder when it is not the bare name, status chip. The short hash sits on one copy button beside it, since a summary is itself a control. Opened, the evidence runs across the list. A list with no folder drops the column. Code spans in the evidence are plain text.

The lists split by the share of a page they render to (elements against 10,000, bytes against 600,000) instead of Markdown length. With its parts, Batocera goes from 12 pages, 18,854 elements and 1.21 MB to 10 pages, 13,671 and 1.11 MB. A part that folds back into its platform page redirects there, from the record in website/src/data/platform-parts.json.
2026-10-27 21:41:00 +01:00
Abdessamad Derraz e370970363 feat: write a finding again on new evidence
A finding line is written again when the records behind it change, so an open finding cites the oracle runs rather than a frame comparison. A contradicted validation shows the checks the runs prove, content and size, beside the profile's list.
2026-10-27 21:16:54 +01:00
Abdessamad Derraz 0e2e014a96 chore: merge the long list tables 2026-10-27 21:15:17 +01:00
Abdessamad Derraz 1409a8db6a feat: list long file pages as filtered tables
MAME's card pages, GroovyMAME and yaps2 read 13,000 to 16,700 words of
cards. Each entry is now a table row (name, members, status, short
hash) that opens to its card in a details block, with the site's filter
box and row counter. Pages from 150 entries and every split part use it.
Visible words: mame-files-1 16,680 to 1,612, groovymame 14,396 to 1,876.

A row costs about five elements, so a part holds 180 cards: MAME goes
from 8 parts to 9, the largest at 567 KB and 8,989 elements.

Each wanted list (TOSEC, No-Intro, Redump) opens on its counts, then one
table folded under a filter box that opens it, in place of a block per
DAT. Only the names are searched: indexed descriptions and hashes put
the PlayStation page ahead of the PlayStation 2 one.

A filtered row stayed visible on a phone: a stacked row's display:block
outweighed the hidden attribute on every stacked table.
2026-10-27 21:14:04 +01:00
Abdessamad Derraz fabd431adb feat: take the host probe lock per run
core_probe takes probe.lock itself around one execution, load check included, and releases it when the run ends. A campaign pass never holds it between runs, so two passes interleave. RETROBIOS_PROBE_LOCK names another file, RETROBIOS_PROBE_LOCK_HELD says the caller holds it. Tests use their own lock file.
2026-10-27 21:02:03 +01:00
Abdessamad Derraz a8340614b8 chore: merge origin/main 2026-10-27 20:57:27 +01:00
Abdessamad Derraz 8adb84696d feat: fold card hashes, one install command
System and emulator file cards show the name, size, download, a short
SHA1 with a copy button, the setups and the emulators. The full hashes
and the other facts sit in one closed fold, each hash with a copy button.

A system's install box shows one command under the OS tabs for the
setup picked in a list, remembered across pages. Without the script the
list of every command shows, open. The Markdown the page budgets are
measured on is unchanged, so no card moves.
2026-10-27 20:56:07 +01:00
Abdessamad Derraz 20dc488e41 feat: list platform files as table rows
Each file of a platform list is one row: name, folder, status chip, a short hash on a button that copies it whole, and a details cell. The details hold every declared hash, the size, the served copy and the readers. A reference's role is one word (glue, port, upstream, build) defined once in a legend instead of a phrase after every reference.

Visible words: Batocera 2,433 to 2,332, AmberELEC 1,301 to 1,299, RomM 1,607 to 1,596, RetroArch 2,207 to 2,088, RetroDECK arcade part 2 19,127 to 2,104. Rows weigh more Markdown, so 38 more systems move to pages of their own under the existing budgets.
2026-10-27 20:44:05 +01:00
Abdessamad Derraz 74b49e8c04 refactor: return source ref parts with roles
source_ref_items() gives each part of a source_ref with the role of the code it cites. _source_ref_markdown() joins them as before, so every other page renders the same text.
2026-10-27 20:43:59 +01:00
Abdessamad Derraz b6ed5ba0a1 feat: group tested rows by the file they vary
Rows sit under the file under test. A title names the game and what sets the test apart from the others of its file (build, clock, region, files tried), never cut. Zero counts leave the summary. A legend explains the dots. The chips say depends on the file and same with every file. Ran on shows the short host.
2026-10-27 19:48:16 +01:00
Abdessamad Derraz 0d510eb4bd chore: merge core versions read from code 2026-10-27 19:42:04 +01:00
Abdessamad Derraz e7d0c0183c feat: show each test as one row with its proof
The Tested block opens on one summary line, then a row per test: a short title, a result chip, a dot per variant and the date. Method, claim, computer, core build and the record wait in the row's details. Tests from before the boot oracle fold under one line, benchmarks get their own table. beetle_saturn shows under 120 words before anything is opened.
2026-10-27 19:40:16 +01:00
Abdessamad Derraz 91b7bd94d0 feat: refuse placeholder versions in the schema
core_version is a string the code states, null or keyed by build mode,
and the schema refuses Git, SVN, unknown, master, main, latest, dev and
HEAD, alone or annotated. version_basis is defined per kind with the
fields each one needs, and kind none requires a null version.

tests/test_version_basis.py reads a recorded fixture of forge answers
offline and fails on drift, holds every profile to a basis (tag versions
in describe form, aliases equal to their profile, none as null), checks
that a pin advance reads the version again or is refused, and that the
schema refuses placeholders.
2026-10-27 19:23:17 +01:00
Abdessamad Derraz c8eb3a79e3 feat: say when the system runs another version
The installer reads the version the system states in its own file,
without running anything: batocera.version, recalbox.version (a release
only), RetroBat's system/version.info, VERSION_ID in os-release on Lakka
and EmuELEC. When it differs from the version the list was built for, a
line says so before the install and again with the result, and the new
version event carries it for the screen. An unknown version says
nothing, and the install never stops on it.
2026-10-27 19:10:44 +01:00
Abdessamad Derraz 25c356669b fix: record each platform's own list version
install/<platform>.json wrote the constant "1.0" as the version of the
list it was built from. The manifest now carries the version the platform
YAML records, and generate_pack refuses a YAML without one.

inherits: no longer passes a parent's version to a child: Lakka was
published as RetroArch v1.22.2 in its manifest, pack name and site page.
lakka.yml records v6.1, the Lakka-LibreELEC tag its target list is read
at. The MiSTer scraper refuses a BiosDB without a timestamp instead of
writing an empty version.

A pack's manifest.json records list_version, so an install from the pack
alone knows it too. Manifests regenerated twice and compared: only
version and generated changed.
2026-10-27 19:10:39 +01:00
Abdessamad Derraz 77a1a7285a feat: write the romset table from the profiles
The FAQ table of arcade romsets was typed by hand and told readers to
collect MAME 0.160 and FinalBurn Neo 1.0.0.2 sets for cores whose source
states 0.159 and 1.0.0.03. generate_site now writes it between generated
markers from the profiles, and a test holds the page to the generator.
An emulator page names where its version is read (the file, the tag and
the commits after it, the package), and says when the code states none
or the version could not be confirmed. HBMAME no longer claims a MAME
romset. tools.md and profiling.md document version_basis and the two
modes.
2026-10-27 18:59:30 +01:00
Abdessamad Derraz 07c82c885f refactor: read the fbneo version from version.h
FBNeo bumps src/burn/version.h without tagging: the scraper wrote the
last tag, v1.0.0.02, while the source said 1.0.0.03. It now reads the
header the way the libretro core prints it. The merge no longer writes
core_version: the hashes come from finalburnneo/FBNeo's tip, which the
libretro/FBNeo pin does not name, so the version stays the one
version_basis reads at that pin. A MAME release profile is written with
its version_basis.
2026-10-27 18:48:16 +01:00
Abdessamad Derraz 6aa236440a feat: read core_version from its recorded basis
version_basis says where a profile's version is read: a pattern in a
file, the nearest tag and its distance, an alias, a package, none, or
unverified with the reason. profile_sync --verify-version reads it again
at the pin and exits 1 on a difference, --set-version writes what the
code states. A pin advance reads the version at the new pin in the same
write and is refused when it cannot.

upstream.describe counts the commits after the nearest tag in a treeless
clone kept under the cache, and graph_ancestry answers a pair GitHub
refuses to compare.
2026-10-27 18:48:11 +01:00
Abdessamad Derraz 70c3bbd853 feat: settle a file only when the run uses it
hle_fallback needs the file to change the run: Beetle PSX hashes every file of its folder, so opened is not used. PCSX-ReARMed hides the BIOS logo by default: its BIOS load line stands for the bios stage. A check runs again only when the oracle entries of its content and core change, and the rules run on every pass from the kept records.
2026-10-27 17:58:48 +01:00
Abdessamad Derraz a4af2ffda4 fix: name --offline when given, keep page budget
--offline given with the variable set is named as the option, and the
runs the screen starts get it only when the user gave it. The first
install page stays under its 400 words, the 3.8 runs skip only the
pack test that needs generate_pack.
2026-10-27 16:57:34 +01:00
Abdessamad Derraz a17406edae feat: let the campaign read another content list
--content-file names the test content list for run and references.
2026-10-27 15:53:25 +01:00
Abdessamad Derraz 3f93da6394 docs: show boot verdicts on the measurement pages
The Tested blocks and the explorer show each variant's verdict. A frame comparison from before the oracle shows no verdict. The SQLite variants table gains verdict columns. The method page says what each verdict means and how a file is proven.
2026-10-27 15:52:29 +01:00
Abdessamad Derraz b95bf466e8 feat: add the PlayStation reference frames
Bust-A-Move 2 (USA) on Beetle PSX, Beetle PSX HW, SwanStation, PCSX-ReARMed and PCSX1, each frame reviewed on its PNG. A boot needs a screen of the game: the BIOS screens alone are not one.
2026-10-27 15:52:29 +01:00
Abdessamad Derraz 38dd7401fd feat: keep the campaign timer as an opt-in
campaign.py install-timer writes the autorun units and enables nothing unless --enable is passed. The docs say a contributor who wants a schedule on their own machine installs the timer.
2026-10-27 15:46:20 +01:00
Abdessamad Derraz 44ab3febf4 feat: settle profile fields from boot verdicts
The campaign runs the full set, no BIOS, and per file the file alone with zeros, its first half and other regions' dumps under its name. required, hle_fallback and validation are read from the verdicts, never from frames that differ. Benchmarks run only with --bench. The timer command is gone: a maintainer starts each pass. references lists stable screens for the oracle.
2026-10-27 15:45:22 +01:00
Abdessamad Derraz e4c7ded489 feat: judge each probe run with a boot oracle
core_probe writes record version 4. Each run is boots, bios_error, crash, hang or unknown, from reference frames, core error patterns and screens, and the process status. A crash or a timeout is a verdict now, rebuilt from the events the worker streamed. --vary accepts @zeros:SIZE and @head:SIZE:PATH, and run takes --repeat.
2026-10-27 15:45:22 +01:00
Abdessamad Derraz 0fa4552dae fix: refuse only the axis a list lacks
The MinUI family's lists record cores and regions and no system: every
narrowing was refused with a remedy that fetched the same list. Each
option is now refused by the axis missing, with what works, and a pack
names its platform's list only when that list records the axis. Line
mode names what c chooses and skips a question with one answer, and
Options leaves out parallel downloads when nothing is downloaded.
2026-10-27 15:37:24 +01:00
Abdessamad Derraz aa6ee6db1c feat: add the boot verdict to the record schema
Record version 4 carries verdict and verdict_evidence on every repetition and variant, verdicts on a check result, and inputs.oracle. Version 3 records stay valid.
2026-10-27 15:25:30 +01:00
Abdessamad Derraz a334ace439 fix: name cards by files no menu rewrites
A KNULLI card renamed in its menu was read as a Batocera SHARE drive.
KNULLI is now named by about.info first, Batocera by its own hostname,
and a drive two systems fit is said instead of guessed. Runs without
questions see mounted cards as the screen does and stop on two setups.
The card marker table of the wiki is generated.
2026-10-27 14:40:28 +01:00
Abdessamad Derraz 9c8dbcd99a chore: merge the automatic test campaign 2026-10-27 14:05:18 +01:00
Abdessamad Derraz 332c217155 feat: pass GITHUB_TOKEN to declared API steps
A step marked github_token: true receives the token. autorun accepts the key only on check_freshness.py, refresh_stale.py and python -m scripts.scraper.* tracked in this repository, refuses it on any other step and in the campaign job. The nightly steps declare it.
2026-10-27 13:41:40 +01:00
Abdessamad Derraz 57436839a8 fix: start each open finding with a capital
The statement of a finding follows a full stop on the Measurements page.
2026-10-27 13:07:33 +01:00
Abdessamad Derraz 5f3c836db9 chore: merge origin/main into the campaign work 2026-10-27 13:02:18 +01:00
Abdessamad Derraz 37d2c14e1d chore: keep campaign prose to the style rules
No clause semicolons in the campaign docstring and content list, and a help text for every option.
2026-10-27 13:02:15 +01:00
Abdessamad Derraz a67dc339bc chore: merge main into the audit 17 answers
# Conflicts:
#	install.ps1
#	install.sh
#	install/launchers/index.json
2026-10-27 12:48:04 +01:00
Abdessamad Derraz c138270edc refactor: keep commit rules in a tracked module
The rules lived only in the local commit-msg hook, which CI and other clones do not have, so autorun could only check a subject's shape there. scripts/commit_rules.py now holds them, the hook calls it, and autorun checks every job subject with it when the job list loads.
2026-10-27 12:41:06 +01:00
Abdessamad Derraz 31c574aab0 fix: line mode and screen share one decision
Answers code_audit_17 and tui_audit_10: the folder found after o, SHARE
drive roots, one same-choice command, a counted confirm step in line
mode, counts from the narrowed list, read-only advice without sudo,
systems recorded in packs, Ctrl-Z stopping the whole job, and the
wording, layout and docs items.
2026-10-27 12:24:26 +01:00
Abdessamad Derraz f5e98bfda6 refactor: fetch builds into bare mirrors
Host git only fetches into bare mirrors no sandbox ever sees, https only, with no user or system config and overrides emptying hooks, fsmonitor, ssh and proxy commands and every filter driver the mirror defines. Each build is a git archive of the commit and of each submodule commit from its own mirror. Old working-tree clones are deleted. The core copied out must be a regular file inside the build folder, opened without following links.
2026-10-27 12:19:55 +01:00
Abdessamad Derraz fdcf120a7d refactor: build cores from an exported tree
make runs on a git archive export of the commit and its submodules, the clone and its .git never bound into the sandbox. Host-side git runs with no user or system config, no hooks, no fsmonitor and the file and ext transports refused.
2026-10-27 12:07:16 +01:00
Abdessamad Derraz fd449fdae3 refactor: refuse unsandboxed campaign runs
Without a working bubblewrap the pass is refused and nothing third-party runs, probes require the sandbox and builds are refused, unless --allow-unsandboxed is given. A job report holding an alert raises an autorun alert.
2026-10-27 11:53:35 +01:00
Abdessamad Derraz e440073562 refactor: build the core sandbox as an allowlist
bwrap now binds only /usr, the /lib* and /bin links and the files of /etc the loader, fonts, TLS, the clock and user lookups read. Home, /tmp and /mnt are empty, the core, content and staged system directory read-only, the scratch folder the one writable path, every namespace unshared. --require-sandbox refuses to run without it. The worker imports nothing from the repository.
2026-10-27 11:53:35 +01:00
Abdessamad Derraz 960cc4a14e refactor: escape campaign values on the site
Values a run produced (core output, romfetch errors, profile names) are written as character references inside <code> or as plain text, so backticks, MDX braces and HTML stay text.
2026-10-27 11:48:22 +01:00
Abdessamad Derraz 95ca227bce refactor: run cores and builds in a sandbox
core_probe starts its worker with only the --env values, the worker clears its own environment before loading the core, and a working bubblewrap runs it with a read-only tree, no network and the user's credentials masked. The record names the isolation used. A timeout kills the whole process group. campaign.py starts core_probe, git, make and romfetch with the allowlist, make inside the same sandbox. --record refuses a folder that is not a checkout root.
2026-10-27 11:48:22 +01:00
Abdessamad Derraz e5cc1b5255 refactor: keep tokens out of autorun job steps
Every step, gate and recomputation runs with an allowlisted environment (PATH, HOME, LANG, LC_*, TZ, TMPDIR and the variables its env: key names). Names holding TOKEN, SECRET or PASS are dropped and refused in the registry. Only the push and the gh calls that read CI get GITHUB_TOKEN.
2026-10-27 11:48:22 +01:00
Abdessamad Derraz ee7319d838 feat: add the real-condition campaign runner
campaign.py takes buildbot nightlies (or a build of the pinned commit), the test content of tests/campaign/content.yml through romfetch, and runs core_probe for the files, boot, path, alias, variants and bench checks. A fixed rule table settles hle_fallback, required, path and aliases with the record ids as evidence, the rest goes to measurements/findings.jsonl. Resumable state, load, disk and budget guards, retries, content reference counting with an LRU cap, and an autorun job after the nightly.
2026-10-27 11:38:01 +01:00