Commit Graph
1883 Commits
Author SHA1 Message Date
Abdessamad Derraz fdcf120a7d refactor: build cores from an exported tree
make runs on a git archive export of the commit and its submodules, the clone and its .git never bound into the sandbox. Host-side git runs with no user or system config, no hooks, no fsmonitor and the file and ext transports refused.
2026-10-27 12:07:16 +01:00
Abdessamad Derraz fd449fdae3 refactor: refuse unsandboxed campaign runs
Without a working bubblewrap the pass is refused and nothing third-party runs, probes require the sandbox and builds are refused, unless --allow-unsandboxed is given. A job report holding an alert raises an autorun alert.
2026-10-27 11:53:35 +01:00
Abdessamad Derraz e440073562 refactor: build the core sandbox as an allowlist
bwrap now binds only /usr, the /lib* and /bin links and the files of /etc the loader, fonts, TLS, the clock and user lookups read. Home, /tmp and /mnt are empty, the core, content and staged system directory read-only, the scratch folder the one writable path, every namespace unshared. --require-sandbox refuses to run without it. The worker imports nothing from the repository.
2026-10-27 11:53:35 +01:00
Abdessamad Derraz 95ca227bce refactor: run cores and builds in a sandbox
core_probe starts its worker with only the --env values, the worker clears its own environment before loading the core, and a working bubblewrap runs it with a read-only tree, no network and the user's credentials masked. The record names the isolation used. A timeout kills the whole process group. campaign.py starts core_probe, git, make and romfetch with the allowlist, make inside the same sandbox. --record refuses a folder that is not a checkout root.
2026-10-27 11:48:22 +01:00
Abdessamad Derraz e5cc1b5255 refactor: keep tokens out of autorun job steps
Every step, gate and recomputation runs with an allowlisted environment (PATH, HOME, LANG, LC_*, TZ, TMPDIR and the variables its env: key names). Names holding TOKEN, SECRET or PASS are dropped and refused in the registry. Only the push and the gh calls that read CI get GITHUB_TOKEN.
2026-10-27 11:48:22 +01:00
Abdessamad Derraz ee7319d838 feat: add the real-condition campaign runner
campaign.py takes buildbot nightlies (or a build of the pinned commit), the test content of tests/campaign/content.yml through romfetch, and runs core_probe for the files, boot, path, alias, variants and bench checks. A fixed rule table settles hle_fallback, required, path and aliases with the record ids as evidence, the rest goes to measurements/findings.jsonl. Resumable state, load, disk and budget guards, retries, content reference counting with an LRU cap, and an autorun job after the nightly.
2026-10-27 11:38:01 +01:00
Abdessamad Derraz faa79c5920 refactor: check job subjects with the repo hook
The runner carried its own copy of the refused vocabulary, which put those words in a tracked file. It now runs the commit-msg hook on each job's subject when the job list loads, and checks only the shape where no hook exists.
2026-10-27 11:08:19 +01:00
Abdessamad Derraz 07c34781e7 chore: merge the autorun runner and jobs 2026-10-27 10:41:53 +01:00
Abdessamad Derraz 343a4e8bd7 feat: add the autorun job runner
Runs the jobs of automation/jobs.yml under one host lock, with load and disk guards, one idle retry on a host misread, artifacts computed twice, explicit-path commits in .cache/autorun/worktree, an opt-in push (--push) and a CI circuit breaker. First jobs: nightly refresh chain, large-files asset dry run, disk cleanup and check, weekly bios integrity. install-timers writes systemd user units without enabling them.
2026-10-27 10:40:10 +01:00
Abdessamad Derraz 79c27b3798 feat: read saved findings in refresh_stale
--findings FILE takes a check_freshness.py --json report written earlier, kept to the --only areas, so one full check can drive a refresh of a few areas.
2026-10-27 10:40:05 +01:00
Abdessamad Derraz 0fbf920e45 chore: merge one-file measurement records 2026-10-27 10:24:41 +01:00
Abdessamad Derraz a9301fbdc0 chore: merge line mode on the screen's decisions
# Conflicts:
#	tests/test_install_tui.py
2026-10-27 10:20:31 +01:00
Abdessamad Derraz 77bdf7c206 fix: line mode answers in the merged tests
The TERM probe test answers the setup question, first-install keeps
its button name on one line and under 400 words, and --page of the
installer doc script is named on the page.
2026-10-27 10:19:37 +01:00
Abdessamad Derraz 11c65f40fc chore: align card depth test and document page 2026-10-27 10:14:56 +01:00
Abdessamad Derraz 102108f849 fix: hold launcher tests off the host's arkos 2026-10-27 10:14:55 +01:00
Abdessamad Derraz da68f19411 refactor: keep one full record per line
Each line of measurements/<emulator>.jsonl now holds the whole run:
every repetition, core logs, per-frame hashes and raw timings. The gzip
data files, the data schema and the split and join code are gone, and
core_probe --record appends one line. record_version moves to 3.

The seven records are rewritten once: each new line equals its old line
joined with its data file, and rebuilds the first-layout file its id
hashes.
2026-10-27 09:57:39 +01:00
Abdessamad Derraz 8c1887fd0c chore: merge main into the audit answers 2026-10-27 09:57:16 +01:00
Abdessamad Derraz 559494ae43 fix: line mode reads the screen's decisions
Answers code_audit_16 and tui_audit_9: share drives give their bios
folder, card markers stop at mount points, a model finds its family,
the folder question quits on q, pages move by the rows drawn, and a
folder the installer proposed is created without the guard.
2026-10-27 09:55:50 +01:00
Abdessamad Derraz 9cf0f9177d chore: merge installer tables from code
# Conflicts:
#	install.ps1
#	install.sh
#	install/launchers/index.json
#	scripts/measurement_records.py
#	tests/test_install_tui.py
#	wiki/installer.md
2026-10-27 09:48:44 +01:00
Abdessamad Derraz 47ed945b4a feat: write installer tables from install.py
scripts/generate_installer_doc.py rewrites the marked tables of
wiki/installer.md: options from build_parser(), events from EVENT_KEYS,
exit codes from the sys.exit calls, RETROBIOS_PAD values and the keys of
each route from TUI_ROUTES, the one table the bar, the help and the stop
line now read. Prose cells are kept, a row the code gains stops the
script until its text is written, and --check runs in the tests. The
site drops the markers.

The full screen falls back to the line questions where the terminal's
terminfo moves no cursor the ANSI way (vt52, glass terminals), and --tui
names why. The line mode draws no escape code with TERM unset, dumb or
such a terminal. Ctrl-Z typed on the screen stops the run as SIGTSTP
does, the terminal handed back and redrawn on fg. A window too small
names 40x12 and its own size.

Pty tests check the modes, the cursor and the main screen after Ctrl-C,
Ctrl-D, SIGTERM, SIGHUP and Ctrl-Z then fg. Tests from a mutmut run over
the device filter, the card bounds and the WSL paths raise its score
from 62/71 to 71/71. The parser lists its options in the page's order,
with NAME, PATH, NAMES and N as arguments.
2026-10-27 09:47:44 +01:00
Abdessamad Derraz 9c34e8891f chore: merge origin/main 2026-10-27 09:11:43 +01:00
Abdessamad Derraz 571e50f2a8 refactor: read measurements from the local tree
The site compiles the index and the SQLite files from measurements/
instead of a pinned clone. The pin, the sync script, the CI clone steps,
the freshness area and its refresh handler are gone. core_probe --record
writes into measurements/ of the checkout by default, and a test fails
when a line or data file HEAD holds changed or vanished.
2026-10-27 09:11:12 +01:00
Abdessamad Derraz 11a4698e06 chore: merge line and screen setup parity 2026-10-27 08:43:49 +01:00
Abdessamad Derraz 4fbbd2c760 fix: one setup decision for line mode and screen
Line mode finds mounted cards, shows its menu and names the folder for a
pack, asks the folder instead of taking a default, and lists setups in the
screen's order. Devices have one spelling each, matched without case,
spaces or punctuation, and a device several systems run lists them all.
--tui --dest reads a card as --plain does. Card boundaries read every
parent up to a mount point from mountinfo. --offline with --yes counts the
files it leaves out as pending, never as failed. The P3 points of
code_audit_15 and tui_audit_8 follow.
2026-10-27 08:43:08 +01:00
Abdessamad Derraz 087e5b4d73 feat: show measurements on the site
Each emulator with records gets a Tested block: claim, result, method, computer, core build and the permalink of its record at the pin. A Measurements page filters every test in the browser from the compact index, and the Data page lists measurements.sqlite with its tables. One SQLite file per emulator is linked from its page. wiki/measurements.md says how a test runs, held to the Start here budgets.

Both workflows fetch the pinned commit alone with git before the build, and the deploy fails without it.
2026-10-27 08:30:16 +01:00
Abdessamad Derraz 6109d7abd5 feat: split probe records into line and sidecar
A record is now one line of records/<emulator>.jsonl in the measurements repository, under 20 KB, and a gzip sidecar holding every repetition, the core logs and the raw timings, its sha256 in the line. record_version 2, core-probe-data.schema.json for the sidecar. core_probe --record appends to a measurements clone and refuses a directory inside this repository's tracked tree.

The seven records of issue 85 moved to Abdess/retrobios-measurements: each line plus its sidecar rebuilds the first file, and keeps its first id.
2026-10-27 08:30:07 +01:00
Abdessamad Derraz e627f0cf49 chore: merge the core probe and issue 85 runs 2026-10-27 07:02:05 +01:00
Abdessamad Derraz fecc6d9d32 chore: merge round 5 search fixes 2026-10-27 06:54:51 +01:00
Abdessamad Derraz 57c5d3b7ce chore: check the PlayStation opening names a BIOS
The opening line follows the profiles' required and hle_fallback data: it reads "No file needed with ... Only ... require scph5501.bin" since most PlayStation cores run without the BIOS. The test asserted the words "Start with"; it now asserts what it meant, a linked scph BIOS before the first card.
2026-10-27 06:54:13 +01:00
Abdessamad Derraz 07f785cfd7 fix: rank a console first for its own name
The PlayStation 2 page holds every word of "playstation bios" and pagefind ignores the words a page has that the query lacks, so its own body matches put it first. A console whose every word another system page's consoles also hold is now written again in an `exact` field (weight 20). File names reach the `files` field as their parts: whole names let "ds" reach twelve dsi_* names on the DSi page. `consoles` weighs 100 so an emulator named after its console (Saturn.emu) comes second, and `aliases` 10 so the Nintendo page keeps "gba bios". The gate runs both directions of the PlayStation, Game Boy, DS and Wii pairs and Saturn.
2026-10-27 06:54:08 +01:00
Abdessamad Derraz c2d745fbfa chore: merge one card reading and offline wording 2026-10-27 06:12:18 +01:00
Abdessamad Derraz 74f4e301a6 fix: one card reading and one offline wording
card_choice is the one reading of a card's root for --dest, the setup
list with --dest, a card listed, browsed, typed or numbered: a blank
stick named for KNULLI, ROCKNIX or a system no card file names gets its
folder below the root and the same note on the screen and in line mode,
kept on Create the folder? and the plan. The MinUI family refuses a blank
--dest. A card mounted as BIOS is that card, a folder inside another
system's card is refused up to the drive's mount point, a list named for
another system's card gets a note, a refused --dest emits an error
event. AmberELEC and EmuELEC get a card setup.

Offline, every surface says missing, not failed: no retry line, no
--from after --from, Nothing installed when nothing was, --check counts
what needs the internet, a suggested command keeps --offline.

Drive letters under WSL for every path option, shown as D:\.

The setup list names each card system's devices and puts the cards after
RetroArch when nothing is found (Onion: 4 presses at 40x12, was 10), the
filter and the line menu take a device name, one match is chosen in one
press. Line mode names the BIOS folder and the card picked, leaves the
save directory files out before its menu, offers f) to copy from a
folder. Confirm shows no Install while it counts, Back from the systems
lands on Install, help keeps its column, one spelling per device.
2026-10-27 06:10:48 +01:00
Abdessamad Derraz 88324d595e fix: keep the host system out of a from test
On a WSL host the in-process run turned drive letters on for every test after it, and three TUI tests failed in the combined suite.
2026-10-27 06:10:31 +01:00
Abdessamad Derraz 4522f3ba01 fix: keep the paths of a tree no folder lists
A repository that answers 404 (TeknoParrotUI) was listed folder by folder,
every listing failed, and the tree read as holding nothing: a fetch without
its cached tree dropped its recorded paths. It is now unread, so the record
keeps what it held.
2026-10-27 05:55:10 +01:00
Abdessamad Derraz 8b12685c16 feat: settle a probe check per observable
A check is inconclusive when the compared observable differs between repetitions, a run fails to load or a file changes during it. A disagreement on another observable stays flagged without voiding the comparison. Core log lines name the staged directories instead of the host paths.
2026-10-27 05:53:52 +01:00
Abdessamad Derraz 68f42bf8e2 fix: judge each build's refs in its own tree
A profile keyed by build mode compared a part in every repository and kept
the best outcome: xenia-edge's libretro tree carries emulator.cc offset,
and its lines vouched for standalone refs that had moved 4 lines. A part
of an entry with a mode is read in that build's repository, and a part
naming none in the first build declared, as _realign_part decides.
2026-10-27 05:41:27 +01:00
Abdessamad Derraz 6a611cd77a chore: cover the round 5 search package 2026-10-27 05:40:09 +01:00
Abdessamad Derraz 9b2cddfbe5 fix: structured data names releases, not pins
The home page is the site: named RetroBIOS, part of nothing. A source repository states the core_version of its build when it names a release, never a commit id or a pin, and no placeholder description.
2026-10-27 05:40:03 +01:00
Abdessamad Derraz 15f7c72077 feat: setup tab, letter crumbs, console words
get/ pages join the tab of the setup pages of the same name, so no page is excluded from every tab and the tabs keep their names. Groups filing emulators by first letter are no crumb. A console page carries "<name> BIOS" in its terms field. The search gate runs the console, checker, schema, license and iOS queries.
2026-10-27 05:39:59 +01:00
Abdessamad Derraz 9690579f34 feat: record env files by name and sha256
A path given to --env is a detail of the host that ran the probe. The record keeps the file name and its sha256, which is what the run depended on.
2026-10-27 05:34:31 +01:00
Abdessamad Derraz d0de6dde30 fix: rebuild source paths from the written pins
load_emulator_profiles caches by directory, and the run had loaded the
profiles before writing them, so the record rebuilt after a write kept the
pins just replaced: linapple, then ten profiles in one advance. The cache
is cleared first. The record is rebuilt for those ten.
2026-10-27 05:24:04 +01:00
Abdessamad Derraz 00b034612f feat: add headless libretro core probe
core_probe.py loads a libretro core with no window, runs it for a
number of frames and reports the files it opened (strace), its log
lines, frame, audio and save-state hashes and its speed. bench repeats
a run after warm-up and gives median, mean, deviation and range; check
swaps one system file between runs and compares one observable.

--record writes a JSON record with the core build, frontend commit,
host and every input by hash, validated against
core-probe-record.schema.json. A busy host is refused, and a
repetition that disagrees with the first is flagged.
2026-10-27 05:18:10 +01:00
Abdessamad Derraz 951bd070bf feat: advance profiles that cite no source
_needs_review held every skipped report, so a profile with no source_ref
never moved: anarch sat behind a CI-only commit through every bulk advance.
Its source pin now moves; its upstream pin does not, since nothing was
compared there.
2026-10-27 04:26:28 +01:00
Abdessamad Derraz e96d1326a3 fix: skip the line mode pty run without pyte
The offline line mode test opened a pty with pyte missing and errored where its neighbours skip. A comment in install.py also carried a clause semicolon that the prose style test refuses.
2026-10-27 04:20:23 +01:00
Abdessamad Derraz 625bee35cc fix: record missing data dirs in the run
main() starts each run with no notices, so the mocked run records the
missing directory itself.
2026-10-27 04:16:33 +01:00
Abdessamad Derraz 4b11a43a86 feat: version check reads tags and numbers
A project that tags without releases (eightyone, higan) was never compared,
and a core_version naming no tag (parallel-launcher 9.0.5 beside v9.0-6)
silenced the passed-release warning. The newest tag stands in when it is a
version number, and an untagged core_version is compared by number within
one scheme. --emulator is repeatable: a stored value kept only the last.
2026-10-27 04:16:31 +01:00
Abdessamad Derraz d1d88fed27 feat: one card list and one card check
Every system that runs from its SD card comes from card_setups(),
derived from the device rules: the setup list, its filter, --platform
and the line menus give each its own row and ask for its card. The
RetroArch row no longer lists handheld systems.

card_root_folder and card_choice share one identity check: another
system's card is refused for every system, KNULLI and ROCKNIX included,
and the same --dest gives the same folder from every entry point. WSL
drive letters are read as typed, the line question lists only that
system's cards and asks again after no.

Offline at a terminal, files left out count as missing: the run ends
with code 1 or 2, as the full screen does. Continue on the systems list
opens Confirm. Wording and layout points of code_audit_13 and
tui_audit_6.
2026-10-27 02:55:29 +01:00
Abdessamad Derraz 7abe5b8309 fix: say offline files are not downloaded
The line mode under --offline, with files that need the internet,
ended on Done with 0 files installed. It now counts them as not
downloaded and says --offline is why, not an answer.
2026-10-27 00:45:57 +01:00
Abdessamad Derraz fe1e98917b fix: offer only what an install can do
Back from the systems list with nothing ticked keeps the ticks it
opened with, and None says First system once nothing is left to untick.
Confirm under --offline with nothing on hand has no Install, and the line
mode no longer asks to download what --offline never fetches. The line
mode counts the data files before its question.

Screen words: the checking clock starts at zero, a pack installs from
the pack, the copies found are said on a line of their own, the screen
of misplaced files says the copies stay, the save folder gets a line of
its own, the shared note is one line, a size keeps its unit, the same
save folder remedy on both screens, A Tick on the Miyoo bar, no Home
key in the help of a pad, and a system name said once with a pack.
2026-10-27 00:17:26 +01:00
Abdessamad Derraz 44f94ab703 fix: ask a card setup for its own card
A card system chosen on a computer, from --platform, the setup list, its
filter or the line-mode menu, always asks for its card. The card screen
lists only the cards whose files say they run that system, offers Look
again, and checks a browsed, typed or --dest folder as --dest is checked.
The line-mode Card root prompt lists the mounted drives, asks again on a
missing or foreign root, and asks before creating several folders. On the
device the detected folder is offered, and the bound card is forgotten
when another setup is chosen.
2026-10-27 00:04:56 +01:00