Commit Graph
397 Commits
Author SHA1 Message Date
Abdessamad Derraz c29c456206 fix: name the python install command per system
Without Python the bootstrap told every system to run sudo apt install python3. It now reads Termux, macOS and /etc/os-release to name pkg, xcode-select, apt, dnf, pacman, zypper or apk, and in Termux offers to run pkg install python, which needs no root.
2026-10-21 10:04:54 +02:00
Abdessamad Derraz d16f8fd17a fix: accept downloads by cryptographic hash only
A crc32 and a size can be met by crafted bytes, so a remote download
matched on them could land in bios/. verify_content now requires every
sha256, sha1 and md5 the entry declares to match, and an entry declaring
none is reported as not verifiable from the network and nothing is
written. crc32 and size still find a file the database already holds.
2026-10-21 10:04:20 +02:00
Abdessamad Derraz 7cebbfe2ec feat: fill standalone emulator folders
With no platform detected, a machine running standalone emulators (ES-DE with DuckStation and PCSX2) was offered RetroArch's folder. The installer now fills the emulator folders of the standalone copies list that exist here, from the EmuDeck then RetroArch file lists, asking first at a terminal.
2026-10-21 10:03:58 +02:00
Abdessamad Derraz f4010bfd18 docs: state what each .info declares, not a grade
Ten profile notes graded a .info (correct, accurate, still, missing, omits). Each now says what the file declares and what the code reads. test_no_grade_on_another_projects_file reads every profile line that names a .info, core-info or System.dat.
2026-10-21 10:02:16 +02:00
Abdessamad Derraz 0b0480a121 fix: name standalone folders instead of skipped
Every install ended on 'Standalone copies skipped', on machines with no standalone emulator as on one whose DuckStation lacked its BIOS. The line now lists the emulator folders that exist here, and nothing is said when there are none.
2026-10-21 10:02:15 +02:00
Abdessamad Derraz 6c076884ed fix: let auto_fetch prove sha256 and crc32 entries
verify_content read sha1 then md5 only, so an entry pinned by sha256 or
crc32 could be searched and never accepted, and one without any hash was
announced as searched. The strongest declared hash now decides, crc32
counts with its size, the database lookup reads sha256 and crc32, and an
entry with nothing to prove a download by is reported, not searched.
2026-10-21 10:02:05 +02:00
Abdessamad Derraz e9f945857c docs: describe the native export without verdicts 2026-10-21 10:01:46 +02:00
Abdessamad Derraz ff8a390ce6 fix: say when --dest does not exist
A mistyped --dest was created with every missing parent and reported done. The run now names the missing folder and the nearest one that exists, asks at a terminal before creating more than one level, and prints the folder it created. The free-space check now runs before the folder is made and ends with 2, nothing installed.
2026-10-21 10:01:34 +02:00
Abdessamad Derraz 0b772fe255 docs: compare scraper output in the repo tmp/ 2026-10-21 10:01:07 +02:00
Abdessamad Derraz f3586001ab fix: fail freshness on unknown, name the refresher
check_freshness exited 0 when no upstream answered. An unknown row now
holds the exit code, and a profile skipped by nature is reported skipped
so it does not hold it forever. --only names the valid areas, a stale
native row names its platform, and the summary names refresh_stale.py.
2026-10-21 10:00:36 +02:00
Abdessamad Derraz e257504e20 docs: point the CI table to each workflow's paths 2026-10-21 10:00:21 +02:00
Abdessamad Derraz 7e926ac909 docs: say which entries skip the name steps 2026-10-21 09:59:53 +02:00
Abdessamad Derraz 630d5d85fb docs: name the system names --system takes 2026-10-21 09:59:33 +02:00
Abdessamad Derraz 6bca75b23e docs: name the registry keys the tools read 2026-10-21 09:59:22 +02:00
Abdessamad Derraz cc57d44de5 fix: order inherited rescrapes, end on verify
refresh_stale ran a parent's rescrape and its children's truth and
export jobs side by side, so a child read the parent mid-rewrite. A job
now starts once the rescrapes of its ancestors have ended, and a child
no row names gets its export rewritten after them. A rescrape chain ends
on verify.py --platform, whose summary closes the report.
2026-10-21 09:58:58 +02:00
Abdessamad Derraz f997bdc81a fix: check a recalbox zip by its composite only
Bios::Scan takes Zip::Md5Composite for every .zip (Bios.cpp:64-68). The resolver already honoured zip_digest: composite, but digest_verdict accepted the container's own md5 first, so the pack check passed bytes Recalbox marks red. The page described that order.
2026-10-21 09:57:06 +02:00
Abdessamad Derraz 563ce1ebf2 fix: read fbneo dats at one checked commit
The FBNeo import recorded the listing's blob sha for bytes fetched from a
branch URL a CDN caches, and --fetch <revision> still read the tip. The
revision is now resolved to a commit, listed and downloaded there, each
DAT is held against its blob sha, and a DAT read at a named revision is
labelled with it without replacing the tip's recorded blobs.
2026-10-21 09:55:50 +02:00
Abdessamad Derraz d8e9ad23d3 docs: describe short md5 handling as the code does
No platform YAML holds an md5 shorter than 32 characters: the Batocera scraper completes one when a single collection md5 starts with it. The pages said some entries carry 29 characters and that scrapers store them as-is.
2026-10-21 09:54:57 +02:00
Abdessamad Derraz 10c94369df fix: no sudo advice for a read-only mount
EROFS was advised like a permission refusal, and sudo fails the same way on a read-only mount. The advice now reads the mount flag and says to check and remount the card or drive.
2026-10-21 09:54:55 +02:00
Abdessamad Derraz 730f900ff5 fix: detect emudeck on macos
EmuDeck's macOS build reads ~/.config/EmuDeck/settings.sh like the Linux one (functions/all.sh:3-4,24 and vars.sh:2 at dddb3022). The settings and ~/Emulation/bios rules now run on darwin too.
2026-10-21 09:53:37 +02:00
Abdessamad Derraz 12cc71b634 fix: hold the pipeline's fixed outputs for a run
The pipeline locked only --output-dir while it also writes database.json,
install/, README.md and docs/. A run on a scratch output directory could
start beside the run on dist/ and rewrite them. Every run now holds the
repository's .lock first.
2026-10-21 09:53:15 +02:00
Abdessamad Derraz a6d37e40ae docs: show the handy output verify prints 2026-10-21 09:52:57 +02:00
Abdessamad Derraz 43337c6719 fix: match release packs by exact name
release_record rebuilt nothing from the name builder: a substring match
took RetroArch_v1.22.2_AndroidArm64V8A_BIOS_Pack.zip for the RetroArch
pack. The record now refuses any pack that is not a registered platform's
full pack and stores the platforms each pack serves, which the readers
use instead of the pack name.
2026-10-21 09:51:55 +02:00
Abdessamad Derraz f2628c1170 docs: drop hand-kept --derive figures from tools 2026-10-21 09:51:51 +02:00
Abdessamad Derraz 6877512af7 feat: carry new upstream names and hashes in json
--detect-new-files and --watch-hashes were printed only by the text report and refused with --json, so check_freshness --profiles never saw a name or hash the cited code gained. Both now live on the report, and check_freshness passes them and marks such a profile stale.
2026-10-21 09:51:14 +02:00
Abdessamad Derraz 6c2204a7a8 chore: merge tooling follow-up
# Conflicts:
#	tests/test_audit_regressions.py
2026-10-21 06:50:20 +02:00
Abdessamad Derraz 378a9e4da5 chore: merge audit fixes (integrity)
# Conflicts:
#	install/batocera.json
#	install/emudeck.json
#	install/lakka.json
#	install/recalbox.json
#	install/retroarch.json
#	install/retrobat.json
#	install/retrodeck.json
#	install/retropie.json
#	install/rocknix.json
#	install/romm.json
#	scripts/generate_pack.py
#	wiki/tools.md
2026-10-21 06:48:42 +02:00
Abdessamad Derraz 57ac252482 feat: check the profiles a branch adds or edits
profile_sync gains --changed (added or edited profiles, uncommitted
ones included) and --added, both against the merge base of --base
(origin/main by default). CONTRIBUTING runs --changed --strict instead
of one --emulator call per new profile, and the CI job runs --added
in place of its own diff loop.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 35b4cacb5d chore: merge audit fixes (installer)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 06:02:02 +02:00
Abdessamad Derraz 1c4686ab93 feat: place RomM firmware by its own template
RomM 5.4 reads each platform's firmware where filesystem.structure.firmware
in config.yml says, bios/{platform} by default, and suggests {platform}/bios
for a {platform}/roms library (rommapp/romm 5.4.0,
backend/config/config_manager.py:58,101-109,205-225,564-568,1105-1124).
The installer only wrote bios/<platform>/. It now reads the template from
config.yml beside the library, takes --romm-firmware when that file is
elsewhere, refuses a platform-first library with no template, and detects
RomM by /romm/config/config.yml too.
2026-10-21 05:53:23 +02:00
Abdessamad Derraz 734643b8fc chore: merge audit fixes (site)
# Conflicts:
#	wiki/testing-guide.md
2026-10-21 05:53:15 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00
Abdessamad Derraz 53da2a95d2 chore: merge audit fixes (docs)
# Conflicts:
#	tests/test_doc_surfaces.py
#	wiki/installer.md
#	wiki/testing-guide.md
2026-10-21 05:50:25 +02:00
Abdessamad Derraz 43c6df58ee chore: merge audit fixes (region)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 05:48:59 +02:00
Abdessamad Derraz a67265e903 fix: let the site search land on cards
The search splits a page at its headings. Emulator cards were titled in
bold and platform systems folded into blocks under one heading, so a
pasted hash opened the top of a page or the systems overview of a
100-block platform page. Each file card is now a level 4 heading kept
out of the table of contents, and each platform system a heading.

Card lines that list other pages (platforms, packs, source citations,
collection paths) leave the search index: 24.7 MB down to 18.1 MB, the
names, hashes and notes still searchable.
2026-10-21 05:47:31 +02:00
Abdessamad Derraz f4a225488d chore: merge audit fixes (scrapers) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz eb44e3f898 chore: merge audit fixes (profiles) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 73d73e0f2a chore: merge audit fixes (profsync) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz e03fe338ee chore: merge audit fixes (tests) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 89c533e9ba fix: detect Batocera and Knulli by build files
No Batocera build writes /etc/batocera-version: batocera-system.mk:21-22
(batocera.linux 38845725, unchanged since batocera-30) writes
/usr/share/batocera/batocera.version. Knulli (knulli-linux 57afa662) adds
OS_NAME=knulli to os-release, writes /usr/share/knulli/knulli.version and
removes /usr/share/batocera, and no Knulli source writes /etc/knulli-release.
Neither was detected. A Knulli board is named and not narrowed by
Batocera's per-board lists, and an unlisted board is named.
2026-10-21 05:44:27 +02:00
Abdessamad Derraz 5858ebf53e docs: restate the recipe store counts
The reimport kept the sets holding missing catalog dumps: 24,101 entries
in 2,148 distinct recipes.
2026-10-21 05:38:58 +02:00
Abdessamad Derraz edda080c59 fix: read muOS RetroArch from its own config
muOS runs RetroArch on a config copied from its default, which reads
/run/muos/storage/bios, bound to MUOS/bios on the second card when that
card holds one, and loads cores from /opt/muos/share/core (MustardOS/internal
f41f4a89, script/control/retroarch.sh:8-20, script/device/bind.sh:116-150,
retroarch.default.cfg:2806,3151). The installer wrote to /mnt/mmc/MUOS/bios
and --installed-cores found no core. The config now decides both.
2026-10-21 05:35:45 +02:00
Abdessamad Derraz bdcc8f0fb6 fix: lower the pipeline floor to Python 3.9
The 3.12 guard rested on generate_readme f-strings that have since been
rewritten. Every script compiles under CPython 3.9.25, 3.10 and 3.11, and
only generate_pack's parenthesized context managers fail under 3.8. Under
3.9 the offline pipeline (--skip-packs) completed with no change to the
committed artifacts, the 15 manifests matched 3.12 byte for byte, and the
RomM pack was identical. CI now compiles every script under a real 3.9.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz 15cb52d551 fix: find EmuDeck Windows where its scripts look
functions/all.ps1:4-14 (emudeck-we 037460ef) reads the APPDATA
settings.ps1, else the one under the profile, and takes C:\Emulation when
neither names a folder. Only the first was read and a forced platform fell
back to %USERPROFILE%\Emulation\bios. A network install names its share
through $VolName (JSONtoPS1.ps1:166-172), and a UTF-16 settings file is
decoded.
2026-10-21 05:31:35 +02:00
Abdessamad Derraz cdb2cfa6f0 feat: find RetroBat through the folder it records
RetroBat.exe writes its folder to HKCU\SOFTWARE\RetroBat
LatestKnownInstallPath on every start (retrobat-executable 702da66a,
Program.cs:57 and 99, RegistryTools.cs:15-50). Only C:\RetroBat and the
profile folder were looked at, so an install on D: was missed and a forced
--platform retrobat wrote into C:\RetroBat\bios. The value is read
through winreg, or reg.exe under WSL, before the fixed folders.
2026-10-21 05:28:35 +02:00
Abdessamad Derraz 8f86842117 feat: keep romsets holding a missing catalog dump
The recipe snapshot kept only the archives a profile or platform names,
so provenance_report --missing named no set for dumps a public romset
carries (fm7740sx holds the FM77AV40SX DICROM, EXTSUB and KANJI2 ROMs).
The importer now also keeps the sets holding a catalog entry the
collection lacks. Reimported from mame0289 and the cached FBNeo DATs:
MAME 1726 to 2148 recipes, FBNeo 104 to 111, none dropped.
2026-10-21 05:27:42 +02:00
Abdessamad Derraz eb385a4039 docs: note the profile verdict check in the guide 2026-10-21 05:27:09 +02:00
Abdessamad Derraz 309231afc2 docs: say what --skip-packs still runs
The help said "Only regenerate DB and verify" while steps 1b, 4d, 7 and
8 still ran and rewrote derivable.json, README.md and docs/. It now names
the steps it skips and points to --skip-docs.
2026-10-21 05:27:07 +02:00
Abdessamad Derraz b464ed9202 fix: count prose that holds the pin as review work
The triage, --changed-only and --strict counted refs only. Prose that
--realign-prose would move or refuse blocks every recale and bump, and
blastem read 0 to review. Each report now carries that reading. The
triage also gets a header, a closing count, and one sampling notice per
run instead of one per profile.
2026-10-21 05:24:17 +02:00
Abdessamad Derraz 5e70fe7ff5 docs: state each verification rule once
architecture.md drew path_suffix ahead of SHA1 and listed a core
resolution that missed the retroarch list entry and built_by. Both now
link to the one statement that matches common.py. The Recalbox table
read hashMatchMandatory into verify.py, which never reads it, and
md5_composite hashes member contents, not member MD5s
(Zip.cpp:71-96). The validation index is built from the platform's
resolved profiles only. The requirement table gains the fields that
make a platform YAML reversible, and pages that name a platform list
say what changes rather than judging it.
2026-10-21 05:22:38 +02:00