Commit Graph
240 Commits
Author SHA1 Message Date
Abdessamad Derraz ce1f165d2f chore: merge round 2 fixes (tests)
# Conflicts:
#	install.ps1
#	install.py
#	install.sh
2026-10-21 10:29:19 +02:00
Abdessamad Derraz c8c4f3f486 chore: merge round 2 fixes (installer) 2026-10-21 10:17:44 +02:00
Abdessamad Derraz d7afac0993 chore: merge round 2 fixes (profile names) 2026-10-21 10:14:59 +02:00
Abdessamad Derraz de8ab51d6e fix: derive pack install guides from the registry
The pack README's one-line install was written per platform: ROCKNIX and MiSTer had none, EmuDeck's Windows edition was told to open Konsole, and RomM users were sent to library/bios/ whatever their firmware template. The automatic section now follows each platform's install.detect systems, the EmuDeck and RomM manual steps name every layout, and the download page lists EmuDeck under Windows and macOS.
2026-10-21 10:10:15 +02:00
Abdessamad Derraz d63c881648 chore: drop label comments across scripts
CommentsSayWhy read three scripts while 22 comments in 13 other files
named the next statement. They are removed or replaced by the reason,
and the test reads every tracked script and install.py.
2026-10-21 10:06:39 +02:00
Abdessamad Derraz d16f8fd17a fix: accept downloads by cryptographic hash only
A crc32 and a size can be met by crafted bytes, so a remote download
matched on them could land in bios/. verify_content now requires every
sha256, sha1 and md5 the entry declares to match, and an entry declaring
none is reported as not verifiable from the network and nothing is
written. crc32 and size still find a file the database already holds.
2026-10-21 10:04:20 +02:00
Abdessamad Derraz e255348f52 fix: read mame hiscore.dat where the cores open it
mame2000 opens hiscore.dat, history.dat and mameinfo.dat from IMAMEBASEPATH, the content directory (src/libretro/fileio.c:1002-1014, libretro.c:1044-1052). mame2014 and mame2015 open hiscore.dat through emu_file with no search path, the working directory (src/emu/hiscore.c:330-337, fileio.c:141-143). With no load_from the entries claimed system/ and the packs shipped files neither core opens there. ANoteOutsideTheSystemDirCarriesLoadFrom holds libretro notes and load_from together.
2026-10-21 10:02:30 +02:00
Abdessamad Derraz 62a4f19ce3 fix: keep ares labels off other dumps' names
ares finds firmware by sha256 (desktop-ui/settings/firmware.cpp:116-136) and names no file. Its FDS entry (RP2C33-01, crc32 1c7ae5d5) was labelled disksys.rom, the name nestopia, fceumm and four more cores read for RP2C33A-01A: ROCKNIX and EmuDeck got ares's revision at disksys.rom. It now carries MAME's member name rp2c33-01.bin.

The LaserActive v1.05 entry, the PAC-S10, PAC-N10, PAC-N1, PCE-LP1 and Neo Geo Pocket aliases are names RetroBat's launcher passes (Ares.Generator.cs:435-555): mega-ld.cpp:16-17 holds only the v1.04 US and v1.02 Japan hashes. RetroBat declares each name in its own list.

AHashScanLabelNamesNoOtherDump compares every found_by: content label with other profiles' entries of that name through the hash indexes.
2026-10-21 10:02:30 +02:00
Abdessamad Derraz 6c076884ed fix: let auto_fetch prove sha256 and crc32 entries
verify_content read sha1 then md5 only, so an entry pinned by sha256 or
crc32 could be searched and never accepted, and one without any hash was
announced as searched. The strongest declared hash now decides, crc32
counts with its size, the database lookup reads sha256 and crc32, and an
entry with nothing to prove a download by is reported, not searched.
2026-10-21 10:02:05 +02:00
Abdessamad Derraz fe2fcf6034 fix: create tmp/tests when the test package loads
Most modules write under tmp/, which a fresh clone lacks, and only three
created it on import. A module run alone failed with FileNotFoundError on
a temporary directory.
2026-10-21 10:01:23 +02:00
Abdessamad Derraz 7a120d7a51 fix: generate the site before CI runs the suite
docs/ is gitignored and was built after the tests, so the three guards
that read rendered pages skipped on every CI run. The site is generated
first, the suite runs whatever that step returns, and the guards fail
in CI when the pages are absent.
2026-10-21 09:59:22 +02:00
Abdessamad Derraz 9bb1a73f91 fix: hold every data tree, present or not
export_native and cross_reference read data/ without the shared hold, so
a refresh could swap a tree mid-walk. hold_data_dirs skipped a cache not
yet on disk, so a refresh could install it between verify and the packs
of one pipeline run. Both now hold every registered tree.
2026-10-21 09:56:49 +02:00
Abdessamad Derraz 8721dafd2a fix: load every script under the 3.9 floor
A module-level alias written with a PEP 604 union compiled under 3.9 and
raised TypeError when check_release_assets.py, and check_release.py
through it, were loaded. The floor test now imports every module under
scripts/ with a real 3.9 interpreter, which CI gives pyyaml and
jsonschema.
2026-10-21 09:53:34 +02:00
Abdessamad Derraz 2f5b4bd446 chore: resolve the merge markers left in a test 2026-10-21 06:50:36 +02:00
Abdessamad Derraz 6c2204a7a8 chore: merge tooling follow-up
# Conflicts:
#	tests/test_audit_regressions.py
2026-10-21 06:50:20 +02:00
Abdessamad Derraz 76e44a74b4 refactor: drop comments that restate the code
Thirty one-line labels in verify.py, generate_site.py and
generate_truth.py named the statement under them ("# Load registry",
"# Copy branding assets"). Comments that give a reason stay, as do the
module section headers. A test reads the three files for new ones.
2026-10-21 06:20:50 +02:00
Abdessamad Derraz 57ac252482 feat: check the profiles a branch adds or edits
profile_sync gains --changed (added or edited profiles, uncommitted
ones included) and --added, both against the merge base of --base
(origin/main by default). CONTRIBUTING runs --changed --strict instead
of one --emulator call per new profile, and the CI job runs --added
in place of its own diff loop.
2026-10-21 06:09:55 +02:00
Abdessamad Derraz 6b4010907d feat: list the six BizHawk DSi NAND slots
FirmwareDatabase.cs:405-410 declares NDS+NAND_JPN, EUR, USA, AUS, CHN
and KOR, each with a placeholder hash and 251658240 + 64 bytes. In DSi
mode DecideNAND takes the first one set, limited to the regions a
DSi-enhanced header allows, and stops the load with none
(MelonDS.cs:114, 304, 484, 684-706). DSi mode is off by default, so the
entries are optional, like DSi_Firmware.bin and the DSi BIOS slots.
2026-10-21 06:07:30 +02:00
Abdessamad Derraz 7056dd84ea fix: state why WinUAE needs the CD extended ROMs
A missing extended ROM only raises a notice (memory.cpp:1833-1849), but
the machine then runs on the Kickstart alone. cdstrap, cd.device,
cdtv.device and cdfs.library are resident modules of the CD32 and CDTV
extended ROMs and absent from their Kickstarts, the CD32 and base CDTV
models have no floppy drive (cfgfile.cpp:9375-9379, 9405-9406), so no CD
is read or booted. Both entries stay required. The AROS replacement
stands in only for a missing Kickstart (memory.cpp:3055-3063), so the
CDTV entry loses hle_fallback.
2026-10-21 06:06:35 +02:00
Abdessamad Derraz d678e6acfa fix: declare the FreeIntv overlays on the core
FreeIntv declared no data tree while its alias freeintv_ts_overlay,
the same code, declared freeintv_overlays. The core reads
freeintv_overlays/<rom>.png only with the multi-screen option and draws
its embedded keypad image without one (src/libretro.c:278-280, 294,
321-325), so the tree is optional. A test holds every alias to the trees
of its core.
2026-10-21 06:06:07 +02:00
Abdessamad Derraz 734643b8fc chore: merge audit fixes (site)
# Conflicts:
#	wiki/testing-guide.md
2026-10-21 05:53:15 +02:00
Abdessamad Derraz ce10323f7c chore: align two tests with the merged lock flow
The lock refusal moved to stderr and pipeline labels lost their /8 in
two branches that met at merge.
2026-10-21 05:53:01 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00
Abdessamad Derraz 53da2a95d2 chore: merge audit fixes (docs)
# Conflicts:
#	tests/test_doc_surfaces.py
#	wiki/installer.md
#	wiki/testing-guide.md
2026-10-21 05:50:25 +02:00
Abdessamad Derraz a67265e903 fix: let the site search land on cards
The search splits a page at its headings. Emulator cards were titled in
bold and platform systems folded into blocks under one heading, so a
pasted hash opened the top of a page or the systems overview of a
100-block platform page. Each file card is now a level 4 heading kept
out of the table of contents, and each platform system a heading.

Card lines that list other pages (platforms, packs, source citations,
collection paths) leave the search index: 24.7 MB down to 18.1 MB, the
names, hashes and notes still searchable.
2026-10-21 05:47:31 +02:00
Abdessamad Derraz 73d73e0f2a chore: merge audit fixes (profsync) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz e03fe338ee chore: merge audit fixes (tests) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz bdcc8f0fb6 fix: lower the pipeline floor to Python 3.9
The 3.12 guard rested on generate_readme f-strings that have since been
rewritten. Every script compiles under CPython 3.9.25, 3.10 and 3.11, and
only generate_pack's parenthesized context managers fail under 3.8. Under
3.9 the offline pipeline (--skip-packs) completed with no change to the
committed artifacts, the 15 manifests matched 3.12 byte for byte, and the
RomM pack was identical. CI now compiles every script under a real 3.9.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz 309231afc2 docs: say what --skip-packs still runs
The help said "Only regenerate DB and verify" while steps 1b, 4d, 7 and
8 still ran and rewrote derivable.json, README.md and docs/. It now names
the steps it skips and points to --skip-docs.
2026-10-21 05:27:07 +02:00
Abdessamad Derraz 94a230a063 chore: frame the triage rows in a refusal test 2026-10-21 05:26:53 +02:00
Abdessamad Derraz 738577b5a4 fix: run verify and generate_pack from any dir
Started outside the repository root, both ended on a FileNotFoundError
traceback: the database and the data registry name files from the root.
They now move to the root after reading the paths the caller typed, which
stay relative to where they were typed.
2026-10-21 05:25:29 +02:00
Abdessamad Derraz 0457bd8f40 fix: resolve the pipeline output dir once
Every step runs from the repository root, but the lock and the purge of
stale packs read --output-dir from the caller's directory. Started from
scripts/, the run locked and purged scripts/dist/ and wrote to dist/,
leaving the stale packs there. The directory is now made absolute before
the lock, dist/ at the root by default.
2026-10-21 05:24:25 +02:00
Abdessamad Derraz 82d5ff77cf docs: sentence-case and split README paragraphs
Quick Install was the one Title Case heading left in the published
text. The Files sentence was glued to the paragraph above it, two
paragraphs before the table whose column it describes. It now names
the column and sits just above the table.
2026-10-21 05:24:25 +02:00
Abdessamad Derraz 4fa598645e fix: print pack and lock errors on stderr
The lock refusal, "no pack for", "no manifest for" and "no pack to
verify" went to stdout, so a run redirected to a log exited 1 with
nothing on the terminal. A test reads both scripts and refuses an ERROR
line printed to stdout.
2026-10-21 05:22:05 +02:00
Abdessamad Derraz a160a5bc6b fix: name the hash formats --from-md5 accepts
A value of another length was looked up as an MD5 and reported NOT FOUND,
a SHA-256 was dropped, and an input with no hash ended on "No valid
hashes found in input". Digests are now read by length (CRC32, MD5 and
its 29-digit Batocera form, SHA-1, SHA-256), SHA-256 is looked up in the
database, and any other value is refused with the formats accepted.
2026-10-21 05:20:59 +02:00
Abdessamad Derraz bc68e517cb fix: suggest the closest system for --platform
--platform retroarch --system sony-playstaton listed the first ten systems
of the alphabet. The warning now names the closest systems and the
command that lists the platform's own, as --system alone already did.
2026-10-21 05:19:27 +02:00
Abdessamad Derraz 443aab68b7 fix: name an untracked bios file as uncommitted
The release-source check called any uncommitted repo_path gitignored,
pointing at the release assets instead of a missing commit, and its
skip outside a checkout could not fire. It now asks git which paths are
ignored, and a new check requires bios/ to hold no untracked file.
2026-10-21 05:11:23 +02:00
Abdessamad Derraz 8008439033 fix: keep manifests and target lists apart
Both are written as <platform>.json. --manifest-targets aimed at install/
replaced install/retroarch.json with a target table, and --manifest aimed
at install/targets/ did the reverse, both with exit 0. Each mode now stops
when the output directory holds the other kind and names the directory to
use.
2026-10-21 05:06:36 +02:00
Abdessamad Derraz 5fc2423ca5 fix: apply --required-only to data directories
The Required packs and manifests carried every data tree, the FreeIntv
overlays included. A data_directories entry now accepts required: false,
and the platform, emulator and manifest walks leave such a tree out under
--required-only. FreeIntv reads its overlays only with the multi-screen
option and draws a placeholder keypad without one (src/libretro.c:294-296,
350-369).
2026-10-21 05:05:49 +02:00
Abdessamad Derraz 82acd55ebc fix: guide installer users with their own command
Names are matched on letters and digits, so the site's display names
work, and a typo gets the closest names instead of a full list. Hints
and advice give back the command the user ran (curl or PowerShell form,
every flag kept), --dest alone follows detection and says whose list it
installs, Ctrl+C at a question exits 130, and --cores-dir names a core
folder. install.ps1 runs in its own scope so irm | iex leaves the
session alone, and an override that causes a fingerprint mismatch is
named. --check lists hash mismatches first, missing required files are
grouped by system, and --help ends with examples.
2026-10-21 04:47:50 +02:00
Abdessamad Derraz 7095aa61e7 fix: make maintainer tool output actionable
--emulator accepts a buildbot core name from a profile's cores:, and a
mistyped target gets the closest names instead of every target. diff_truth
shows destinations and reports unpaired entries as evidence, not removals.
The verify summary counts discrepancies, a shared destination names both
systems, a truth hash conflict names both cores, and pipeline steps carry
their place in the real run instead of N/8. Every GitHub API caller takes
the token from github_token(), which falls back to gh auth token.
2026-10-21 04:42:01 +02:00
Abdessamad Derraz 40bc07aa3a feat: record files a launcher never opens
unread_on names the platforms whose launcher points the emulator at other files. RetroBat writes absolute BIOS paths into standalone BlastEm's config, so the default names were reported as required and missing there. Such an entry leaves that platform's gap report and pack, and its emulator page says where it is never opened.
2026-10-20 10:27:15 +02:00
Abdessamad Derraz ef8b372810 chore: drop copies only an install provides
A file that arrives with the emulator is already on the user's machine
and the packs leave it out. 4558 collected copies (1.4 GB) were reached
only by such entries, with no platform, other profile entry or install
manifest needing them. They are removed, and two release-asset copies
leave the ignore list. bundled_only.py now lists every copy of a listed
file, since the database keeps one path per content, and pipeline step
4d fails while one remains.
2026-10-20 10:17:08 +02:00
Abdessamad Derraz 954f213d42 fix: one id per card, and render the site locally
A copy filed under two consoles of one maker was listed twice with the same id, and a data directory listed the same bytes twice, so the site deploy refused 20 pages. The first card keeps the id. The pipeline only generated the pages, so it now renders them with mkdocs build --strict and runs validate_site as the deploy does. The resumed-speed test read the clock and failed on a slow runner, it now fixes the elapsed time.
2026-10-20 07:05:01 +02:00
Abdessamad Derraz 6df087e3e9 fix: a platform's own data trees leave its pack
The platform layer names data trees for its systems: retroarch.yml carries dolphin-sys and ppsspp-assets, and Lakka inherits them, while Lakka's core packages install both trees into the system directory. A profile records that with installed_on on its data directory, and the pack and the install manifest now leave those trees out for that platform.
2026-10-20 06:36:32 +02:00
Abdessamad Derraz 1ff2bfc3c0 fix: the API schema holds each endpoint's fields
The data page promises that v1 fields are only ever added, and the envelope schema left items unconstrained, so a renamed field passed validation. Each kind now lists the fields its items carry and their types.
2026-10-20 06:18:25 +02:00
Abdessamad Derraz 1bbb1d4749 fix: emulator JSON-LD names every repository read
The structured data of an emulator page published one repository and one commit. A profile whose builds live in two repositories, or a port with an upstream_commit, now lists each repository with its own pin.
2026-10-20 06:17:40 +02:00
Abdessamad Derraz 08396be3c6 fix: the collection DAT names sets by subfolder
The collection DAT named its sets "Sony - PlayStation", and the data page said it audits a copy of bios/. clrmamepro and RomVault looked for a folder of that name and reported the 330 sets missing. A Logiqx set name writes a subfolder with a backslash: Sony\PlayStation.
2026-10-20 06:17:16 +02:00
Abdessamad Derraz f7d57dd91d feat: a platform package can differ from upstream
bundled describes the emulator's own release, and a platform package can differ both ways: Batocera's hcl package installs the binary alone and sends users to fetch the data, while its blueMSX core package copies the machines and databases beside the core. not_bundled_on and installed_on name those platforms, and the builder reads them through one rule.
2026-10-20 06:11:10 +02:00
Abdessamad Derraz a07821d810 feat: list files only an emulator install provides
A file installed with its emulator is already on the user's machine and the packs leave it out. bundled_only.py lists the collected copies nothing else needs: no platform declares them, no entry of any build needs them, no install manifest ships them.
2026-10-20 06:05:20 +02:00