Commit Graph
2467 Commits
Author SHA1 Message Date
Abdessamad Derraz a1a9da59af chore: merge audit fixes (export)
# Conflicts:
#	exports/recalbox.json
#	exports/retropie.json
2026-10-21 05:53:05 +02:00
Abdessamad Derraz ce10323f7c chore: align two tests with the merged lock flow
The lock refusal moved to stderr and pipeline labels lost their /8 in
two branches that met at merge.
2026-10-21 05:53:01 +02:00
Abdessamad Derraz 9f1dbaa6c2 chore: merge audit fixes (packcli)
# Conflicts:
#	README.md
#	scripts/generate_pack.py
#	scripts/pipeline.py
2026-10-21 05:51:12 +02:00
Abdessamad Derraz 53da2a95d2 chore: merge audit fixes (docs)
# Conflicts:
#	tests/test_doc_surfaces.py
#	wiki/installer.md
#	wiki/testing-guide.md
2026-10-21 05:50:25 +02:00
Abdessamad Derraz 43c6df58ee chore: merge audit fixes (region)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 05:48:59 +02:00
Abdessamad Derraz f4a225488d chore: merge audit fixes (scrapers) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz eb44e3f898 chore: merge audit fixes (profiles) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 73d73e0f2a chore: merge audit fixes (profsync) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz e03fe338ee chore: merge audit fixes (tests) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz 5b7f8526a8 refactor: read the gap report through its helper
test_launcher_layer looked for find_undeclared_files in the extras
collector, which now reaches it through undeclared_report.
2026-10-21 05:43:14 +02:00
Abdessamad Derraz 28435230a7 fix: read each named core's own entry in recalbox
An element's mandatory flag came from the one truth entry its md5 paired
with. saturn/saturn_bios.bin names yabause, yabasanshiro and kronos,
which require it, and was paired with the beetle_saturn entry, which
does not. Each named core's entries for the file, found by name, now
decide, except a namesake elsewhere in the same tree (bluemsx KANJI.rom
under Shared Roms is not the one under MSX2+ - European).
2026-10-21 05:39:17 +02:00
Abdessamad Derraz 5858ebf53e docs: restate the recipe store counts
The reimport kept the sets holding missing catalog dumps: 24,101 entries
in 2,148 distinct recipes.
2026-10-21 05:38:58 +02:00
Abdessamad Derraz e83f3e9d8a chore: regenerate manifests and native exports 2026-10-21 05:37:14 +02:00
Abdessamad Derraz bdcc8f0fb6 fix: lower the pipeline floor to Python 3.9
The 3.12 guard rested on generate_readme f-strings that have since been
rewritten. Every script compiles under CPython 3.9.25, 3.10 and 3.11, and
only generate_pack's parenthesized context managers fail under 3.8. Under
3.9 the offline pipeline (--skip-packs) completed with no change to the
committed artifacts, the 15 manifests matched 3.12 byte for byte, and the
RomM pack was identical. CI now compiles every script under a real 3.9.
2026-10-21 05:32:50 +02:00
Abdessamad Derraz d2141ac0bb fix: give the MAME clone test a parsed set
The fixture parsed no BIOS root set, which the scraper now refuses.
2026-10-21 05:28:38 +02:00
Abdessamad Derraz 8f86842117 feat: keep romsets holding a missing catalog dump
The recipe snapshot kept only the archives a profile or platform names,
so provenance_report --missing named no set for dumps a public romset
carries (fm7740sx holds the FM77AV40SX DICROM, EXTSUB and KANJI2 ROMs).
The importer now also keeps the sets holding a catalog entry the
collection lacks. Reimported from mame0289 and the cached FBNeo DATs:
MAME 1726 to 2148 recipes, FBNeo 104 to 111, none dropped.
2026-10-21 05:27:42 +02:00
Abdessamad Derraz eb385a4039 docs: note the profile verdict check in the guide 2026-10-21 05:27:09 +02:00
Abdessamad Derraz 309231afc2 docs: say what --skip-packs still runs
The help said "Only regenerate DB and verify" while steps 1b, 4d, 7 and
8 still ran and rewrote derivable.json, README.md and docs/. It now names
the steps it skips and points to --skip-docs.
2026-10-21 05:27:07 +02:00
Abdessamad Derraz 94a230a063 chore: frame the triage rows in a refusal test 2026-10-21 05:26:53 +02:00
Abdessamad Derraz 738577b5a4 fix: run verify and generate_pack from any dir
Started outside the repository root, both ended on a FileNotFoundError
traceback: the database and the data registry name files from the root.
They now move to the root after reading the paths the caller typed, which
stay relative to where they were typed.
2026-10-21 05:25:29 +02:00
Abdessamad Derraz 61f0d033b5 chore: split a test docstring sentence 2026-10-21 05:25:10 +02:00
Abdessamad Derraz 76618af460 docs: shorten two script docstrings
_enrich_hashes repeated "ground truth" five times with capitalised
insistence for a three-step lookup, and the MAME ZIP rebuild listed
three guarantees of which two were the same.
2026-10-21 05:24:34 +02:00
Abdessamad Derraz 0457bd8f40 fix: resolve the pipeline output dir once
Every step runs from the repository root, but the lock and the purge of
stale packs read --output-dir from the caller's directory. Started from
scripts/, the run locked and purged scripts/dist/ and wrote to dist/,
leaving the stale packs there. The directory is now made absolute before
the lock, dist/ at the root by default.
2026-10-21 05:24:25 +02:00
Abdessamad Derraz 82d5ff77cf docs: sentence-case and split README paragraphs
Quick Install was the one Title Case heading left in the published
text. The Files sentence was glued to the paragraph above it, two
paragraphs before the table whose column it describes. It now names
the column and sits just above the table.
2026-10-21 05:24:25 +02:00
Abdessamad Derraz b464ed9202 fix: count prose that holds the pin as review work
The triage, --changed-only and --strict counted refs only. Prose that
--realign-prose would move or refuse blocks every recale and bump, and
blastem read 0 to review. Each report now carries that reading. The
triage also gets a header, a closing count, and one sampling notice per
run instead of one per profile.
2026-10-21 05:24:17 +02:00
Abdessamad Derraz 905316061e fix: read retropie and emudeck targets at the pin
The RetroPie target scraper read master and the EmuDeck one the default
branch, while both platform files are pinned on a commit: a module flag
changed in between filtered packages the pinned list does not describe.
Both now read the revision pinned_revision returns, which also accepts the
codeload URL the RetroPie scraper records. EmuDeck Windows stays on the
emudeck-we default branch: the platform file pins no revision of it.
2026-10-21 05:24:04 +02:00
Abdessamad Derraz 06ae666f92 fix: describe WinUAE kickstarts by its own code
Every entry took its names from RetroBat's Uae.Generator.cs, which
the RetroBat launchers: records already hold. Kickstarts are now
labels of the ROM scan, and kick.rom, the name the loader opens under
its data path, is listed.
2026-10-21 05:23:14 +02:00
Abdessamad Derraz 5e70fe7ff5 docs: state each verification rule once
architecture.md drew path_suffix ahead of SHA1 and listed a core
resolution that missed the retroarch list entry and built_by. Both now
link to the one statement that matches common.py. The Recalbox table
read hashMatchMandatory into verify.py, which never reads it, and
md5_composite hashes member contents, not member MD5s
(Zip.cpp:71-96). The validation index is built from the platform's
resolved profiles only. The requirement table gains the fields that
make a platform YAML reversible, and pages that name a platform list
say what changes rather than judging it.
2026-10-21 05:22:38 +02:00
Abdessamad Derraz 9edf785513 docs: refresh installer and release pages
The installer output example showed 5865 files and no Not installable
line, while the RetroArch manifest gives 8069 and omits 11. Exit code 2
also comes from every option argparse refuses. RETROBIOS_REF holds a
file list only until the next release deletes its tag, and large files
always come from large-files. The release commit now carries the
manifests the pipeline regenerates. A RetroPie 4.8 image ships Python
3.7.3, below the bootstrap's floor, and the pages say what to use.
2026-10-21 05:22:28 +02:00
Abdessamad Derraz 32bda05be3 docs: align the testing and profiling guides
The test skeleton indexed verify_platform's dict result as a list and
stopped on KeyError: 0. The verification discipline and the profiling
guide hedged rules the project holds as firm. The profile field table
marked as required fields the schema does not require, and now names
the pin that profile_sync --strict needs.
2026-10-21 05:22:20 +02:00
Abdessamad Derraz 4fa598645e fix: print pack and lock errors on stderr
The lock refusal, "no pack for", "no manifest for" and "no pack to
verify" went to stdout, so a run redirected to a log exited 1 with
nothing on the terminal. A test reads both scripts and refuses an ERROR
line printed to stdout.
2026-10-21 05:22:05 +02:00
Abdessamad Derraz 0061041466 fix: claim the pack output before refreshing data
generate_pack refreshed the data directories before probing its output
lock. The refresh waits for every run holding a tree, and a pipeline holds
them all: a second build sat silent for hours instead of being refused.
The output lock is now taken first, and a refresh that has to wait for a
tree names it.
2026-10-21 05:21:54 +02:00
Abdessamad Derraz a160a5bc6b fix: name the hash formats --from-md5 accepts
A value of another length was looked up as an MD5 and reported NOT FOUND,
a SHA-256 was dropped, and an input with no hash ended on "No valid
hashes found in input". Digests are now read by length (CRC32, MD5 and
its 29-digit Batocera form, SHA-1, SHA-256), SHA-256 is looked up in the
database, and any other value is refused with the formats accepted.
2026-10-21 05:20:59 +02:00
Abdessamad Derraz 7bcd41e0c4 fix: let taiwan and asia-ntsc answer each other
kronos tags the ST-V BIOS epr-19854.ic8 taiwan, mednafen_stv tags it
asia-ntsc, the SMPC area Mednafen names for Taiwan (ss.cpp:1119,
2363-2364). As siblings under asia they never matched, so a request in
one spelling dropped the file tagged in the other, and --region
taiwan,us dropped the Saturn sega_101.bin. AREA_TERRITORIES relates
each area to the territory it serves, china for asia-pal included, in
region.py and its installer copy.
2026-10-21 05:20:08 +02:00
Abdessamad Derraz e318bb4503 fix: read pcsx_rearmed us slot as the default
CheckCdrom sets PSX_REGION_US before it reads the licence, and only
"Inc." or "Euro pe" move it (libpcsxcore/misc.c:422-453). A Latin
American disc boots the US slot, so its four BIOS images carry
latin-america and --region br,jp keeps them.
2026-10-21 05:20:03 +02:00
Abdessamad Derraz 4e8d375074 fix: serve australian mega cd discs their pal bios
BlastEm and Kega Fusion pick the European Mega CD BIOS for every disc
whose region code is the PAL one, Australian discs included: BlastEm
reads E or bit 8 of the header region (romdb.c:176-215), Fusion the
security byte 0x64 at 0x20B. Both now carry oceania, and a repo-wide
test holds every European Mega CD BIOS to it.

Fusion picks its export Master System BIOS from the CountryOrder
setting or a CRC32 list, never from the cart, so bios_U.sms and
bios_E.sms carry no region.
2026-10-21 05:20:00 +02:00
Abdessamad Derraz 7e6b80d501 fix: keep amiberry to the names its code reads
kick02019.AROS and its .ext were Recalbox configgen names, shipped at
the BIOS root of four packs whose code never opens them. Recalbox
keeps them through its own amiga/bios declarations. amiga-os-310.rom
is a ROM scan label, now found_by content.
2026-10-21 05:19:56 +02:00
Abdessamad Derraz bc68e517cb fix: suggest the closest system for --platform
--platform retroarch --system sony-playstaton listed the first ten systems
of the alphabet. The warning now names the closest systems and the
command that lists the platform's own, as --system alone already did.
2026-10-21 05:19:27 +02:00
Abdessamad Derraz 959934d43c refactor: compute the gap report once per build
The region grouping and phase 2 of each pack and manifest called
find_undeclared_files with the same arguments: 30 reports for the 15
manifests of --all --manifest, 2.8 s, now 15 and 1.7 s. The 15 manifests
are byte-identical before and after.
2026-10-21 05:18:53 +02:00
Abdessamad Derraz 805ca3bb6b fix: pass over a profile revision that won't parse
One unparsable commit in a profile's history stopped the walk and
marked every older prose citation unknown, which refused the recale and
bump for good (lindbergh-loader, df7346b3). The revision is now passed
over unless the pins differ on its two sides while the field's
citations changed across it.
2026-10-21 05:18:49 +02:00
Abdessamad Derraz 247093d7d2 fix: name BizHawk firmware by its own database
BizHawk finds firmware by SHA-1 scan or a user specification, so the
profile is found_by content. 38 entries used the names RetroBat's
generator writes; they now carry FirmwareDatabase.cs recommended
names, and the pooled NDS firmware entry is split per option.
2026-10-21 05:17:34 +02:00
Abdessamad Derraz bca76e8ace fix: judge a cached large file from one read
_fetch_asset hashed the cache copy, then hashed the path again for the
release check. A copy another run promoted in between answered the second
question, and the caller was told the cache held nothing when it held
the file asked for. Both checks now read one snapshot, taken while the
inode stays the same.
2026-10-21 05:17:16 +02:00
Abdessamad Derraz b275f278ca fix: type the model notes filter 2026-10-21 05:17:16 +02:00
Abdessamad Derraz 79d3a86db7 chore: drop an unused import in export tests 2026-10-21 05:17:10 +02:00
Abdessamad Derraz 31566ce618 fix: read a past pin only for its own repository
A source_commit string from before the field was keyed by mode pinned
the standalone repository, yet was read as a writing pin for every mode.
xenia-edge then saw its libretro pin move under unchanged prose, and
the recale and bump were refused.
2026-10-21 05:17:01 +02:00
Abdessamad Derraz 4d514336c9 fix: close a data dir refresh on its counts
refresh_data_dirs printed one line per entry and no total, a dry run named
the cached tag only, and cutting a quoted ETag at 12 characters kept one
quote. The run now ends on refreshed / up to date / failed counts, a dry
run shows the remote tag beside the cached one, and tags lose their quotes
before they are cut.
2026-10-21 05:14:55 +02:00
Abdessamad Derraz 64b2967ffb fix: read a source_ref role per part, prose too
The role a part writes (source or upstream) was kept per path, first
met first kept, so a later part naming the other role was read in the
other repository. In prose the word source made a citation external
and the word upstream was dropped. The role now travels on each part.
2026-10-21 05:14:47 +02:00
Abdessamad Derraz d9548e3522 fix: count a recalbox element kept as written
An element whose md5 the truth contradicts without naming another is
written back whole. It was counted as an entry the format cannot keep.
It now counts as kept, and the md5 it keeps is named as withheld.
2026-10-21 05:14:44 +02:00
Abdessamad Derraz 842d313a88 fix: keep truth model notes out of the native rows
check_freshness --only native rebuilds the truth model to compare it, and
the hash conflict notes generate_truth reports opened the freshness output.
They are dropped there, and any other stderr line still passes through.
2026-10-21 05:13:59 +02:00
Abdessamad Derraz 4a6b654bdc fix: keep a romm entry whole without a size
RomM compares file_size_bytes with the entry's size before any hash. A
new hash the truth gives no size for went out beside the old size and
was reported as a size removal. The entry now stays as written and the
correction is withheld and named.
2026-10-21 05:13:46 +02:00