A cached bios_db.json.zip that was not a zip raised BadZipFile out of
run(): --all stopped and the platforms after it were never exported.
fetch() also cached any 200 reply, so an error page replaced a good
archive and was served on every later run, and an IncompleteRead or an
oversized reply escaped the same way. An archive is now read before it
is cached, and an unreadable file fails its own platform with the
--refresh-cache remedy.
System.dat, RomM, Recalbox and RetroBat were rebuilt from our data when
their file was not cached: libretro's quoted names, Recalbox's comments
and one of its two zxspectrum blocks were lost, and the run said OK.
needs_original now follows native_sources, so every format we patch
fails without the file it patches and writes nothing.
dist/ is not committed and exports/ is. On a fresh clone, or with a
mistyped --truth-dir, the export copied each platform's file back
uncorrected, said OK and replaced the corrected export in exports/.
check_freshness then reported a stale export instead of the missing
model. The platform now fails, writes nothing, and names the
generate_truth command.
The page called the original "their file" and listed hashes "corrected" and additions "left out", a verdict on work the page only proposes to extend. It now says what each updated file changes in neutral terms built from the report's numbers, gives the reason a file is not added, credits the effort a list takes to keep current, and invites an issue when a change does not fit a platform. A test keeps the judging words off the page.
exports/ now holds the twelve platforms' own files, corrected, with a
report of every change, read by the site's corrected files page. Two
summaries are reworded for the page: an addition the format cannot
carry is counted as left out, and RetroPie names each skipped package
by sentence instead of joining reasons with semicolons. Its parser no
longer reads "e.g" as a file name.
export_native now writes into exports/ by default, and beside each
platform's corrected file a report naming every change, where the run
printed five per kind. A new site page lists each platform with its
original file, a download of the corrected one and every corrected,
filled or withheld value. Each platform page links to its row.
When the truth takes over an entry without a size of its own, the format writes it without the platform's size: System.dat's neocdz.zip lost size 953078 and no counter said so. Removed sizes are now counted and named, and the summary counts read 1 size, 2 sizes.
Six platforms are read from a branch: System.dat, batocera-systems.json, checkBIOS.sh, the MiSTer BiosDB, the RetroPie scriptmodules and the RetroDECK component manifests. Their YAML named the branch, so the native export patched whatever the branch had become since the scrape. Each scraper now resolves the branch head once, reads every file at that commit and writes it into source. The export follows a recorded tarball and a recorded tree as it already followed a raw file URL.
Profiles, the wiki, comments, docstrings, command output and schema descriptions carried em-dashes, clause-joining semicolons, unicode arrows, box-drawing banners and check-mark glyphs. Each sentence was rewritten rather than having its character swapped. Hashes, paths, source_ref values and every cited file:line are unchanged, which a comparison of each parsed profile against the previous revision confirms.
validate_pr comments now say PASS and FAIL in words, and source_ref values in a profile's structured blocks render as code. test_prose_style fails if any of these comes back, in tracked files or in the generated site.
RomM kept hashes the truth contradicts beside the corrected one, RetroDECK kept a cleared md5, Recalbox counted changes on an element it writes back whole, and two record formats counted fills no record could hold.
slots ended a mistyped --platform on a traceback; generate_truth,
refresh_data_dirs and auto_fetch exited 0 having done nothing for it.
All six now stop on require_platform before any work.
argparse completed a prefix into the only flag starting with it:
slots.py --emulator read profiles from a directory named after the
emulator and reported nothing, and pipeline.py --all built every pack
variant. Every parser now refuses an abbreviation.
An export wrote into its platform directory without removing what an
earlier run left there, so RetroPie scriptmodules and RetroDECK
manifests the current truth no longer corrects stayed beside a
report saying they were untouched. Files the run does not produce
are now removed first.
The export cache wrote an original's bytes, then recorded the URL
that produced them under a separate lock, so two runs wanting two
revisions of one file could pair one's bytes with the other's URL.
Both now change under the one index lock.
Batocera, ROCKNIX and Recalbox find a file through the core that wants
it. Additions now carry that core under the platform's own name: the
emulator/core pair checkBios reads (ROCKNIX's from its emulators
package), or Recalbox's core attribute. One with no nameable core, or
filed in a system gated on another core, is counted as not stated.
Loading the emulator profiles is the most expensive step of every
command here, and all forty call sites used the pure-Python scanner
while libyaml sat unused in the same wheel. One shared yaml_load picks
the C loader when pyyaml ships it: the 375 profiles parse in 0.18s
instead of 1.39s, and verify --platform retroarch drops from 2.17s to
0.73s. The loader class is the same restricted one safe_load uses.
es_bios.xml was parsed straight from the network while install.py
already refused a document declaring entities; both now share one
guard. Scrapers reach it through a single path bootstrap in the
package rather than two ad-hoc ones.
Run ruff check --fix: remove unused imports (F401), fix f-strings
without placeholders (F541), remove unused variables (F841), fix
duplicate dict key (F601).
Run isort --profile black: normalize import ordering across all files.
Run ruff format: apply consistent formatting (black-compatible) to
all 58 Python files.
3 intentional E402 remain (imports after require_yaml() must execute
after yaml is available).
Abdessamad Derraz