Ranking contested core claims sorted the whole gap report, which
reordered every manifest. Only the claims on one contested path move now,
grouped where the first of them stood, and two spellings of a path are
one group where the filesystem folds case (RetroBat's 64DD_IPL_DEV.bin).
An entry without a destination (EmuDeck's system:md5 whitelist) no
longer counts as a pack path in verify, and step 5 prints one status for
counts and paths together. A full offline run compared all 12 archives
and 930 emulator packs path by path, every one equal.
The 3.12 guard rested on generate_readme f-strings that have since been
rewritten. Every script compiles under CPython 3.9.25, 3.10 and 3.11, and
only generate_pack's parenthesized context managers fail under 3.8. Under
3.9 the offline pipeline (--skip-packs) completed with no change to the
committed artifacts, the 15 manifests matched 3.12 byte for byte, and the
RomM pack was identical. CI now compiles every script under a real 3.9.
The help said "Only regenerate DB and verify" while steps 1b, 4d, 7 and
8 still ran and rewrote derivable.json, README.md and docs/. It now names
the steps it skips and points to --skip-docs.
Every step runs from the repository root, but the lock and the purge of
stale packs read --output-dir from the caller's directory. Started from
scripts/, the run locked and purged scripts/dist/ and wrote to dist/,
leaving the stale packs there. The directory is now made absolute before
the lock, dist/ at the root by default.
The lock refusal, "no pack for", "no manifest for" and "no pack to
verify" went to stdout, so a run redirected to a log exited 1 with
nothing on the terminal. A test reads both scripts and refuses an ERROR
line printed to stdout.
Step 5 compared (ok, total) and the core file count. RetroArch, Lakka
and RetroPie read 526/526 on both sides while verify credited four paths
below SGB1.sfc and SGB2.sfc that the pack never writes: a declared file
blocks them as a directory. verify now places the platform files in the
builder's order with the same path rules, and a user_provided or
external declaration keeps its slot on both sides, so no core file takes
it in the report only.
verify --destinations writes the paths each pack holds by the report,
and step 5 compares them with the archive members (notes and data tree
members set aside). Every emulator profile, in each build it has, is
built in memory (PackRecord, nothing read or written) and compared with
verify --emulator path by path. Cost measured here: about 1 s per
platform archive and 11 s for the 930 emulator packs.
--emulator accepts a buildbot core name from a profile's cores:, and a
mistyped target gets the closest names instead of every target. diff_truth
shows destinations and reports unpaired entries as evidence, not removals.
The verify summary counts discrepancies, a shared destination names both
systems, a truth hash conflict names both cores, and pipeline steps carry
their place in the real run instead of N/8. Every GitHub API caller takes
the token from github_token(), which falls back to gh auth token.
A file that arrives with the emulator is already on the user's machine
and the packs leave it out. 4558 collected copies (1.4 GB) were reached
only by such entries, with no platform, other profile entry or install
manifest needing them. They are removed, and two release-asset copies
leave the ignore list. bundled_only.py now lists every copy of a listed
file, since the database keeps one path per content, and pipeline step
4d fails while one remains.
A copy filed under two consoles of one maker was listed twice with the same id, and a data directory listed the same bytes twice, so the site deploy refused 20 pages. The first card keeps the id. The pipeline only generated the pages, so it now renders them with mkdocs build --strict and runs validate_site as the deploy does. The resumed-speed test read the clock and failed on a slow runner, it now fixes the elapsed time.
PR validation runs validate_schemas.py --source-only and the pipeline did not, so a profile with an unquoted numeric core name passed every step and failed CI on push. It is step 0 now, skipped where jsonschema is not installed.
check_buildbot does not fail the run, yet the summary printed it FAILED right above Pipeline COMPLETE. It now prints WARN and a line saying the difference does not fail the run. The total is aligned under the step durations.
The consistency check compared the platform's own files only. It now also compares verify's core files and game data with the builder's count of files taken from the cores.
Profiles, the wiki, comments, docstrings, command output and schema descriptions carried em-dashes, clause-joining semicolons, unicode arrows, box-drawing banners and check-mark glyphs. Each sentence was rewritten rather than having its character swapped. Hashes, paths, source_ref values and every cited file:line are unchanged, which a comparison of each parsed profile against the previous revision confirms.
validate_pr comments now say PASS and FAIL in words, and source_ref values in a profile's structured blocks render as code. test_prose_style fails if any of these comes back, in tracked files or in the generated site.
Lakka builds an image named switch while RetroArch files switch as
nintendo-switch. One canonical name taken across the registry put
Lakka's model under RetroArch's target. generate_truth now writes each
platform under its own target name, and diff_truth --target reads it.
The Wanted page listed 79 dumps the collection already produces,
such as a Kickstart's byte-swapped words or the halves of a C128
character ROM. The pipeline now records which missing dumps can be
derived, in derivable.json, and the page moves them to a section of
their own that names the held file and the transform.
argparse completed a prefix into the only flag starting with it:
slots.py --emulator read profiles from a directory named after the
emulator and reported nothing, and pipeline.py --all built every pack
variant. Every parser now refuses an abbreviation.
The pipeline held its output directory for the run but let each step
lock the data directory trees on its own, so another session's
refresh could swap a tree between the packs and the manifests. The
trees are now held shared from right after the pipeline's own
refresh until it exits.
generate_truth named a narrowed model's directory after the target
as typed, so three aliases of one target wrote three copies, and
the pipeline rebuilt the slug by hand to diff it. One function now
names the directory from the canonical target for both.
Step 2 refreshed the data directories, then the pack build and the
manifest build refreshed them again, so a tree tracking a moving
branch could differ between the pack and the manifest. generate_pack
gains --no-refresh-data, which packs the cached trees and still
fetches release assets; the online pipeline passes it to both builds.
Abdessamad Derraz