Commit Graph
56 Commits
Author SHA1 Message Date
Abdessamad Derraz 01ca625378 feat: rescrape retrobat at its pinned commit
The scraper read emulatorLauncher master on every run, so an extractor
change was diffed together with upstream's commits since the pin. It now
rereads the commit platforms/retrobat.yml pins. --revision master moves
it, and check_freshness and refresh_stale pass that through the
registry's rescrape_args.

The extractors also read master (f681f435): Kronos and YabaSanshiro
fall back to sega_101.bin then mpr-17933.bin when the chosen BIOS is
absent, Ymir writes a forced BIOS only under its override flag, and a
folder created below bios (Ymir's cdb) is a passes_path record. At the
pin the 187 records are unchanged.
2026-11-11 11:40:05 +01:00
Abdessamad Derraz 6191d4783a fix: point each Amiberry build pin at its repo
KNULLI builds amiberry from Batocera's package through its batocera
submodule: the reference now names knulli-linux at 20260511, the tag
platforms/knulli.yml is read at, and the package file below the
submodule. check_freshness follows a path .gitmodules declares to the
commit the tree pins, which reads AMIBERRY_VERSION = v7.1.0 there.

RetroBat pins amiberry 8.3.0.0 only in emulators/versions.xml on its
download server, read by emulatorLauncher/Installer.cs. No repository
holds it, so amiberry-8.3.0 keeps pin_frozen, which says so, and no
pin_build.
2026-11-10 07:10:43 +01:00
Abdessamad Derraz 7080459011 fix: read No-Intro from a mirror that rebuilds 2026-11-06 14:59:40 +01:00
Abdessamad Derraz 395910ed2b fix: one-line errors for --record-coreinfo 2026-11-06 14:45:26 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz 25b11fa497 chore: merge site fixes, fourth pass 2026-11-06 10:33:56 +01:00
Abdessamad Derraz 1396350795 fix: count build file trees as pinned
check_source_paths read the trees profiles pin from source and upstream
only, so the two platform trees the record now holds for pin_build
files read as orphaned and the record as stale.
2026-11-06 10:14:02 +01:00
Abdessamad Derraz ada7738fa7 feat: pin_installer writes the installer pins
Every change to install.py meant a sha256sum pasted by hand into
install.sh and install.ps1, and the tests that caught a stale pin
failed on two bare hashes. scripts/pin_installer.py writes both pins
(--check only reports), the pin tests name it, check_freshness reads
the pins through it and refresh_stale runs it for a stale pin row.
2026-11-06 09:02:18 +01:00
Abdessamad Derraz 320db75490 fix: check_freshness aligns details after remotes
render padded the local value but not the remote one, so in an area
where some rows carry a remote value their detail started fifteen
columns later than the rows without one. The remote value is padded to
its column, and a row without one leaves the same blank span.
2026-11-06 08:56:32 +01:00
Abdessamad Derraz 25b9c85208 fix: refuse an unreadable native sources index
_load_sources read a truncated .sources.json as empty: fetch served
the cached original of an older pin as current, the next download
rewrote the index with one entry, and check_freshness said no URL was
recorded. It now raises SourcesIndexUnreadable naming the remedy, and
check_freshness reports one ERROR row for the index.
2026-11-06 08:15:15 +01:00
Abdessamad Derraz 19898cb6ea chore: merge site and docs fixes, third pass 2026-11-06 06:38:47 +01:00
Abdessamad Derraz 27991c84ba fix: refuse a missing platforms directory 2026-11-06 03:25:20 +01:00
Abdessamad Derraz ab1866fd66 feat: compare the site npm pins with the registry
The ci area of check_freshness read the PyPI and action pins only. The
exact versions website/package.json pins (astro, starlight) now get a row
each, STALE once npm publishes a newer release.
2026-11-06 03:17:31 +01:00
Abdessamad Derraz 3011ee498e feat: record the build file behind a held pin
pin_frozen was prose, and no tool read the build file it cites, so a platform building another revision left the profile holding one nobody ships. pin_build records the platform, its repository, the revision read, the file, the variable and its value. check_freshness reads the file again where the platform's YAML now stands and reports STALE when the value moved. 36 of the 46 held profiles carry it, each value read from the build file at the platform's scraped revision. The other ten name no build file and are counted on one row: six RetroBat binary packages and four holds not tied to a platform build.
2026-11-05 19:54:48 +01:00
Abdessamad Derraz 9ae3919d34 fix: spell cited commands so a paste runs them
Messages in check_freshness, generate_pack, common, profile_sync and fileless_audit named commands as export_native.py --platform x, which fails from the repository root. They now read python scripts/<name>.py like the rest of the report.
2026-11-05 19:19:04 +01:00
Abdessamad Derraz b90f8bb0b9 chore: merge tooling batch 2026-11-05 15:57:19 +01:00
Abdessamad Derraz 2c81e1a1af feat: watch the rom folder pins 2026-11-05 12:09:10 +01:00
Abdessamad Derraz c34ed4d2a1 feat: watch the game catalogs for new DATs 2026-11-05 12:06:04 +01:00
Abdessamad Derraz 8dcf531625 fix: report pinned trees the source record lacks 2026-11-05 11:57:01 +01:00
Abdessamad Derraz 92ee0cf118 fix: align freshness details in one column 2026-11-05 11:49:45 +01:00
Abdessamad Derraz 651b1404db fix: skip the core-info row when offline 2026-11-05 11:49:31 +01:00
Abdessamad Derraz 571e50f2a8 refactor: read measurements from the local tree
The site compiles the index and the SQLite files from measurements/
instead of a pinned clone. The pin, the sync script, the CI clone steps,
the freshness area and its refresh handler are gone. core_probe --record
writes into measurements/ of the checkout by default, and a test fails
when a line or data file HEAD holds changed or vanished.
2026-10-27 09:11:12 +01:00
Abdessamad Derraz effa9e332f feat: read measurements at a pinned commit
measurements-pin.json names the measurements repository and the commit read. sync_measurements.py reads the clone in .cache/measurements at that commit, checks every line against its id and sidecar, and writes the site index and the SQLite databases. --fetch refuses a remote that changed or removed a line the pin holds, --bump moves the pin. validate_schemas checks every line and sidecar against their schemas, check_freshness flags a pin behind the remote head.
2026-10-27 08:30:11 +01:00
Abdessamad Derraz de55360b56 feat: flag a stale source-paths record
check_freshness reports source-paths.json STALE when it records a tree no profile pins any more, and tools.md names sitesources.py --fetch where pins move.
2026-10-23 20:04:12 +02:00
Abdessamad Derraz 055627f39c chore: merge round 2 fixes (tooling) 2026-10-21 10:31:04 +02:00
Abdessamad Derraz f3586001ab fix: fail freshness on unknown, name the refresher
check_freshness exited 0 when no upstream answered. An unknown row now
holds the exit code, and a profile skipped by nature is reported skipped
so it does not hold it forever. --only names the valid areas, a stale
native row names its platform, and the summary names refresh_stale.py.
2026-10-21 10:00:36 +02:00
Abdessamad Derraz 6877512af7 feat: carry new upstream names and hashes in json
--detect-new-files and --watch-hashes were printed only by the text report and refused with --json, so check_freshness --profiles never saw a name or hash the cited code gained. Both now live on the report, and check_freshness passes them and marks such a profile stale.
2026-10-21 09:51:14 +02:00
Abdessamad Derraz d6fc66274f fix: let check_freshness read the profile report
check_freshness ran profile_sync with --json and the detection flags,
which profile_sync refuses, so --profiles always reported an unreadable
report. The version check now lives on the report and travels in the
JSON and markdown outputs; check_freshness asks for --json
--check-version only.
2026-10-21 06:09:15 +02:00
Abdessamad Derraz f4a225488d chore: merge audit fixes (scrapers) 2026-10-21 05:44:32 +02:00
Abdessamad Derraz b275f278ca fix: type the model notes filter 2026-10-21 05:17:16 +02:00
Abdessamad Derraz 842d313a88 fix: keep truth model notes out of the native rows
check_freshness --only native rebuilds the truth model to compare it, and
the hash conflict notes generate_truth reports opened the freshness output.
They are dropped there, and any other stderr line still passes through.
2026-10-21 05:13:59 +02:00
Abdessamad Derraz 9c2f49720d feat: flag a changed .info of a profiled core
The coreinfo area compared .info names only: a firmwareN_path added to a
profiled core's .info came out OK. check_freshness now compares each
profiled .info's git blob with platforms/_core_info.json, recorded on
2026-10-21 from the current listing, and --record-coreinfo CORE records
one as reviewed.
2026-10-21 05:12:34 +02:00
Abdessamad Derraz 9ad9b7ab3a fix: diff every field a platform rescrape writes
diff_platform compared only version, files and core lists: a rescrape
that changed launchers, dual_mode_cores or a system's core, native_id or
data_directories came out identical. Every other field is now compared,
without the revision read (source) and the source_ref lines that move with it.
2026-10-21 05:10:05 +02:00
Abdessamad Derraz 7095aa61e7 fix: make maintainer tool output actionable
--emulator accepts a buildbot core name from a profile's cores:, and a
mistyped target gets the closest names instead of every target. diff_truth
shows destinations and reports unpaired entries as evidence, not removals.
The verify summary counts discrepancies, a shared destination names both
systems, a truth hash conflict names both cores, and pipeline steps carry
their place in the real run instead of N/8. Every GitHub API caller takes
the token from github_token(), which falls back to gh auth token.
2026-10-21 04:42:01 +02:00
Abdessamad Derraz f8e0b33314 feat: report a published export that fell behind
exports/ is committed but CI cannot rebuild it, since the originals it
patches live in the local cache. The native freshness check now runs
the export offline and compares it with the committed files, and
refresh_stale writes the export again after refreshing an original.
2026-10-20 09:28:14 +02:00
Abdessamad Derraz 77389236e6 feat: record a release with no genuine copy
Paid Android emulators announce versions that circulate only rebuilt or re-signed. The profile stays on the newest unmodified build, and version_source.no_genuine_copy names the release searched for, so check_freshness stops flagging it until the page announces a later one.
2026-10-20 04:49:24 +02:00
Abdessamad Derraz eaa40d7384 fix: offline target lists are skipped, not ok
Offline, check_freshness labelled every scraped target list static and counted it OK although nothing was compared. Other sections say SKIPPED offline, and now this one does too.
2026-10-20 04:43:45 +02:00
Abdessamad Derraz 282f8d5d0a feat: watch the release page of off-forge profiles
About forty profiles describe projects published outside any forge (Altirra, ePSXe, DraStic, mednafen, xroar). profile_sync skipped them as an unsupported host and check_freshness reported the same UNKNOWN line every run, so a new release adding a firmware went unseen. A profile can now name the page announcing its releases and a pattern for the version: --check-version reads it, and check_freshness --profiles calls the profile stale when the page announces another version than core_version.
2026-10-20 04:28:27 +02:00
Abdessamad Derraz 4231902a6c docs: write prose without dashes or semicolons
Profiles, the wiki, comments, docstrings, command output and schema descriptions carried em-dashes, clause-joining semicolons, unicode arrows, box-drawing banners and check-mark glyphs. Each sentence was rewritten rather than having its character swapped. Hashes, paths, source_ref values and every cited file:line are unchanged, which a comparison of each parsed profile against the previous revision confirms.

validate_pr comments now say PASS and FAIL in words, and source_ref values in a profile's structured blocks render as code. test_prose_style fails if any of these comes back, in tracked files or in the generated site.
2026-10-19 06:47:31 +02:00
Abdessamad Derraz 04deb95a35 docs: give every command-line option a help text 2026-10-19 04:23:34 +02:00
Abdessamad Derraz 3c5570f8f0 fix: regenerate the truth before a native refresh 2026-10-18 14:34:42 +02:00
Abdessamad Derraz fd6db086d3 fix: keep full lists in check_freshness JSON 2026-10-18 08:40:44 +02:00
Abdessamad Derraz e1b6aad51b fix: refuse abbreviated flags in every script
argparse completed a prefix into the only flag starting with it:
slots.py --emulator read profiles from a directory named after the
emulator and reported nothing, and pipeline.py --all built every pack
variant. Every parser now refuses an abbreviation.
2026-10-18 05:30:58 +02:00
Abdessamad Derraz 55631a2fbc fix: end listings quietly when the reader leaves 2026-10-17 20:50:22 +02:00
Abdessamad Derraz e93fe09a88 fix: read the buildbot system index for freshness 2026-10-17 20:44:47 +02:00
Abdessamad Derraz 8375e11680 fix: check the names a single-target platform runs 2026-10-17 20:43:51 +02:00
Abdessamad Derraz 08f296138f fix: read target overrides as the builder does 2026-10-17 19:38:47 +02:00
Abdessamad Derraz b023a9fd53 feat: watch the emulatorjs release romm installs 2026-10-17 15:16:39 +02:00
Abdessamad Derraz 426d3c619d fix: let a platform's only target filter nothing 2026-10-17 14:48:35 +02:00
Abdessamad Derraz 43647eb781 fix: keep freshness values whole 2026-10-17 12:44:45 +02:00