Commit Graph
177 Commits
Author SHA1 Message Date
Abdessamad Derraz ae7feba539 fix: type the lists and returns mypy flagged
The data tree record carries the needed flag since 072380c26, and
the manifest test still built the three-part one. profile_sync's triage
lists and _emit's returns, site_insights' GoatCounter answers and
make_demo's blocks are annotated, and the baseline loses the entry
that no longer occurs.
2026-11-06 22:30:55 +01:00
Abdessamad Derraz 8bdf19c7dd feat: progress lines for online profile_sync 2026-11-06 15:02:53 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz 420162c79b fix: rank uncheckable profiles first in triage
--strict fails on a profile no declared repository answers for, yet --triage scored it 0 under the clean profiles and left it out of the count to review. It now weighs as a pin to set again and is counted.
2026-11-06 08:37:36 +01:00
Abdessamad Derraz 63fafaf0f9 fix: read teknoparrot and simple64 from mirrors
teknogods/TeknoParrotUI and simple64/simple64 answer 404 on the API and the web, and neither organisation lists them now. simple64's README at the pin archives the project for gopher64. Each profile declares upstream_gone and a fork that keeps its default branch at the analysed commit: thecrixux/TeknoParrotUI master is b75872ad, simple64-vault/simple64 main is d8c969c7. realign_prose reads a source declared gone through that mirror, which unblocks five teknoparrot notes citations that anchor unchanged.
2026-11-06 08:37:12 +01:00
Abdessamad Derraz 8dc3ac82c7 fix: date each prose citation on its own
realign_prose dated a whole field from the last commit that changed any of its citations. idtech4a++ corrected two notes citations by hand in the commit that advanced its pin, so the untouched Game.java:69-72 and GameResourceUrl.java:14-118 read as written under the current pin and were never offered again. Each citation is now followed through the history on its own. The two idtech4a++ ranges move to 72-75 and 15-121, read at ce49510f.
2026-11-06 08:24:49 +01:00
Abdessamad Derraz 3bad1ad35e fix: an uncached tip offline is unknown
Offline, resolve_head answered None for a tip the cache never stored, as for a branch the forge does not have. select_views then dropped xenios-jp/XeniOS and judged its citations against has207/xenia-edge. The tip and the dated commit now raise NotCachedError, and the profile is skipped as unread.
2026-11-06 03:27:04 +01:00
Abdessamad Derraz 0350f42351 chore: merge site and docs batch 2026-11-05 22:22:37 +01:00
Abdessamad Derraz f9a786896e fix: restore the profile when the prose read fails
_apply_writes promoted the bumped profile, then read its prose under the new pin. A NotCachedError, RateLimitError or UpstreamError from that read left the profile written with a pin the guard would refuse. The original bytes are now put back before the error goes on.
2026-11-05 19:30:04 +01:00
Abdessamad Derraz f8dce8f8c3 fix: keep upstream HEAD for a pin past the fork
_follow_fork_base swapped the fork merge base in as upstream HEAD even when upstream_commit was newer than that base, as for melonds and easyrpg whose upstream is the standalone build they document. melonds read its Qt refs 825 commits back and reported 4 to review. A pin that descends from the base now keeps the HEAD, melonds shows 0 to review.
2026-11-05 19:29:23 +01:00
Abdessamad Derraz 651cabf412 fix: rank orphaned and moved pins first in triage
drift_score ignored orphaned pins and moved repositories although triage_state says they are fixed before any ref is read: pcem ranked last at 0 with an upstream_commit on no branch. Each now weighs more than a GONE ref.
2026-11-05 19:27:36 +01:00
Abdessamad Derraz a3c853316a fix: name the dirty profiles profile_sync refuses
The refusal said the directory carried uncommitted changes without naming the profile, among about 375. It now lists the files git status reports, and says that --force writes over them.
2026-11-05 19:27:12 +01:00
Abdessamad Derraz a06f5ad2ba fix: fetch the project main an upstream lacks
A remote named upstream added as CONTRIBUTING said and never fetched made validate_pr --changed compare with the fork's main and exit 0 on nothing. It now fetches the project's main when the remote does not hold it. profile_sync names the fetch when --base is absent, and CONTRIBUTING adds git fetch upstream.
2026-11-05 19:20:50 +01:00
Abdessamad Derraz 9ae3919d34 fix: spell cited commands so a paste runs them
Messages in check_freshness, generate_pack, common, profile_sync and fileless_audit named commands as export_native.py --platform x, which fails from the repository root. They now read python scripts/<name>.py like the rest of the report.
2026-11-05 19:19:04 +01:00
Abdessamad Derraz d5e5146819 feat: hold bumps of profiles one platform ships
A bulk advance moved fheroes2, supertux and supermariowar past the
revision EmuELEC compiles, and nothing stopped it: bump_commit refused
built_by only. It now leaves a profile a single registered platform ships
until it carries pin_frozen or the new pin_follows, which says that
platform builds the branch tip.

hode, bermuda, bennugd-monolithic and ccleste are held on the
PKG_VERSION of their EmuELEC v4.8 package.mk, equal to their pins.
rigelengine was one commit past it (README.md only) and goes back to
f05996f9.
2026-11-05 12:23:41 +01:00
Abdessamad Derraz bef8b50669 fix: compare --changed with the project main line
In a fork's clone origin/main is the fork: once pushed it is HEAD, and
profile_sync --changed --strict checked nothing and passed while the PR's
CI then failed. The default base is now the line validate_pr.py reads:
upstream/main or origin/main when that remote is the project, the
project's main fetched otherwise.
2026-11-05 12:16:28 +01:00
Abdessamad Derraz 593260a5ed fix: name orphaned pins on the triage line
needs_review counts an orphaned pin and a moved repository, but the
triage line only added stale prose and unchecked refs, so pcem read
"1 to review" for an upstream_commit outside its branch history.
2026-11-05 11:59:33 +01:00
Abdessamad Derraz 739f651a16 feat: check refs into a submodule
idtech4a++ cites xash3d-fwgs filesystem/filesystem.c, starship and
ghostship Torch src/Companion.cpp: files of submodules of the declared
repository. No blob of the declared tree answers, so the parts stayed
external and a bump moved the submodule commit with nothing checked.
A project word now also resolves to the one submodule named after it,
and a file under a submodule is read in the repository .gitmodules
names, at the commit the declared revision pins it to.
2026-11-05 11:58:22 +01:00
Abdessamad Derraz 91b7bd94d0 feat: refuse placeholder versions in the schema
core_version is a string the code states, null or keyed by build mode,
and the schema refuses Git, SVN, unknown, master, main, latest, dev and
HEAD, alone or annotated. version_basis is defined per kind with the
fields each one needs, and kind none requires a null version.

tests/test_version_basis.py reads a recorded fixture of forge answers
offline and fails on drift, holds every profile to a basis (tag versions
in describe form, aliases equal to their profile, none as null), checks
that a pin advance reads the version again or is refused, and that the
schema refuses placeholders.
2026-10-27 19:23:17 +01:00
Abdessamad Derraz 6aa236440a feat: read core_version from its recorded basis
version_basis says where a profile's version is read: a pattern in a
file, the nearest tag and its distance, an alias, a package, none, or
unverified with the reason. profile_sync --verify-version reads it again
at the pin and exits 1 on a difference, --set-version writes what the
code states. A pin advance reads the version at the new pin in the same
write and is refused when it cannot.

upstream.describe counts the commits after the nearest tag in a treeless
clone kept under the cache, and graph_ancestry answers a pair GitHub
refuses to compare.
2026-10-27 18:48:11 +01:00
Abdessamad Derraz 68f42bf8e2 fix: judge each build's refs in its own tree
A profile keyed by build mode compared a part in every repository and kept
the best outcome: xenia-edge's libretro tree carries emulator.cc offset,
and its lines vouched for standalone refs that had moved 4 lines. A part
of an entry with a mode is read in that build's repository, and a part
naming none in the first build declared, as _realign_part decides.
2026-10-27 05:41:27 +01:00
Abdessamad Derraz d0de6dde30 fix: rebuild source paths from the written pins
load_emulator_profiles caches by directory, and the run had loaded the
profiles before writing them, so the record rebuilt after a write kept the
pins just replaced: linapple, then ten profiles in one advance. The cache
is cleared first. The record is rebuilt for those ten.
2026-10-27 05:24:04 +01:00
Abdessamad Derraz 951bd070bf feat: advance profiles that cite no source
_needs_review held every skipped report, so a profile with no source_ref
never moved: anarch sat behind a CI-only commit through every bulk advance.
Its source pin now moves; its upstream pin does not, since nothing was
compared there.
2026-10-27 04:26:28 +01:00
Abdessamad Derraz 4b11a43a86 feat: version check reads tags and numbers
A project that tags without releases (eightyone, higan) was never compared,
and a core_version naming no tag (parallel-launcher 9.0.5 beside v9.0-6)
silenced the passed-release warning. The newest tag stands in when it is a
version number, and an untagged core_version is compared by number within
one scheme. --emulator is repeatable: a stored value kept only the last.
2026-10-27 04:16:31 +01:00
Abdessamad Derraz 44378ae5c9 feat: name a release newer than the pinned tag
A profile pinned on its release tag read as clean forever: armsx1, armsx2
and x1-box sat on old tags while newer releases were out. --check-version
now names the latest release when it follows both the declared tag and
the pin, unless pin_frozen holds the pin on purpose.
2026-10-26 23:25:51 +01:00
Abdessamad Derraz f3a161b60e fix: keep unread trees in the source paths record
A sitesources run that GitLab rate-limited dropped about twenty trees it
never read from website/src/data/source-paths.json. A tree key names a
pinned sha, so the last record's entry for an unread tree still holds and
is carried over, in sitesources and in profile_sync's refresh.
2026-10-26 22:52:23 +01:00
Abdessamad Derraz 9dbe68f990 fix: skip the gained-names check on held pins 2026-10-26 22:16:43 +01:00
Abdessamad Derraz 45d611332a feat: hold a bump when cited files gained names
Refs that still anchor say nothing of what the code added around them:
mame2003_plus's zn.c gained two Taito G-NET BIOS sets while its cited
lines only shifted. --rebase-refs and --bump-commit now refuse a profile
whose cited files gained a file name or a hash literal since the pin, or
could not be read, until --accept-changed says the additions were read.
With this, --all --rebase-refs --fresh is safe to run as is.
2026-10-26 21:55:23 +01:00
Abdessamad Derraz 32633c4e5c fix: let any refusal confirm a declared dead forge
citron's taken domain now resolves to an unrelated host that fails TLS,
so the pass reported an upstream error where the profile already states
the forge is gone. Any forge error now agrees with upstream_gone; a rate
limit still does not.
2026-10-26 21:53:47 +01:00
Abdessamad Derraz fa3651d14d feat: refresh source paths after a profile write
Each pin advance left website/src/data/source-paths.json behind and the
site's freshness test red until sitesources.py --fetch ran by hand.
profile_sync now rebuilds it after any run that wrote a profile, online
and not in a dry run.
2026-10-26 21:07:08 +01:00
Abdessamad Derraz 0c77ef074e feat: add --fresh and name why a keyed pin waits
Branch tips, tags and releases are cached for six hours, and the
EKA2L1, trzy/Supermodel and VICE heads read as behind their real HEAD.
--fresh asks the forges again, and is refused with --offline.

A recale on a profile whose pins are keyed by build mode said the prose
had to move first. It now says the pins are set by hand.
2026-10-24 04:08:59 +02:00
Abdessamad Derraz fa7977d2c3 fix: ask again for a tip older than the pin
A branch tip is cached for six hours. scummvm-standalone was pinned on
6f7b060d while the cache still held 77d80d7, its ancestor, so profile_sync
showed a HEAD behind the pin and a bump would have moved it backwards. A
tip that is a strict ancestor of the pin is now asked for again.
2026-10-24 02:00:21 +02:00
Abdessamad Derraz 5a3f621e94 fix: read the mirror of a gone source
select_views stopped on the first repository that answered as gone, so
yuzu's source_mirror was never read and the profile stayed unchecked. A
gone repository is now passed over, and a profile declared gone that
answers only through its mirror is not reported as a contradiction.
2026-10-23 19:13:48 +02:00
Abdessamad Derraz 82bf236a5a feat: hold an upstream pin a port vendors
upstream_frozen says why upstream_commit stays: the port vendors that
upstream tree, synced to that release, or merged that commit. The view
is then read at its own pin, the way pin_frozen holds source_commit, and
a bump leaves it. Outside a GitHub fork network nothing else could tell
aps3e's vendored RPCS3 1f080485 from an upstream read on a date.
2026-10-23 17:30:20 +02:00
Abdessamad Derraz 01465ecf6b fix: ignore a release older than the tag
--check-version said pcsx2x6's v0.2.22 was older than its code because
the pin holds release v0.2.20, the latest the project published. A
release that is an ancestor of the declared tag is not a later one.
2026-10-23 17:26:25 +02:00
Abdessamad Derraz 19f22da75f fix: write prose and quoted refs as written
A folded run rewritten whole keeps its separators (boom3, mame), a
double-quoted source_ref is matched as the loader reads it and escaped on
write (etlegacy), two files sharing a name take their build mode in the
field path (supermodel Games.xml), and a project word ending the line
before a path marks the citation external (genesis_plus_gx_expanded).
2026-10-23 17:23:01 +02:00
Abdessamad Derraz 6f8d3dede3 fix: read prose where its path is written
realign_prose stripped a path prefix, or read a path two repositories
share, in the repository whose pin moved, while the profile meant the
other. hatarib's hatari/src/tos.c became hatari/hatari's src/tos.c, and
freej2me_plus's fork citations were read in hex007/freej2me: 32 and 8
citations asked to be read again although their files were unchanged.
The path as written now decides first, and a shared path belongs to the
repository declared first, as in attribution.

test_a_frozen_build_files_its_content_with_the_build_that_follows paired
every pin_frozen profile with every live one of the same source. It now
skips platform builds (built_by), and requires the frozen systems to be
a subset: a later release can add a board (advancemame's coh1002v).
2026-10-23 16:41:04 +02:00
Abdessamad Derraz 861b89dd44 feat: follow a fork's base as its upstream head
For a port that is a GitHub fork of its upstream, the upstream code it
carries is the merge base of the two, not upstream's HEAD. The bump moved
upstream_commit to upstream HEAD on snes9x, kronos, np2kai and others
whose pin was that base. The upstream view's head is now the merge base
GitHub's compare names, the way _follow_submodule already reads a
vendored submodule. Outside a fork network the HEAD stays.
2026-10-23 16:25:56 +02:00
Abdessamad Derraz 2d48cde8d6 fix: judge each recaled citation at its own place
A sentence written in two fields was located over the whole file, found
twice and left in both: the writer now searches the lines of its own
scalar. A structured ref was judged pending when its old text appeared
anywhere, so a chain of shifts (tr1x cut2 508 to 531, cut3 531 to 552)
held the pin on the values the previous entries took: it is now looked
up at its own field.
2026-10-23 16:21:04 +02:00
Abdessamad Derraz 97c960484c fix: read the file a detected literal names
A printf conversion glued to a name, a name with spaces cut to its last
word, an extension list, and an identifier in code were reported as new
files.
2026-10-23 16:21:04 +02:00
Abdessamad Derraz 5fba72f1a3 fix: map a file edited line for line as changed 2026-10-23 16:21:04 +02:00
Abdessamad Derraz 42dfc87e24 fix: recale wrapped and comment prose citations
The recale skipped runs in YAML comments and runs wrapped over two
lines of a literal block, while the pin check counted them: the run
wrote nothing and said nothing. Both are now rewritten at their located
position. A citation that still cannot move is named on stderr and the
profile is left untouched.
2026-10-23 16:21:04 +02:00
Abdessamad Derraz e63bdd23c2 fix: name only the pin moves a bump keeps
A bump rolled back for stale prose still printed the pins it would have
set (yaps2: upstream_commit -> 7fe1e55 after "nothing was written").
2026-10-23 15:57:19 +02:00
Abdessamad Derraz d1a6e96820 fix: check the prose a bump leaves behind
The prose guard read citations under the pins a profile held before the
write. A bump that moves upstream_commit alone could then leave prose the
report had judged in the fork under the new upstream pin: ten profiles
(dolphin-mmjr2, hatari, xenios among them) came out of a bump with prose
to read again. The written file is now read again and the original bytes
put back when a citation is left under an older pin.

A pin_frozen reason was printed as a tag name ("pinned to tag EmuELEC,
the only platform..."); it now reads "pin held (...)".
2026-10-23 15:50:51 +02:00
Abdessamad Derraz 9dd2a3d95f fix: date-order tags and scope the write guard
check_version read latest_tag as the first entry of GitHub's REST
listing, which sorts by name (nesoid: 2.5 before 2.5-4, gargoyle:
stable-20090825 before 2026.1.1), and found the declared tag only on that
first page of 100. The newest tag now comes from GraphQL by commit date,
and the declared one is asked for by name on GitHub, GitLab and Forgejo.

The clean-tree guard of the write modes looked at the whole emulators
directory: a loop of --emulator X --rebase-refs refused every profile
after the first it wrote. It now looks at the selected profiles only.
2026-10-23 15:40:56 +02:00
Abdessamad Derraz 18fc5ce53f fix: leave platform-built pins to a reader
bump_commit advanced any profile whose refs all anchored at HEAD. On a
built_by profile the pin is the commit a platform compiles (mame0258,
mame0278, gsplus-legacy at recalbox gsplus.mk:8-9), so HEAD is a build
nobody ships under that name. The bump now refuses those profiles.
2026-10-23 15:35:51 +02:00
Abdessamad Derraz de6f616bb9 docs: split a docstring sentence in profile_sync 2026-10-23 15:34:11 +02:00
Abdessamad Derraz a293887145 feat: place the pin against its version tag
--check-version said only that core_version names a tag other than the
pin. It now counts the pin against that tag (after it, before it, or off
its history), and names the latest release when the pin already contains
it. Of 52 warnings, 43 are builds between releases; ares, boytacean,
nesoid and primehack name an older release than their code, vkquake and
vkquake2 a later one.
2026-10-23 15:24:21 +02:00
Abdessamad Derraz d458cf653a fix: wait out a 429 and keep the pass going
A single 429 from git.libretro.com on cemu's tree ended profile_sync --all
with a traceback and no report for the other 646 profiles. upstream now
waits a 429 out (Retry-After up to 120 s, else 5, 20, 60 s), as
base_scraper already did, and stops asking a host that outlasts the
waits. profile_sync reports that profile as rate limited, fails the run,
and carries on with the other forges.
2026-10-23 15:02:48 +02:00
Abdessamad Derraz 36427edbb2 fix: point source refs at paths held at the pin
trident cites the Panda3DS fork 3dsTrident vendors as a submodule, now
its upstream at the submodule pin 69095ee6. pce-emu, uqm and stonesoup
wrote paths from a subdirectory (PCE.emu/, sc2/, crawl-ref/), nesoid
named two Java files without their directory; lines unchanged, each read
at the pin.

The pages no longer take a binary citation (DesktopKinect.dll, an
address continuing a BAM.dll range) for a source path, and a Forgejo
tree is read page by page: mirrors/Ryujinx holds 4704 entries, 1000 a
page.
2026-10-23 13:11:48 +02:00