Commit Graph
2174 Commits
Author SHA1 Message Date
Abdessamad Derraz f4b45c641d feat: run the PR checks with one command 2026-11-06 14:55:03 +01:00
Abdessamad Derraz af6936055b docs: quote the pip lines CI pins for PR checks 2026-11-06 14:51:00 +01:00
Abdessamad Derraz bd50b08133 docs: fetch platform files before the PR tests 2026-11-06 14:49:06 +01:00
Abdessamad Derraz 747d40e988 docs: give the clone command and size for packs 2026-11-06 14:47:47 +01:00
Abdessamad Derraz 8100d671f9 chore: merge origin/main 2026-11-06 13:02:20 +01:00
Abdessamad Derraz c1b0149a81 chore: merge site insights 2026-11-06 12:56:13 +01:00
Abdessamad Derraz b23840bc21 feat: add site insights report
Search Console queries by page, file names searched against the
collection, GoatCounter totals and GitHub traffic in one Markdown report.
2026-11-06 12:55:57 +01:00
Abdessamad Derraz f55e473183 chore: one clause per sentence in a comment 2026-11-06 12:55:13 +01:00
Abdessamad Derraz 5e90a455e4 feat: add EmuELEC hardware targets
The v4.8 release publishes four images (Amlogic-ng, GameForce,
OdroidGoAdvance, OdroidM1, read from its asset names). The metapackage
(packages/sx05re/emuelec/package.mk:8-77) builds the S922X cores, dolphinSA
and mame only on Amlogic-ng, Amlogic-no, RK356x and OdroidM1, and removes
duckstation on the handhelds with a sed that replaces the first match only:
pcsx_rearmed, written twice, keeps its 64-bit build. Each package is mapped
to its cores by the .so or program its install step copies, PKG_ARCH is
applied as scripts/build:28-30 does, and es_systems.cfg is edited per
device as emuelec-emulationstation/package.mk:114-143 does. Amlogic-ng and
OdroidM1 list 124 cores, the handhelds 117.
2026-11-06 12:09:05 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz 25b11fa497 chore: merge site fixes, fourth pass 2026-11-06 10:33:56 +01:00
Abdessamad Derraz d0c65dca22 chore: merge pack and verify fixes 2026-11-06 10:28:27 +01:00
Abdessamad Derraz c0e4bd8379 chore: merge resolver fixes, fourth pass 2026-11-06 10:28:27 +01:00
Abdessamad Derraz a29b836730 chore: merge installer fixes, fourth pass 2026-11-06 10:28:14 +01:00
Abdessamad Derraz 1396350795 fix: count build file trees as pinned
check_source_paths read the trees profiles pin from source and upstream
only, so the two platform trees the record now holds for pin_build
files read as orphaned and the record as stale.
2026-11-06 10:14:02 +01:00
Abdessamad Derraz e491ff1bc6 chore: merge tooling fixes, fourth pass 2026-11-06 09:50:16 +01:00
Abdessamad Derraz f9f7ad3b92 docs: quote the bundled line in the verify excerpt
reading-reports quotes a RetroDECK run, which now prints the bundled count and 30 missing game data instead of 38. The comment introducing the bundled names also loses its semicolon.
2026-11-06 09:46:37 +01:00
Abdessamad Derraz 2c60b5f20d docs: refresh_stale names pin_installer plainly
The docstring line naming pin_installer.py read as a clause after a
semicolon, which the prose test refuses.
2026-11-06 09:43:07 +01:00
Abdessamad Derraz 21f4122c83 fix: collisions keep the platform's order
Walking the systems sorted reordered the declarations doubled.py
reports from, so the export report named the CoCo reader first. The
declarations keep the platform's order again, and the packed file is
picked from the sorted walk the builder makes.
2026-11-06 09:43:06 +01:00
Abdessamad Derraz 79a21ba1f1 feat: name the files in emulator search snippets
A search for gba_bios.bin or scph5500.bin landed on a page described as
"It reads 4 files, none of them required". The snippet now names the
console and the files, required first, in the longest form that fits.
2026-11-06 09:28:03 +01:00
Abdessamad Derraz 856cb5618f fix: link a build file only where it is recorded
Every source link on the site leads to a path the source record saw at
a commit. The Build file row linked pin_build files unchecked, eight of
them at commits the record did not hold. build_source_paths now records
the build files read at a commit, and the row links only those: a tag
can move and stays plain text. The record run also picked up two cited
paths it had not seen before (pom2 CMakeLists.txt, pcem configure.ac).
2026-11-06 09:14:42 +01:00
Abdessamad Derraz 462b61a4bc refactor: compute slot choices once per manifest
generate_manifest called slot_choices, and pack_overrides called it again with the same five arguments. pack_overrides takes the choices when the caller holds them. Measured on generate_pack.py --all --manifest --offline: 48 calls and 7.0 s in slot_choices before, 24 calls and 4.3 s after (wall 45.7 s and 42.4 s, one run each on a loaded host). The manifests written are identical.
2026-11-06 09:11:37 +01:00
Abdessamad Derraz 9633ce3dd4 refactor: normalize each system id once
SelectionIndex.build and system_group called _norm_system_id 1.7 million times per manifest run for about a thousand ids, each through the manufacturer prefix loop. The function reads only its argument and two constant tables, so its answer is cached. Measured on the call volume of one manifest run (630 ids, 1,686,618 calls): 1.53 s uncached, 0.07 s cached. The end-to-end manifest run moves inside its own noise on this host (45.9 to 50.6 s across four runs).
2026-11-06 09:08:09 +01:00
Abdessamad Derraz 43cb0f84ab feat: add AmberELEC hardware targets
AmberELEC's metapackage adds LIBRETRO_CORES_EXTRA (bsnes, Mesen, melonDS, geolith) only on RG552, RG353 and RG503, and deletes those cores from es_systems.cfg on every other device (packages/amberelec/package.mk:23-31, 45-58 at a21975bf). The new target scraper reads the Makefile's device images, that test and es_systems.cfg the way the platform scraper does: RG351P, RG351V and RG351MP list 125 cores, the other three 136. The installer reads the device from COREELEC_DEVICE in os-release (scripts/image:168).
2026-11-06 09:04:51 +01:00
Abdessamad Derraz 55e2c8328e feat: pack readme names spellings left out
A pack folded for Windows and macOS keeps one spelling of each case twin. Extracted on a Steam Deck or Linux, the core asking for the other spelling finds nothing, and only the installer added it. Where the platform also runs on a filesystem that keeps case, the README now lists each spelling left out beside the one it holds: 12 for EmuDeck and 10 for RetroArch, the manifest's case_variants.
2026-11-06 09:02:51 +01:00
Abdessamad Derraz ada7738fa7 feat: pin_installer writes the installer pins
Every change to install.py meant a sha256sum pasted by hand into
install.sh and install.ps1, and the tests that caught a stale pin
failed on two bare hashes. scripts/pin_installer.py writes both pins
(--check only reports), the pin tests name it, check_freshness reads
the pins through it and refresh_stale runs it for a stale pin row.
2026-11-06 09:02:18 +01:00
Abdessamad Derraz 4efcb3da64 fix: group core files under their pack path
On a platform that writes bios/ in front of its destinations (RetroDECK, AmberELEC, EmuELEC), a core extra was grouped, dropped and held under its bare destination while the platform's declaration of the same file used bios/<name>. Beetle PSX's scph5501.bin was kept by the --region japan pack while the declaration was dropped and the installer dropped the path. extra_region_key keys every extra with the platform's declared root, in the builder, the manifest selection, verify's replay and the omissions. The same key gives the declared entries the systems of the core files at their path: every --system case of RetroDECK differed between pack and installer before, none does now.
2026-11-06 09:00:06 +01:00
Abdessamad Derraz 725325f0a1 fix: announce a data tree at its archive size
A tarball tree was announced at its extracted size: yaps2 at 6.5 MB for a 21.2 MB download, Cemu's game profiles at 9.9 KB for 21.4 MB. refresh_data_dirs records the size of each archive it downloads (a tree current from before is measured, not refetched), the install lists carry it as archive_size, and the plan, the menu and the data event count it.
2026-11-06 08:59:10 +01:00
Abdessamad Derraz 01ce92232d docs: say how an uploaded ZIP is read
CONTRIBUTING and the site's contributing page now say that a ZIP no
list declares as a whole is read file by file, and that the files go in
unzipped.
2026-11-06 08:58:40 +01:00
Abdessamad Derraz 5407eafe5b feat: validate_pr reads an undeclared ZIP's files
A card's BIOS folder zipped to upload it was hashed as one file and
reported not recognised, its contributor asked where the dump came from
and told to place it under bios/. A ZIP no list or profile declares as a
whole is now read member by member: each is named held, declared and not
held, or not recognised, and the contributor is asked for the files
unzipped. A declared archive (a MAME or FBNeo set) is never opened.
2026-11-06 08:57:15 +01:00
Abdessamad Derraz 320db75490 fix: check_freshness aligns details after remotes
render padded the local value but not the remote one, so in an area
where some rows carry a remote value their detail started fifteen
columns later than the rows without one. The remote value is padded to
its column, and a row without one leaves the same blank span.
2026-11-06 08:56:32 +01:00
Abdessamad Derraz 8b7e5ceef4 fix: refresh_stale knows the romfolders area
check_freshness gained a romfolders area and refresh_stale did not:
its rows were dropped from a saved report without a word, and --only
romfolders was refused. A rom-folder pin is moved by hand, so the area
is listed for a human, a test holds the two area lists equal, and the
--only help names every area.
2026-11-06 08:56:14 +01:00
Abdessamad Derraz ea58f6f5ce fix: refresh_stale --dry-run honours --json
--dry-run printed its text plan and returned before --json was read,
so a caller parsing the output as JSON failed. With both flags the plan
is printed as JSON: planned jobs with their commands and follow-ups,
the rows left to a human, and the count ignored.
2026-11-06 08:55:51 +01:00
Abdessamad Derraz bf206d5b42 feat: slots names who declares a collided path
A collision line named two files and neither the system declaring each
nor the one the pack ships, so the reader went back to the platform
file. find_collisions walks the systems in the builder's sorted order,
takes the builder's preferred declaration, and the line and the JSON
now name each system and the file packed. RetroDECK's bios/disk.rom
reads as the CoCo ROM packed, the PC-88 one left out, as its manifest
holds.
2026-11-06 08:55:26 +01:00
Abdessamad Derraz a343be0da1 fix: validate_pr json says failed, not new
The JSON report put "outcome": "new" on every file without another
outcome, so a path that does not exist, or a file that failed a check,
was listed as a new contribution. A failed file now reads "failed".
2026-11-06 08:54:23 +01:00
Abdessamad Derraz a1ddee8e5e fix: a lock reader refuses a missing directory
artifact_lock created its directory even for a shared reader, so a
mistyped path given to --verify-packs or release_record was left as a
directory holding a .lock, and split_pack created the parent before
printing a bare errno. A reader now raises ArtifactDirectoryMissing,
which run_cli prints on one line, and split_pack checks its target
before taking the lock.
2026-11-06 08:54:06 +01:00
Abdessamad Derraz 19dddedff6 fix: a data lock wait names its holder
A refresh blocked on a data directory lock printed only that another
run held it, then waited with no limit. The line now names the lock
file and the lsof command that shows its holder, and for a refresh,
the generate_pack --no-refresh-data and --offline runs that read the
cache without waiting.
2026-11-06 08:53:26 +01:00
Abdessamad Derraz 46a4a26a54 fix: keep bundled game data out of missing
verify counted game data installed with its emulator as missing, which cross_reference files as bundled: 24 of Batocera's 427. Bundled firmware already left the lists, and the line naming it never printed. Every bundled entry is now counted on that line, required firmware named and the rest under --verbose.
2026-11-06 08:50:29 +01:00
Abdessamad Derraz 8aa1a5fdad fix: a release asset absent on disk stays held
The previous commit counted a name with no resolved copy as not held, which also dropped release assets the builder holds while no copy is on disk (test_install_selection AHeldFileTheInstallerCannotFetch). Only the case-insensitive name match now leaves out copies in another emulator's tree, as resolve_local_file does.
2026-11-06 08:42:53 +01:00
Abdessamad Derraz 63be8cc3bc fix: find tab-indented batocera-systems entries
The entry pattern required four spaces of indent. EmuELEC indents 23 systems with one tab, so render appended each again whole and the published export carried 21 duplicated keys, the md5 fills only in the second copy. Any indent opens an entry now, and an entry written again keeps its own. The EmuELEC export is regenerated: one copy of each system, the amiga entry also gaining the eight WHDLoad Kickstarts the profiles added since 2026-10-22.
2026-11-06 08:39:40 +01:00
Abdessamad Derraz 420162c79b fix: rank uncheckable profiles first in triage
--strict fails on a profile no declared repository answers for, yet --triage scored it 0 under the clean profiles and left it out of the count to review. It now weighs as a pin to set again and is counted.
2026-11-06 08:37:36 +01:00
Abdessamad Derraz 63fafaf0f9 fix: read teknoparrot and simple64 from mirrors
teknogods/TeknoParrotUI and simple64/simple64 answer 404 on the API and the web, and neither organisation lists them now. simple64's README at the pin archives the project for gopher64. Each profile declares upstream_gone and a fork that keeps its default branch at the analysed commit: thecrixux/TeknoParrotUI master is b75872ad, simple64-vault/simple64 main is d8c969c7. realign_prose reads a source declared gone through that mirror, which unblocks five teknoparrot notes citations that anchor unchanged.
2026-11-06 08:37:12 +01:00
Abdessamad Derraz 4377cbb2af fix: another tree's casing does not hold a file
dosbox-x's et4000.bin matched only pcem's et4000.BIN. resolve_local_file, asked as dosbox-x, refuses another emulator's tree in its case-insensitive step, so the pack and installer leave it out, while the gap report and verify's core files resolved without the asker and counted it held. Both now resolve as the profile, and the manifests of Batocera and KNULLI no longer flag it held.
2026-11-06 08:36:42 +01:00
Abdessamad Derraz a0ecca0024 fix: collect abbreviated prose citations
A location written without its file, (:507), reads in the file named last in the scalar, either cited or named as 'path:' before a list. A dotless file cited with its directory (tools/errorgen:20) is a citation too. devilutionx, corsixth, starship, plastic, eden-nightly, theforceengine, lightspark and higan_sfc gain 33 citations that bump and realign now follow. Two read that way pointed at another file and now name theirs: devilutionx assets.cpp:511-518, corsixth app.lua:1591.
2026-11-06 08:33:16 +01:00
Abdessamad Derraz 21dd14c851 fix: a verified large file keeps its bytes
fetch_large_file verified a cached copy and returned its name, which another run replaces when the release serves a re-upload: the caller then packed bytes it never verified. The returned path is now a hard link under .cache/large/.by-sha1/<sha1>/, kept only if it holds the inode that was hashed. Revisions no name holds are dropped a day later. restore_large_files hashes each copy before moving it into place.
2026-11-06 08:30:55 +01:00
Abdessamad Derraz 04aad25683 feat: pipeline compares split parts to the pack
Step 5 compared every narrowed pack with verify and never the --split parts with the full pack, so a part writing a path the full pack settles passed every check. check_split_packs builds each platform's parts in memory and compares their union with the full pack, path for path and byte for byte.
2026-11-06 08:27:48 +01:00
Abdessamad Derraz da9e1f568c fix: refuse an offline export with no originals
pipeline.py --offline --with-export, listed as a standard command, passes
no --fetch to step 2e: on a fresh clone the database, verify and truth
were built before the export failed on every platform whose own file it
patches. The run is refused before step 0 when .cache/upstream-native/
holds nothing for such a platform, and tools.md says how to fill it.
2026-11-06 08:27:32 +01:00
Abdessamad Derraz 556855421a fix: keep release notes to checked claims
The v2026.10.10 notes said a pack holds every file its emulators open, so
a game never stops on a missing file, and that hashes never come from a
forum list: the gap analysis lists files no pack carries, and in md5
mode a pack keeps the platform's own md5 (verification-modes.md). The
generated text and the README now point to the gap analysis, and
release_notes.py refuses a whats-new text holding a judging or
contradicted phrase.
2026-11-06 08:25:25 +01:00
Abdessamad Derraz 8dc3ac82c7 fix: date each prose citation on its own
realign_prose dated a whole field from the last commit that changed any of its citations. idtech4a++ corrected two notes citations by hand in the commit that advanced its pin, so the untouched Game.java:69-72 and GameResourceUrl.java:14-118 read as written under the current pin and were never offered again. Each citation is now followed through the history on its own. The two idtech4a++ ranges move to 72-75 and 15-121, read at ce49510f.
2026-11-06 08:24:49 +01:00
Abdessamad Derraz 3e2a5279a1 fix: split parts hold what the full pack holds
Each part was built alone: the Other_Cores part never saw the platform's destinations and each part had its own case fold. RetroDECK's parts wrote bios/SGB1.sfc/ members beside the file bios/SGB1.sfc, RetroBat's wrote 21 openmsx/ twins of openMSX/ files, and a path two systems declare got each system's file. The full pack is now built in memory first and a part writes a declared file or core extra only where the full pack writes the same bytes.
2026-11-06 08:21:52 +01:00