Commit Graph
4417 Commits
Author SHA1 Message Date
Abdessamad Derraz 25b11fa497 chore: merge site fixes, fourth pass 2026-11-06 10:33:56 +01:00
Abdessamad Derraz d0c65dca22 chore: merge pack and verify fixes 2026-11-06 10:28:27 +01:00
Abdessamad Derraz c0e4bd8379 chore: merge resolver fixes, fourth pass 2026-11-06 10:28:27 +01:00
Abdessamad Derraz a29b836730 chore: merge installer fixes, fourth pass 2026-11-06 10:28:14 +01:00
Abdessamad Derraz 1396350795 fix: count build file trees as pinned
check_source_paths read the trees profiles pin from source and upstream
only, so the two platform trees the record now holds for pin_build
files read as orphaned and the record as stale.
2026-11-06 10:14:02 +01:00
Abdessamad Derraz 8a14a1f75c docs: regenerate README counts
et4000.bin is no longer counted held for dosbox-x on Batocera and KNULLI.
2026-11-06 09:58:47 +01:00
Abdessamad Derraz e491ff1bc6 chore: merge tooling fixes, fourth pass 2026-11-06 09:50:16 +01:00
Abdessamad Derraz 3672c6de4a docs: list the AmberELEC target scraper 2026-11-06 09:46:37 +01:00
Abdessamad Derraz f9f7ad3b92 docs: quote the bundled line in the verify excerpt
reading-reports quotes a RetroDECK run, which now prints the bundled count and 30 missing game data instead of 38. The comment introducing the bundled names also loses its semicolon.
2026-11-06 09:46:37 +01:00
Abdessamad Derraz 508dc4d039 docs: list the new test modules
test_twin_declaration_status and test_verify_data_dir_notice join the testing guide table, and test_large_file_cache calls unittest.main after its last class.
2026-11-06 09:44:28 +01:00
Abdessamad Derraz 2c60b5f20d docs: refresh_stale names pin_installer plainly
The docstring line naming pin_installer.py read as a clause after a
semicolon, which the prose test refuses.
2026-11-06 09:43:07 +01:00
Abdessamad Derraz 21f4122c83 fix: collisions keep the platform's order
Walking the systems sorted reordered the declarations doubled.py
reports from, so the export report named the CoCo reader first. The
declarations keep the platform's order again, and the packed file is
picked from the sorted walk the builder makes.
2026-11-06 09:43:06 +01:00
Abdessamad Derraz 79a21ba1f1 feat: name the files in emulator search snippets
A search for gba_bios.bin or scph5500.bin landed on a page described as
"It reads 4 files, none of them required". The snippet now names the
console and the files, required first, in the longest form that fits.
2026-11-06 09:28:03 +01:00
Abdessamad Derraz adcfa283fb docs: keep the FAQ under its word ceiling
The --core answer added 14 words to a page held at 450. It now names the
command in as many words as the line it replaced, and the --region line
goes: what --region keeps is on the installer page.
2026-11-06 09:14:47 +01:00
Abdessamad Derraz 856cb5618f fix: link a build file only where it is recorded
Every source link on the site leads to a path the source record saw at
a commit. The Build file row linked pin_build files unchecked, eight of
them at commits the record did not hold. build_source_paths now records
the build files read at a commit, and the row links only those: a tag
can move and stays plain text. The record run also picked up two cited
paths it had not seen before (pom2 CMakeLists.txt, pcem configure.ac).
2026-11-06 09:14:42 +01:00
Abdessamad Derraz 462b61a4bc refactor: compute slot choices once per manifest
generate_manifest called slot_choices, and pack_overrides called it again with the same five arguments. pack_overrides takes the choices when the caller holds them. Measured on generate_pack.py --all --manifest --offline: 48 calls and 7.0 s in slot_choices before, 24 calls and 4.3 s after (wall 45.7 s and 42.4 s, one run each on a loaded host). The manifests written are identical.
2026-11-06 09:11:37 +01:00
Abdessamad Derraz ef6607302d chore: rebuild the site when siteinputs changes
Every page module reads its inputs from scripts/siteinputs.py, so the
deploy workflow names it with the other site modules.
2026-11-06 09:11:02 +01:00
Abdessamad Derraz 9633ce3dd4 refactor: normalize each system id once
SelectionIndex.build and system_group called _norm_system_id 1.7 million times per manifest run for about a thousand ids, each through the manufacturer prefix loop. The function reads only its argument and two constant tables, so its answer is cached. Measured on the call volume of one manifest run (630 ids, 1,686,618 calls): 1.53 s uncached, 0.07 s cached. The end-to-end manifest run moves inside its own noise on this host (45.9 to 50.6 s across four runs).
2026-11-06 09:08:09 +01:00
Abdessamad Derraz 333f1ac3ca fix: resume a cut data archive
A data archive cut mid-download was deleted on every failure: the retry and the next run started from zero, while the run said nothing was lost. The archive is now kept beside its tree with the validator its server sent, and the rest is asked for with Range and If-Range. Checked against the release: fbneo-samples stopped at 16 MB, the next run downloaded 34.6 MB of 50.5.
2026-11-06 09:04:52 +01:00
Abdessamad Derraz 43cb0f84ab feat: add AmberELEC hardware targets
AmberELEC's metapackage adds LIBRETRO_CORES_EXTRA (bsnes, Mesen, melonDS, geolith) only on RG552, RG353 and RG503, and deletes those cores from es_systems.cfg on every other device (packages/amberelec/package.mk:23-31, 45-58 at a21975bf). The new target scraper reads the Makefile's device images, that test and es_systems.cfg the way the platform scraper does: RG351P, RG351V and RG351MP list 125 cores, the other three 136. The installer reads the device from COREELEC_DEVICE in os-release (scripts/image:168).
2026-11-06 09:04:51 +01:00
Abdessamad Derraz 55e2c8328e feat: pack readme names spellings left out
A pack folded for Windows and macOS keeps one spelling of each case twin. Extracted on a Steam Deck or Linux, the core asking for the other spelling finds nothing, and only the installer added it. Where the platform also runs on a filesystem that keeps case, the README now lists each spelling left out beside the one it holds: 12 for EmuDeck and 10 for RetroArch, the manifest's case_variants.
2026-11-06 09:02:51 +01:00
Abdessamad Derraz ada7738fa7 feat: pin_installer writes the installer pins
Every change to install.py meant a sha256sum pasted by hand into
install.sh and install.ps1, and the tests that caught a stale pin
failed on two bare hashes. scripts/pin_installer.py writes both pins
(--check only reports), the pin tests name it, check_freshness reads
the pins through it and refresh_stale runs it for a stale pin row.
2026-11-06 09:02:18 +01:00
Abdessamad Derraz 4efcb3da64 fix: group core files under their pack path
On a platform that writes bios/ in front of its destinations (RetroDECK, AmberELEC, EmuELEC), a core extra was grouped, dropped and held under its bare destination while the platform's declaration of the same file used bios/<name>. Beetle PSX's scph5501.bin was kept by the --region japan pack while the declaration was dropped and the installer dropped the path. extra_region_key keys every extra with the platform's declared root, in the builder, the manifest selection, verify's replay and the omissions. The same key gives the declared entries the systems of the core files at their path: every --system case of RetroDECK differed between pack and installer before, none does now.
2026-11-06 09:00:06 +01:00
Abdessamad Derraz 8107564631 fix: gearcoleco warns on an unknown CRC32
At be1a3716 find_firmware rejects a candidate by size alone (platforms/libretro/libretro.cpp:811-816), and an OS-7, EOS or SmartWriter image whose CRC32 differs is logged and loaded (lines 875-876, 928-935). The three entries keep crc32 in validation with on_mismatch warn, so another 8 KiB OS-7 revision such as colecoa.rom no longer reads as rejected.
2026-11-06 08:59:15 +01:00
Abdessamad Derraz 725325f0a1 fix: announce a data tree at its archive size
A tarball tree was announced at its extracted size: yaps2 at 6.5 MB for a 21.2 MB download, Cemu's game profiles at 9.9 KB for 21.4 MB. refresh_data_dirs records the size of each archive it downloads (a tree current from before is measured, not refetched), the install lists carry it as archive_size, and the plan, the menu and the data event count it.
2026-11-06 08:59:10 +01:00
Abdessamad Derraz 01ce92232d docs: say how an uploaded ZIP is read
CONTRIBUTING and the site's contributing page now say that a ZIP no
list declares as a whole is read file by file, and that the files go in
unzipped.
2026-11-06 08:58:40 +01:00
Abdessamad Derraz 5407eafe5b feat: validate_pr reads an undeclared ZIP's files
A card's BIOS folder zipped to upload it was hashed as one file and
reported not recognised, its contributor asked where the dump came from
and told to place it under bios/. A ZIP no list or profile declares as a
whole is now read member by member: each is named held, declared and not
held, or not recognised, and the contributor is asked for the files
unzipped. A declared archive (a MAME or FBNeo set) is never opened.
2026-11-06 08:57:15 +01:00
Abdessamad Derraz 320db75490 fix: check_freshness aligns details after remotes
render padded the local value but not the remote one, so in an area
where some rows carry a remote value their detail started fifteen
columns later than the rows without one. The remote value is padded to
its column, and a row without one leaves the same blank span.
2026-11-06 08:56:32 +01:00
Abdessamad Derraz 8b7e5ceef4 fix: refresh_stale knows the romfolders area
check_freshness gained a romfolders area and refresh_stale did not:
its rows were dropped from a saved report without a word, and --only
romfolders was refused. A rom-folder pin is moved by hand, so the area
is listed for a human, a test holds the two area lists equal, and the
--only help names every area.
2026-11-06 08:56:14 +01:00
Abdessamad Derraz ea58f6f5ce fix: refresh_stale --dry-run honours --json
--dry-run printed its text plan and returned before --json was read,
so a caller parsing the output as JSON failed. With both flags the plan
is printed as JSON: planned jobs with their commands and follow-ups,
the rows left to a human, and the count ignored.
2026-11-06 08:55:51 +01:00
Abdessamad Derraz bf206d5b42 feat: slots names who declares a collided path
A collision line named two files and neither the system declaring each
nor the one the pack ships, so the reader went back to the platform
file. find_collisions walks the systems in the builder's sorted order,
takes the builder's preferred declaration, and the line and the JSON
now name each system and the file packed. RetroDECK's bios/disk.rom
reads as the CoCo ROM packed, the PC-88 one left out, as its manifest
holds.
2026-11-06 08:55:26 +01:00
Abdessamad Derraz a343be0da1 fix: validate_pr json says failed, not new
The JSON report put "outcome": "new" on every file without another
outcome, so a path that does not exist, or a file that failed a check,
was listed as a new contribution. A failed file now reads "failed".
2026-11-06 08:54:23 +01:00
Abdessamad Derraz a1ddee8e5e fix: a lock reader refuses a missing directory
artifact_lock created its directory even for a shared reader, so a
mistyped path given to --verify-packs or release_record was left as a
directory holding a .lock, and split_pack created the parent before
printing a bare errno. A reader now raises ArtifactDirectoryMissing,
which run_cli prints on one line, and split_pack checks its target
before taking the lock.
2026-11-06 08:54:06 +01:00
Abdessamad Derraz 19dddedff6 fix: a data lock wait names its holder
A refresh blocked on a data directory lock printed only that another
run held it, then waited with no limit. The line now names the lock
file and the lsof command that shows its holder, and for a refresh,
the generate_pack --no-refresh-data and --offline runs that read the
cache without waiting.
2026-11-06 08:53:26 +01:00
Abdessamad Derraz 47e0e8d120 fix: database paths must be committed or ignored
The manifest guard compared repo_path with database.json only, so a
git mv under bios/ committed without regenerating either left both
naming the old path and every test passed. A new test requires each
database path to be tracked or gitignored, and a manifest entry whose
repo_path git neither holds nor ignores now fails as untracked.
2026-11-06 08:52:58 +01:00
Abdessamad Derraz 1b7ea4212f fix: tests write temporary files under tmp/
264 TemporaryDirectory() and mkdtemp() calls without dir= wrote to the
/tmp tmpfs, and a full tmpfs failed modules no change touched. The test
package now points tempfile and TMPDIR at tmp/tests, and sets
GIT_CEILING_DIRECTORIES there so git run in a scratch directory does
not find the checkout above it. The package test asserts both.
2026-11-06 08:52:31 +01:00
Abdessamad Derraz 46a4a26a54 fix: keep bundled game data out of missing
verify counted game data installed with its emulator as missing, which cross_reference files as bundled: 24 of Batocera's 427. Bundled firmware already left the lists, and the line naming it never printed. Every bundled entry is now counted on that line, required firmware named and the rest under --verbose.
2026-11-06 08:50:29 +01:00
Abdessamad Derraz 9a805fd240 docs: name the methods that count an export
adding-a-platform listed what a format can hold and none of what counts the written file. lands, states, fills, clears, names_anew and withheld are what seven exporters override so the summary matches the diff; a test reads the page for every one an exporter overrides.
2026-11-06 08:50:29 +01:00
Abdessamad Derraz eb99131a6e fix: stop advising --save-dir a list cannot serve
RetroPie, ROCKNIX, AmberELEC, EmuELEC and RomM keep their save directory in no fixed place, and their lists leave those files out with no hash or address. The run said to name the folder with --save-dir, and said it again when --save-dir was given. It now says the list holds none of them.
2026-11-06 08:45:03 +01:00
Abdessamad Derraz d1d94f5979 fix: keep card and closed-folder tests apart
The --dest refusal for a path under a file stopped two fixtures: muOS's MUOS marker was written as a file with the BIOS folder below it, and the folder that cannot be created was a path under a file. The marker is now a folder and that case uses a folder this user cannot write.
2026-11-06 08:44:22 +01:00
Abdessamad Derraz 8aa1a5fdad fix: a release asset absent on disk stays held
The previous commit counted a name with no resolved copy as not held, which also dropped release assets the builder holds while no copy is on disk (test_install_selection AHeldFileTheInstallerCannotFetch). Only the case-insensitive name match now leaves out copies in another emulator's tree, as resolve_local_file does.
2026-11-06 08:42:53 +01:00
Abdessamad Derraz 63be8cc3bc fix: find tab-indented batocera-systems entries
The entry pattern required four spaces of indent. EmuELEC indents 23 systems with one tab, so render appended each again whole and the published export carried 21 duplicated keys, the md5 fills only in the second copy. Any indent opens an entry now, and an entry written again keeps its own. The EmuELEC export is regenerated: one copy of each system, the amiga entry also gaining the eight WHDLoad Kickstarts the profiles added since 2026-10-22.
2026-11-06 08:39:40 +01:00
Abdessamad Derraz 420162c79b fix: rank uncheckable profiles first in triage
--strict fails on a profile no declared repository answers for, yet --triage scored it 0 under the clean profiles and left it out of the count to review. It now weighs as a pin to set again and is counted.
2026-11-06 08:37:36 +01:00
Abdessamad Derraz 23d878c526 chore: stub the export cache check in a test
test_a_targeted_export_reads_a_full_model runs pipeline.py --offline
--with-export, which is now refused when .cache/upstream-native/ is
empty, as in any fresh checkout. The test is about the truth directory
the export reads.
2026-11-06 08:37:19 +01:00
Abdessamad Derraz 328bfbe53b chore: run the catalog row test from its file
The class sat after unittest.main(), so running test_doc_pages.py on its
own skipped it.
2026-11-06 08:37:16 +01:00
Abdessamad Derraz 63fafaf0f9 fix: read teknoparrot and simple64 from mirrors
teknogods/TeknoParrotUI and simple64/simple64 answer 404 on the API and the web, and neither organisation lists them now. simple64's README at the pin archives the project for gopher64. Each profile declares upstream_gone and a fork that keeps its default branch at the analysed commit: thecrixux/TeknoParrotUI master is b75872ad, simple64-vault/simple64 main is d8c969c7. realign_prose reads a source declared gone through that mirror, which unblocks five teknoparrot notes citations that anchor unchanged.
2026-11-06 08:37:12 +01:00
Abdessamad Derraz 4377cbb2af fix: another tree's casing does not hold a file
dosbox-x's et4000.bin matched only pcem's et4000.BIN. resolve_local_file, asked as dosbox-x, refuses another emulator's tree in its case-insensitive step, so the pack and installer leave it out, while the gap report and verify's core files resolved without the asker and counted it held. Both now resolve as the profile, and the manifests of Batocera and KNULLI no longer flag it held.
2026-11-06 08:36:42 +01:00
Abdessamad Derraz a0ecca0024 fix: collect abbreviated prose citations
A location written without its file, (:507), reads in the file named last in the scalar, either cited or named as 'path:' before a list. A dotless file cited with its directory (tools/errorgen:20) is a citation too. devilutionx, corsixth, starship, plastic, eden-nightly, theforceengine, lightspark and higan_sfc gain 33 citations that bump and realign now follow. Two read that way pointed at another file and now name theirs: devilutionx assets.cpp:511-518, corsixth app.lua:1591.
2026-11-06 08:33:16 +01:00
Abdessamad Derraz b3f8f64395 fix: refuse options a listing does not read
Two listing options printed one list, and --list-platforms, --list-targets and --list-emulators printed the same output with --system, --region or --core as without, code 0. A run now prints one list, and an option a listing ignores is refused.
2026-11-06 08:33:01 +01:00
Abdessamad Derraz 21dd14c851 fix: a verified large file keeps its bytes
fetch_large_file verified a cached copy and returned its name, which another run replaces when the release serves a re-upload: the caller then packed bytes it never verified. The returned path is now a hard link under .cache/large/.by-sha1/<sha1>/, kept only if it holds the inode that was hashed. Revisions no name holds are dropped a day later. restore_large_files hashes each copy before moving it into place.
2026-11-06 08:30:55 +01:00