Commit Graph
4436 Commits
Author SHA1 Message Date
Abdessamad Derraz de8f20f409 fix: count listed systems in the --system hint
A mistyped --system said '--list-systems lists all 579' and the listing
prints 532: the hint counted every spelling across profiles, and
system_index merges the ones that differ in case or separator before
printing. Both verify.py and generate_pack.py now pass that count to
suggest_names, whose listed parameter exists for this case.
2026-11-06 14:55:40 +01:00
Abdessamad Derraz c9e5ea67e7 fix: count a missing core file once in the summary
The closing summary of verify --all grouped platform to names: sfiii2a.zip,
missing on 15 platforms, was printed 15 times on a 1 060-character line and
counted 24 files for 10 distinct ones. It now groups by file and names the
platforms by the shorter side (every platform but BizHawk, MinUI, MiSTer
FPGA).
2026-11-06 14:54:38 +01:00
Abdessamad Derraz 82bccd9dae fix: refuse Python 3.8 in every build script
Only generate_pack.py and pipeline.py checked the interpreter. verify.py,
slots.py, validate_pr.py, cross_reference.py and the probe tools started
under 3.8 and stopped halfway on a str.removeprefix AttributeError. The
check now lives in common.py, which every build script imports, and in
core_probe.py and region_check.py, which do not. generate_site.py imports
common before siterender, which raised a TypeError at load under 3.8.
2026-11-06 14:52:50 +01:00
Abdessamad Derraz c4ae8d9c6e fix: type pack helpers as ZipFile or PackRecord
Six helpers that write a pack member declared zipfile.ZipFile while every
call site also passes the PackRecord the pipeline builds emulator and
narrowed packs with. mypy reported the 19 call sites as arg-type errors.
With the union in the signature a helper that calls writestr or write
without its isinstance branch is a union-attr error in CI, instead of an
AttributeError in the pipeline's record pass.
2026-11-06 14:49:19 +01:00
Abdessamad Derraz b56ec21728 feat: check type hints in CI against a baseline
Every signature carries annotations and nothing read them. mypy 1.18.2
over install.py and scripts/ reports 993 errors today, among them the
six pack helpers typed zipfile.ZipFile that every caller also hands a
PackRecord. The errors sit in mypy-baseline.txt and the schema job fails
on any error outside it, or on an entry that no longer occurs. Third-party
packages are left out so the result does not depend on the machine: 6 s
locally, run twice and on 3.12 and 3.13 with the same errors.
2026-11-06 14:47:20 +01:00
Abdessamad Derraz a99682beeb chore: merge origin/main 2026-11-06 13:46:16 +01:00
Abdessamad Derraz b8d9b2a90c fix: send searched words only as file names
A search that found nothing sent any short text, which a name could be.
Its words now go out only as a file name or a hash, every other search
is counted as "other" without them.
2026-11-06 13:11:15 +01:00
Abdessamad Derraz 1a29b802ce chore: merge removal of the archived status 2026-11-06 13:10:29 +01:00
Abdessamad Derraz 8100d671f9 chore: merge origin/main 2026-11-06 13:02:20 +01:00
Abdessamad Derraz c1b0149a81 chore: merge site insights 2026-11-06 12:56:13 +01:00
Abdessamad Derraz b23840bc21 feat: add site insights report
Search Console queries by page, file names searched against the
collection, GoatCounter totals and GitHub traffic in one Markdown report.
2026-11-06 12:55:57 +01:00
Abdessamad Derraz ebe9745e7f feat: count visits without cookies
Pages, referring domains and a few actions (searches that found
nothing, downloads, missing addresses, "Did this page help?") are sent
to GoatCounter. Nothing is stored on the device, and nothing is sent
under Do Not Track or Global Privacy Control. wiki/privacy.md says what
is sent.
2026-11-06 12:55:57 +01:00
Abdessamad Derraz f55e473183 chore: one clause per sentence in a comment 2026-11-06 12:55:13 +01:00
Abdessamad Derraz dafbff26d1 chore: merge emuelec targets 2026-11-06 12:17:17 +01:00
Abdessamad Derraz 44565a07bb feat: read the EmuELEC device in the installer
EmuELEC's scripts/image writes COREELEC_DEVICE to os-release (v4.8, line
189), as a GameForce image shows. The installer reads it the way it does
for AmberELEC and applies install/targets/emuelec.json. MiSTer FPGA takes
EmuELEC's place in the test of a platform without targets.
2026-11-06 12:09:09 +01:00
Abdessamad Derraz 5e90a455e4 feat: add EmuELEC hardware targets
The v4.8 release publishes four images (Amlogic-ng, GameForce,
OdroidGoAdvance, OdroidM1, read from its asset names). The metapackage
(packages/sx05re/emuelec/package.mk:8-77) builds the S922X cores, dolphinSA
and mame only on Amlogic-ng, Amlogic-no, RK356x and OdroidM1, and removes
duckstation on the handhelds with a sed that replaces the first match only:
pcsx_rearmed, written twice, keeps its 64-bit build. Each package is mapped
to its cores by the .so or program its install step copies, PKG_ARCH is
applied as scripts/build:28-30 does, and es_systems.cfg is edited per
device as emuelec-emulationstation/package.mk:114-143 does. Amlogic-ng and
OdroidM1 list 124 cores, the handhelds 117.
2026-11-06 12:09:05 +01:00
Abdessamad Derraz 14cf0a13f2 chore: keep the platform status from returning 2026-11-06 11:52:51 +01:00
Abdessamad Derraz 4d3f589ef4 refactor: drop the archived platform status
RetroPie was the only platform with status: archived, which kept it out
of verify --all, slots, the pipeline and list_platforms unless
--include-archived was typed. release_pack already decides which packs a
release publishes and the installer serves every platform, so the status
set nothing apart. The field, its schema entry and the flag are gone, and
RetroPie is covered by every full run like any other platform.
2026-11-06 11:52:34 +01:00
Abdessamad Derraz 2e98be9366 fix: accept the bundled line in the report excerpt
verify now prints a line for files installed with their emulator, and
reading-reports.md quotes it. The shapes the excerpt may hold did not
list it.
2026-11-06 11:10:33 +01:00
Abdessamad Derraz 25b11fa497 chore: merge site fixes, fourth pass 2026-11-06 10:33:56 +01:00
Abdessamad Derraz d0c65dca22 chore: merge pack and verify fixes 2026-11-06 10:28:27 +01:00
Abdessamad Derraz c0e4bd8379 chore: merge resolver fixes, fourth pass 2026-11-06 10:28:27 +01:00
Abdessamad Derraz a29b836730 chore: merge installer fixes, fourth pass 2026-11-06 10:28:14 +01:00
Abdessamad Derraz 1396350795 fix: count build file trees as pinned
check_source_paths read the trees profiles pin from source and upstream
only, so the two platform trees the record now holds for pin_build
files read as orphaned and the record as stale.
2026-11-06 10:14:02 +01:00
Abdessamad Derraz 8a14a1f75c docs: regenerate README counts
et4000.bin is no longer counted held for dosbox-x on Batocera and KNULLI.
2026-11-06 09:58:47 +01:00
Abdessamad Derraz e491ff1bc6 chore: merge tooling fixes, fourth pass 2026-11-06 09:50:16 +01:00
Abdessamad Derraz 3672c6de4a docs: list the AmberELEC target scraper 2026-11-06 09:46:37 +01:00
Abdessamad Derraz f9f7ad3b92 docs: quote the bundled line in the verify excerpt
reading-reports quotes a RetroDECK run, which now prints the bundled count and 30 missing game data instead of 38. The comment introducing the bundled names also loses its semicolon.
2026-11-06 09:46:37 +01:00
Abdessamad Derraz 508dc4d039 docs: list the new test modules
test_twin_declaration_status and test_verify_data_dir_notice join the testing guide table, and test_large_file_cache calls unittest.main after its last class.
2026-11-06 09:44:28 +01:00
Abdessamad Derraz 2c60b5f20d docs: refresh_stale names pin_installer plainly
The docstring line naming pin_installer.py read as a clause after a
semicolon, which the prose test refuses.
2026-11-06 09:43:07 +01:00
Abdessamad Derraz 21f4122c83 fix: collisions keep the platform's order
Walking the systems sorted reordered the declarations doubled.py
reports from, so the export report named the CoCo reader first. The
declarations keep the platform's order again, and the packed file is
picked from the sorted walk the builder makes.
2026-11-06 09:43:06 +01:00
Abdessamad Derraz 79a21ba1f1 feat: name the files in emulator search snippets
A search for gba_bios.bin or scph5500.bin landed on a page described as
"It reads 4 files, none of them required". The snippet now names the
console and the files, required first, in the longest form that fits.
2026-11-06 09:28:03 +01:00
Abdessamad Derraz adcfa283fb docs: keep the FAQ under its word ceiling
The --core answer added 14 words to a page held at 450. It now names the
command in as many words as the line it replaced, and the --region line
goes: what --region keeps is on the installer page.
2026-11-06 09:14:47 +01:00
Abdessamad Derraz 856cb5618f fix: link a build file only where it is recorded
Every source link on the site leads to a path the source record saw at
a commit. The Build file row linked pin_build files unchecked, eight of
them at commits the record did not hold. build_source_paths now records
the build files read at a commit, and the row links only those: a tag
can move and stays plain text. The record run also picked up two cited
paths it had not seen before (pom2 CMakeLists.txt, pcem configure.ac).
2026-11-06 09:14:42 +01:00
Abdessamad Derraz 462b61a4bc refactor: compute slot choices once per manifest
generate_manifest called slot_choices, and pack_overrides called it again with the same five arguments. pack_overrides takes the choices when the caller holds them. Measured on generate_pack.py --all --manifest --offline: 48 calls and 7.0 s in slot_choices before, 24 calls and 4.3 s after (wall 45.7 s and 42.4 s, one run each on a loaded host). The manifests written are identical.
2026-11-06 09:11:37 +01:00
Abdessamad Derraz ef6607302d chore: rebuild the site when siteinputs changes
Every page module reads its inputs from scripts/siteinputs.py, so the
deploy workflow names it with the other site modules.
2026-11-06 09:11:02 +01:00
Abdessamad Derraz 9633ce3dd4 refactor: normalize each system id once
SelectionIndex.build and system_group called _norm_system_id 1.7 million times per manifest run for about a thousand ids, each through the manufacturer prefix loop. The function reads only its argument and two constant tables, so its answer is cached. Measured on the call volume of one manifest run (630 ids, 1,686,618 calls): 1.53 s uncached, 0.07 s cached. The end-to-end manifest run moves inside its own noise on this host (45.9 to 50.6 s across four runs).
2026-11-06 09:08:09 +01:00
Abdessamad Derraz 333f1ac3ca fix: resume a cut data archive
A data archive cut mid-download was deleted on every failure: the retry and the next run started from zero, while the run said nothing was lost. The archive is now kept beside its tree with the validator its server sent, and the rest is asked for with Range and If-Range. Checked against the release: fbneo-samples stopped at 16 MB, the next run downloaded 34.6 MB of 50.5.
2026-11-06 09:04:52 +01:00
Abdessamad Derraz 43cb0f84ab feat: add AmberELEC hardware targets
AmberELEC's metapackage adds LIBRETRO_CORES_EXTRA (bsnes, Mesen, melonDS, geolith) only on RG552, RG353 and RG503, and deletes those cores from es_systems.cfg on every other device (packages/amberelec/package.mk:23-31, 45-58 at a21975bf). The new target scraper reads the Makefile's device images, that test and es_systems.cfg the way the platform scraper does: RG351P, RG351V and RG351MP list 125 cores, the other three 136. The installer reads the device from COREELEC_DEVICE in os-release (scripts/image:168).
2026-11-06 09:04:51 +01:00
Abdessamad Derraz 55e2c8328e feat: pack readme names spellings left out
A pack folded for Windows and macOS keeps one spelling of each case twin. Extracted on a Steam Deck or Linux, the core asking for the other spelling finds nothing, and only the installer added it. Where the platform also runs on a filesystem that keeps case, the README now lists each spelling left out beside the one it holds: 12 for EmuDeck and 10 for RetroArch, the manifest's case_variants.
2026-11-06 09:02:51 +01:00
Abdessamad Derraz ada7738fa7 feat: pin_installer writes the installer pins
Every change to install.py meant a sha256sum pasted by hand into
install.sh and install.ps1, and the tests that caught a stale pin
failed on two bare hashes. scripts/pin_installer.py writes both pins
(--check only reports), the pin tests name it, check_freshness reads
the pins through it and refresh_stale runs it for a stale pin row.
2026-11-06 09:02:18 +01:00
Abdessamad Derraz 4efcb3da64 fix: group core files under their pack path
On a platform that writes bios/ in front of its destinations (RetroDECK, AmberELEC, EmuELEC), a core extra was grouped, dropped and held under its bare destination while the platform's declaration of the same file used bios/<name>. Beetle PSX's scph5501.bin was kept by the --region japan pack while the declaration was dropped and the installer dropped the path. extra_region_key keys every extra with the platform's declared root, in the builder, the manifest selection, verify's replay and the omissions. The same key gives the declared entries the systems of the core files at their path: every --system case of RetroDECK differed between pack and installer before, none does now.
2026-11-06 09:00:06 +01:00
Abdessamad Derraz 8107564631 fix: gearcoleco warns on an unknown CRC32
At be1a3716 find_firmware rejects a candidate by size alone (platforms/libretro/libretro.cpp:811-816), and an OS-7, EOS or SmartWriter image whose CRC32 differs is logged and loaded (lines 875-876, 928-935). The three entries keep crc32 in validation with on_mismatch warn, so another 8 KiB OS-7 revision such as colecoa.rom no longer reads as rejected.
2026-11-06 08:59:15 +01:00
Abdessamad Derraz 725325f0a1 fix: announce a data tree at its archive size
A tarball tree was announced at its extracted size: yaps2 at 6.5 MB for a 21.2 MB download, Cemu's game profiles at 9.9 KB for 21.4 MB. refresh_data_dirs records the size of each archive it downloads (a tree current from before is measured, not refetched), the install lists carry it as archive_size, and the plan, the menu and the data event count it.
2026-11-06 08:59:10 +01:00
Abdessamad Derraz 01ce92232d docs: say how an uploaded ZIP is read
CONTRIBUTING and the site's contributing page now say that a ZIP no
list declares as a whole is read file by file, and that the files go in
unzipped.
2026-11-06 08:58:40 +01:00
Abdessamad Derraz 5407eafe5b feat: validate_pr reads an undeclared ZIP's files
A card's BIOS folder zipped to upload it was hashed as one file and
reported not recognised, its contributor asked where the dump came from
and told to place it under bios/. A ZIP no list or profile declares as a
whole is now read member by member: each is named held, declared and not
held, or not recognised, and the contributor is asked for the files
unzipped. A declared archive (a MAME or FBNeo set) is never opened.
2026-11-06 08:57:15 +01:00
Abdessamad Derraz 320db75490 fix: check_freshness aligns details after remotes
render padded the local value but not the remote one, so in an area
where some rows carry a remote value their detail started fifteen
columns later than the rows without one. The remote value is padded to
its column, and a row without one leaves the same blank span.
2026-11-06 08:56:32 +01:00
Abdessamad Derraz 8b7e5ceef4 fix: refresh_stale knows the romfolders area
check_freshness gained a romfolders area and refresh_stale did not:
its rows were dropped from a saved report without a word, and --only
romfolders was refused. A rom-folder pin is moved by hand, so the area
is listed for a human, a test holds the two area lists equal, and the
--only help names every area.
2026-11-06 08:56:14 +01:00
Abdessamad Derraz ea58f6f5ce fix: refresh_stale --dry-run honours --json
--dry-run printed its text plan and returned before --json was read,
so a caller parsing the output as JSON failed. With both flags the plan
is printed as JSON: planned jobs with their commands and follow-ups,
the rows left to a human, and the count ignored.
2026-11-06 08:55:51 +01:00
Abdessamad Derraz bf206d5b42 feat: slots names who declares a collided path
A collision line named two files and neither the system declaring each
nor the one the pack ships, so the reader went back to the platform
file. find_collisions walks the systems in the builder's sorted order,
takes the builder's preferred declaration, and the line and the JSON
now name each system and the file packed. RetroDECK's bios/disk.rom
reads as the CoCo ROM packed, the PC-88 one left out, as its manifest
holds.
2026-11-06 08:55:26 +01:00