Commit Graph
1203 Commits
Author SHA1 Message Date
Abdessamad Derraz a07821d810 feat: list files only an emulator install provides
A file installed with its emulator is already on the user's machine and the packs leave it out. bundled_only.py lists the collected copies nothing else needs: no platform declares them, no entry of any build needs them, no install manifest ships them.
2026-10-20 06:05:20 +02:00
Abdessamad Derraz ddc7432825 feat: packs leave out what the emulator installs
A file or tree installed with the emulator is already on the user's machine, so packs, install manifests and verify's pack count leave it out. bundled is scoped by mode like any entry: Dolphin's Sys and PPSSPP's assets ship with the standalone releases, not with the libretro cores, so they now follow the libretro build only and still reach RetroArch packs. PrimeHack, standalone only, ships its own Sys.
2026-10-20 06:04:10 +02:00
Abdessamad Derraz c589ed49b8 fix: the site says 1 file, not 1 files
Counts were printed with a fixed plural and the site said "1 files" 234 times, "1 issues" and "1 entries" too. A shared helper picks the noun, and the generated-pages prose test refuses the form.
2026-10-20 05:57:05 +02:00
Abdessamad Derraz 976b164536 feat: the gaps table names the accepted dump held
A hash mismatch row printed the hash the emulator accepts and stopped there, although the collection holds that exact dump for most rows. A column now links the held copy.
2026-10-20 05:55:41 +02:00
Abdessamad Derraz 49c7bacc46 fix: a file copy is listed on its folder's page
The database keeps one path per content, so a copy filed under another manufacturer was on no page of its own: the Channel F BIOS sat under Arcade > FBNeo and Fairchild had no system page. Every folder holding a copy now lists it, read from the full paths by_path_suffix records, and the index counts each content once.
2026-10-20 05:54:57 +02:00
Abdessamad Derraz 77ae9a7f74 fix: every accepted md5 is its own search word
Recalbox lists accepted md5s joined by commas, and platform pages printed the list as one code span. The site search took it for a single word, so 198 accepted values could not be found. Each value now has its own span.
2026-10-20 05:52:59 +02:00
Abdessamad Derraz dec0fdc163 feat: each citation names the code that asks
A profile that reads a libretro core and the original emulator cited both trees alike, and readers could not tell whether the core or the emulator asked for a file. A cited path now opens with source or upstream, the repository its lines are in. Emulator code a port carries is attributed upstream when the original holds the same value on the cited line.

profile_sync --attribute-refs writes the words from where the lines are found and follows a word already written. The site links each citation to that repository's pin, labels it libretro core or upstream emulator, and says on each file card whose code asks for it. A recale keeps the word, and mame_ref_audit reads the tree it names.
2026-10-20 05:52:13 +02:00
Abdessamad Derraz adea35e12a fix: a recale keeps the repository word of a ref
A source_ref written as `stella src/a.c:120` names the repository its lines are in. The path is read without the word, and --rebase-refs wrote the new span back without it, losing the attribution.
2026-10-20 05:46:09 +02:00
Abdessamad Derraz 10a1f475b0 feat: prose citations link to the pinned source
Notes, file notes and platform details carried file:line citations as plain text although profile_sync keeps each on the profile's pin. A citation now links when the profile reads one repository and the path is rooted or already used by a structured source_ref. Platform detail source_refs go through the same renderer as file cards.
2026-10-20 05:44:08 +02:00
Abdessamad Derraz 01b310f724 fix: release_notes options say what they do 2026-10-20 05:12:01 +02:00
Abdessamad Derraz 17e35d1af9 feat: file cards name the packs that install them
A card named only the platforms whose own list declares the file. A RetroBat user looking up scph5501.bin found every platform but RetroBat, whose pack installs it for six cores. The cards now read the install manifests and add each pack that places the file, with the destination it gives it.
2026-10-20 04:56:23 +02:00
Abdessamad Derraz 91403aa149 fix: site systems match under every spelling
Platform pages matched emulators to systems by the exact id, so the RetroArch page listed none for sega-mega-drive (genesis_plus_gx says sega-megadrive), and listed every emulator of a system instead of those the platform runs. Emulator cards linked only ids a platform uses: 2,057 of 4,215 system lines had no link, now 327, the rest having no page. Ids are compared as _norm_system_id reads them, and a system only profiles name is placed by the files they file under it.
2026-10-20 04:55:16 +02:00
Abdessamad Derraz 810396941d feat: write the release notes from the record
The pack table, its parts sentence and the install commands were typed by hand from figures release.json already held. A copied cell or an old collection total at the top is how issue 81 happened. release_notes.py renders them and appends the hand-written account of what changed.
2026-10-20 04:51:56 +02:00
Abdessamad Derraz 8bd2d319b2 feat: the pipeline runs the schema check first
PR validation runs validate_schemas.py --source-only and the pipeline did not, so a profile with an unquoted numeric core name passed every step and failed CI on push. It is step 0 now, skipped where jsonschema is not installed.
2026-10-20 04:50:04 +02:00
Abdessamad Derraz 77389236e6 feat: record a release with no genuine copy
Paid Android emulators announce versions that circulate only rebuilt or re-signed. The profile stays on the newest unmodified build, and version_source.no_genuine_copy names the release searched for, so check_freshness stops flagging it until the page announces a later one.
2026-10-20 04:49:24 +02:00
Abdessamad Derraz 8d295273a5 fix: validate_pr checks the size a name expects
1024 bytes named scph5501.bin, a name every declaration sizes at 524288, were reported "File size OK" and offered a .variants slot at the repository root. The review now warns when no declaration of the name accepts the size, and places a variant beside the folder that holds the name.
2026-10-20 04:48:07 +02:00
Abdessamad Derraz 1ae2854474 fix: a truth hash conflict is reported once
Generating the model for every platform met the same two GameIndex.yaml declarations once per platform, and check_freshness opened on six identical warnings.
2026-10-20 04:46:03 +02:00
Abdessamad Derraz 50790a2988 fix: the pipeline summary says what does not fail
check_buildbot does not fail the run, yet the summary printed it FAILED right above Pipeline COMPLETE. It now prints WARN and a line saying the difference does not fail the run. The total is aligned under the step durations.
2026-10-20 04:44:30 +02:00
Abdessamad Derraz eaa40d7384 fix: offline target lists are skipped, not ok
Offline, check_freshness labelled every scraped target list static and counted it OK although nothing was compared. Other sections say SKIPPED offline, and now this one does too.
2026-10-20 04:43:45 +02:00
Abdessamad Derraz 2bb4dc9187 fix: the download bar fits the terminal
A 2 GB part drew a 98-column line, and on an 80-column console every redraw wrapped and left a copy behind. The bar now shrinks to the width and goes below ten columns of room.
2026-10-20 04:43:12 +02:00
Abdessamad Derraz 3f029578af fix: a scraper that cannot start says what to do
Scrapers resolve their tag or branch head in the constructor, which ran outside the try in scraper_cli: a refused GitHub call ended in a traceback. The error is now printed with the next step, setting GITHUB_TOKEN when the API refused an anonymous call.
2026-10-20 04:36:24 +02:00
Abdessamad Derraz 18e093182d docs: name the registries behind assets and dedup
The guides said files over 50 MB become release assets and that RPG Maker and ScummVM were the trees kept out of dedup. .gitignore decides the first, four committed files are over 50 MB, and NODEDUP_DIRS holds eleven trees.
2026-10-20 04:35:02 +02:00
Abdessamad Derraz 77c4857a46 fix: a site preview writes its own config
generate_site --docs-dir sent the pages elsewhere but always rewrote the tracked mkdocs.yml, outside the lock taken on that folder, with the navigation of the work in progress. Two previews into two folders both wrote it. Another folder now gets <name>.mkdocs.yml beside it, docs_dir set and its paths rebased, which mkdocs builds with -f.
2026-10-20 04:30:03 +02:00
Abdessamad Derraz 282f8d5d0a feat: watch the release page of off-forge profiles
About forty profiles describe projects published outside any forge (Altirra, ePSXe, DraStic, mednafen, xroar). profile_sync skipped them as an unsupported host and check_freshness reported the same UNKNOWN line every run, so a new release adding a firmware went unseen. A profile can now name the page announcing its releases and a pattern for the version: --check-version reads it, and check_freshness --profiles calls the profile stale when the page announces another version than core_version.
2026-10-20 04:28:27 +02:00
Abdessamad Derraz c07f0d9205 refactor: parse emulator profiles once per process
The alias-free profile set was parsed separately from the full one, so a process asking for both read the 534 profiles twice, 1.6 s of a site build. It is now a filter over the full set.
2026-10-20 03:37:57 +02:00
Abdessamad Derraz 7edd438af1 fix: refuse a target no platform has
With --all, an unknown --target skipped every platform that has targets and ran the others (BizHawk, MiSTer) unfiltered, so verify, slots and the pipeline exited 0 on a typo. A platform without targets is now left out like the rest, and a target no platform has is refused.
2026-10-20 03:27:41 +02:00
Abdessamad Derraz c7c2f260f3 fix: check a forced rehash against the last one
The pipeline rebuilds the database with --force, which discarded the previous digests and wrote whatever the CPU computed, on a host that has given an incorrect SHA-1 under load. The forced run still rehashes every file, but a digest that contradicts the cache for an unchanged mtime and size is computed again, and the two readings that agree win, with a warning naming the file.
2026-10-20 03:24:55 +02:00
Abdessamad Derraz d5560244b4 feat: list the OpenDingux, Miyoo and webOS targets
The buildbot publishes .so cores for OpenDingux (mips32, its beta and RS-90 builds), RetroFW, MiyooCFW and webOS (armv7a, aarch64), 7 to 206 cores each. None was a target, so a handheld or TV user could not narrow a pack or an install to the board. They are scraped now, with opendingux, retrofw, miyoo and webos as aliases.
2026-10-20 02:28:14 +02:00
Abdessamad Derraz 2fc0547a50 fix: check the banned vocabulary in tracked text
The project bans a short list of marketing words in what it publishes, and no test read for them: the site's platform table described Recalbox's interface with one. The line now says what Recalbox does, and test_prose_style fails on any of those words in a tracked file.
2026-10-20 02:25:12 +02:00
Abdessamad Derraz b402a8b917 docs: install markdown with the test dependencies
CONTRIBUTING and the testing guide installed pyyaml and jsonschema only, while the CI test job also installs markdown. On a contributor's machine the wiki heading test skipped and the run read OK. Both name it now, and a test compares them to the packages the tests import.
2026-10-20 02:24:36 +02:00
Abdessamad Derraz 392a5d68cc fix: strict sync fails a profile it cannot check
profile_sync skipped a profile whose repository no longer answers, which carries no pin, or whose declared death the forge contradicts, with empty counts. --strict counted refs to review only, so the CI job meant to stop an unchecked new profile passed a mistyped repository. Such skips now fail a strict run. An unsupported host, a declared takedown and a profile without refs stay accepted.
2026-10-20 02:23:56 +02:00
Abdessamad Derraz 7bfe68b777 fix: an unanswered search is not a missing file
auto_fetch caught every network error in its repository and archive.org steps and reported the file as needing a community contribution. Offline or rate-limited, --create-issues asked the public for files the tool never looked for, and the errors counter stayed at 0. A 404 is still an answer. Any other failure now marks the file as not fully searched and keeps it out of issues.
2026-10-20 02:23:17 +02:00
Abdessamad Derraz 9fa1f45e92 fix: raise on a registry missing a platform file
list_registered_platforms skipped a registered platform whose YAML was missing and returned an empty list without a registry. verify --all --strict, every --all build, the CI matrix and the site then ran over fewer platforms and exited 0, and install.py kept serving the stale manifest. Both now raise and name the file.
2026-10-20 02:22:37 +02:00
Abdessamad Derraz c42c756b4a fix: group packs on everything that shapes them
group_identical_platforms fingerprinted each file by destination, sha1 and md5. Two platforms whose lists differed only in required, zipped_file, size or crc32, or in how the frontend checks a file, would have shared the representative's pack and verify result. The fingerprint now carries every field of every entry and the platform's verification_mode and hash_rule. No current platform is regrouped.
2026-10-20 02:14:14 +02:00
Abdessamad Derraz 23a999153d fix: read PR hashes the way the resolver does
load_profile_hashes and load_platform_hashes read hash fields by hand: crc32 was never indexed and a comma list of sha256 stayed one key. A contribution a profile names by crc32 alone (clk's Electron os.rom, among 626 entries) or by one of mesence's four sha256 was reported as referenced nowhere. Both indexes now read through parse_hash_list, and a crc32 counts where no stronger hash is declared, bounded by the declared size.
2026-10-20 02:12:54 +02:00
Abdessamad Derraz ac47faf7f2 fix: the pack check applies the frontend's rule
The integrity step compared a member's md5 by hand and ignored hash_rule, so under RomM's rule (size, then any of md5, sha1 and crc32) it would refuse a revision the builder rightly shipped on its sha1. The rule now lives in digest_verdict, which the path check and the pack member check both call. The member's archive digest is computed without the path cache, whose key a buffer's reused address made collide.
2026-10-20 02:11:53 +02:00
Abdessamad Derraz 52b2970e1c fix: report a size the export removes
When the truth takes over an entry without a size of its own, the format writes it without the platform's size: System.dat's neocdz.zip lost size 953078 and no counter said so. Removed sizes are now counted and named, and the summary counts read 1 size, 2 sizes.
2026-10-20 01:25:25 +02:00
Abdessamad Derraz 19a7128ec6 fix: compare md5 when keeping Recalbox text
The attribute pattern took letters only, so md5 never entered the comparison and an element whose digest the truth corrects kept Recalbox's text. A test written before the change caught it. The RetroDECK fixture now names the core that reads its addition, as the placement rule requires.
2026-10-20 01:25:25 +02:00
Abdessamad Derraz c5fd079b78 fix: keep the text of unchanged Recalbox elements
Every <bios> element the model held was rendered again, so attribute order, alignment, a line break inside md5 and the case of a digest changed in 222 elements that stated the same thing. The diff deleted 451 lines for 136 corrections. An element whose attributes say what Recalbox wrote now keeps Recalbox's own text: 175 lines, the corrections.
2026-10-20 01:23:29 +02:00
Abdessamad Derraz cef6d943e2 fix: a Batocera note stays with its own item
Notes were keyed by file path, so a comment before one of the items reading an archive came back before every item reading it: bk's '# MAME' twelve times. A comment after the list's opening bracket matched neither rule and was dropped (enterprise's ep128emu-core). Notes are now keyed by file and archive member, written once, and a head-line comment goes with the first item.
2026-10-20 01:22:27 +02:00
Abdessamad Derraz e74d182d32 fix: a RetroDECK addition needs its component core
Additions took the component their system's entries sit in, whatever core reads them. 104 armsx2 and yaps2 files went into PCSX2's manifest, beetle_psx and rustation dumps into DuckStation's, a directxbox image into xemu's, all counted as added. An addition now lands only where a core reading it is that component's emulator: 132 additions instead of 581.
2026-10-20 01:21:38 +02:00
Abdessamad Derraz 132c69a4b2 fix: a ROCKNIX runner follows its build
ROCKNIX registers DuckStation twice, as the libretro core under retroarch and as duckstation-sa under its own emulator. The exporter looked a standalone runner up as a core name, so standalone DuckStation's 98 additions were written for users of the libretro core, PS2 images included, and standalone ARMSX2 was refused. A libretro runner now answers with the retroarch pair, a standalone with its emulator's own.
2026-10-20 01:20:45 +02:00
Abdessamad Derraz 7aa9b3fcaa fix: the gap CLI sets bundled files apart
cross_reference printed 20 runtime components bundled with their emulator as [MISSING] and counted them among the files to source. The collection leaves them out by design, as verify and the gaps page already say. They are now listed as [BUNDLED] and counted on their own line.
2026-10-20 01:18:09 +02:00
Abdessamad Derraz c6db200beb fix: the global gap report keeps standalone files
With no platform named, cross_reference passed no standalone cores, so every dual profile was laid out as libretro only. The CLI and the site's gaps page left out 133 files that Batocera and RetroBat run standalone and that no platform declares (eka2l1 90, dolphin 21, cemu 11). Both now pass the union of every platform's standalone cores.
2026-10-20 01:17:35 +02:00
Abdessamad Derraz 185529ce75 fix: pack only what BizHawk's database names
EmuHawk finds every firmware by SHA1 against FirmwareDatabase.FirmwareRecords, and its cores are its own builds. The registry and the scraper both listed libretro cores whose systems overlap BizHawk's, so the pack shipped Clock Signal, O2EM, FBNeo, VICE and Citra files into Firmware/ that EmuHawk never reads. Both lists are now empty, as MiSTer's are: the database alone decides.
2026-10-20 00:33:38 +02:00
Abdessamad Derraz 7e23815545 fix: reports hold the data trees they read
generate_readme, generate_site and slots resolve against data/ but took no hold when run on their own, as the documented commands do. A refresh swapping a tree mid-read then gave counts from half of it. Each now holds the trees shared, like verify and the builder.
2026-10-20 00:32:26 +02:00
Abdessamad Derraz d1ff233990 fix: profile_sync reports in English
The summary line and the markdown table header were the only French strings among the project's command output.
2026-10-20 00:29:57 +02:00
Abdessamad Derraz 7eb4a36711 fix: step 5 compares the cores' share of a pack
The consistency check compared the platform's own files only. It now also compares verify's core files and game data with the builder's count of files taken from the cores.
2026-10-20 00:28:40 +02:00
Abdessamad Derraz 89050ea5d9 fix: count core files as the pack places them
The report counted claims, not files: several cores claim one path (rom.key by three Amiga cores, openbios.bin with and without a region) and the builder ships one file there. RetroArch read 1130 core files, 48 more than its pack. Game data ignored the in_pack marker, so RomM read 351 game data files in a pack that holds none. verify now takes both counts from its replay of the builder's placement, one per pack path.
2026-10-20 00:28:40 +02:00
Abdessamad Derraz 6bfd73f4f8 fix: a truncated tree does not prove a file gone
When the forge truncates the HEAD tree, the rename search cannot see every path, yet a bare filename it did not find was reported GONE. eden's fs_paths.h and path_util.cpp exist under src/common/fs/. Such a citation is now AMBIGUOUS with the reason, as the realign path already says unclear and never absent.
2026-10-20 00:15:39 +02:00