Commit Graph
1501 Commits
Author SHA1 Message Date
Abdessamad Derraz dc3bf41d26 chore: merge the copy rewrite 2026-10-22 14:36:32 +02:00
Abdessamad Derraz 009413944e docs: open the deep pages on their question
Each reference and contributor page now says what it answers and gives
a real case before its rules: scph5501.bin for the check modes and the
data model, DuckStation's entry for profiles, RomM for a platform. The
verification example is the RetroBat d64_1.rom case the gaps page lists
instead of a scph5501.bin case the data cannot produce.

Emulator pages open on what the emulator reads, from the profile, and
keep their technical notes folded. System pages say what a card holds.
The data model shows a real database entry and links the one
resolution chain. Gaps tables name what they count ("Discrepancy",
"Where the files come from"), and the core inventory is titled Cores
by platform so cross-reference keeps one meaning.
2026-10-22 14:31:57 +02:00
Abdessamad Derraz 8b66041b5a chore: fail on a wiki page left out of the nav
A wiki page is published as soon as it exists, but one missing from the nav is reachable only by a link, and MkDocs reports that at info level. getting-started.md stays out on purpose.
2026-10-22 14:28:21 +02:00
Abdessamad Derraz 70d761682c docs: plain copy on the newcomer pages
The Start here pages ran 460 to 2,100 words and the first command of
Get your files sat 254 words down. Each page now opens on its action,
with short sentences and one fact per block. The facts a newcomer does
not act on moved one level down: the legal analysis to wiki/legal.md,
the project terms to wiki/reference-terms.md, the annotated run and the
installer failures to the installer page, collection figures to a new
Explore index.

tests/test_site_copy.py holds the budgets on the generated pages, and
the testing guide states them with their reasons.

Four statements the repository contradicts are rewritten: prboom.wad
is compiled into PrBoom (emulators/prboom.yml), Batocera reports a
mismatch as UNTESTED (batocera-systems checkBios), 13 hle_fallback
entries are required, and the gaps headline splits missing firmware
from game data and unread entries.
2026-10-22 14:23:54 +02:00
Abdessamad Derraz 3164a815bc fix: confirm the backup holder after writing
Between the check and the creation of the run folder, retrobios-backup could be swapped for a link. The holder's device and inode are confirmed unchanged once the run folder exists. Directory file descriptors would close the window entirely but do not exist on Windows; the remaining window is the instant between that confirmation and the move.
2026-10-22 14:10:30 +02:00
Abdessamad Derraz a2b98af88e fix: never reuse a backup folder of another run
When every dated name in retrobios-backup already existed, the loop ended on the last one and wrote into it. It now raises, so the next location is tried and the file stays in place if none works. A holder another user owns is refused on systems with user ids.
2026-10-22 14:09:19 +02:00
Abdessamad Derraz 14f042c540 chore: merge the MAME required rule 2026-10-22 14:07:13 +02:00
Abdessamad Derraz cdaf4f4ea1 fix: never write a backup through a link
A BIOS folder can sit on a share other people write to. A retrobios-backup that is a link is no longer written through (the backup goes inside the BIOS folder instead), run folders are created for the user only and carry a marker, and --restore only takes a folder holding that marker, walking it without following links. Beginner pages label their links to the installer page as details.
2026-10-22 14:07:13 +02:00
Abdessamad Derraz d60fe814ed feat: read required from romload for MAME sets
A set whose every ROM is NO_DUMP or ROM_OPTIONAL only warns in
handle_missing_file, so the machine runs without it. The parser now
keeps undumped and optional ROMs for that decision, the merge and the
release profiles write required false for such sets, and the text
patcher rewrites the required line it used to leave. A curated false
over a set with dumped ROMs is kept, except over no contents, which is
what an undumped set leaves once a later release dumps it.

pSxMAME builds mamedev/mame itself at 0.133 and left the current
refresh, which would have moved it onto 0.289 refs. Release profiles
keep their citations double-quoted on regeneration.
2026-10-22 14:06:18 +02:00
Abdessamad Derraz 5d868ee19b chore: merge the installer backups 2026-10-22 14:00:25 +02:00
Abdessamad Derraz 468b5e1d3e feat: keep a replaced BIOS file beside its folder
A file in place whose contents differ (another revision, a personal
dump) was overwritten and lost. It is now moved to
retrobios-backup/<date-time>/ beside the folder written to, under the
same path, before the verified copy replaces it. Beside, not inside:
DuckStation, SwanStation and Rustation read every file of the BIOS
folder and BizHawk walks its firmware folder recursively.

Identical files are never kept. --dry-run counts what would be kept,
the summary and the summary event name the folder, files and bytes,
and --restore FOLDER puts the files back. Data trees and standalone
copies keep their replaced files the same way.
2026-10-22 14:00:00 +02:00
Abdessamad Derraz 274fc37673 chore: merge the site weight and speed work 2026-10-22 13:46:50 +02:00
Abdessamad Derraz be5be996db feat: hold the site to page and index budgets
validate_site.py fails a page over 1 MB of HTML, a search index over
8.5 MB (1.7 MB gzip), a skipped heading level, a table header without
text, a script from another host and a diagram drawn at runtime. The
limits sit above what the site now measures (0.88 MB, 7.2 MB, 1.43 MB),
each with its reason, and each has a test.
2026-10-22 13:13:55 +02:00
Abdessamad Derraz 36ac7b0cd6 feat: split the heavy site pages
A manufacturer page over 500 KB of Markdown becomes a summary: its
largest consoles move to pages of their own, cut into parts when one
console is still too large, and every console keeps its heading there.
MAME's 1,564 cards move to four pages, its page keeping a row per entry
with the card's old anchor. RetroDECK's arcade list gets its own page.
site.js follows a moved card by content or by its row.

The largest page drops from 10.5 MB to 0.88 MB. Emulator groups of the
navigation open by initial (65 KB to 41 KB before the content). File
cards are level 3 under a level 2, and the hook keeps the table of
contents of card pages to level 2: platform pages no longer list every
system id. Key/value cards name their columns. The sweep runs before
the decoration, which kept every page no generator wrote any more.
2026-10-22 13:13:52 +02:00
Abdessamad Derraz 2d2addd6d2 fix: align the suite with the site layout
The redesign moved the technical questions to technical-faq.md, made the Get your files rows generated from each platform key and reduced getting-started.md to links. Tests now read those places: the pack-row test reads the rendered page. The new site scripts go through run_cli and refuse abbreviated options. The three maintenance image tools are exempt by name from the build dependency rule: their output is committed and the build only reads it. siteguides.py now triggers the site deploy, and the recipe counts match the store.
2026-10-22 13:11:07 +02:00
Abdessamad Derraz 33041b6bd1 feat: find a file by any hash, out of the search
Hashes, link lists, install commands, member tables and long reference
tables leave the site search: the index drops from 21.0 MB (4.88 MB
gzip) to 6.8 MB (1.33 MB gzip). Names and prose stay searchable.

files/?hash= takes a SHA1, MD5, CRC32 or SHA256 and reads one shard of
api/v1/lookup/ (about 22 KB gzip, was 675 KB) listing the collection
card, the platforms that declare the value and the profiles that check
it. files/?sha1= still opens the card, and a sha1- anchor missing from
a page is looked up there, so a card can move between pages.

Excluded blocks are section elements: the search plugin compares
elements by tag, so a div ended at the first nested div.
2026-10-22 12:46:40 +02:00
Abdessamad Derraz 42a0498d20 chore: run site tools through run_cli
The image and diagram scripts accepted abbreviated options and bypassed
run_cli. The FAQ checks read faq.md only, while the questions on hashes
and checks moved to technical-faq.md.
2026-10-22 12:46:34 +02:00
Abdessamad Derraz baf38ffbee refactor: compute coverage once, index proofs
EntryCategories asked every core in turn: 3.8 million proofs for 7,430
entries. ProofIndex looks each key up once, and verify --all --json is
byte-identical (24 s to 17 s). generate_site and generate_readme now share
one coverage, cached in tmp/coverage-cache keyed by its inputs, so the
README step drops from 25 s to 3 s.
2026-10-22 12:23:26 +02:00
Abdessamad Derraz f67f4ead1b chore: align the merged site with its checks
The two redesign branches and the archive repair met in the testing guide and the prose rules: list test_archive_recipes, drop three clause semicolons, write the forbidden glyphs as escapes in the image test. README and mkdocs.yml regenerated from the merged generator.
2026-10-22 12:04:42 +02:00
Abdessamad Derraz f39bcebb22 chore: merge the site visual system 2026-10-22 12:00:19 +02:00
Abdessamad Derraz db1d679e4e chore: merge the site funnel 2026-10-22 11:59:37 +02:00
Abdessamad Derraz 6353ee53df feat: lay the site out as a reading funnel
Five tabs from Start here to Contribute, a level on every page, a device picker on Get your files that lists every registered platform, and platform pages that open on where the files go, the command, the pack and how to check. FAQ and troubleshooting keep the questions asked before installing, the technical ones move to their own page, and getting started points to the pages that took its content.
2026-10-22 11:50:53 +02:00
Abdessamad Derraz 1b9ea86e1b feat: draw site images from data, host them
scripts/sitediagrams.py writes the explainer, install flow, verification modes, MinUI card, pipeline and data layer diagrams as inline SVG from the registry, the platform files, the install manifests and the pipeline step list. A page asks for one with an image comment, which the decoration pass resolves. The two mermaid diagrams of the architecture page are replaced, so no page loads mermaid from unpkg.

Platform logos and system icons are copied once by fetch_site_images.py with their source and owner recorded, and the build rewrites the remote links to the copies. validate_site refuses an image from another host. Three registry logos named a personal avatar instead of the project's (RomM, ROCKNIX, MiSTer FPGA) and MyMinUI's was a GitHub identicon, now dropped.

One 1200x630 sharing card per section, a 404 page with search and the two entry pages, and a list of screenshots that need a real device.
2026-10-22 11:45:47 +02:00
Abdessamad Derraz 67a4eb9c64 chore: merge the device set repair 2026-10-22 11:19:47 +02:00
Abdessamad Derraz 57cbff781a chore: merge the remaining platform core profiles 2026-10-22 11:15:32 +02:00
Abdessamad Derraz 15b962185a chore: admit duckstation-legacy in two checks 2026-10-22 11:15:04 +02:00
Abdessamad Derraz 1abb8f678f feat: check archives against their set recipes
A held archive whose name has a recipe fails when it carries three or more members outside every recipe of that name. Exempt: containers a platform pins by md5, members a platform pins by zipped_file, merged clones in their own folder, NAOMI device_ref ROMs, samples and artwork.
2026-10-22 11:11:02 +02:00
Abdessamad Derraz 87a4e8e1b6 fix: send unreleased platforms to the installer
A release recorded without a platform's pack linked its button to that release, which serves no pack for AmberELEC, EmuELEC, KNULLI, MinUI, NextUI or MyMinUI. The cell now names the release and points to the installer, which reads the file lists on main. Core folder links resolve each core name to the profile page that describes it (mednafen_vb is beetle_vb).
2026-10-22 11:04:25 +02:00
Abdessamad Derraz 4501601769 fix: show where each platform's pack goes
Platform pages labelled base_destination, the root the platform's own list writes from, as the BIOS path, so EmuDeck read bios/ where the pack goes into ~/Emulation/bios/. The row now gives the registry's extract_to, the folder the README and the pack guide name. A platform that names no file (MinUI and its forks) showed 0/0 coverage and an empty systems table; its page now lists each core folder with the files the pack puts in it.
2026-10-22 10:56:35 +02:00
Abdessamad Derraz 45c442b413 chore: merge the pro1000 set and clone check 2026-10-22 08:16:15 +02:00
Abdessamad Derraz 36f7d60d88 feat: fail on a declared archive with no copy
The layout check skipped any archive it found no copy of, and it looked
copies up by name only, so 23 MAME device sets held through the clone map
were never checked and pro1000.zip was never reported absent.
2026-10-22 08:15:45 +02:00
Abdessamad Derraz 3276754102 fix: end profile history at a deleted namesake 2026-10-22 08:15:15 +02:00
Abdessamad Derraz 3e6da220b8 chore: name the remedy for an unheld pin 2026-10-22 08:06:29 +02:00
Abdessamad Derraz e1aac191e7 fix: skip platforms with no native list
export_native counted MinUI, NextUI and MyMinUI as failures although the registry marks them native_list: none: their code names no BIOS file, so there is no list of theirs to correct.
2026-10-22 07:52:08 +02:00
Abdessamad Derraz 1263f71e78 chore: merge the bios set layouts 2026-10-22 07:02:57 +02:00
Abdessamad Derraz d768a11cbd feat: check every declared archive layout is held 2026-10-22 06:33:52 +02:00
Abdessamad Derraz cd7d993cac feat: settle entries by their system's cores
A system that lists the cores it runs is settled by those cores alone, so an unprofiled core of another system no longer leaves its entries open. Readers whose core runs the entry's system are named first, and cores that only share the file name are listed as such.
2026-10-22 06:30:56 +02:00
Abdessamad Derraz 4bc90cf06e chore: merge the elec core names 2026-10-22 06:29:22 +02:00
Abdessamad Derraz 9648b9fc3b feat: read elec ports and per-system cores
EmuELEC's ports system has one emulator, `ports`, the dispatch to a
script ports.yaml generates. The scraper now reads each port's engine
(the libretro core handed to emuelecRunEmu.sh with -C, else port_exe),
so devilutionx, sonic2013, soniccd, sonic-mania, reminiscence and
rigelengine are examined. Standalone emulators keep the spelling the
launch code dispatches on instead of invented lowercase forms, and
media players and runtimes go to the overrides' remove_cores.

Each system of the AmberELEC, EmuELEC and KNULLI lists now carries the
cores its ES list offers it, in menu order, under `cores:`.
2026-10-22 06:28:24 +02:00
Abdessamad Derraz a0b27d612d fix: align tests with the ten-value pack result
Two test modules still unpacked nine values from verify_pack_against_platform. The category evidence joined readers with semicolons, which the prose rule refuses on generated pages, and three comments used clause semicolons.
2026-10-22 06:23:15 +02:00
Abdessamad Derraz 8ba65de1ca chore: merge the game data filter 2026-10-22 06:19:55 +02:00
Abdessamad Derraz 48a9a61819 feat: let the installer skip game data
install.py --no-game-data leaves out the files and omissions the manifest
marks game data, in an install, --check and --dry-run, and names them with
their category. The --events summary gains left_out, one group per
category with every destination. A list without categories is refused.
2026-10-22 06:19:10 +02:00
Abdessamad Derraz 8b952968d6 feat: add a --no-game-data pack narrowing
The install manifest marks each game data file "category": "game_data":
a platform entry only game data profile entries read, or a core file its
profile declares game data. Firmware carries no category. Every manifest
says "categorized": true so the installer can refuse an older list.

generate_pack.py --no-game-data cuts those files from the platform list
and the core profiles before the build, names each one with the cores that
read it as game data, and tags the pack _NoGameData through _narrowings.
verify.py and the pack check take the same cut, so --verify-packs and the
report describe the pack the builder wrote. Modes without a platform
(--emulator, --system alone, --from-md5) and --all-variants refuse it.
2026-10-22 06:19:05 +02:00
Abdessamad Derraz a75ea05b86 feat: document what settles each category
Each missing platform entry now carries its evidence: the profile entries that read it with their source reference, the cores of its system examined without finding a reader, or why the question stays open. verify --verbose prints it, --json carries it and the platform pages show it.
2026-10-22 05:43:06 +02:00
Abdessamad Derraz f66adfdcbc feat: categorize missing platform entries
A platform list can name a game's data files or a file no core it runs opens. Verify gives each missing entry a category read from the profiles of the platform's cores: firmware, game data or unread. Only firmware gaps carry a severity and fail --strict; the others are counted on their own line, and the pack check no longer counts them as pack errors. Missing firmware stays a conformance error.
2026-10-22 05:38:57 +02:00
Abdessamad Derraz b7b407698f fix: count files the collection lacks apart
The pack check failed a pack for a declared file nothing in the collection resolves to, though the builder cannot ship it and verify already reports it missing. Such destinations are now counted as not in the collection; a held file absent from the pack stays an error.
2026-10-22 04:28:43 +02:00
Abdessamad Derraz e05844e611 chore: merge the handheld port 2026-10-22 04:12:09 +02:00
Abdessamad Derraz 841017c887 chore: merge the minui family platforms 2026-10-22 04:09:17 +02:00
Abdessamad Derraz 43bdf83973 feat: add minui, nextui and myminui platforms
minarch hands every core SDCARD_PATH/Bios/<TAG> as its system directory
and opens no file in it. The scrapers read each pak's launch.sh (core,
tag, devices), each device's core makefile (the project a library is
built from: MyMinUI's dosbox_libretro.so is dosbox-pure) and the minarch
statement, at a pinned commit. The installer recognises the card from
.system/version.txt and refuses a folder inside it.
2026-10-22 04:04:36 +02:00
Abdessamad Derraz e7edae0101 chore: merge android retroarch folder per build
# Conflicts:
#	install.ps1
#	install.sh
2026-10-22 04:00:16 +02:00