mirror of
https://github.com/Abdess/retroarch_system.git
synced 2026-11-06 01:53:56 -06:00
docs: drop hand counts from reference pages
This commit is contained in:
Abdessamad Derraz committed
1 parent
b2e313d5f1
commit
c92b8eb7da
4 files changed
+60
-61
No files matched your search
+49
-51
@@ -12,9 +12,9 @@ contents. The pack builder joins the three and ships every file a platform's
|
||||
emulators need, and the verifier checks the result the way the platform itself
|
||||
would.
|
||||
|
||||
Example: libretro's `System.dat` names 526 RetroArch files. The cores RetroArch
|
||||
runs load 7,650 more, and the RetroArch pack carries both (counts of
|
||||
[What the packs do not hold](../gaps.md) on 2026-10-22).
|
||||
Example: libretro's `System.dat` names the files RetroArch checks. The cores
|
||||
RetroArch runs load many more, and the RetroArch pack carries both.
|
||||
[What the packs do not hold](../gaps.md) gives the current counts.
|
||||
|
||||
## Directory structure
|
||||
|
||||
@@ -26,30 +26,41 @@ emulators/ one YAML profile per core/engine
|
||||
platforms/ one YAML config per platform (scraped from upstream)
|
||||
_shared.yml shared file groups across platforms
|
||||
_registry.yml platform metadata (logos, scrapers, status, install config)
|
||||
_guides.yml the guide text of each platform's page
|
||||
_data_dirs.yml data directory definitions (Dolphin Sys, PPSSPP...)
|
||||
targets/ hardware target configs + _overrides.yml
|
||||
provenance/ dump-catalog snapshots (redump, no-intro, tosec)
|
||||
recipes/ romset recipes read from MAME and FBNeo DATs
|
||||
exports/ each platform's own file, corrected, with a report of the changes
|
||||
measurements/ records of real runs, one JSONL file per emulator
|
||||
scripts/ all tooling (Python and pyyaml, jsonschema for schema checks)
|
||||
scraper/ upstream scrapers (libretro, batocera, recalbox...)
|
||||
scraper/targets/ hardware target scrapers (retroarch, batocera, emudeck, retropie)
|
||||
scraper/targets/ hardware target scrapers (one *_targets_scraper.py per platform)
|
||||
exporter/ native format exporters (batocera, recalbox, emudeck...)
|
||||
install/ JSON install manifests per platform
|
||||
targets/ JSON target manifests per platform (cores per architecture)
|
||||
install.py the installer, with install.sh and install.ps1 to start it
|
||||
ports/ launchers that run the installer on handhelds
|
||||
automation/ jobs.yml, the unattended jobs scripts/autorun.py runs
|
||||
data/ cached data directories (not BIOS, fetched at build)
|
||||
schemas/ JSON schemas for platform and emulator YAML (checked in CI)
|
||||
tests/ test suite with synthetic fixtures
|
||||
wiki/ hand-written documentation sources
|
||||
docs_assets/ logos, icons and images the site uses
|
||||
website/ Astro Starlight project; pages, data and navigation
|
||||
are generated into it (gitignored, rebuilt in CI)
|
||||
_mame_clones.json MAME parent/clone set mappings
|
||||
database.json file index built from bios/ (SHA1 primary key)
|
||||
derivable.json files a held dump already contains, from provenance_report.py
|
||||
release.json files, sizes and assets of each pack of the last release
|
||||
allowed_signers public key that signs SHA256SUMS.txt
|
||||
dist/ generated packs (gitignored)
|
||||
.cache/ hash cache and large file downloads (gitignored)
|
||||
```
|
||||
|
||||
`docs/` is never edited by hand: `generate_site.py` rebuilds it from
|
||||
`platforms/`, `emulators/`, `database.json`, and the `wiki/` sources.
|
||||
Documentation changes belong in `wiki/` or in the generator.
|
||||
The generated pages in `website/` are never edited by hand: `generate_site.py`
|
||||
rebuilds them from `platforms/`, `emulators/`, `database.json`, and the `wiki/`
|
||||
sources. Documentation changes belong in `wiki/` or in the generator.
|
||||
|
||||
The site search splits each page at its headings and sends a result to the
|
||||
heading above the match. Every file card is therefore a level 3 heading with
|
||||
@@ -59,26 +70,26 @@ not the top of the page. On pages of cards (systems, emulators, platforms) the
|
||||
table of contents stops at level 2, and a page of cards under one or two
|
||||
headings has none (`scripts/sitestarlight.py`).
|
||||
|
||||
The browser downloads and indexes the whole search index before it shows a
|
||||
first result, so the index holds names and prose only. Hashes, lists of other
|
||||
pages, install commands, member tables and long reference tables sit in
|
||||
`data-search-exclude` blocks. Those blocks are `section` elements: the search
|
||||
plugin compares elements by tag, and a `div` stopped excluding at the first
|
||||
`div` closed inside it. A hash goes to [Find a file](../files/index.md), which
|
||||
The search is Pagefind, which loads its index in pieces: the words typed pick
|
||||
the chunks it fetches. The index holds names and prose only. Hashes, lists of
|
||||
other pages, install commands, member tables and long reference tables carry
|
||||
`data-pagefind-ignore`. A hash goes to [Find a file](../files/index.md), which
|
||||
reads one shard of `api/v1/lookup/` named by the first two characters of the
|
||||
value.
|
||||
|
||||
A page has a budget. A manufacturer page over 500 KB of Markdown becomes a
|
||||
summary: its largest consoles move to pages of their own, cut into parts when
|
||||
one console alone is over the budget, and every console keeps its heading on
|
||||
the summary. An emulator with more than 400 entries (MAME) lists them in a
|
||||
table on its page, each row holding the old anchor of the card, and the cards
|
||||
move to pages of 400. A platform system whose list is over 200 KB moves to a
|
||||
page of its own. A card anchor (`#sha1-...`) that a page no longer holds is
|
||||
looked up by content, so a link cited before a split still opens the card.
|
||||
`validate_site.py` fails a page over 1 MB of HTML, a search index over 8.5 MB
|
||||
(1.7 MB compressed), a skipped heading level, a table header without text, a
|
||||
script from another host and a diagram drawn at runtime.
|
||||
A page has a budget, set by constants in `scripts/generate_site.py`. A
|
||||
manufacturer page over `SYSTEM_PAGE_BUDGET` becomes a summary: its largest
|
||||
consoles move to pages of their own, cut into parts when one console alone is
|
||||
over the budget, and every console keeps its heading on the summary. An
|
||||
emulator with more than `EMULATOR_CARDS_PER_PAGE` entries (MAME) lists them in
|
||||
a table on its page, each row holding the old anchor of the card, and the cards
|
||||
move to pages of `EMULATOR_CARDS_PER_PART`. A platform system whose list passes
|
||||
`PLATFORM_SECTION_BUDGET` moves to a page of its own. A card anchor
|
||||
(`#sha1-...`) that a page no longer holds is looked up by content, so a link
|
||||
cited before a split still opens the card. `validate_site.py` fails a page over
|
||||
`MAX_PAGE_BYTES` of HTML or `MAX_PAGE_ELEMENTS` elements, a search index over
|
||||
`MAX_PAGEFIND_TOTAL_BYTES`, a skipped heading level, a table header without
|
||||
text, a script from another host and a diagram drawn at runtime.
|
||||
|
||||
## Data flow
|
||||
|
||||
@@ -105,10 +116,9 @@ truth.py generates diff_truth.py export_native.py
|
||||
emulator profiles scraped platform own file, updated
|
||||
```
|
||||
|
||||
Pipeline runs all steps in sequence: DB, provenance report, data dirs,
|
||||
MAME/FBNeo hashes, verify, packs, install manifests, target manifests,
|
||||
consistency check, pack integrity, README, site. See [tools](tools.md)
|
||||
for the full pipeline reference.
|
||||
`pipeline.py` runs every step in order, from the schema check to the rendered
|
||||
site. [Adding a platform](adding-a-platform.md#step-8-validate-the-full-pipeline)
|
||||
lists the steps, and [tools](tools.md) the options.
|
||||
|
||||
<!-- image: pipeline -->
|
||||
|
||||
@@ -171,8 +181,8 @@ platform. Where the emulator source states something else, the profile and
|
||||
`_shared.yml` carry it.
|
||||
Another name for the same file is an `aliases:` entry in the profile, never a
|
||||
second entry in a platform YAML or in `_shared.yml`. A name a platform's own
|
||||
launcher reads, such as the 22 names Batocera's DuckStation generator tries
|
||||
per region, describes the platform and lives in its `launchers:` records (see
|
||||
launcher reads, such as the names Batocera's DuckStation generator tries
|
||||
for each region, describes the platform and lives in its `launchers:` records (see
|
||||
[launchers](adding-a-platform.md#launchers)).
|
||||
|
||||
Some entries in a platform YAML are not the platform's own: files a
|
||||
@@ -327,28 +337,16 @@ How files enter `bios/`, and why.
|
||||
| `embedded` (default) | file is in the `bios/` directory, included in packs |
|
||||
| `external` | file has a `source_url`, downloaded at pack build time |
|
||||
| `user_provided` | user must provide the file (instructions included in pack) |
|
||||
| `large_file`, `release` | file is an asset of the `large-files` release, listed in `.gitignore`, fetched at build time |
|
||||
|
||||
## Verification severity
|
||||
## Verification severity and discrepancies
|
||||
|
||||
How missing or mismatched files are reported:
|
||||
|
||||
| Mode | required + missing | optional + missing | hash mismatch |
|
||||
|------|-------------------|-------------------|--------------|
|
||||
| existence | WARNING | INFO | N/A |
|
||||
| md5 | CRITICAL | WARNING | UNTESTED |
|
||||
|
||||
Files with `hle_fallback: true` are downgraded to INFO when missing
|
||||
(the emulator has a software fallback).
|
||||
|
||||
## Discrepancy detection
|
||||
|
||||
When a file passes platform verification (MD5 match) but fails
|
||||
emulator-level validation (wrong CRC32, wrong size), a DISCREPANCY is reported.
|
||||
The pack generator searches the repo for a variant that satisfies both.
|
||||
If none exists, the platform version is kept and the discrepancy stays
|
||||
reported. A filename or destination never overrides a declared content hash
|
||||
during resolution. Whether a mismatch is packed is then decided by the
|
||||
platform's own verification mode.
|
||||
How a missing or mismatched file is reported, and what happens when the
|
||||
platform's check and the emulator's check disagree, follow the platform's
|
||||
verification mode: see the [severity matrix](verification-modes.md#severity-matrix)
|
||||
and [discrepancy detection](verification-modes.md#discrepancy-detection).
|
||||
A filename or destination never overrides a declared content hash during
|
||||
resolution.
|
||||
|
||||
## Security
|
||||
|
||||
@@ -408,7 +406,7 @@ not available on the target. Without `--target`, all systems are included.
|
||||
|
||||
## MAME clone map
|
||||
|
||||
`_mame_clones.json` at repo root maps MAME clone ROM names to their canonical parent.
|
||||
`_mame_clones.json`, beside the `bios/` tree it describes, maps MAME clone ROM names to their canonical parent.
|
||||
When a clone ZIP was deduplicated, `resolve_local_file` uses this map to find the canonical file.
|
||||
|
||||
## Install manifests
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ This is the most contested area, and the weakest part of the project's position.
|
||||
- **Keys come from consumer hardware.** They are extracted from retail devices their owners bought. *Chamberlain v. Skylink* (2004) held that a 1201 claim requires some nexus to copyright infringement, in a dispute over a garage-door opener. Later cases have narrowed its reach, and it is a weak foundation for a general post-sale right.
|
||||
- **Trade secret is not a live issue.** Keys embedded in millions of shipped devices and widely republished are not kept secret by reasonable measures.
|
||||
|
||||
### Recent firmware (Switch 19.0.0, PS3UPDAT, PSVUPDAT)
|
||||
### Recent firmware (Switch firmware, PS3UPDAT, PSVUPDAT)
|
||||
|
||||
- **Free download is not a licence.** Manufacturers publish these updates on open CDNs without authentication, which makes them easy to obtain but does not grant a right to redistribute them, and PlayStation firmware ships under an end-user licence agreement. The practical argument is that mirroring a file the manufacturer already gives away for free causes no identifiable loss, not that doing so is expressly permitted.
|
||||
- **Functional necessity.** Emulators cannot run the software these systems were built for without this firmware. That is why the files are collected. It is a statement of purpose, not a legal defence in itself.
|
||||
|
||||
@@ -22,7 +22,7 @@ everyday words (BIOS, emulator, core, hash) are in the [glossary](glossary.md).
|
||||
Every entry has a `source_ref`, the line of code that proves it, at the
|
||||
revision named by `source_commit` (the pin). Example:
|
||||
`emulators/duckstation.yml` lists `scph5501.bin` at
|
||||
`src/core/bios.cpp:47`, commit `4122fed9`.
|
||||
`src/core/bios.cpp:47`, read at its `source_commit`.
|
||||
|
||||
**Alias**
|
||||
: Another name the same emulator accepts for a file, listed under `aliases:`
|
||||
@@ -92,9 +92,9 @@ everyday words (BIOS, emulator, core, hash) are in the [glossary](glossary.md).
|
||||
|
||||
**hle_fallback**
|
||||
: A flag on a profile entry: the emulator has HLE for this file, so its
|
||||
absence is reported at INFO. It does not make the file optional: 13 of
|
||||
the 778 entries that carry it are `required: true`, among them Beetle
|
||||
PSX's `scph5501.bin`.
|
||||
absence is reported at INFO. It does not make the file optional: a few
|
||||
entries that carry it are `required: true`, among them Yabause's
|
||||
`saturn_bios.bin`.
|
||||
|
||||
**Variant**
|
||||
: Another version of a BIOS file (a revision, a region or another dump), kept
|
||||
@@ -117,7 +117,8 @@ everyday words (BIOS, emulator, core, hash) are in the [glossary](glossary.md).
|
||||
|
||||
**Storage tier**
|
||||
: Where a file comes from: `embedded` (in `bios/`), `external` (downloaded at
|
||||
build time) or `user_provided`.
|
||||
build time), `user_provided`, or `large_file` and `release` (an asset of
|
||||
the `large-files` release).
|
||||
|
||||
**Target**
|
||||
: A hardware architecture a setup runs on, such as `switch`, `rpi4` or
|
||||
|
||||
@@ -171,8 +171,8 @@ not a single digest gets a new `hash_rule` value, never a new mode.
|
||||
ROCKNIX derives its BIOS checker from Batocera's, so the logic has the same
|
||||
shape: `md5sum()` on the file, `checkInsideZip()` for archive entries, and an
|
||||
`altmd5` field that accepts a second hash for the same entry. Every entry the
|
||||
platform declares is mandatory in its own data, so the platform YAML marks all
|
||||
38 files `required: true` and none of them use `zippedFile`.
|
||||
platform declares is mandatory in its own data, so the platform YAML marks every
|
||||
file `required: true`, and none of them uses `zippedFile`.
|
||||
|
||||
### KNULLI, AmberELEC and EmuELEC verification
|
||||
|
||||
@@ -207,8 +207,8 @@ no required/optional distinction: an entry either matches or it does not.
|
||||
|
||||
Scope note: only the BiosDB entries are in scope. They carry an explicit `url`
|
||||
because MiSTer cannot host them, which is exactly what the user has to supply.
|
||||
The 208 `games/` files shipped by the `Distribution_MiSTer` repository carry no
|
||||
URL and install themselves, so they are not part of the pack.
|
||||
The `games/` files shipped by the `Distribution_MiSTer` repository carry no URL
|
||||
and install themselves, so they are not part of the pack.
|
||||
|
||||
|
||||
## SHA1 mode
|
||||
|
||||
Reference in new issue
Block a user