docs: put the project's working rules in the wiki

Principles, collection rules and the inverted model in architecture.md, research and citation rules in profiling.md, platform and scraper rules in their pages, tool conventions in tools.md, test and commit expectations in testing-guide.md, installer constraints in installer.md. Each rule sits where a reader looks for it, with its reason. Also fixes the stale release-job line, the suite command without -t ., and the unsourceable definition.
This commit is contained in:
Abdessamad Derraz committed 2026-10-21 04:03:31 +02:00
1 parent 392aacb207
commit 348c56e94c
9 files changed
+407 -10

No files matched your search

+11
View File
@@ -328,6 +328,13 @@ If a file already exists at the output path, the CLI preserves fields that the
scraper does not generate (e.g. `data_directories`, manually added metadata).
Only the `systems` section is replaced.
The YAML is the scraper's output and is never edited by hand: the next scrape
would undo the edit, and the file would stop describing the platform. Anything
an export has to write back (the platform's own system id, path, name) is
recorded by the scraper and never reconstructed later, because several of the
platform's systems can fold into one of ours. A rescrape can add systems, so
read `verify.py --platform` before committing the regenerated YAML.
Verify the result:
```bash
@@ -451,6 +458,10 @@ For platforms that only run on one target (e.g. RetroBat on Windows, RomM in the
browser), create a static YAML file in `platforms/targets/` instead of a scraper.
Set `target_scraper: null` in the registry.
Its target writes `cores: null`, which means every core the platform declares.
A core list written by hand there falls behind the scraped platform and drops
cores the pack ships.
## Step 7: Add install detection (optional)
The `install` section in `_registry.yml` tells `install.py` how to detect
+10
View File
@@ -211,6 +211,16 @@ System ID consistency matters for cross-platform operations. The same console mu
Scraped data reflects what the upstream declares, which may not match reality. The scraper's job is faithful transcription of upstream data, not correction. Corrections happen in the emulator profiles (source-verified) and in `_shared.yml` (curated). If a scraper meets a hash or filename that differs from the emulator source, log a warning and still include the upstream value. The divergence will surface during truth diffing.
A few rules keep that transcription honest:
- An entry the upstream file does not carry is marked: `shared_group`,
`mirror_of` or `curated`. Unmarked, it would be counted and exported as
something the platform declares.
- A scraper that reads nothing raises. An empty input (a refused API call, a
moved file) must not rewrite a platform or target file as empty.
- Names that YAML would turn into numbers or dates are written quoted, so the
next scrape produces the same file.
## Creating a target scraper
Target scrapers determine which emulator cores are available on each hardware
+4 -4
View File
@@ -331,7 +331,7 @@ The diff reports:
### Export to native formats
Rewrite a platform's own BIOS file, corrected. The output is the file its
Write a platform's own BIOS file, updated from the emulators' code. The output is the file its
maintainers keep, not our data in their syntax: hashes the truth can prove
are applied, entries the truth says nothing about are left alone, and files
the truth knows and the platform lacks are added.
@@ -342,11 +342,11 @@ python scripts/export_native.py --platform recalbox --fetch # es_bios.xml
python scripts/export_native.py --all --fetch # every platform, into exports/
```
The corrected files are committed under `exports/<platform>/`, and
The updated files are committed under `exports/<platform>/`, and
`exports/<platform>.json` names every change the run made. The site's
[updated platform files](../exports.md) page is built from those reports:
one row per platform, a download link, and each corrected hash, filled hash
and withheld correction listed in full.
one row per platform, a download link, and each updated hash, added hash
and change not applied listed in full.
Seven of the formats carry code as well as data: Batocera's and ROCKNIX's
scripts, EmuDeck's shell library, BizHawk's C# database, RetroDECK's
+128 -3
View File
@@ -1,5 +1,37 @@
# Architecture
## Principles
Every part of the toolchain follows from a few rules.
- **The emulator's source code is the authority.** A profile records what the
code opens, under which names, what it checks and whether it runs without
the file. Documentation, `.info` files and platform lists are starting
points to verify. When they disagree with the code, the code wins.
- **Every statement rests on a fact anyone can check.** A field, a deletion or
a wording names what settles it: a `file:line` at a pinned revision, a hash,
a command and its output. Where no fact settles a question, the answer is
"not verified" until one is found. A sentence on this site once said
Batocera matches md5 prefixes. Its script compares the full string
(`batocera-systems:1088`).
- **The collection serves the user.** A pack carries what the user has to
obtain to run their emulators. When something forces the user or a
contributor to work around the toolchain, the toolchain is changed rather
than the workaround documented.
- **Profiles document the emulator, not this repository.** A profile can be
taken into another project as is. No field refers to this repository's
folders, file names or database, nor to one platform's conventions. The
toolchain resolves a profile against the collection.
- **Other projects are peers.** A finding about another project says what
changes, never who erred, and comes with its evidence. Our data can be
wrong too, and an issue that shows it is as welcome as a finding sent
upstream.
- **An inconsistency found is fixed now.** A fix comes with a test that fails
without it, so the same mistake cannot return unnoticed.
- **Collect everything a working setup needs.** BIOS, firmware, keys, fonts,
boot and DSP ROMs, databases, data files. Age and licence are not criteria.
A file the collection lacks is an acquisition target, not an end state.
## Directory structure
```
@@ -109,6 +141,23 @@ even if the platform does not declare them. Files the platform declares but no
profile references are kept as well (flagged during cross-reference), since the
upstream may cover cases not yet profiled.
A platform YAML is regenerated by its scraper and never edited by hand: an
edit would be lost at the next scrape, and the file would stop describing the
platform. What the platform gets wrong is answered in the other two layers.
Another name for the same file is an `aliases:` entry in the profile, never a
second entry in a platform YAML or in `_shared.yml`. A name a platform's own
launcher reads, such as the 22 names Batocera's DuckStation generator tries
per region, describes the platform and lives in its `launchers:` records (see
[launchers](adding-a-platform.md#launchers)).
Some entries in a platform YAML are not the platform's own: files a
`_shared.yml` group injects (`shared_group`), copies the RomM scraper mirrors
to a twin system (`mirror_of`) and additions the libretro scraper makes
(`curated`). `native_layer()` removes them wherever the question is what the
platform itself declares: the export, the truth diff and the site's
comparison tables. Counted as the platform's, they were written into
System.dat as if libretro had declared them.
```mermaid
graph TD
PY[Platform YAML<br/>scraped from upstream] --> PG[Pack generation]
@@ -124,6 +173,25 @@ graph TD
style ZIP fill:#2d333b,stroke:#adbac7,color:#adbac7
```
## The inverted model
Packs are built today from what each platform declares, completed by what the
cores need. The direction the project works toward is the inverse: generate
each platform's file from what the emulators' code needs, file by file, and
offer it to that platform's maintainers as a source they can check and adopt.
The [updated platform files](../exports.md) page publishes these files, every
change traceable to code, presented as suggestions. Keeping a BIOS list in
step with every emulator release is a large amount of work, and the files are
written to make it lighter.
An entry a platform declares is never removed from its file on the strength of
a missing match. When this was measured, an entry the generated data did not
pair was still read in 1189 of 2776 cases, by an alias, a hash, a directory
entry or a core outside the resolved set. A removal needs positive proof:
every core of the system profiled, each with a reference showing the closed
set of names it opens. Until then the entry is listed as a removal candidate
with the cores examined (see [export_native.py](tools.md#export_nativepy)).
## Dump-catalog provenance
Snapshots of the dump-preservation catalogs (Redump, No-Intro, TOSEC) live in
@@ -182,6 +250,64 @@ the bytes, for both the platform and the emulator pack paths. Its fixture
holds a romset, because without one the comparison never reaches the rebuild
path and passes while the real packs still move.
## Collection rules
How files enter `bios/`, and why.
- **Variants keep every layer served.** A different dump under an existing
name goes to `.variants/`. The primary carries the name the emulator's
source uses, and a dump a platform pins under an older name stays a variant,
so each layer receives the bytes it checks. A destination that already
exists is something a platform depends on: a `segaboot.gcm` that Batocera
requires was once overwritten by the dump RetroDECK requires, and is now
kept beside it.
- **One content, one file.** A byte-identical copy under another name is
removed, and the name it carried becomes a profile alias. Trees addressed by
path keep their equal files, since equality is part of their layout: those
folders are `NODEDUP_DIRS` in `scripts/dedup.py`.
- **Archives the project builds are TorrentZip.** `build_torrentzip()` makes
an archive's bytes depend on its contents alone, so anyone can rebuild it.
An upstream romset whose MD5 a platform pins stays verbatim in `.variants/`.
A member renamed by hand in an existing archive gives bytes that match no
distributed romset and that nobody can regenerate.
- **A pack carries what the user must obtain.** A file the emulator installs
itself (`bundled: true`) is not kept once nothing else needs it, and step 4d
of the pipeline fails on such a copy. Its sha1 and size are written into the
profile first, so the entry keeps the file's identity.
- **A platform's older revision needs the core's own claim first.** When a
platform pins an older revision of a data file, collecting it makes the
platform's hash the only content proof at that destination, and a core
profile without a hash then receives the old file. The profile first
declares the hash of the file at its own `source_commit`
(`find_pinned_revision.py` finds the revision behind a pinned md5).
- **A transformation of a held dump is collected by reproducing it.** A byte
swap, a permutation, a concatenation or a published patch is replayed from
its source and checked against the expected hash, and the profile states
what the file derives from.
- **A dump says what it is.** The version string inside a ROM names its model,
whatever folder it arrived in. Dumps once filed for one handheld announced
another model in their own strings.
- **Game data lives under the tree its engine reads.** Engines open generic
names in their own folders: `pak0.pak` is Quake II's under `baseq2/` and
Half-Life's under `valve/`. Each game's files are stored under that tail so
they resolve by path, and `tests/test_game_data_homonyms.py` fails when one
game's entry reaches another game's file by name.
- **A file a platform pins by hash is collected into `bios/`.** A data
directory cache that happens to hold it is not enough, because the installer
cannot fetch from the cache.
- **Collected means committed.** A file indexed by the database but never
committed gives the installer a 404, so
`git ls-files --others --exclude-standard bios` is empty before manifests
are regenerated.
- **MAME and FBNeo archives are named the way the emulator opens them.** A
MAME device's archive takes the short name of its `DEFINE_DEVICE_TYPE`, not
the `ROM_START` label. An FBNeo archive is named after its `RomDesc`, once
the driver's `GetZipName` function strips the system prefix (`msx_msx`
opens `msx.zip`). BIOS root sets change between MAME versions and are listed
from each version's source. A distributed non-merged archive also holds the
ROMs of every device the machine pulls in, which is what a `-listxml` DAT
describes and why [romset recipes](tools.md#romset-recipes) come from DATs.
## Storage tiers
| Tier | Meaning |
@@ -223,9 +349,8 @@ platform's own verification mode.
files, encryption, excessive expansion and suspicious compression ratios.
- standalone-emulator copies require explicit `--standalone-copies` consent.
- CI writes only what its job needs: `validate.yml` holds `pull-requests: write`
for the validation comment and labels, `deploy-site.yml` holds the Pages
deploy identity, and the release job holds `contents: write` but runs only
when someone dispatches it.
for the validation comment and labels, and `deploy-site.yml` holds the Pages
deploy identity. No workflow builds or publishes a release.
- `safe_extract_zip()` prevents zip-slip path traversal attacks
- `deterministic_zip` rebuilds MAME ZIPs so same ROMs always produce the same hash
+16
View File
@@ -465,6 +465,22 @@ Standalone copies are stricter, since they write outside the tree the user
selected: a symlink already sitting at the destination is skipped rather than
followed.
## Changing the installer
`install.py` runs on the user's machine, often an older handheld or a stock
RetroPie image, so it stays on Python 3.8 and the standard library alone.
Tests refuse any 3.9 syntax or API, and one runs the installer end to end under
a real 3.8 interpreter, which CI installs beside 3.12 so the check cannot skip.
The build scripts need 3.12 and are not bound by this.
Every change to `install.py` recomputes the SHA-256 pinned in `install.sh` and
`install.ps1` in the same commit. The tests compare the pins with the file.
Narrowing by system, core, region or target replays the pack builder's rules
from fields the manifest carries, and a parity test compares an installer
selection with the pack built the same way. Given a manifest without those
fields, the installer refuses to narrow rather than install everything.
## Exit status
| Code | Meaning |
+145 -1
View File
@@ -67,6 +67,24 @@ fresh pass for each new profile. When fbneo_neogeo was profiled, the
NeoGeo subset referenced BIOS files that the main arcade analysis
hadn't encountered. A fresh look avoids carrying over blind spots.
A declared upstream that answers 404 is not proof the project is gone. The
account can differ from the Android package id (`aenu.aps3e` lives at
`aenu1/aps3e`), a repository can be renamed, and a live repository can belong
to the parent project rather than the one that publishes the package. A code
search on the package id usually names the real repository.
A closed-source emulator is profiled from what it ships: decompiled IL for a
.NET or Unity build, the string table and import directory of a native binary,
the debug information a library was not stripped of. A source drop published
after the fact can be older than every shipped build, so its option and file
name strings are compared with a genuine binary before it is trusted.
An Android package is fingerprinted before a byte of it is read. Mirrors carry
rebuilt and re-signed copies of paid emulators, and a profile read from one
describes someone else's modification. Read the signing certificate, compare
it with a build known to be genuine, and prefer an older genuine build to a
newer repack. The profile names the rejected copies.
### 2. Trace file loading
Read the code flow, tracing from the entry point.
@@ -93,6 +111,23 @@ object. BlastEm reads a list of ROM paths from a configuration
structure. same_cdi opens CD-i BIOS files through a machine
initialization routine. The loading flow varies widely between emulators.
Check which build reaches the code. When one tree builds a desktop and an
Android app, the Android project can remove sources from its compile and wire
other entry points, so a file the shared code opens may never be read on
Android. A platform header can also leave a symbol undefined whatever the
comment above it says. Read the project file and the macro block before
calling a load path live, and scope the file in its `note:` when only one
build reads it.
Everything the code opens from outside goes in the profile, not only BIOS
images: fonts, keyboard maps, NV memory, samples, DSP firmware, game databases,
data directories, native libraries loaded by name. For a library shipped
inside a package, the test is whether the emulator's own code names it.
ArcadeFlashWeb ships a whole Qt deployment, and its profile lists the few files
its source names, such as the Flash runtime passed on the command line.
Operating system runtimes (`VCRUNTIME140.dll`) are prerequisites written in
`notes:`, never files.
### 3. Determine required vs optional
This is decided by code behavior, not by judgment:
@@ -113,6 +148,11 @@ are marked with `hle_fallback: true` and `required: false`. The file
still ships in packs (better experience for the user), but its absence
does not raise alarms during verification.
MAME is read the same way. A missing ROM is fatal only when it is neither
`ROM_OPTIONAL` nor `NO_DUMP` (`src/emu/romload.cpp`), so a set made only of
`NO_DUMP` entries is not required, and MAME hashes the members of an archive,
never the archive: those hashes go in `contents:`.
### 3b. Decide whether the file carries a region
Set `region:` only when the **code branches on a territory to select the
@@ -169,6 +209,18 @@ the regions No-Intro gives the same dump. Each disagreement is a line to read
in the source, not a correction to apply: the field records the code's
selection, and a Hong Kong PS2 is an NTSC-J console.
### 3c. Record the search order
When several files can fill one slot, `priority:` records the order the code
looks for them, lowest first, read from the source alone. Two cores can rank
the same file differently: pcsx1 walks `scph1001`, `scph5501`, `scph7001` in
that order while DuckStation prefers `scph5501` (5) to `scph1001` (10). The
two uses of the field then answer differently. Choosing one file per slot
(`--one-per-slot`) keeps every candidate, since dropping one would drop
another core's first choice. Ordering a list keeps the best rank any core
gives and notes the disagreement beside it, because a list drops nothing and
the file the emulator prefers is the one a reader copies first.
### 4. Document divergences
When the libretro port differs from the upstream:
@@ -225,6 +277,41 @@ read at. Profiles written before this field existed carry `profiled_date`
and `core_version` instead. They gain `source_commit` as they are
re-verified.
### Citations and pins
Every choice in a profile carries a `source_ref` to the original emulator's
code, and to the wrapper's too when the wrapper changes the path. Line numbers
only hold at one revision, so a few rules keep them checkable.
- **The pin and the references move together.** A profile rewritten from a
reading of the current head records that head's sha in `source_commit` in
the same edit. References updated without the pin describe one revision
while the pin names another, and the next comparison reads the old code at
line numbers that only make sense in the new.
- **Paths start at the repository root.** `src/xenia/ui/imgui_drawer.cc:39`
anchors, `ui/imgui_drawer.cc:39` reports GONE although the file exists, and
the failure looks like deleted code. A bare file name anchors only while one
file in the tree carries it.
- **Citations in prose count.** Every `file:line` in notes and free text is
checked like a `source_ref`, and a pin does not advance while one of them
still describes the old revision.
- **Line numbers are read, not copied.** Search tools disagree on whether lines
count from 0 or 1. Read the cited span at the pinned revision before writing
it.
- **A frozen line is pinned to its tag.** A profile documenting a release that
lives as a tag in a still active repository (`pcsx2-legacy` at PCSX2 v1.6.0)
is checked against its own revision and never rebased onto the head.
- **A fork's base comes from the forge.** `upstream_commit` of a fork is the
merge base the compare API returns. A vendored `version.h` can name the
project the fork once came from. When a package ships prebuilt binaries
built from different commits of its own repository, the pin is the revision
whose sources describe the shipped bytes, found by hashing each binary
against the history.
- **Upstream moved, the profile is read again.** When the code a profile cites
is rewritten or moved, the profile is re-profiled at the new revision. A
profile held on a revision nobody ships documents a program that no longer
exists.
### Writing style
Notes in a profile describe what the core does, kept focused on:
@@ -264,6 +351,50 @@ A few field conventions that protect the toolchain:
the bytes can be extracted and hashed. If the source performs no hash
check at all, the hash is omitted from the profile. The .info or docs
may list an MD5, but source confirmation makes it more reliable.
A data file the build generates is rebuilt from the tree and compared
byte for byte with the official release before its hash is recorded
(PrBoom+ builds `prboom-plus.wad` with its own tool). A firmware URL that
embeds an MD5, as Sony's update URLs do, names the revision the code
expects, and records `md5:` without `validation: [md5]`. A hash that comes
from neither the code nor the shipped file is not written, even labelled
informative.
- A name a platform's launcher reads (a Batocera generator, a RetroBat
launcher) is not an alias: it describes the platform, and the platform's
scraper records it under `launchers:`. `aliases:` holds the names the
emulator itself opens.
- For a file the code finds by content (`found_by: content`), `name:` is a
label. It reuses the name other profiles give that exact dump, so it never
collides with another dump's established name: ares labels borrowed from
other cores once put its revision on a path where RetroBat pins a different
dump.
- `bundled: true` keeps a file out of packs because the user already has it.
`not_bundled_on`, `installed_on` and `unread_on` refine that per platform
from that platform's own package or launcher, each with the reference that
shows it.
- `unsourceable:` stops a different file with the same name from being served
in the entry's place. It never means the file cannot be found, and the entry
stays an acquisition target. XM7's kanji ROMs carry it because the
collection's `KANJI.ROM` files are other machines' fonts.
- `path:` is the full destination, file name included. A directory-only value
collapses several entries into one count. On a profile that is
`type: standalone`, `mode: standalone` sits at the top only: repeated on each
file it hides them from the cross-reference.
- An entry of a multi-system profile carries `system:`, read from the code.
The toolchain never guesses it, and an entry without one is counted as
unattributed.
- Every name in `cores:`, and every numeric file name, is a quoted string:
`name: 01` reads back as `1`. `cores:` lists every name the buildbot and the
platforms use for the core, since a name no profile claims is a core that
target filtering skips.
- Metadata goes in fields, never in parentheses inside a string.
### 6. Validate
@@ -287,6 +418,19 @@ value to compare (`validation: [crc32]` and no `crc32`), a `data_directories`
ref the registry does not name, a `path:` written in this repository's layout
instead of the emulator's, a citation no declared revision can reach.
Three more checks catch what the schema cannot see:
- `python scripts/profile_sync.py --emulator NAME` reports 0 to re-read. A
batch of GONE references on a new profile means the path prefix, not
upstream drift.
- In `python scripts/verify.py --emulator NAME --json`, the status counts add
up to the number of entries. An entry the collection was never meant to
hold that reports OK is a homonym being served, and needs `unsourceable:`.
- A profile that becomes `standalone + libretro` joins every RetroArch pack.
Generate the RetroArch manifest with and without it into a scratch
`--output-dir` and compare the destinations: a standalone layout written
into `path:` shows up as files at destinations no core reads.
### Lessons learned
These are patterns that have come up while building profiles. Sharing
@@ -417,6 +561,6 @@ partitions), and `game_count`.
| `contents` | structure of files inside a BIOS ZIP (`name`, `description`, `size`, `crc32`) |
| `storage` | `embedded` (default), `external`, `user_provided`, or `large_file`/`release` for files stored as release assets (listed in `.gitignore`) |
| `agnostic` | true if any file under the system path within size constraints satisfies the requirement |
| `unsourceable` | reason why the file cannot be sourced (acknowledged gap) |
| `unsourceable` | why the collection cannot hold these exact bytes. It stops a different file with the same name from being served in the entry's place, while every hash proof still runs. The entry stays an acquisition target |
| `destination` | target path within the BIOS directory |
+56 -1
View File
@@ -182,7 +182,13 @@ disambiguated by a hash suffix on purpose.
output or formatting.
5. **Run the full suite.** After adding your test, run `python -m unittest
discover tests -v` to verify nothing else broke.
discover -s tests -t . -v` to verify nothing else broke. `-t .` imports
`tests/__init__.py`, which keeps the suite offline.
A fix comes with a test that fails without it. Write the test first, watch it
fail on the current code, then fix: a test written after the fix may never
have exercised the case it names. When a reading of the code finds a fact a
script could have established, the script gains that check in the same change.
Example skeleton:
@@ -240,6 +246,55 @@ If a change passes tests but breaks the pipeline, it's worth investigating befor
Ideally, tests, code, and documentation ship together. When profiles and platform configs are involved, updating them in the same change helps keep everything in sync.
A run that dies with a segmentation fault or an exception the code cannot
raise is run again once, alone, before anything is investigated. Only a
failure that reproduces is a defect in the code.
## Prose and tone
`tests/test_prose_style.py` holds every tracked file, and the generated site,
to the way the project writes:
- No em-dash or en-dash, no semicolon joining two clauses, no unicode arrows,
no ellipsis character, no box-drawing banners, no emoji or check marks.
Rewrite the sentence rather than swapping the character.
- None of the marketing vocabulary the test lists, and no superlatives. A
number with its criterion says more.
- Headings in sentence case, never repeating the site name.
Formatting stays: bold labels, lists, tables and admonitions make a page
easier to use. When markup shows raw on the site, the rendering is fixed and
the formatting kept.
What the test cannot read is tone. Text that names another project says what
changes, not who erred: "hash updated to the emulator source", "files added",
"required flags aligned with the code". No "wrong", "bogus" or "phantom" about
someone else's work. Notes in a profile say what the core does, nothing about
what it is not. A reply on an issue or a pull request thanks once, gives the
facts, describes the script or the file rather than the person, and leaves
the door open.
## Landing a change
A commit subject is English, 50 characters at most, `type: description` with a
lowercase word after the colon. The types are `feat`, `refactor`, `chore`,
`docs` and `fix`. The message does not use bug, regression, revert, dead code,
broken, wrong, hack, workaround or implement (derived forms included), nor the
marketing vocabulary above, and carries no Co-Authored-By trailer. A body
explains the why when the subject cannot: the scenario that failed, the
upstream `file:line` that decides it, what users will see.
```
feat: add retrobat platform support
refactor: extract shared utils to common.py
```
Files are staged by name, never with `git add -A` or `git add .`, which sweep in
deletions and work that is not yours. Generated artifacts (`database.json`,
`install/*.json`, `README.md`, `CONTRIBUTING.md`, `exports/`) are committed
with the data they derive from, never alone: a manifest committed before the
files it points at gives the installer a 404.
## CI integration
The `validate.yml` workflow runs `python -m unittest discover -s tests -t . -v` on both
+25 -1
View File
@@ -67,6 +67,30 @@ Pipeline flags:
`--check-buildbot` runs the whole pipeline online. For the freshness check
alone, run `python scripts/check_buildbot_system.py`.
## Conventions every tool keeps
- **A flag is applied or refused.** A mode that accepts a flag it does not
apply leaves the user believing they filtered. Four such cases were found
(`verify --emulator --region` among them), and a test now runs every mode
against every flag.
- **Every dimension that narrows an artifact appears in its name.** Region,
target, required-only, standalone and source variant are part of the pack
and manifest names. `--manifest --target switch` once overwrote the
manifest every installer downloads with a narrowed one.
- **A step that did not run says SKIPPED**, never OK.
- **verify.py and generate_pack.py read the same functions.** A count that
differs between the report and the pack is a defect, which is what step 5
checks.
- **Generators write through `write_if_changed()`.** A new timestamp alone
rewrites nothing, so a diff on a generated file always means content.
- **One run per output directory.** Pack generation takes a lock on
`<output-dir>/.lock`, and a second run on the same folder stops at once
rather than read a half-written ZIP. Experiments use a scratch
`--output-dir` under `tmp/`.
- **Read the exit status before the output.** A generator that crashed leaves
the previous artifact in place, which reads exactly like a run that changed
nothing.
## Individual tools
### generate_db.py
@@ -238,7 +262,7 @@ python scripts/diff_truth.py --all # diff all platforms
### export_native.py
Rewrite each platform's own BIOS file, corrected: System.dat, es_bios.xml,
Write each platform's own BIOS file, updated from the emulators' code: System.dat, es_bios.xml,
batocera-systems, checkBIOS.sh, FirmwareDatabase.cs, the RetroDECK manifests,
MiSTer's BiosDB, RetroPie's scriptmodules. Every platform in the registry,
archived included.
+12
View File
@@ -104,6 +104,10 @@ wrong hash beside a right one does not withhold the file. `verify.py`, the
pack builder and the install manifest apply that rule. ZIP files are not
opened, only the container is checked.
`verification_mode` answers whether the frontend reads the bytes, and
`hash_rule` answers which comparison it makes. A frontend whose acceptance is
not a single digest gets a new `hash_rule` value, never a new mode.
### ROCKNIX verification
**Source**: `rocknix-systems`, function `checkBios`
@@ -266,6 +270,14 @@ When platform verification passes but emulator validation fails, the file has a
This happens most often in existence-mode platforms where any file with the right name is
accepted.
A local file that contradicts a declared hash is handled by the platform's
own mode, never by one global rule. An existence frontend never reads the
bytes, so the file is packed and the disagreement reported: a stale hash in
System.dat must not remove a file RetroArch would have loaded. An md5 or sha1
frontend would reject it, so it is left out of the pack and counted as
`unsafe excluded`, which is distinct from `missing`. The pack builder and the
pack verification apply the same rule.
### Variant search
`validated_choice()` (`scripts/validation.py`) decides the file a platform