mirror of
https://github.com/Abdess/retroarch_system.git
synced 2026-11-16 06:44:43 -06:00
chore: merge main into Sony pages
This commit is contained in:
commit
0bd206da4d
410 files changed
+146998
-134303
No files matched your search
@@ -725,7 +725,7 @@ This executes in sequence, numbered as the run prints them (`STEP_LABELS` in
|
||||
- 4d `bundled_only.py --strict`, copies only an emulator install provides
|
||||
- 5 Consistency check: verify and the pack agree on counts and on the paths each archive holds
|
||||
- 6 Pack integrity: extract the ZIPs and check hashes per platform mode
|
||||
- 7 `generate_readme.py`, regenerate README and CONTRIBUTING
|
||||
- 7 `generate_readme.py`, regenerate README and CONTRIBUTING, then `generate_installer_doc.py` rewrites the tables and run samples of `wiki/installer.md`
|
||||
- 8 `generate_site.py`, regenerate the site pages and navigation
|
||||
- 8b `npm run build` in `website/`, then `validate_site.py`
|
||||
|
||||
|
||||
+89
-32
@@ -393,18 +393,18 @@ lists record these files as omitted.
|
||||
|
||||
<!-- generated: dry-run -->
|
||||
|
||||
A RetroArch dry run into an empty folder, with the list of 2026-10-27, printed this:
|
||||
A RetroArch dry run into an empty folder, with the list of 2026-11-03, printed this:
|
||||
|
||||
```
|
||||
Fetching file index for retroarch...
|
||||
106 files read from RetroArch's save directory go to a folder per core under /home/me/.config/retroarch/saves
|
||||
10994 files (6.7 GB)
|
||||
3269 BIOS files (4.1 GB), 5089 game data files (2.5 GB) and 2636 data files (38.5 MB).
|
||||
10995 files (6.7 GB)
|
||||
3269 BIOS files (4.1 GB), 5089 game data files (2.5 GB) and 2637 data files (39.6 MB).
|
||||
Game data: files a game or a game engine reads, of no use to the other cores. Data files: the files an emulator reads from a data folder of its own, such as Dolphin's Sys and PPSSPP's assets.
|
||||
Not installable: 37 files are not in the collection, 1 of them required.
|
||||
Not installable: 38 files are not in the collection, 2 of them required.
|
||||
|
||||
Checking existing files...
|
||||
0 in place, 10994 missing, 0 wrong copies
|
||||
0 in place, 10995 missing, 0 wrong copies
|
||||
```
|
||||
<!-- /generated: dry-run -->
|
||||
|
||||
@@ -423,7 +423,7 @@ the installer can fetch them.
|
||||
<!-- generated: run-date -->
|
||||
|
||||
Each line a first install prints, and what it means. The run below is a
|
||||
Batocera install into an empty folder, from the list of 2026-10-27.
|
||||
Batocera install into an empty folder, from the list of 2026-11-03.
|
||||
The figures change with each release.
|
||||
<!-- /generated: run-date -->
|
||||
|
||||
@@ -466,7 +466,7 @@ Fetching file index for batocera...
|
||||
14723 files (9.2 GB)
|
||||
3179 BIOS files (4.4 GB) and 11544 game data files (4.8 GB).
|
||||
Game data: files a game or a game engine reads, of no use to the other cores.
|
||||
Not installable: 425 files are not in the collection, 79 of them required.
|
||||
Not installable: 426 files are not in the collection, 80 of them required.
|
||||
|
||||
Checking existing files...
|
||||
0 in place, 14723 missing, 0 wrong copies
|
||||
@@ -480,7 +480,9 @@ On a first install nothing is in place. On a later run the line counts what
|
||||
is already there, and only the rest is downloaded. A download of 1 GB or more
|
||||
is confirmed first, the size and the folder in the question.
|
||||
|
||||
The second phase downloads what is missing, one line per file:
|
||||
The second phase downloads what is missing. At a terminal one progress line
|
||||
moves with the count and the size. A pipe or a log gets one line per file,
|
||||
as the log of the run below:
|
||||
|
||||
<!-- generated: run-download -->
|
||||
|
||||
@@ -503,7 +505,10 @@ command again picks up where it stopped.
|
||||
Done. 14723 files installed.
|
||||
Installed: 3179 BIOS files and 11544 game data files.
|
||||
BIOS folder: /userdata/bios
|
||||
425 files were not installed: the collection does not hold them, or its copy fails their check.
|
||||
426 files were not installed: the collection does not hold them, or its copy fails their check.
|
||||
Needed to start an emulator (1 file):
|
||||
capcom-cps3 (fbneo): sfiii2a.zip
|
||||
Each emulator's page says what these files are: https://abdess.github.io/retrobios/emulators/
|
||||
Not available here: DARK.GOB, DARKWAR.RTL, DARKWAR.WAD and 76 more, game data their games read. Copy them from your own copy of the game into /userdata/bios/, /userdata/bios/Source/, /userdata/bios/base/ and 10 more folders.
|
||||
Game data: files a game or a game engine reads, of no use to the other cores.
|
||||
To check this later, run the same command with --check.
|
||||
@@ -516,7 +521,7 @@ would save, once that saving is over half.
|
||||
|
||||
<!-- generated: bulk-hint -->
|
||||
|
||||
RetroArch with `--system sony-playstation`, from the list of 2026-10-27:
|
||||
RetroArch with `--system sony-playstation`, from the list of 2026-11-03:
|
||||
|
||||
```
|
||||
duckstation pulls 316.5 MB of this 329.0 MB download for sony-playstation. --core pcsx1 keeps 11.0 MB of it and saves 318.0 MB (96%).
|
||||
@@ -525,9 +530,10 @@ RetroArch with `--system sony-playstation`, from the list of 2026-10-27:
|
||||
|
||||
`--dry-run` prints the same line under its plan.
|
||||
|
||||
When some files came from this machine rather than the network, a line under
|
||||
`Done` says where each came from (`38 files copied from E:\, 3 copied from
|
||||
the wrong place in the BIOS folder, 4 downloaded.`). See
|
||||
When some files came from this machine rather than the network, the run says
|
||||
where each came from: on the `Done` line when all came from one place
|
||||
(`Done. 22 files copied from /home/me/old.`), on a line under it otherwise
|
||||
(`2 files copied from /home/me/old, 2 downloaded.`). See
|
||||
[Copy from an old card](#copy-from-an-old-card).
|
||||
|
||||
The word `Done` and the BIOS folder are the confirmation. Any other ending
|
||||
@@ -565,20 +571,27 @@ replaces it and keeps the old one, see
|
||||
### Why a file is refused { #why-a-file-is-refused }
|
||||
|
||||
Each file `--check` flags comes with the reason it found and the place each
|
||||
core opens it:
|
||||
core reads it. A RetroArch folder holding a copy of each kind, checked
|
||||
against a list of four files:
|
||||
|
||||
<!-- generated: check-refused -->
|
||||
|
||||
```
|
||||
3 files to download:
|
||||
2 of these files are in the BIOS folder in the wrong place: a run without --check copies them where the cores open them, and leaves your copies as they are.
|
||||
wrong copy: scph5501.bin
|
||||
another region: the contents of scph5500.bin (japan)
|
||||
opened at /home/me/.config/retroarch/system/scph5501.bin by beetle_psx, pcsx_rearmed
|
||||
read by beetle_psx, pcsx_rearmed at /home/me/.config/retroarch/system/scph5501.bin
|
||||
missing: gba_bios.bin
|
||||
nested folder: it is at system/system/gba_bios.bin, a system folder inside system
|
||||
opened at /home/me/.config/retroarch/system/gba_bios.bin by mgba
|
||||
a run without --check copies it into place
|
||||
read by mgba at /home/me/.config/retroarch/system/gba_bios.bin
|
||||
missing: pcsx/extra.bin
|
||||
still zipped: it is inside PSX BIOS.zip
|
||||
opened at /home/me/.config/retroarch/system/pcsx/extra.bin by pcsx_rearmed
|
||||
a run without --check copies it into place
|
||||
read by pcsx_rearmed at /home/me/.config/retroarch/system/pcsx/extra.bin
|
||||
```
|
||||
<!-- /generated: check-refused -->
|
||||
|
||||
| Reason | What the folder holds |
|
||||
|--------|-----------------------|
|
||||
@@ -699,12 +712,15 @@ other contents is kept aside first, as a download would replace it (see
|
||||
[Files you already had](#files-you-already-had)). What no folder holds is
|
||||
downloaded as usual, or taken from the pack with `--from-pack`. Several
|
||||
`--from` are read in the order given, and the end of the run says where each
|
||||
file came from:
|
||||
file came from. A run of a list of four files, two of them in `old`:
|
||||
|
||||
<!-- generated: from-run -->
|
||||
|
||||
```
|
||||
Done. 44 files installed.
|
||||
38 files copied from E:\, 4 downloaded, 2 still missing.
|
||||
Done. 4 files installed.
|
||||
2 files copied from /home/me/old, 2 downloaded.
|
||||
```
|
||||
<!-- /generated: from-run -->
|
||||
|
||||
`--check --from FOLDER` writes nothing and says how many files the folder
|
||||
would give (`E:\ holds 38 of these files: a run without --check copies
|
||||
@@ -731,10 +747,11 @@ by `scripts/generate_installer_doc.py`):
|
||||
<!-- generated: line-menu -->
|
||||
|
||||
```
|
||||
RetroArch: 10994 files, 6.7 GB.
|
||||
3269 BIOS files (4.1 GB), 5089 game data files (2.5 GB) and 2636 data files (38.5 MB).
|
||||
RetroArch: 10995 files, 6.7 GB.
|
||||
3269 BIOS files (4.1 GB), 5089 game data files (2.5 GB) and 2637 data files (39.6 MB).
|
||||
Game data: files a game or a game engine reads, of no use to the other cores. Data files: the files an emulator reads from a data folder of its own, such as Dolphin's Sys and PPSSPP's assets.
|
||||
BIOS folder: /home/me/.config/retroarch/system
|
||||
Not available here, and needed to start its emulator: sfiii2a.zip (Capcom Cps3). Your own copy goes in the BIOS folder.
|
||||
Not available here: jazz2/Source/Anims.j2a (Jazz Jackrabbit 2), game data its game reads. Copy it from your own copy of the game into /home/me/.config/retroarch/system/jazz2/Source/.
|
||||
Enter) install everything
|
||||
c) choose systems, cores and regions
|
||||
@@ -745,10 +762,11 @@ RetroArch: 10994 files, 6.7 GB.
|
||||
|
||||
Counted again for your choices. Nothing is written before Enter.
|
||||
|
||||
RetroArch: 5905 files, 4.2 GB.
|
||||
3269 BIOS files (4.1 GB) and 2636 data files (38.5 MB). 5089 game data files left out.
|
||||
RetroArch: 5906 files, 4.2 GB.
|
||||
3269 BIOS files (4.1 GB) and 2637 data files (39.6 MB). 5089 game data files left out.
|
||||
Game data: files a game or a game engine reads, of no use to the other cores. Data files: the files an emulator reads from a data folder of its own, such as Dolphin's Sys and PPSSPP's assets.
|
||||
BIOS folder: /home/me/.config/retroarch/system
|
||||
Not available here, and needed to start its emulator: sfiii2a.zip (Capcom Cps3). Your own copy goes in the BIOS folder.
|
||||
Same choice without questions: python3 install.py --platform retroarch --dest /home/me/.config/retroarch/system --no-game-data
|
||||
Enter) install everything
|
||||
c) choose systems, cores and regions
|
||||
@@ -826,6 +844,31 @@ python install.py --platform recalbox --region us,eu,jp
|
||||
python install.py --platform emuelec --no-game-data
|
||||
```
|
||||
|
||||
### For the games on this card { #for-the-games-on-this-card }
|
||||
|
||||
When the card holds games in its rom folders, the plan opens on **For the
|
||||
games on this card: yes**, and Install takes only what those games need. One
|
||||
press on the row installs everything instead. Line mode asks the same: Enter
|
||||
installs for the games, `e` everything, `r` only what starts. On the command
|
||||
line it is `--for-games`, and `--for-games FOLDER` reads a games folder on a
|
||||
computer, one subfolder per system.
|
||||
|
||||
The rule is that every game must start. Per system, the run keeps what the
|
||||
system's default core reads. One region is kept only where the game's own
|
||||
content proves it: a cartridge's crc32 found in No-Intro, a disc serial found
|
||||
in Redump, or a header the core reads. A file name such as `(Europe)` proves
|
||||
nothing. Where a core forces a region by an option, falls back to another
|
||||
region, or uses its own disc list, every region stays. An arcade set gets the
|
||||
BIOS sets of its parent chain (`neogeo.zip` for `mslug.zip`).
|
||||
|
||||
A game that is not recognised gets every BIOS of its system, and the run names
|
||||
it. So does an arcade set the core does not know. **Only what starts**
|
||||
(`--required-only`) leaves out the files the cores mark optional.
|
||||
|
||||
The games are read without hashing whole discs, and a cache keeps what was
|
||||
read, so a second run over the same card reads only new files. Offline, with
|
||||
no games index on this machine, the run keeps every file and says so.
|
||||
|
||||
### Firmware without game data
|
||||
|
||||
Some platform lists name the files a game reads beside the firmware its
|
||||
@@ -919,6 +962,8 @@ what the options name, everything by default.
|
||||
| <span id="option-core"></span>`--core NAMES` | Keep the platform's own list and only the extra files these cores load. `--list-cores` prints the names |
|
||||
| <span id="option-emulator"></span>`--emulator NAMES` | Install what these emulators read on the detected platform, and nothing else: their own files and the platform's files of the systems they run. Each emulator page on the site gives this command. `--list-emulators` prints the names, buildbot names such as `mednafen_psx` included |
|
||||
| <span id="option-region"></span>`--region LIST` | Preferred regions, best first (`us,eu,jp`): keep one BIOS per region where a system has several. Accepts the names and aliases of `generate_pack.py --region` |
|
||||
| <span id="option-for-games"></span>`--for-games FOLDER` | Install only what the games on the card need. See [For the games on this card](#for-the-games-on-this-card) |
|
||||
| <span id="option-required-only"></span>`--required-only` | With `--for-games`, only the files the games cannot start without |
|
||||
| <span id="option-no-game-data"></span>`--no-game-data` | Leave out game data (a game's own files, not firmware) and install the firmware only. What it leaves out is named. See [Firmware without game data](#firmware-without-game-data) |
|
||||
| <span id="option-declared-only"></span>`--declared-only` | Keep only the files the platform's own list names (`cores: null` in the manifest), not the extra files cores load. With `--check`, the answer is about the list the platform itself checks. Refused with `--core`, the listings it does not change, `--standalone-copies` and `--tui` |
|
||||
| <span id="option-list-systems"></span>`--list-systems` | Print a platform's systems with the files and size each holds, the files other systems share too counted apart |
|
||||
@@ -1056,6 +1101,8 @@ gives for the device, below the card's root instead of its mount point:
|
||||
| `.tmp_update/onionVersion/version.txt` | Onion | `BIOS`, RetroArch's list |
|
||||
| `.allium/` | Allium | `BIOS`, RetroArch's list |
|
||||
| `Koriki/version.txt` | Koriki | `BIOS`, RetroArch's list |
|
||||
| `bgmusic/` beside `bios/`, on a drive that is not a fixed disk | ArkOS | `bios`, RetroArch's list |
|
||||
| `launchimages/` beside `bios/`, on a drive that is not a fixed disk | ArkOS | `bios`, RetroArch's list |
|
||||
| `gamedata/`, `homebrew/` beside `bios/`, on a drive that is not a fixed disk | AmberELEC | `bios`, AmberELEC's list |
|
||||
| `bezels/default.cfg` beside `bios/`, on a drive that is not a fixed disk | EmuELEC | `bios`, EmuELEC's list |
|
||||
| `system/batocera.conf` holding `system.hostname=BATOCERA` or no hostname line, and no KNULLI file above | Batocera SHARE drive | `bios`, Batocera's list |
|
||||
@@ -1635,14 +1682,20 @@ The build scripts need 3.9 and are not bound by this.
|
||||
Every change to `install.py` recomputes the SHA-256 pinned in `install.sh` and
|
||||
`install.ps1` in the same commit. The tests compare the pins with the file.
|
||||
|
||||
The tables of options, events, exit codes and keys on this page sit between
|
||||
`generated` markers and are written from `install.py` by
|
||||
`scripts/generate_installer_doc.py`: the options from its parser, the events
|
||||
from `EVENT_KEYS`, the codes from its `sys.exit` calls, the keys from
|
||||
`TUI_ROUTES`. The prose cells stay as written. A row the code gains stops the
|
||||
script until its text is added inside the markers, and `--check`, which a
|
||||
test runs, fails on a table behind the code. `--page FILE` renders another
|
||||
copy of the page, which the tests use.
|
||||
The tables of options, events, exit codes, keys and card markers, the line
|
||||
menu and the run samples on this page sit between `generated` markers and are
|
||||
written from `install.py` by `scripts/generate_installer_doc.py`: the options
|
||||
from its parser, the events from `EVENT_KEYS`, the codes from its `sys.exit`
|
||||
calls, the keys from `TUI_ROUTES`, the card markers from its card rules. The
|
||||
run samples (a dry run, a first install, a check, the reasons a file is
|
||||
refused, a copy from an old card) are printed by `install.py` itself, in a
|
||||
sandbox under `tmp/` served on the loopback from the committed
|
||||
`install/*.json`. A new list therefore rewrites this page, and `--check`
|
||||
fails until the script has run. Pipeline step 7 runs it after the README.
|
||||
The prose cells stay as written. A row the code gains stops the script until
|
||||
its text is added inside the markers, and `--check`, which a test runs, fails
|
||||
on a table behind the code. `--page FILE` renders another copy of the page,
|
||||
which the tests use.
|
||||
|
||||
The [handheld](#on-a-handheld-the-retrobios-port) archives each carry
|
||||
`install.py`, the one the one-line command of the same commit downloads:
|
||||
@@ -1714,9 +1767,13 @@ other schemas under `api/v1/schemas/`.
|
||||
| `scanned` | `folder`, `files`, `truncated`, `other_drives` | Each folder named with `--from`, once read and before `detect`. `files` counts the files found, `truncated` is true when the search stopped at 200,000 of them, and `other_drives` lists the folders below it on another drive, which were not entered |
|
||||
| `version` | `platform`, `list_version`, `installed_version`, `message` | Before `choices`, when the system here runs another version than the one the list was built for: `list_version` is the list's, `installed_version` the one the system's version file states, and `message` says it and what it means. Not written when either version is unknown. The run goes on |
|
||||
| `choices` | `platform`, `systems` | Before `checking`: the systems the run's list lets one choose among, sorted by id, `other` last for the files no system names. The run's other options apply (`--core`, `--region`, `--no-game-data`, the files left out for an unknown save directory), `--system` does not: these are the systems a screen lists to choose from, and the count a `--system` is out of |
|
||||
| `set_aside` | `platform`, `dests` | Right after `choices`: the files of the list the run sets aside before any choice, in list order, those read from a save directory whose place is not known among them. A screen chooses its cores among the rest, the cores `--core` accepts |
|
||||
| `file` | `platform`, `dest`, `systems`, `size`, `state` | Each file of the list after the check. `state` is `present` (hash verified), `wrong_hash` or `missing` |
|
||||
| `system` | `platform`, `system`, `name`, `files`, `present`, `missing`, `wrong_hash`, `bytes_needed` | Each system, sorted by id, and with `--system` only the systems named. Files with no system count under `other`. `name` is the platform's own name for it |
|
||||
| `checking` | `files_done`, `files_total` | While the files already in the folder are read and hashed, at most every half second and once at the end. `files_total` counts the files found in place |
|
||||
| `games_reading` | `files_done`, `files_total` | With `--for-games`, while the games of the card are read (a cartridge hashed, a disc's header read), at most every half second and once at the end |
|
||||
| `games` | `platform`, `folder`, `games`, `recognised`, `found`, `fallback`, `not_games`, `required_only` | With `--for-games`, once the games are read, before `choices`. `found` lists per system its `folder`, `games`, `recognised` and the `regions` kept (empty for every region). `fallback` names each game that got every BIOS of its system, with a `reason` (`not_recognised`, `not_read`, `region_unknown`, `unknown_set`, `no_parent`) |
|
||||
| `games_unavailable` | `platform`, `reason`, `message` | With `--for-games`, what the games need could not be read (`offline`, `network`, `invalid`, `stale`, `no_sets`, `no_folder`): the run keeps every file of the list, except for a `FOLDER` that does not exist, where it stops |
|
||||
| `check` | `platform`, `name`, `bios_dir`, `files`, `present`, `missing`, `wrong_hash`, `bytes_needed`, `data_missing`, `not_installable` | The platform's totals. `data_missing` counts data-tree files to fetch, `not_installable` the files the collection cannot supply |
|
||||
| `declared` | `platform`, `files`, `present`, `missing`, `wrong_hash`, `bytes_needed`, `dests` | Right after `check`, the same counts over the files the platform's own list names (`cores: null`), and their `dests` in the order of the `file` events. A setup that checks its own list reads its answer here, with or without `--declared-only` |
|
||||
| `data` | `platform`, `files`, `bytes` | Right after `declared`: the data-tree files to fetch (the `data_missing` of `check`) and the bytes they write once in place |
|
||||
|
||||
@@ -54,7 +54,7 @@ Every module in `tests/`, by the area it guards.
|
||||
| Packs and manifests | synthetic + real packs | `test_artifact_lock`, `test_atomic_artifacts`, `test_batocera_cores`, `test_deterministic_zip`, `test_emulator_pack_counts`, `test_finalize_pack_names`, `test_large_file_cache`, `test_libretro_layout`, `test_manifest_entry`, `test_minui_platform`, `test_no_game_data`, `test_pack_conformance_dirs`, `test_pack_core_extras`, `test_pack_counts`, `test_pack_destinations`, `test_pack_failures`, `test_pack_file_names`, `test_pack_grouping`, `test_pack_integrity`, `test_pack_member_identity`, `test_pack_readme`, `test_platform_inheritance`, `test_platform_keys`, `test_release_packs`, `test_required_only_archives`, `test_required_only_twins`, `test_save_root`, `test_split_extras`, `test_split_names`, `test_split_pack`, `test_system_pack_extras`, `test_system_pack_parity`, `test_torrentzip`, `test_which_pack` |
|
||||
| Slots and regions | synthetic + profiles | `test_region`, `test_region_agnostic_scan`, `test_region_groups`, `test_region_held`, `test_region_platform_index`, `test_region_required_only`, `test_slot`, `test_slots`, `test_slots_shipped_file` |
|
||||
| Hardware targets and systems | synthetic + target files | `test_extract_targets`, `test_lakka_targets`, `test_libretro_slugs`, `test_system_ids`, `test_system_selection`, `test_target_names`, `test_target_scrapers`, `test_target_systems`, `test_target_verify` |
|
||||
| Installer and downloads | loopback server | `test_check_release`, `test_download`, `test_install`, `test_install_cores`, `test_install_experience`, `test_install_app`, `test_install_from`, `test_install_offline`, `test_install_selection`, `test_install_tui` |
|
||||
| Installer and downloads | loopback server | `test_check_release`, `test_download`, `test_install`, `test_install_cores`, `test_install_experience`, `test_install_app`, `test_install_from`, `test_install_games`, `test_install_offline`, `test_install_selection`, `test_install_tui` |
|
||||
| Handheld launchers | a pty as the console, fake installer and firmware tools, stubbed PortMaster dialogs | `test_handheld_launchers`, `test_port` |
|
||||
| Running cores | parsers on fixed inputs, a libretro core built from C in the test when a compiler exists | `test_core_probe` |
|
||||
| Measurement records | synthetic record files, the committed `measurements/` against HEAD, the **Proof** line naming the BIOS a core ran, and that no page shows a Tested block, a region outcome or a time per frame | `test_measurements` |
|
||||
@@ -65,6 +65,7 @@ Every module in `tests/`, by the area it guards.
|
||||
| Automation runner, disk cleanup, asset sync and bit-flip repair | fake jobs, git repositories with a bare origin and session worktrees, a committed file with one bit flipped, canned CI runs in place of `gh` | `test_autorun` |
|
||||
| Truth and native export | synthetic + cached originals | `test_export_counts`, `test_export_native_layer`, `test_export_report`, `test_exporters`, `test_truth_data_dir_readers`, `test_truth_diff`, `test_truth_layout` |
|
||||
| Scrapers, upstream and provenance | inline sources, synthetic | `test_batocera_md5_list`, `test_check_freshness`, `test_contributing_text`, `test_coverage_cache`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_dedup`, `test_doc_pages`, `test_doc_surfaces`, `test_elec_scrapers`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_launchers`, `test_launchers_emudeck`, `test_launchers_recalbox`, `test_launchers_retrobat`, `test_launchers_retrodeck`, `test_launchers_retropie`, `test_launchers_rocknix`, `test_mame_devices`, `test_mame_parser`, `test_mame_ref_audit`, `test_optional_dependencies`, `test_profile_sync`, `test_prose_style`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_romset_recipes`, `test_schema_semantic_gate`, `test_scraper_contract`, `test_site_availability`, `test_site_bootstrap`, `test_site_content_data`, `test_site_copy`, `test_site_cores`, `test_site_density_cards`, `test_site_emulator_page`, `test_site_entry_tables`, `test_site_exports`, `test_site_funnel`, `test_site_images`, `test_site_picker`, `test_site_platform_density`, `test_site_platform_hashes`, `test_site_r2_components`, `test_site_r2_css`, `test_site_r2_emulators`, `test_site_r2_entry`, `test_site_r2_gate`, `test_site_r2_nav`, `test_site_r2_p0`, `test_site_r2_reference`, `test_site_r2_systems`, `test_site_r2_wiki`, `test_site_r3_collection`, `test_site_r3_components`, `test_site_r3_css`, `test_site_r3_emulators`, `test_site_r3_entry`, `test_site_r3_nav`, `test_site_r3_p0`, `test_site_r3_reference`, `test_site_r3_systems`, `test_site_r3_wiki`, `test_site_r4_components`, `test_site_r4_css`, `test_site_r4_emulators`, `test_site_r4_entry`, `test_site_r4_merge`, `test_site_r4_p0`, `test_site_r4_reference`, `test_site_r4_search`, `test_site_r4_systems`, `test_site_r4_wiki`, `test_site_r5_components`, `test_site_r5_css`, `test_site_r5_emulators`, `test_site_r5_entry`, `test_site_r5_reference`, `test_site_r5_search`, `test_site_r5_systems`, `test_site_r5_wiki`, `test_site_render`, `test_site_system_links`, `test_site_validation`, `test_site_weight`, `test_site_write`, `test_source_ref_targets`, `test_upstream`, `test_upstream_index`, `test_validate_pr`, `test_validate_pr_flow`, `test_validate_pr_hashes`, `test_version_basis`, `test_workflow_paths` |
|
||||
| Games on the card | small DATs, listxml and setup sources, a fake fetcher, fixture install lists, and the committed data against its schemas | `test_arcade_parents`, `test_game_catalogs`, `test_game_headers`, `test_game_regions`, `test_games_index`, `test_romfolders_card`, `test_romfolders_es` |
|
||||
|
||||
## Test architecture
|
||||
|
||||
|
||||
+42
-2
@@ -47,9 +47,10 @@ Pipeline steps:
|
||||
| 4b | Generate install manifests | `--skip-packs` |
|
||||
| 4c | Generate target manifests | `--skip-packs` |
|
||||
| 4d | Collected copies only an emulator install provides (`bundled_only.py --strict`, gates the run) | - |
|
||||
| 4e | Games index: what the installer reads for the games on a card (`generate_games_index.py`, fails under its coverage floor) | - (reads the committed install lists) |
|
||||
| 5 | Consistency check: counts, paths per archive, every emulator pack | if verify or pack skipped |
|
||||
| 6 | Pack integrity (extract + hash) | `--skip-packs` |
|
||||
| 7 | Generate README | `--skip-docs` |
|
||||
| 7 | Generate README and CONTRIBUTING, then the installer page's tables and run samples (`generate_installer_doc.py`) | `--skip-docs` |
|
||||
| 8 | Generate site | `--skip-docs` |
|
||||
| 8b | Render the site with `npm run build --prefix website`, then `validate_site.py`, as the deploy does | `--skip-docs`, or Node not set up (`npm ci --prefix website`) |
|
||||
|
||||
@@ -1301,7 +1302,7 @@ before a release (wiki/release-process.md). Its lines stay in
|
||||
| `validate_pr.py` | Validate BIOS files in pull requests, post markdown report |
|
||||
| `validate_schemas.py` | Validate the data contracts: schemas and semantic invariants. `--source-only` checks `emulators/` and `platforms/` alone, which is what PR validation runs |
|
||||
| `generate_event_schema.py` | Write `schemas/install-events.schema.json` from `EVENT_KEYS` in `install.py` (`--check` exits 1 when the file is stale, which `validate_schemas.py` also reports) |
|
||||
| `generate_installer_doc.py` | Rewrite the generated tables of `wiki/installer.md` (options, events, exit codes, keys per route) from `install.py` (`--check` exits 1 when a table is stale, `--page FILE` writes another copy of the page) |
|
||||
| `generate_installer_doc.py` | Rewrite the generated blocks of `wiki/installer.md` from `install.py`: the tables (options, events, exit codes, keys per route, card markers), the line menu, and the run samples `install.py` prints in a sandbox against the committed `install/*.json`, so a new list rewrites the page (`--check` exits 1 when a block is stale, `--page FILE` writes another copy of the page). Pipeline step 7 |
|
||||
| `auto_fetch.py` | Fetch missing BIOS files from known sources (4-step pipeline). A download is accepted only when every cryptographic hash the entry declares (sha256, sha1, md5) matches. crc32 and size find a file the database already holds, never accept a download. An entry with no cryptographic hash is reported and not searched |
|
||||
| `list_platforms.py` | List active platforms (`--include-archived` adds the archived ones, used by CI) |
|
||||
| `download.py` | Download a pack from GitHub releases, every part checked (Python, stdlib only) |
|
||||
@@ -1359,6 +1360,11 @@ Further options, by script:
|
||||
| `scraper/coreinfo_scraper.py` | `--compare-db database.json` lists the `.info` firmware the database lacks |
|
||||
| `scraper/mame_hash_scraper.py` | `--release mameNNNN --profile FILE --source-commit SHA --core NAME --built-by PLATFORM` writes the profile of a MAME build a platform compiles. `--devices --listxml FILE` documents in `mame.yml` every device set with dumped ROMs that the release's `-listxml` names (`isdevice="yes"`), with `--dry-run` to preview |
|
||||
| `scraper/romset_dat_importer.py` | `--cache-dir` (downloaded listxml and DATs) |
|
||||
| `scraper/game_dat_importer.py` | `--source no-intro\|redump` (repeatable), `--pack FILE` imports a local ZIP, directory or DAT, `--refresh` asks the servers again, `--cache-dir`, `--output-dir`, `--dry-run` |
|
||||
| `scraper/arcade_parent_importer.py` | `--profile NAME` (one core version), `--emulators-dir`, `--database` (BIOS roots checked against it by archive name), `--output-dir`, `--cache-dir`, `--offline`, `--dry-run` |
|
||||
| `scraper/romfolders` | `--sources DIR` (default `platforms/romfolders/_sources`), `--output-dir`, `--dry-run` |
|
||||
| `game_rules.py` | `--regions`, `--extras`, `--headers` (files checked against the profiles and the header readers), `--emulators-dir`, `--json` |
|
||||
| `generate_games_index.py` | `--output-dir` (default `install/games`), `--check`, `--quiet` |
|
||||
|
||||
## Installation tools
|
||||
|
||||
@@ -1603,6 +1609,40 @@ on the database entry and renders a verified dump badge on the system pages.
|
||||
It never overrides the emulator source code: see
|
||||
[architecture](architecture.md#dump-catalog-provenance).
|
||||
|
||||
## Games on the card
|
||||
|
||||
The installer's games-on-card mode reads the rom folders of a card, identifies
|
||||
each game and installs only the BIOS those games need. Its data is generated
|
||||
from sources and published under `install/games/` by pipeline step 4e.
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.romfolders --list # setups and the format each is read in
|
||||
python -m scripts.scraper.romfolders --all # platforms/romfolders/<setup>.yml at the pins of _sources/
|
||||
python -m scripts.scraper.romfolders --setup batocera --offline
|
||||
python -m scripts.scraper.game_dat_importer --fetch --floor # games/catalogs/: No-Intro and Redump game DATs
|
||||
python -m scripts.scraper.game_dat_importer --floor # same from .cache/game-dats/ only
|
||||
python -m scripts.scraper.arcade_parent_importer --all # games/arcade/: game set to BIOS parent, per core
|
||||
python scripts/game_rules.py # games/regions.yml and games/extras.yml against the profiles
|
||||
python scripts/generate_games_index.py # install/games/ (pipeline step 4e)
|
||||
python scripts/generate_games_index.py --check # exit 1 when a published file would change
|
||||
```
|
||||
|
||||
| Source | Written by | Holds |
|
||||
|--------|-----------|-------|
|
||||
| `platforms/romfolders/_sources/<setup>.yml` | hand, each fact with its `source_ref` | pins, roms root, case rule, folders never scanned |
|
||||
| `platforms/romfolders/<setup>.yml` | `scraper/romfolders` | folders, extensions and default core as the OS writes them |
|
||||
| `games/catalogs/<source>/<system>.json` | `scraper/game_dat_importer.py` | crc32, size, sha1, region and title per game, disc serials |
|
||||
| `games/headers.yml` | hand, cited | header fields each core reads and the region each value means |
|
||||
| `games/regions.yml`, `games/extras.yml` | hand, cited | region routes, the cases that keep every region, per-game files |
|
||||
| `games/arcade/<profile>.json` | `scraper/arcade_parent_importer.py` | BIOS parent and device sets of each game set, from the DAT the pin names or, for mame2000, mame2003 and mame2003-plus, from the driver code |
|
||||
|
||||
Folder names resolve to systems through `SYSTEM_ALIASES` and `_norm_system_id`
|
||||
in `common.py`, the one alias table, and core names to profiles through the
|
||||
profiles' `cores:`. The fallback set of a folder, what an unrecognised game
|
||||
keeps, is computed from `install/<platform>.json` with the installer's own
|
||||
core narrowing. Every published file is listed in `install/games/index.json`
|
||||
with its size, its gzip size and its sha256.
|
||||
|
||||
## Target scrapers
|
||||
|
||||
Located in `scripts/scraper/targets/`. Each inherits `BaseTargetScraper` and implements `fetch_targets()`.
|
||||
|
||||
Reference in new issue
Block a user