mirror of
https://github.com/Abdess/retroarch_system.git
synced 2026-04-13 12:22:33 -05:00
Compare commits
42 Commits
auto/updat
...
40ff2b5307
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
40ff2b5307 | ||
|
|
d0dd05ddf6 | ||
|
|
d4b0af0a38 | ||
|
|
038c3d3b40 | ||
|
|
427fef5669 | ||
|
|
75e34898ee | ||
|
|
94c3ac9834 | ||
|
|
319a1d2041 | ||
|
|
00d7b57884 | ||
|
|
caf6285a04 | ||
|
|
529cb8a915 | ||
|
|
1146fdf177 | ||
|
|
4fbb3571f8 | ||
|
|
0be68edad0 | ||
|
|
1ffc4f89ca | ||
|
|
f1ebfff5bd | ||
|
|
425ea064ae | ||
|
|
6818a18a42 | ||
|
|
c11de6dba6 | ||
|
|
c4f3192020 | ||
|
|
e2d0510f4e | ||
|
|
74269bab84 | ||
|
|
1e6b499602 | ||
|
|
9b785ec785 | ||
|
|
d415777f2c | ||
|
|
eafabd20f3 | ||
|
|
2aca4927c0 | ||
|
|
17777f315b | ||
|
|
692484d32d | ||
|
|
a8430940f9 | ||
|
|
1f073f521d | ||
|
|
903c49edcf | ||
|
|
d3a2224dd2 | ||
|
|
f898f26847 | ||
|
|
2712307420 | ||
|
|
54022e9db1 | ||
|
|
4db9e4350c | ||
|
|
6864ce6584 | ||
|
|
12196b6445 | ||
|
|
7551e41a7b | ||
|
|
7b484605d4 | ||
|
|
b587381f05 |
113
CONTRIBUTING.md
113
CONTRIBUTING.md
@@ -1,14 +1,109 @@
|
||||
# Contributing to RetroBIOS
|
||||
|
||||
## Add a BIOS file
|
||||
## Types of contributions
|
||||
|
||||
1. Fork this repository
|
||||
2. Place the file in `bios/Manufacturer/Console/filename`
|
||||
3. Variants (alternate hashes): `bios/Manufacturer/Console/.variants/`
|
||||
4. Create a Pull Request - checksums are verified automatically
|
||||
- **Add a BIOS file** - a great way to get started. Fork, add the file, open a PR.
|
||||
- **Create an emulator profile** - document what a core actually loads from source code. See the [profiling guide](https://abdess.github.io/retrobios/wiki/profiling/).
|
||||
- **Add a platform** - integrate a new frontend (scraper + YAML config). See [adding a platform](https://abdess.github.io/retrobios/wiki/adding-a-platform/).
|
||||
- **Add or fix a scraper** - parse upstream sources for BIOS requirements. See [adding a scraper](https://abdess.github.io/retrobios/wiki/adding-a-scraper/).
|
||||
- **Fix a bug or improve tooling** - Python scripts in `scripts/`, single dependency (`pyyaml`).
|
||||
|
||||
## File conventions
|
||||
## Local setup
|
||||
|
||||
- Files >50 MB go in GitHub release assets (`large-files` release)
|
||||
- RPG Maker and ScummVM directories are excluded from deduplication
|
||||
- See the [documentation site](https://abdess.github.io/retrobios/) for full details
|
||||
```bash
|
||||
git clone https://github.com/Abdess/retrobios.git
|
||||
cd retrobios
|
||||
pip install pyyaml
|
||||
|
||||
# run tests
|
||||
python -m unittest tests.test_e2e -v
|
||||
|
||||
# run full pipeline (DB + verify + packs + consistency check)
|
||||
python scripts/pipeline.py --offline
|
||||
```
|
||||
|
||||
Requires Python 3.10 or later.
|
||||
|
||||
## Adding a BIOS file
|
||||
|
||||
1. Place the file in `bios/Manufacturer/Console/filename`.
|
||||
2. Alternate versions (different hash, same purpose) go in `bios/Manufacturer/Console/.variants/`.
|
||||
3. Files over 50 MB go as assets on the `large-files` GitHub release (git handles them better that way).
|
||||
4. RPG Maker and ScummVM directories are excluded from deduplication - please keep their structure as-is.
|
||||
5. Open a pull request. CI validates checksums automatically and posts a report.
|
||||
|
||||
## Commit conventions
|
||||
|
||||
Format: `type: description` (50 characters max, lowercase start).
|
||||
|
||||
Allowed types: `feat`, `refactor`, `chore`, `docs`, `fix`.
|
||||
|
||||
```
|
||||
feat: add panasonic 3do bios files
|
||||
docs: update architecture diagram
|
||||
fix: resolve truncated md5 matching
|
||||
chore: remove unused test fixtures
|
||||
refactor: extract hash logic to common.py
|
||||
```
|
||||
|
||||
Keep messages factual. No marketing language, no superfluous adjectives.
|
||||
|
||||
## Code and documentation quality
|
||||
|
||||
The codebase runs on Python 3.10+ with a single dependency (`pyyaml`). All modules
|
||||
include `from __future__ import annotations` at the top. Type hints on every function
|
||||
signature, `pathlib` instead of `os.path`, and dataclasses where a plain class would
|
||||
just hold attributes.
|
||||
|
||||
On performance: O(1) or O(n) algorithms are preferred. If something needs O(n^2), a
|
||||
comment explaining why helps future readers. List comprehensions over explicit loops,
|
||||
generators when iterating large datasets, and standard default arguments
|
||||
(`def f(items=None)` over `def f(items=[])`).
|
||||
|
||||
File I/O uses context managers. ZIP extraction goes through `safe_extract_zip()` in
|
||||
`common.py`, which prevents zip-slip path traversal.
|
||||
|
||||
The code stays lean. Comments that describe *why* age better than comments that
|
||||
restate *what*. Unused variables can be deleted rather than renamed with an underscore.
|
||||
|
||||
The same spirit applies to documentation and emulator profiles. Straightforward
|
||||
language, honest labels ("untested" when something is untested).
|
||||
|
||||
When a bug is spotted while working on something else, fixing it in the same PR
|
||||
keeps things tidy. Features ideally ship complete in one pass: code, tests, a
|
||||
passing pipeline run, and any documentation updates together.
|
||||
|
||||
## Pull request process
|
||||
|
||||
CI runs four checks on every PR:
|
||||
|
||||
| Check | What it does |
|
||||
|-------|-------------|
|
||||
| `validate-bios` | hashes changed BIOS files against the database, posts a validation report |
|
||||
| `validate-configs` | schema-validates platform YAML configs |
|
||||
| `run-tests` | runs the full E2E test suite |
|
||||
| `label-pr` | auto-labels the PR based on changed file paths |
|
||||
|
||||
All four checks must pass. One maintainer approval is required before merge.
|
||||
|
||||
## Project philosophy
|
||||
|
||||
This repository is an archive. The goal is a complete collection of every file a
|
||||
user needs to run every supported console in an emulator, with zero friction. That
|
||||
scope goes beyond BIOS ROMs: firmware updates, system fonts, data files, encryption
|
||||
keys, hiscore databases, anything the emulator loads from disk rather than generating
|
||||
itself. In a hundred years the pack should still work out of the box.
|
||||
|
||||
Licensing considerations are secondary to completeness. If an emulator needs a
|
||||
file to function, the file belongs in the collection. The project's legal position is
|
||||
documented in the FAQ section of the site.
|
||||
|
||||
Integrity matters. Every file, hash, and metadata field is cross-checked
|
||||
against the emulator's source code. Upstream references like System.dat,
|
||||
`.info` files, and wiki pages are valuable and generally accurate, though
|
||||
they can occasionally fall out of date. When an upstream source and the
|
||||
code disagree, the code at runtime is the tiebreaker.
|
||||
|
||||
## Documentation
|
||||
|
||||
Full reference docs, profiling guides, and architecture details are on the [documentation site](https://abdess.github.io/retrobios/).
|
||||
|
||||
36
README.md
36
README.md
@@ -2,7 +2,7 @@
|
||||
|
||||
Complete BIOS and firmware packs for Batocera, BizHawk, EmuDeck, Lakka, Recalbox, RetroArch, RetroBat, RetroDECK, RetroPie, and RomM.
|
||||
|
||||
**7,244** verified files across **387** systems, ready to extract into your emulator's BIOS directory.
|
||||
**7,241** verified files across **396** systems, ready to extract into your emulator's BIOS directory.
|
||||
|
||||
## Quick Install
|
||||
|
||||
@@ -27,14 +27,14 @@ Pick your platform, download the ZIP, extract to the BIOS path.
|
||||
|
||||
| Platform | BIOS files | Extract to | Download |
|
||||
|----------|-----------|-----------|----------|
|
||||
| Batocera | 359 | `/userdata/bios/` | [Download](../../releases/latest) |
|
||||
| Batocera | 362 | `/userdata/bios/` | [Download](../../releases/latest) |
|
||||
| BizHawk | 118 | `Firmware/` | [Download](../../releases/latest) |
|
||||
| EmuDeck | 161 | `Emulation/bios/` | [Download](../../releases/latest) |
|
||||
| Lakka | 448 | `system/` | [Download](../../releases/latest) |
|
||||
| Recalbox | 346 | `/recalbox/share/bios/` | [Download](../../releases/latest) |
|
||||
| RetroArch | 448 | `system/` | [Download](../../releases/latest) |
|
||||
| RetroBat | 331 | `bios/` | [Download](../../releases/latest) |
|
||||
| RetroDECK | 2007 | `~/retrodeck/bios/` | [Download](../../releases/latest) |
|
||||
| RetroBat | 339 | `bios/` | [Download](../../releases/latest) |
|
||||
| RetroDECK | 2006 | `~/retrodeck/bios/` | [Download](../../releases/latest) |
|
||||
| RetroPie | 448 | `BIOS/` | [Download](../../releases/latest) |
|
||||
| RomM | 374 | `bios/{platform_slug}/` | [Download](../../releases/latest) |
|
||||
|
||||
@@ -44,14 +44,14 @@ BIOS, firmware, and system files for consoles from Atari to PlayStation 3.
|
||||
Each file is checked against the emulator's source code to match what the code actually loads at runtime.
|
||||
|
||||
- **10 platforms** supported with platform-specific verification
|
||||
- **328 emulators** profiled from source (RetroArch cores + standalone)
|
||||
- **387 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
|
||||
- **7,244 files** verified with MD5, SHA1, CRC32 checksums
|
||||
- **9266 MB** total collection size
|
||||
- **329 emulators** profiled from source (RetroArch cores + standalone)
|
||||
- **396 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
|
||||
- **7,241 files** verified with MD5, SHA1, CRC32 checksums
|
||||
- **8144 MB** total collection size
|
||||
|
||||
## Supported systems
|
||||
|
||||
NES, SNES, Nintendo 64, GameCube, Wii, Game Boy, Game Boy Advance, Nintendo DS, Nintendo 3DS, Switch, PlayStation, PlayStation 2, PlayStation 3, PSP, PS Vita, Mega Drive, Saturn, Dreamcast, Game Gear, Master System, Neo Geo, Atari 2600, Atari 7800, Atari Lynx, Atari ST, MSX, PC Engine, TurboGrafx-16, ColecoVision, Intellivision, Commodore 64, Amiga, ZX Spectrum, Arcade (MAME), and 353+ more.
|
||||
NES, SNES, Nintendo 64, GameCube, Wii, Game Boy, Game Boy Advance, Nintendo DS, Nintendo 3DS, Switch, PlayStation, PlayStation 2, PlayStation 3, PSP, PS Vita, Mega Drive, Saturn, Dreamcast, Game Gear, Master System, Neo Geo, Atari 2600, Atari 7800, Atari Lynx, Atari ST, MSX, PC Engine, TurboGrafx-16, ColecoVision, Intellivision, Commodore 64, Amiga, ZX Spectrum, Arcade (MAME), and 362+ more.
|
||||
|
||||
Full list with per-file details: **[https://abdess.github.io/retrobios/](https://abdess.github.io/retrobios/)**
|
||||
|
||||
@@ -59,15 +59,15 @@ Full list with per-file details: **[https://abdess.github.io/retrobios/](https:/
|
||||
|
||||
| Platform | Coverage | Verified | Untested | Missing |
|
||||
|----------|----------|----------|----------|---------|
|
||||
| Batocera | 359/359 (100.0%) | 354 | 5 | 0 |
|
||||
| Batocera | 356/362 (98.3%) | 349 | 7 | 6 |
|
||||
| BizHawk | 118/118 (100.0%) | 118 | 0 | 0 |
|
||||
| EmuDeck | 161/161 (100.0%) | 161 | 0 | 0 |
|
||||
| Lakka | 443/448 (98.9%) | 443 | 0 | 5 |
|
||||
| Recalbox | 276/346 (79.8%) | 273 | 3 | 70 |
|
||||
| RetroArch | 443/448 (98.9%) | 443 | 0 | 5 |
|
||||
| RetroBat | 330/331 (99.7%) | 326 | 4 | 1 |
|
||||
| RetroDECK | 1958/2007 (97.6%) | 1932 | 26 | 49 |
|
||||
| RetroPie | 443/448 (98.9%) | 443 | 0 | 5 |
|
||||
| Lakka | 442/448 (98.7%) | 442 | 0 | 6 |
|
||||
| Recalbox | 277/346 (80.1%) | 274 | 3 | 69 |
|
||||
| RetroArch | 442/448 (98.7%) | 442 | 0 | 6 |
|
||||
| RetroBat | 339/339 (100.0%) | 335 | 4 | 0 |
|
||||
| RetroDECK | 1960/2006 (97.7%) | 1934 | 26 | 46 |
|
||||
| RetroPie | 442/448 (98.7%) | 442 | 0 | 6 |
|
||||
| RomM | 372/374 (99.5%) | 372 | 0 | 2 |
|
||||
|
||||
## Build your own pack
|
||||
@@ -104,7 +104,7 @@ The [documentation site](https://abdess.github.io/retrobios/) provides:
|
||||
- **Per-emulator profiles** with source code references for every file
|
||||
- **Per-system pages** showing which emulators and platforms cover each console
|
||||
- **Gap analysis** identifying missing files and undeclared core requirements
|
||||
- **Cross-reference** mapping files across 10 platforms and 328 emulators
|
||||
- **Cross-reference** mapping files across 10 platforms and 329 emulators
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -130,4 +130,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
||||
|
||||
This repository provides BIOS files for personal backup and archival purposes.
|
||||
|
||||
*Auto-generated on 2026-03-29T21:00:40Z*
|
||||
*Auto-generated on 2026-03-30T20:16:27Z*
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
113
database.json
113
database.json
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"generated_at": "2026-03-30T04:10:12Z",
|
||||
"total_files": 7239,
|
||||
"total_size": 8539795099,
|
||||
"generated_at": "2026-03-30T13:15:58Z",
|
||||
"total_files": 7241,
|
||||
"total_size": 8540057243,
|
||||
"files": {
|
||||
"520d3d1b5897800af47f92efd2444a26b7a7dead": {
|
||||
"path": "bios/3DO Company/3DO/3do_arcade_saot.bin",
|
||||
@@ -40843,16 +40843,6 @@
|
||||
"crc32": "11647ca5",
|
||||
"adler32": "1817f6f4"
|
||||
},
|
||||
"26237b333db4a4c6770297fa5e655ea95840d5d9": {
|
||||
"path": "bios/Pioneer/LaserActive/Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin",
|
||||
"name": "Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin",
|
||||
"size": 131072,
|
||||
"sha1": "26237b333db4a4c6770297fa5e655ea95840d5d9",
|
||||
"md5": "a5a2f9aae57d464bc66b80ee79c3da6e",
|
||||
"sha256": "dca942d977217f703d8d1c6eb1aeb6b32c78ecc421486bbb46c459d385161c94",
|
||||
"crc32": "00eedb3a",
|
||||
"adler32": "94a45a21"
|
||||
},
|
||||
"aa811861f8874775075bd3f53008c8aaf59b07db": {
|
||||
"path": "bios/Pioneer/LaserActive/Pioneer LaserActive Sega PAC Boot ROM v1.04 (1993)(Pioneer - Sega)(US).bin",
|
||||
"name": "Pioneer LaserActive Sega PAC Boot ROM v1.04 (1993)(Pioneer - Sega)(US).bin",
|
||||
@@ -40863,6 +40853,16 @@
|
||||
"crc32": "50cd3d23",
|
||||
"adler32": "72a13133"
|
||||
},
|
||||
"26237b333db4a4c6770297fa5e655ea95840d5d9": {
|
||||
"path": "bios/Pioneer/LaserActive/Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin",
|
||||
"name": "Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin",
|
||||
"size": 131072,
|
||||
"sha1": "26237b333db4a4c6770297fa5e655ea95840d5d9",
|
||||
"md5": "a5a2f9aae57d464bc66b80ee79c3da6e",
|
||||
"sha256": "dca942d977217f703d8d1c6eb1aeb6b32c78ecc421486bbb46c459d385161c94",
|
||||
"crc32": "00eedb3a",
|
||||
"adler32": "94a45a21"
|
||||
},
|
||||
"6973e2593e66fd21627fedccec98d4a364afaaff": {
|
||||
"path": "bios/Pioneer/LaserActive/[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin",
|
||||
"name": "[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin",
|
||||
@@ -40883,6 +40883,26 @@
|
||||
"crc32": "01223dd5",
|
||||
"adler32": "74e98625"
|
||||
},
|
||||
"bc746df0c5d2b779ca41a94954b60d6ac6a7c2a3": {
|
||||
"path": "bios/Pioneer/LaserActive/[BIOS] LaserActive PAC-S1 (Japan) (v1.01).bin",
|
||||
"name": "[BIOS] LaserActive PAC-S1 (Japan) (v1.01).bin",
|
||||
"size": 131072,
|
||||
"sha1": "bc746df0c5d2b779ca41a94954b60d6ac6a7c2a3",
|
||||
"md5": "219dc2aa7cb8d1ad5142c86d50c2ffa5",
|
||||
"sha256": "3300e3771b468b41818b90c2358ff288da69bada92b8247acd793a437b30731c",
|
||||
"crc32": "24ad336e",
|
||||
"adler32": "6993a1dd"
|
||||
},
|
||||
"f7101b40cae0484a0f43f3bbee2d55033ff1e3ec": {
|
||||
"path": "bios/Pioneer/LaserActive/[BIOS] LaserActive PAC-S1 (Japan) (v1.05).bin",
|
||||
"name": "[BIOS] LaserActive PAC-S1 (Japan) (v1.05).bin",
|
||||
"size": 131072,
|
||||
"sha1": "f7101b40cae0484a0f43f3bbee2d55033ff1e3ec",
|
||||
"md5": "515636778430c8e01027234c38c4ddf8",
|
||||
"sha256": "369c1e699abbaecd231a3ab7b98443d28097366ff3cdd978698417195d54ca3c",
|
||||
"crc32": "1493522c",
|
||||
"adler32": "9e972082"
|
||||
},
|
||||
"c1b9202cbe072db12114b223a9ba5374b30718fb": {
|
||||
"path": "bios/Pioneer/LaserActive/[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin",
|
||||
"name": "[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin",
|
||||
@@ -76480,10 +76500,12 @@
|
||||
"c500ff71236068e0dc0d0603d265ae76": "5130243429b40b01a14e1304d0394b8459a6fbae",
|
||||
"f1071cdb0b6b10dde94d3bc8a6146387": "a6120aed50831c9c0d95dbdf707820f601d9452e",
|
||||
"279008e4a0db2dc5f1c048853b033828": "54b8d2c1317628de51a85fc1c424423a986775e4",
|
||||
"a5a2f9aae57d464bc66b80ee79c3da6e": "26237b333db4a4c6770297fa5e655ea95840d5d9",
|
||||
"0e7393cd0951d6dde818fcd4cd819466": "aa811861f8874775075bd3f53008c8aaf59b07db",
|
||||
"a5a2f9aae57d464bc66b80ee79c3da6e": "26237b333db4a4c6770297fa5e655ea95840d5d9",
|
||||
"f69f173b251d8bf7649b10a9167a10bf": "6973e2593e66fd21627fedccec98d4a364afaaff",
|
||||
"f0fb8a4605ac7eefbafd4f2d5a793cc8": "f7412aa822d70a55b2ff3d7095137263dc54f6b6",
|
||||
"219dc2aa7cb8d1ad5142c86d50c2ffa5": "bc746df0c5d2b779ca41a94954b60d6ac6a7c2a3",
|
||||
"515636778430c8e01027234c38c4ddf8": "f7101b40cae0484a0f43f3bbee2d55033ff1e3ec",
|
||||
"761fea207d0eafd4cfd78da7c44cac88": "c1b9202cbe072db12114b223a9ba5374b30718fb",
|
||||
"69489153dde910a69d5ae6de5dd65323": "f2a9ce387019bf272c6e3459d961b30f28942ac5",
|
||||
"3fd0d13282b031f4c017cd6bf6597183": "9dda4cbcc1d3f6d38c10fee3de53b9abd5e47ec0",
|
||||
@@ -91151,18 +91173,24 @@
|
||||
"jopac.bin": [
|
||||
"54b8d2c1317628de51a85fc1c424423a986775e4"
|
||||
],
|
||||
"Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin": [
|
||||
"26237b333db4a4c6770297fa5e655ea95840d5d9"
|
||||
],
|
||||
"Pioneer LaserActive Sega PAC Boot ROM v1.04 (1993)(Pioneer - Sega)(US).bin": [
|
||||
"aa811861f8874775075bd3f53008c8aaf59b07db"
|
||||
],
|
||||
"Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin": [
|
||||
"26237b333db4a4c6770297fa5e655ea95840d5d9"
|
||||
],
|
||||
"[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin": [
|
||||
"6973e2593e66fd21627fedccec98d4a364afaaff"
|
||||
],
|
||||
"[BIOS] LaserActive PAC-N10 (US) (v1.02).bin": [
|
||||
"f7412aa822d70a55b2ff3d7095137263dc54f6b6"
|
||||
],
|
||||
"[BIOS] LaserActive PAC-S1 (Japan) (v1.01).bin": [
|
||||
"bc746df0c5d2b779ca41a94954b60d6ac6a7c2a3"
|
||||
],
|
||||
"[BIOS] LaserActive PAC-S1 (Japan) (v1.05).bin": [
|
||||
"f7101b40cae0484a0f43f3bbee2d55033ff1e3ec"
|
||||
],
|
||||
"[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin": [
|
||||
"c1b9202cbe072db12114b223a9ba5374b30718fb"
|
||||
],
|
||||
@@ -101218,6 +101246,9 @@
|
||||
"g7400.bin": [
|
||||
"5130243429b40b01a14e1304d0394b8459a6fbae"
|
||||
],
|
||||
"[BIOS] LaserActive PAC-S10 (US) (v1.04).bin": [
|
||||
"aa811861f8874775075bd3f53008c8aaf59b07db"
|
||||
],
|
||||
"Battle 1.mid": [
|
||||
"dc5cc32fafa442b09ab2d814ed32074d02597234"
|
||||
],
|
||||
@@ -102371,6 +102402,9 @@
|
||||
"ts1500.zxpand.ovl": [
|
||||
"0334b35f164089df7b2a82d46fa0ec6e43fafa90"
|
||||
],
|
||||
"tvc64.zip": [
|
||||
"abf119cf947ea32defd08b29a8a25d75f6bd4987"
|
||||
],
|
||||
"TVC22_D7.64K": [
|
||||
"abf119cf947ea32defd08b29a8a25d75f6bd4987"
|
||||
],
|
||||
@@ -102413,6 +102447,24 @@
|
||||
"EXOS21.ROM": [
|
||||
"55315b20fecb4441a07ee4bc5dc7153f396e0a2e"
|
||||
],
|
||||
"bk0010.zip": [
|
||||
"4e83a94ae5155bbea14d7331a5a8db82457bd5ae",
|
||||
"34fa37599f2f9eb607390ef2458a3c22d87f09a9",
|
||||
"f087af69044432a1ef2431a72ac06946e32f2dd3",
|
||||
"7e9a30e38d7b78981999821640a68a201bb6df01"
|
||||
],
|
||||
"monit10.rom": [
|
||||
"4e83a94ae5155bbea14d7331a5a8db82457bd5ae"
|
||||
],
|
||||
"bas11m_1.rom": [
|
||||
"34fa37599f2f9eb607390ef2458a3c22d87f09a9"
|
||||
],
|
||||
"b11m_ext.rom": [
|
||||
"f087af69044432a1ef2431a72ac06946e32f2dd3"
|
||||
],
|
||||
"b11m_bos.rom": [
|
||||
"7e9a30e38d7b78981999821640a68a201bb6df01"
|
||||
],
|
||||
"sony-playstation:239665b1a3dade1b5a52c06338011044": [
|
||||
"343883a7b555646da8cee54aadd2795b6e7dd070"
|
||||
],
|
||||
@@ -103185,27 +103237,15 @@
|
||||
"Kickstart v3.0 rev 39.106 (1992)(Commodore)(A4000)[!].rom": [
|
||||
"f0b4e9e29e12218c2d5bd7020e4e785297d91fd7"
|
||||
],
|
||||
"monit10.rom": [
|
||||
"4e83a94ae5155bbea14d7331a5a8db82457bd5ae"
|
||||
],
|
||||
"focal10.rom": [
|
||||
"6386e58bc1bba5e76baec9e8a1ca4b99dc3c573f"
|
||||
],
|
||||
"disk_327.rom": [
|
||||
"28eefbb63047b26e4aec104aeeca74e2f9d0276c"
|
||||
],
|
||||
"b11m_bos.rom": [
|
||||
"7e9a30e38d7b78981999821640a68a201bb6df01"
|
||||
],
|
||||
"b11m_ext.rom": [
|
||||
"f087af69044432a1ef2431a72ac06946e32f2dd3"
|
||||
],
|
||||
"bas11m_0.rom": [
|
||||
"9d76f3eefd64e032c763fa1ebf9cd3d9bd22317a"
|
||||
],
|
||||
"bas11m_1.rom": [
|
||||
"34fa37599f2f9eb607390ef2458a3c22d87f09a9"
|
||||
],
|
||||
"terak.rom": [
|
||||
"273a9933b68a290c5aedcd6d69faa7b1d22c0344"
|
||||
],
|
||||
@@ -107481,10 +107521,12 @@
|
||||
"e20a9f41": "5130243429b40b01a14e1304d0394b8459a6fbae",
|
||||
"a318e8d6": "a6120aed50831c9c0d95dbdf707820f601d9452e",
|
||||
"11647ca5": "54b8d2c1317628de51a85fc1c424423a986775e4",
|
||||
"00eedb3a": "26237b333db4a4c6770297fa5e655ea95840d5d9",
|
||||
"50cd3d23": "aa811861f8874775075bd3f53008c8aaf59b07db",
|
||||
"00eedb3a": "26237b333db4a4c6770297fa5e655ea95840d5d9",
|
||||
"a8cb694c": "6973e2593e66fd21627fedccec98d4a364afaaff",
|
||||
"01223dd5": "f7412aa822d70a55b2ff3d7095137263dc54f6b6",
|
||||
"24ad336e": "bc746df0c5d2b779ca41a94954b60d6ac6a7c2a3",
|
||||
"1493522c": "f7101b40cae0484a0f43f3bbee2d55033ff1e3ec",
|
||||
"76116a02": "c1b9202cbe072db12114b223a9ba5374b30718fb",
|
||||
"4e70e3c0": "f2a9ce387019bf272c6e3459d961b30f28942ac5",
|
||||
"92bcc762": "9dda4cbcc1d3f6d38c10fee3de53b9abd5e47ec0",
|
||||
@@ -124501,6 +124543,10 @@
|
||||
"9451a1a09d8f75944dbd6f91193fc360f1de80ac",
|
||||
"03bbb386cf530e804363acdfc1d13e64cf28af2e",
|
||||
"55315b20fecb4441a07ee4bc5dc7153f396e0a2e",
|
||||
"4e83a94ae5155bbea14d7331a5a8db82457bd5ae",
|
||||
"34fa37599f2f9eb607390ef2458a3c22d87f09a9",
|
||||
"f087af69044432a1ef2431a72ac06946e32f2dd3",
|
||||
"7e9a30e38d7b78981999821640a68a201bb6df01",
|
||||
"343883a7b555646da8cee54aadd2795b6e7dd070",
|
||||
"e38466a4ba8005fba7e9e3c7b9efeba7205bee3f",
|
||||
"ffa7f9a7fb19d773a0c3985a541c8e5623d2c30d",
|
||||
@@ -124623,13 +124669,9 @@
|
||||
"4192c505d130f446b2ada6bdc91dae730acafb4c",
|
||||
"16df8b5fd524c5a1c7584b2457ac15aff9e3ad6d",
|
||||
"f0b4e9e29e12218c2d5bd7020e4e785297d91fd7",
|
||||
"4e83a94ae5155bbea14d7331a5a8db82457bd5ae",
|
||||
"6386e58bc1bba5e76baec9e8a1ca4b99dc3c573f",
|
||||
"28eefbb63047b26e4aec104aeeca74e2f9d0276c",
|
||||
"7e9a30e38d7b78981999821640a68a201bb6df01",
|
||||
"f087af69044432a1ef2431a72ac06946e32f2dd3",
|
||||
"9d76f3eefd64e032c763fa1ebf9cd3d9bd22317a",
|
||||
"34fa37599f2f9eb607390ef2458a3c22d87f09a9",
|
||||
"273a9933b68a290c5aedcd6d69faa7b1d22c0344",
|
||||
"4891d739a8a8b67923681bad4fb67edab2e90e50",
|
||||
"6e89d1227581c76441a53d605f9e324185f1da33",
|
||||
@@ -124956,6 +124998,9 @@
|
||||
"roms/win486/ALI1429G.AMW": [
|
||||
"72c60172fb1ba77c9b24b06b7755f0a16f0b3a13"
|
||||
],
|
||||
".variants/[BIOS] LaserActive PAC-S10 (US) (v1.04).bin": [
|
||||
"aa811861f8874775075bd3f53008c8aaf59b07db"
|
||||
],
|
||||
"rtp/2003/Battle/Arrow.png": [
|
||||
"7aa8d4c377efcea1c9fad01924da1ba7b8575e1e"
|
||||
],
|
||||
|
||||
@@ -5,7 +5,7 @@ source: "https://github.com/libretro/FBNeo"
|
||||
upstream: "https://github.com/finalburnneo/FBNeo"
|
||||
logo: "https://raw.githubusercontent.com/finalburnneo/FBNeo/master/projectfiles/xcode/Emulator/Assets.xcassets/AppIcon.appiconset/icon_512.png"
|
||||
profiled_date: "2026-03-23"
|
||||
core_version: "v1.0.0.03"
|
||||
core_version: "v1.0.0.2"
|
||||
display_name: "Arcade (FinalBurn Neo)"
|
||||
cores:
|
||||
- fbneo
|
||||
@@ -1485,3 +1485,94 @@ files:
|
||||
size: 155000000
|
||||
note: "Two Tigers with Journey CD audio samples (2 WAVs)"
|
||||
source_ref: "src/burn/snd/samples.cpp"
|
||||
|
||||
- name: "coleco.rom"
|
||||
archive: cv_coleco.zip
|
||||
required: true
|
||||
size: 8192
|
||||
crc32: "3aa93ef3"
|
||||
source_ref: "src/burn/drv/coleco/d_coleco.cpp:1079"
|
||||
|
||||
- name: "colecoa.rom"
|
||||
archive: cv_coleco.zip
|
||||
required: true
|
||||
size: 8192
|
||||
crc32: "39bb16fc"
|
||||
source_ref: "src/burn/drv/coleco/d_coleco.cpp:1079"
|
||||
|
||||
- name: "svi603.rom"
|
||||
archive: cv_coleco.zip
|
||||
required: true
|
||||
size: 8192
|
||||
crc32: "19e91b82"
|
||||
source_ref: "src/burn/drv/coleco/d_coleco.cpp:1079"
|
||||
|
||||
- name: "czz50.rom"
|
||||
archive: cv_coleco.zip
|
||||
required: true
|
||||
size: 16384
|
||||
crc32: "4999abc6"
|
||||
source_ref: "src/burn/drv/coleco/d_coleco.cpp:1079"
|
||||
|
||||
- name: "fdsbios.nes"
|
||||
archive: fds_fdsbios.zip
|
||||
required: true
|
||||
size: 8192
|
||||
crc32: "5e607dcf"
|
||||
source_ref: "src/burn/drv/nes/d_nes.cpp:523"
|
||||
|
||||
- name: "st010.bin"
|
||||
archive: snes_st010.zip
|
||||
required: true
|
||||
size: 69632
|
||||
crc32: "aa11ee2d"
|
||||
source_ref: "src/burn/drv/snes/d_snes.cpp:577"
|
||||
|
||||
- name: "st011.bin"
|
||||
archive: snes_st011.zip
|
||||
required: true
|
||||
size: 69632
|
||||
crc32: "34d2952c"
|
||||
source_ref: "src/burn/drv/snes/d_snes.cpp:596"
|
||||
|
||||
- name: "msx.rom"
|
||||
archive: msx_msx.zip
|
||||
required: true
|
||||
size: 32768
|
||||
crc32: "a317e6b4"
|
||||
source_ref: "src/burn/drv/msx/d_msx.cpp:1781"
|
||||
|
||||
- name: "msxj.rom"
|
||||
archive: msx_msx.zip
|
||||
required: true
|
||||
size: 32768
|
||||
crc32: "071135e0"
|
||||
source_ref: "src/burn/drv/msx/d_msx.cpp:1781"
|
||||
|
||||
- name: "kanji.rom"
|
||||
archive: msx_msx.zip
|
||||
required: true
|
||||
size: 262144
|
||||
crc32: "1f6406fb"
|
||||
source_ref: "src/burn/drv/msx/d_msx.cpp:1781"
|
||||
|
||||
- name: "supernova_modbios-japan.u10"
|
||||
archive: skns.zip
|
||||
required: true
|
||||
size: 524288
|
||||
crc32: "b8d3190c"
|
||||
source_ref: "src/burn/drv/pst90s/d_suprnova.cpp:1865"
|
||||
|
||||
- name: "supernova-modbios-korea.u10"
|
||||
archive: skns.zip
|
||||
required: true
|
||||
size: 524288
|
||||
crc32: "1d90517c"
|
||||
source_ref: "src/burn/drv/pst90s/d_suprnova.cpp:1865"
|
||||
|
||||
- name: "mcu"
|
||||
archive: bubsys.zip
|
||||
required: true
|
||||
size: 4096
|
||||
crc32: "00000000"
|
||||
source_ref: "src/burn/drv/konami/d_nemesis.cpp:4539"
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
emulator: LRPS2
|
||||
type: libretro
|
||||
core_classification: community_fork
|
||||
bios_mode: agnostic
|
||||
source: "https://github.com/libretro/ps2"
|
||||
upstream: "https://github.com/PCSX2/pcsx2"
|
||||
profiled_date: "2026-03-25"
|
||||
|
||||
3412
emulators/mame.yml
3412
emulators/mame.yml
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -77,11 +77,14 @@ files:
|
||||
source_ref: "src/SPI.cpp:197-211, src/frontend/Util_ROM.cpp:201-217"
|
||||
|
||||
- name: dsi_nand.bin
|
||||
agnostic: true
|
||||
system: nintendo-dsi
|
||||
description: "DSi NAND dump"
|
||||
required: true
|
||||
source_ref: "src/frontend/Util_ROM.cpp:224-235, src/DSi_NAND.cpp"
|
||||
note: "Uses AES keys from ARM7i BIOS offset 0x8308"
|
||||
size: 251658304
|
||||
storage: large_file
|
||||
source_ref: "src/frontend/Util_ROM.cpp:224-235, src/DSi_NAND.cpp:58-97"
|
||||
note: "Any regional dump works. Nocash footer required (DSi eMMC CID/CPU at EOF-0x40 or 0xFF800). AES keys derived from ARM7i BIOS offset 0x8308."
|
||||
|
||||
- name: dsi_sd_card.bin
|
||||
system: nintendo-dsi
|
||||
|
||||
@@ -1,219 +1,74 @@
|
||||
# PCSX2 emulator BIOS profile
|
||||
# Generated from source analysis of https://github.com/PCSX2/pcsx2
|
||||
# Commit analyzed: HEAD as of 2026-03-17
|
||||
|
||||
emulator: PCSX2
|
||||
type: standalone
|
||||
core_classification: official_port
|
||||
bios_mode: agnostic
|
||||
source: "https://github.com/PCSX2/pcsx2"
|
||||
logo: "https://raw.githubusercontent.com/PCSX2/pcsx2/master/pcsx2-qt/resources/icons/PCSX2logo.svg"
|
||||
profiled_date: "2026-03-18"
|
||||
upstream: "https://github.com/PCSX2/pcsx2"
|
||||
cores:
|
||||
- pcsx2
|
||||
profiled_date: "2026-03-30"
|
||||
core_version: "Git"
|
||||
display_name: "Sony - PlayStation 2 (LRPS2)"
|
||||
display_name: "Sony - PlayStation 2 (PCSX2)"
|
||||
systems: [sony-playstation-2]
|
||||
|
||||
bios_directory: "bios/"
|
||||
bios_detection: "romdir" # scans romdir structure inside binary, looks for RESET/ROMVER/EXTINFO entries
|
||||
bios_selection: "automatic" # scans all files in bios dir matching 4-8 MB size, validates via romdir
|
||||
|
||||
validation:
|
||||
method: "romdir_parse"
|
||||
min_size: 4194304 # 4 MB (MIN_BIOS_SIZE = 4 * _1mb)
|
||||
max_size: 8388608 # 8 MB (MAX_BIOS_SIZE = 8 * _1mb)
|
||||
required_entries: ["RESET", "ROMVER"]
|
||||
optional_entries: ["EXTINFO"]
|
||||
note: "Any file in bios/ between 4-8 MB with valid romdir containing RESET+ROMVER is accepted"
|
||||
|
||||
regions:
|
||||
J: {zone: "Japan", id: 0}
|
||||
A: {zone: "USA", id: 1}
|
||||
E: {zone: "Europe", id: 2}
|
||||
H: {zone: "Asia", id: 4}
|
||||
C: {zone: "China", id: 6}
|
||||
T: {zone: "T10K/COH-H", id: 8}
|
||||
X: {zone: "Test", id: 9}
|
||||
P: {zone: "Free", id: 10}
|
||||
notes: |
|
||||
Filename-agnostic BIOS detection. Scans bios/ for any file between 4-8 MB
|
||||
with valid romdir structure (RESET + ROMVER entries). No hash validation.
|
||||
Companion files (.rom1, .rom2, .nvm, .mec) derive paths from selected BIOS.
|
||||
ROM1 (DVD player) and ROM2 (Chinese extension) silently skipped if missing.
|
||||
NVM and MEC auto-created with defaults if missing.
|
||||
|
||||
files:
|
||||
# -- Main BIOS binary (required) --
|
||||
- name: "<user-selected>.bin"
|
||||
pattern: "*"
|
||||
- name: ps2-0230a-20080220.bin
|
||||
required: true
|
||||
size_range: "4MB-8MB"
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:258-282"
|
||||
note: >
|
||||
PCSX2 does not mandate a specific filename. It scans the entire bios/ directory
|
||||
for any file between 4-8 MB that contains a valid romdir structure (RESET + ROMVER entries).
|
||||
Common filenames follow the SCPH-XXXXX_BIOS_VYYY_REGION_ZZZ.BIN convention but this is
|
||||
not enforced. The file is loaded into the 4 MB ROM region of EE memory.
|
||||
min_size: 4194304
|
||||
max_size: 8388608
|
||||
validation: [size]
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:258-362"
|
||||
note: "Accepts any file 4-8 MB with valid romdir (RESET + ROMVER). Naming convention ps2-VVVVr-YYYYMMDD.bin (version, region, date)."
|
||||
|
||||
# -- ROM1 (optional, DVD player) --
|
||||
- name: "<biosname>.rom1"
|
||||
pattern: "{biosname}.rom1 or {biosbase}.rom1"
|
||||
- name: rom1.bin
|
||||
required: false
|
||||
max_size: 4194304 # 4 MB (Ps2MemSize::Rom1)
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:214-241"
|
||||
note: >
|
||||
DVD player ROM. Loaded via LoadExtraRom("rom1"). PCSX2 tries two naming patterns:
|
||||
1) Full bios path + ".rom1" appended (e.g. scph70004.bin.rom1)
|
||||
2) Bios path with extension replaced (e.g. scph70004.rom1)
|
||||
Mapped to EE memory at ROM1 region (0x1FC00000 + 4MB offset).
|
||||
Contains DVD player and region detection data (DVDID).
|
||||
max_size: 4194304
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:214-241,366"
|
||||
note: "DVD player ROM. Tries {biospath}.rom1 then {biosbase}.rom1. Silently skipped if missing."
|
||||
|
||||
# -- ROM2 (optional, Chinese ROM extension) --
|
||||
- name: "<biosname>.rom2"
|
||||
pattern: "{biosname}.rom2 or {biosbase}.rom2"
|
||||
- name: ROM2.BIN
|
||||
required: false
|
||||
max_size: 4194304 # 4 MB (Ps2MemSize::Rom2)
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:214-241"
|
||||
note: >
|
||||
Chinese ROM extension. Loaded via LoadExtraRom("rom2"). Same naming convention
|
||||
as rom1: tries appended extension first, then replaced extension.
|
||||
Only present on Chinese region consoles.
|
||||
max_size: 4194304
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:214-241,367"
|
||||
note: "Chinese ROM extension. Same naming convention as rom1. Only present on Chinese region consoles."
|
||||
|
||||
# -- NVM / NVRAM (optional, auto-created) --
|
||||
- name: "<biosname>.nvm"
|
||||
pattern: "{biosbase}.nvm"
|
||||
- name: EROM.BIN
|
||||
required: false
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp"
|
||||
note: "Extended ROM. Present in some BIOS dumps but not loaded by PCSX2 code via LoadExtraRom."
|
||||
path: null
|
||||
|
||||
- name: eeprom.dat
|
||||
required: false
|
||||
hle_fallback: true
|
||||
size: 1024 # NVRAM_SIZE = 1024 bytes
|
||||
source_ref: "pcsx2/CDVD/CDVD.cpp:160-238"
|
||||
note: >
|
||||
EEPROM / NVRAM data. Path derived from BiosPath with extension replaced to ".nvm"
|
||||
(cdvdGetNVRAMPath). Contains console configuration: language, timezone, iLink ID,
|
||||
region parameters, OSD settings. Auto-created with defaults if missing.
|
||||
Two NVM layouts exist: v0.00+ (biosVer 0x000) and v1.70+ (biosVer 0x146).
|
||||
|
||||
# -- MEC file (optional, auto-created) --
|
||||
- name: "<biosname>.mec"
|
||||
pattern: "{biosbase}.mec"
|
||||
required: false
|
||||
hle_fallback: true
|
||||
size: 4 # u32 s_mecha_version
|
||||
source_ref: "pcsx2/CDVD/CDVD.cpp:190-204"
|
||||
note: >
|
||||
Mechacon (mechanism controller) version file. 4 bytes containing the mecha version
|
||||
as a u32 value. Auto-created with DEFAULT_MECHA_VERSION (0x00020603) if missing.
|
||||
Path derived from BiosPath with extension replaced to ".mec".
|
||||
|
||||
# -- IRX override (optional, advanced) --
|
||||
- name: "<custom>.irx"
|
||||
pattern: "*.irx"
|
||||
required: false
|
||||
source_ref: "pcsx2/ps2/BiosTools.cpp:243-256,384-385"
|
||||
note: >
|
||||
Custom IOP Reboot eXecutable module. Loaded into ROM at offset 0x3C0000 if
|
||||
EmuConfig.CurrentIRX is set (path length > 3). Injected at IOP reset (PC=0x1630).
|
||||
Used for debugging/development, not needed for normal operation.
|
||||
|
||||
# -- DEV9 EEPROM (optional, network adapter) --
|
||||
- name: "eeprom.dat"
|
||||
required: false
|
||||
hle_fallback: true
|
||||
size: 64 # 64 bytes, mmap'd
|
||||
size: 64
|
||||
source_ref: "pcsx2/DEV9/DEV9.cpp:110-160"
|
||||
note: >
|
||||
DEV9 (network adapter / HDD expansion bay) EEPROM data. Fixed filename "eeprom.dat"
|
||||
opened from working directory. Contains network adapter configuration.
|
||||
Falls back to built-in defaults if file not found. Only relevant when using
|
||||
DEV9 features (online play, HDD).
|
||||
note: "DEV9 network adapter EEPROM. Falls back to built-in defaults if missing."
|
||||
|
||||
common_bios_filenames:
|
||||
# Japan
|
||||
- "SCPH-10000_BIOS_V1_JAP_100.BIN"
|
||||
- "SCPH-15000_BIOS_V3_JAP_120.BIN"
|
||||
- "SCPH-30000_BIOS_V4_JAP_150.BIN"
|
||||
- "SCPH-30001R_BIOS_V7_JAP_160.BIN"
|
||||
- "SCPH-30004R_BIOS_V7_JAP_160.BIN"
|
||||
- "SCPH-35000_BIOS_V5_JAP_160.BIN"
|
||||
- "SCPH-50000_BIOS_V9_JAP_170.BIN"
|
||||
- "SCPH-50004_BIOS_V9_JAP_170.BIN"
|
||||
- "SCPH-70000_BIOS_V12_JAP_200.BIN"
|
||||
- "SCPH-75000_BIOS_V14_JAP_220.BIN"
|
||||
- "SCPH-77000_BIOS_V14_JAP_220.BIN"
|
||||
- "SCPH-90000_BIOS_V18_JAP_230.BIN"
|
||||
# USA
|
||||
- "SCPH-30001_BIOS_V4_USA_150.BIN"
|
||||
- "SCPH-39001_BIOS_V6_USA_160.BIN"
|
||||
- "SCPH-50001_BIOS_V9_USA_170.BIN"
|
||||
- "SCPH-50003_BIOS_V9_USA_170.BIN"
|
||||
- "SCPH-70002_BIOS_V12_USA_200.BIN"
|
||||
- "SCPH-70004_BIOS_V12_USA_200.BIN"
|
||||
- "SCPH-70012_BIOS_V12_USA_200.BIN"
|
||||
- "SCPH-75001_BIOS_V14_USA_220.BIN"
|
||||
- "SCPH-77001_BIOS_V14_USA_220.BIN"
|
||||
- "SCPH-90001_BIOS_V18_USA_230.BIN"
|
||||
# Europe
|
||||
- "SCPH-30002_BIOS_V4_EUR_150.BIN"
|
||||
- "SCPH-30003_BIOS_V4_EUR_150.BIN"
|
||||
- "SCPH-30004_BIOS_V4_EUR_150.BIN"
|
||||
- "SCPH-39002_BIOS_V6_EUR_160.BIN"
|
||||
- "SCPH-39003_BIOS_V6_EUR_160.BIN"
|
||||
- "SCPH-39004_BIOS_V6_EUR_160.BIN"
|
||||
- "SCPH-50002_BIOS_V9_EUR_170.BIN"
|
||||
- "SCPH-50004_BIOS_V9_EUR_170.BIN"
|
||||
- "SCPH-70002_BIOS_V12_EUR_200.BIN"
|
||||
- "SCPH-70003_BIOS_V12_EUR_200.BIN"
|
||||
- "SCPH-70004_BIOS_V12_EUR_200.BIN"
|
||||
- "SCPH-70008_BIOS_V12_EUR_200.BIN"
|
||||
- "SCPH-75002_BIOS_V14_EUR_220.BIN"
|
||||
- "SCPH-75003_BIOS_V14_EUR_220.BIN"
|
||||
- "SCPH-75004_BIOS_V14_EUR_220.BIN"
|
||||
- "SCPH-77002_BIOS_V14_EUR_220.BIN"
|
||||
- "SCPH-77003_BIOS_V14_EUR_220.BIN"
|
||||
- "SCPH-77004_BIOS_V14_EUR_220.BIN"
|
||||
- "SCPH-90002_BIOS_V18_EUR_230.BIN"
|
||||
- "SCPH-90003_BIOS_V18_EUR_230.BIN"
|
||||
- "SCPH-90004_BIOS_V18_EUR_230.BIN"
|
||||
# Asia
|
||||
- "SCPH-50009_BIOS_V9_HK_170.BIN"
|
||||
- "SCPH-70005_BIOS_V12_HK_200.BIN"
|
||||
- "SCPH-70006_BIOS_V12_HK_200.BIN"
|
||||
- "SCPH-70008_BIOS_V12_HK_200.BIN"
|
||||
# China
|
||||
- "SCPH-50009_BIOS_V9_CHN_170.BIN"
|
||||
- "SCPH-70006_BIOS_V12_CHN_200.BIN"
|
||||
- name: GameIndex.yaml
|
||||
path: pcsx2/resources/GameIndex.yaml
|
||||
required: false
|
||||
mode: libretro
|
||||
source_ref: "pcsx2/GameDatabase.cpp:48,880"
|
||||
note: "Game compatibility database. OSD warning if missing."
|
||||
|
||||
memory_layout:
|
||||
ROM: {offset: "0x1FC00000", size: "4 MB", purpose: "Main BIOS binary"}
|
||||
ROM1: {offset: "ROM + 4MB", size: "4 MB", purpose: "DVD player"}
|
||||
ROM2: {offset: "ROM + 8MB", size: "4 MB", purpose: "Chinese ROM extension"}
|
||||
- name: cheats_ws.zip
|
||||
path: pcsx2/resources/cheats_ws.zip
|
||||
required: false
|
||||
mode: libretro
|
||||
source_ref: "pcsx2/VMManager.cpp:340-353"
|
||||
note: "Widescreen patches archive."
|
||||
|
||||
nvm_layout:
|
||||
format_0:
|
||||
applies_to: "BIOS v0.00+"
|
||||
biosVer: 0x000
|
||||
config0: 0x280
|
||||
config1: 0x300
|
||||
config2: 0x200
|
||||
consoleId: 0x1C8
|
||||
ilinkId: 0x1C0
|
||||
modelNum: 0x1A0
|
||||
regparams: 0x180
|
||||
mac: 0x198
|
||||
format_1:
|
||||
applies_to: "BIOS v1.70+"
|
||||
biosVer: 0x146
|
||||
config0: 0x270
|
||||
config1: 0x2B0
|
||||
config2: 0x200
|
||||
consoleId: 0x1F0
|
||||
ilinkId: 0x1E0
|
||||
modelNum: 0x1B0
|
||||
regparams: 0x180
|
||||
mac: 0x198
|
||||
|
||||
notes: |
|
||||
PCSX2 is filename-agnostic for the main BIOS. Detection relies on romdir structure
|
||||
parsing inside the binary itself, not on filename or extension. Any file between 4-8 MB
|
||||
with a valid romdir (containing at least RESET and ROMVER entries) is accepted.
|
||||
|
||||
The ROMVER entry encodes: version (2+2 digits), region letter, console/devel flag,
|
||||
build date (YYYYMMDD), and is used to determine the BIOS description and region.
|
||||
|
||||
Companion files (.nvm, .mec) are auto-created with sane defaults if missing.
|
||||
ROM1/ROM2 are silently skipped if not found - only the main BIOS binary is strictly required.
|
||||
|
||||
PCSX2 no longer ships as a libretro core in official builds. The standalone emulator
|
||||
is the primary distribution channel.
|
||||
|
||||
Devel console BIOSes (< ~2.3 MB) lack the OSD and are handled with NoOSD=true flag.
|
||||
- name: cheats_ni.zip
|
||||
path: pcsx2/resources/cheats_ni.zip
|
||||
required: false
|
||||
mode: libretro
|
||||
source_ref: "pcsx2/VMManager.cpp:375-388"
|
||||
note: "No-interlacing patches archive."
|
||||
|
||||
@@ -1,32 +1,21 @@
|
||||
# RPCS3 emulator firmware profile
|
||||
# Generated from source analysis of https://github.com/RPCS3/rpcs3
|
||||
# Commit analyzed: HEAD as of 2026-03-17
|
||||
|
||||
emulator: RPCS3
|
||||
type: standalone
|
||||
core_classification: official_port
|
||||
source: "https://github.com/RPCS3/rpcs3"
|
||||
logo: "https://raw.githubusercontent.com/RPCS3/rpcs3/master/rpcs3/rpcs3.svg"
|
||||
profiled_date: "2026-03-18"
|
||||
upstream: "https://github.com/RPCS3/rpcs3"
|
||||
cores:
|
||||
- rpcs3
|
||||
profiled_date: "2026-03-30"
|
||||
core_version: "0.0.35"
|
||||
display_name: "RPCS3 (PS3)"
|
||||
display_name: "Sony - PlayStation 3 (RPCS3)"
|
||||
systems: [sony-playstation-3]
|
||||
|
||||
firmware_file: "PS3UPDAT.PUP"
|
||||
firmware_source: "https://www.playstation.com/en-us/support/hardware/ps3/system-software/"
|
||||
firmware_detection: "pup_header" # validates PUP magic bytes, HMAC-SHA1 hash per entry
|
||||
firmware_install: "extracts dev_flash_* TAR packages from PUP into dev_flash/"
|
||||
|
||||
validation:
|
||||
method: "pup_object"
|
||||
magic: "SCEUF"
|
||||
hash_algo: "HMAC-SHA1"
|
||||
source_ref: "rpcs3/Loader/PUP.cpp:8-114"
|
||||
note: "PUP file is validated by magic header, file count, HMAC-SHA1 per entry against PUP_KEY"
|
||||
|
||||
firmware_version:
|
||||
path: "dev_flash/vsh/etc/version.txt"
|
||||
source_ref: "rpcs3/util/sysinfo.cpp:686"
|
||||
note: "Read at startup, displayed as 'Firmware version: X.XX'. Missing = 'Missing Firmware'"
|
||||
files:
|
||||
- name: PS3UPDAT.PUP
|
||||
required: true
|
||||
storage: large_file
|
||||
source_ref: "rpcs3/Loader/PUP.cpp:23-77"
|
||||
note: "PUP firmware package. Validated via SCEUF magic + HMAC-SHA1 per entry. Extracted to dev_flash/ at install time."
|
||||
|
||||
# dev_flash filesystem layout extracted from PUP
|
||||
dev_flash:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
1160
install/bizhawk.json
1160
install/bizhawk.json
File diff suppressed because it is too large
Load Diff
@@ -3,7 +3,7 @@
|
||||
"platform": "emudeck",
|
||||
"display_name": "EmuDeck",
|
||||
"version": "1.0",
|
||||
"generated": "2026-03-29T14:02:36Z",
|
||||
"generated": "2026-03-30T09:46:25Z",
|
||||
"base_destination": "bios",
|
||||
"detect": [
|
||||
{
|
||||
@@ -50,8 +50,8 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"total_files": 270,
|
||||
"total_size": 913637720,
|
||||
"total_files": 290,
|
||||
"total_size": 717377323,
|
||||
"files": [
|
||||
{
|
||||
"dest": "colecovision.rom",
|
||||
@@ -473,24 +473,6 @@
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1000.bin",
|
||||
"sha1": "343883a7b555646da8cee54aadd2795b6e7dd070",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1000.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1001.bin",
|
||||
"sha1": "10155d8d6e6e832d6ea66db9bc098321fb5e8ebf",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1001.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1002a.bin",
|
||||
"sha1": "20b98f3d80f11cbf5a7bfd0779b0e63760ecc62c",
|
||||
@@ -500,132 +482,6 @@
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1002b.bin",
|
||||
"sha1": "76cf6b1b2a7c571a6ad07f2bac0db6cd8f71e2cc",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1002b.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1002c.bin",
|
||||
"sha1": "b6a11579caef3875504fcf3831b8e3922746df2c",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1002c.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "dtlh1100.bin",
|
||||
"sha1": "73107d468fc7cb1d2c5b18b269715dd889ecef06",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/dtlh1100.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph3000.bin",
|
||||
"sha1": "b06f4a861f74270be819aa2a07db8d0563a7cc4e",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph3000.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1001_v20.bin",
|
||||
"sha1": "649895efd79d14790eabb362e94eb0622093dfb9",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1001_v20.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph3500.bin",
|
||||
"sha1": "e38466a4ba8005fba7e9e3c7b9efeba7205bee3f",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph3500.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1001_v21.bin",
|
||||
"sha1": "ca7af30b50d9756cbd764640126c454cff658479",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1001_v21.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph5000.bin",
|
||||
"sha1": "ffa7f9a7fb19d773a0c3985a541c8e5623d2c30d",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph5000.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph7000.bin",
|
||||
"sha1": "77b10118d21ac7ffa9b35f9c4fd814da240eb3e9",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph7000.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph7000w.bin",
|
||||
"sha1": "1b0dbdb23da9dc0776aac58d0755dc80fea20975",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph7000w.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph7001.bin",
|
||||
"sha1": "14df4f6c1e367ce097c11deae21566b4fe5647a9",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph7001.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph7002.bin",
|
||||
"sha1": "8d5de56a79954f29e9006929ba3fed9b6a418c1d",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph7002.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph100.bin",
|
||||
"sha1": "339a48f4fcf63e10b5b867b8c93cfd40945faf6c",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph100.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph101_v44.bin",
|
||||
"sha1": "7771d6e90980408f753891648685def6dd42ef6d",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph101_v44.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph101_v45.bin",
|
||||
"sha1": "dcffe16bd90a723499ad46c641424981338d8378",
|
||||
@@ -653,15 +509,6 @@
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scph1000r.bin",
|
||||
"sha1": "7082bd57141fa0007b3adcd031f7ba23a20108a0",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/scph1000r.bin",
|
||||
"cores": [
|
||||
"DuckStation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ps2_scph18000.bin",
|
||||
"sha1": "d7d6be084f51354bc951d8fa2d8d912aa70abc5e",
|
||||
@@ -2111,6 +1958,42 @@
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "astrocde.zip",
|
||||
"sha1": "8641b09be090c0dc45f4ee5459fec3cc6fb9d78e",
|
||||
"size": 6800,
|
||||
"repo_path": "bios/Bally/Astrocade/astrocde.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "apple2gs.zip",
|
||||
"sha1": "799e2fc90d6bfd8cb74e331e04d5afd36f2f21a1",
|
||||
"size": 174124,
|
||||
"repo_path": "bios/Apple/Apple II/apple2gs.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "casloopy.zip",
|
||||
"sha1": "144ea9d0f20113632dde6d21c62a01c02cf7666d",
|
||||
"size": 478191,
|
||||
"repo_path": "bios/Casio/Loopy/casloopy.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "pv2000.zip",
|
||||
"sha1": "c18f460301177b64bbfcbedc3010e734d0803d4c",
|
||||
"size": 13622,
|
||||
"repo_path": "bios/Casio/PV-2000/pv2000.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "adam.zip",
|
||||
"sha1": "dc4b77cd3ac45cf448e427eff5c0a7b138fd9d81",
|
||||
@@ -2219,6 +2102,303 @@
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "beena.zip",
|
||||
"sha1": "f63eca9a1b1e92ee1582ff5e4c0db55193f97e33",
|
||||
"size": 85176,
|
||||
"repo_path": "bios/Sega/Beena/beena.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "rx78.zip",
|
||||
"sha1": "b3a26e21574395a279a37922238802b349c303b7",
|
||||
"size": 6397,
|
||||
"repo_path": "bios/Bandai/RX-78/rx78.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "segaai.zip",
|
||||
"sha1": "39a0fec854e438f6757882604a8dc56e62e401d9",
|
||||
"size": 343220,
|
||||
"repo_path": "bios/Sega/AI/segaai.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "segaai_soundbox.zip",
|
||||
"sha1": "d90e0b3545e60c59c20a7fb32d0785f6c3abbf60",
|
||||
"size": 47380,
|
||||
"repo_path": "bios/Arcade/MAME/segaai_soundbox.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "lynx48k.zip",
|
||||
"sha1": "64947e9b7d17870839aba5d93217183d480ff897",
|
||||
"size": 25078,
|
||||
"repo_path": "bios/Camputers/Lynx/lynx48k.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "lynx96k.zip",
|
||||
"sha1": "df95ea702606e88b4c906a0233c8fec54c02ff01",
|
||||
"size": 32888,
|
||||
"repo_path": "bios/Camputers/Lynx/lynx96k.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "lynx128k.zip",
|
||||
"sha1": "5300a352007976102d61d53f1e8c48063d2f3026",
|
||||
"size": 24796,
|
||||
"repo_path": "bios/Camputers/Lynx/lynx128k.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "crvision.zip",
|
||||
"sha1": "87526fb12fabbcc01292dd01dc698fdc762ab7dc",
|
||||
"size": 441619,
|
||||
"repo_path": "bios/VTech/CreatiVision/crvision.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "laser310.zip",
|
||||
"sha1": "9fa5f366c4ec43d7c23f03f054733894bf42912f",
|
||||
"size": 42630,
|
||||
"repo_path": "bios/VTech/Laser 310/laser310.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "socrates.zip",
|
||||
"sha1": "6bbaa8f73027eaca2c6988ee3b9b3f3b0b29d18e",
|
||||
"size": 218299,
|
||||
"repo_path": "bios/VTech/Socrates/socrates.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "vsmile.zip",
|
||||
"sha1": "65e526ae3e4795a9186c53c7428d8b946170befe",
|
||||
"size": 3078731,
|
||||
"repo_path": "bios/VTech/V.Smile/vsmile.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "gamate.zip",
|
||||
"sha1": "e99667ea5cfe6a5eceb53faaa39cdda0cbf69c69",
|
||||
"size": 5072,
|
||||
"repo_path": "bios/Bit Corporation/Gamate/gamate.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "gamepock.zip",
|
||||
"sha1": "83e94c56f2fc4ab60de94021a1283cca09d65ee5",
|
||||
"size": 3455,
|
||||
"repo_path": "bios/Epoch/Game Pocket/gamepock.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "gmaster.zip",
|
||||
"sha1": "a89263689f19e046dab6dadea24c5d07270548a6",
|
||||
"size": 1686,
|
||||
"repo_path": "bios/Hartung/Game Master/gmaster.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "gamecom.zip",
|
||||
"sha1": "e0f5e2eced447abf9948342b2facc40179f0f527",
|
||||
"size": 145138,
|
||||
"repo_path": "bios/Tiger/Game.com/gamecom.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "gp32.zip",
|
||||
"sha1": "35def9830e797fb0788740b5da0a7202933554df",
|
||||
"size": 1378301,
|
||||
"repo_path": "bios/GamePark/GP32/gp32.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "fm7.zip",
|
||||
"sha1": "1c29ba16151255cd916f49ea25570956f1353b71",
|
||||
"size": 104276,
|
||||
"repo_path": "bios/Fujitsu/FM Towns/fm7.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "fm77av.zip",
|
||||
"sha1": "b558bb5472cfa727dccab92229c66afa0d64404f",
|
||||
"size": 46921,
|
||||
"repo_path": "bios/Fujitsu/FM Towns/fm77av.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "pegasus.zip",
|
||||
"sha1": "fc10ef402bcac78c70e1cff57d51613fa12202f9",
|
||||
"size": 27305,
|
||||
"repo_path": "bios/Arcade/MAME/pegasus.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "pcw8256.zip",
|
||||
"sha1": "f8a11b4e03bb8c3caee604ca591676eab6535d1b",
|
||||
"size": 1710,
|
||||
"repo_path": "bios/Amstrad/PCW/pcw8256.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "pcw9512.zip",
|
||||
"sha1": "844988dc5a36917b002d0140d3a7692d1407f783",
|
||||
"size": 2625,
|
||||
"repo_path": "bios/Amstrad/PCW/pcw9512.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "vis.zip",
|
||||
"sha1": "beff39b4edb449c66b704e1c64ee6ed58e30a868",
|
||||
"size": 605047,
|
||||
"repo_path": "bios/Tandy/VIS/vis.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "trs80.zip",
|
||||
"sha1": "a9b8931c42303e8dfb475112fc18ff200b7252ce",
|
||||
"size": 14558,
|
||||
"repo_path": "bios/Tandy/TRS-80/trs80.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "trs80m3.zip",
|
||||
"sha1": "b804a031c8db6def59e077a4b6938dcac25093d7",
|
||||
"size": 32854,
|
||||
"repo_path": "bios/Tandy/TRS-80/trs80m3.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "trs80m4.zip",
|
||||
"sha1": "bc58ac28be4854fb198b895c7fd999f599b5a961",
|
||||
"size": 25371,
|
||||
"repo_path": "bios/Tandy/TRS-80/trs80m4.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "trs80m4p.zip",
|
||||
"sha1": "497eea6e0ed80501ca3beee714d71a5815495b07",
|
||||
"size": 4880,
|
||||
"repo_path": "bios/Tandy/TRS-80/trs80m4p.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ti99_4a.zip",
|
||||
"sha1": "e05575b630bea7ff98b9ca1f083d745abb3110b6",
|
||||
"size": 21377,
|
||||
"repo_path": "bios/Texas Instruments/TI-99/ti99_4a.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ti99_speech.zip",
|
||||
"sha1": "b58fe2dbbc254d363c2ec4a459e6ec1d91b2ac86",
|
||||
"size": 31206,
|
||||
"repo_path": "bios/Arcade/Arcade/ti99_speech.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "tutor.zip",
|
||||
"sha1": "273dbb7a93eb0ab83c2e13e9db51b897cc18f838",
|
||||
"size": 34653,
|
||||
"repo_path": "bios/Tomy/Tutor/tutor.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "vg5k.zip",
|
||||
"sha1": "5ad489a58fc457d2bfb7a348f4a312c3b49daad1",
|
||||
"size": 26356,
|
||||
"repo_path": "bios/Arcade/MAME/vg5k.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "qsound.zip",
|
||||
"sha1": "f1ccedb2c6e8f15cfebab52c3923e4b35f4df43f",
|
||||
"size": 2654,
|
||||
"repo_path": "bios/Arcade/Arcade/qsound.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ym2413.zip",
|
||||
"sha1": "900e8c6cd152704a79905b6838fe3229c9499f8c",
|
||||
"size": 286,
|
||||
"repo_path": "bios/Arcade/MAME/ym2413.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ym2608.zip",
|
||||
"sha1": "06fc753d015b43ca1787f4cfd9331b1674202e64",
|
||||
"size": 7609,
|
||||
"repo_path": "bios/Arcade/Arcade/ym2608.zip",
|
||||
"cores": [
|
||||
"MAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "dsi_bios9.bin",
|
||||
"sha1": "db61fa39ddbc5f5ed71fb19cda47609ef0201723",
|
||||
@@ -2246,17 +2426,6 @@
|
||||
"melonDS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "dsi_nand.bin",
|
||||
"sha1": "b48f44194fe918aaaec5298861479512b581d661",
|
||||
"size": 251658304,
|
||||
"repo_path": ".cache/large/dsi_nand.bin",
|
||||
"cores": [
|
||||
"melonDS"
|
||||
],
|
||||
"storage": "release",
|
||||
"release_asset": "dsi_nand.bin"
|
||||
},
|
||||
{
|
||||
"dest": "dsi_sd_card.bin",
|
||||
"sha1": "3b71f43ff30f4b15b5cd85dd9e95ebc7e84eb5a3",
|
||||
@@ -2365,6 +2534,17 @@
|
||||
"shadps4"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "psvita/PSP2UPDAT.PUP",
|
||||
"sha1": "3ae832c9800fcaa007eccfc48f24242967c111f8",
|
||||
"size": 56768512,
|
||||
"repo_path": "bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP",
|
||||
"cores": [
|
||||
"Vita3K"
|
||||
],
|
||||
"storage": "release",
|
||||
"release_asset": "PSP2UPDAT.PUP"
|
||||
},
|
||||
{
|
||||
"dest": "Complex_4627.bin",
|
||||
"sha1": "3944392c954cfb176d4210544e88353b3c5d36b1",
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -3,7 +3,7 @@
|
||||
"platform": "romm",
|
||||
"display_name": "RomM",
|
||||
"version": "1.0",
|
||||
"generated": "2026-03-29T14:04:16Z",
|
||||
"generated": "2026-03-30T09:49:20Z",
|
||||
"base_destination": "bios",
|
||||
"detect": [
|
||||
{
|
||||
@@ -13,8 +13,8 @@
|
||||
}
|
||||
],
|
||||
"standalone_copies": [],
|
||||
"total_files": 545,
|
||||
"total_size": 1197706137,
|
||||
"total_files": 638,
|
||||
"total_size": 1316397979,
|
||||
"files": [
|
||||
{
|
||||
"dest": "3do/3do_arcade_saot.bin",
|
||||
@@ -2623,12 +2623,21 @@
|
||||
"cores": null
|
||||
},
|
||||
{
|
||||
"dest": "psxonpsp660.bin",
|
||||
"sha1": "96880d1ca92a016ff054be5159bb06fe03cb4e14",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/psxonpsp660.bin",
|
||||
"dest": "BB01R4_OS.ROM",
|
||||
"sha1": "decde89fbae90adb591ad2fc553d35f49030c129",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Atari/400-800/BB01R4_OS.ROM",
|
||||
"cores": [
|
||||
"Beetle PSX (Mednafen PSX)"
|
||||
"Atari800"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "XEGAME.ROM",
|
||||
"sha1": "a107db7f16a1129cf9d933c9cf4f013b068c9e82",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/Atari/400-800/XEGAME.ROM",
|
||||
"cores": [
|
||||
"Atari800"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -2640,6 +2649,98 @@
|
||||
"Beetle PSX (Mednafen PSX)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "openbios.bin",
|
||||
"sha1": "389df7981873d9e6e46c84c20cd43af0e4226cf8",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Sony/PlayStation/openbios.bin",
|
||||
"cores": [
|
||||
"Beetle PSX (Mednafen PSX)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "cromwell_1024.bin",
|
||||
"sha1": "4e1c2c2ee308ca4591542b3ca48653f65fae6e0f",
|
||||
"size": 1048576,
|
||||
"repo_path": "bios/Microsoft/Xbox/cromwell_1024.bin",
|
||||
"cores": [
|
||||
"DirectXBox"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "GC/dsp_rom.bin",
|
||||
"sha1": "f4f683a49d7eb4155566f793f2c1c27e90159992",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/Nintendo/GameCube/Sys/GC/dsp_rom.bin",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "GC/dsp_coef.bin",
|
||||
"sha1": "c116d867ba001dcd6bf6d399ff4bf38d340f556c",
|
||||
"size": 4096,
|
||||
"repo_path": "bios/Nintendo/GameCube/.variants/dsp_coef.bin.c116d867",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Wii/shared2/sys/SYSCONF",
|
||||
"sha1": "3256c026284a24fb99d2ec1558d95db3b5dcc2e9",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Nintendo/Wii/SYSCONF",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Wii/title/00000001/00000002/data/setting.txt",
|
||||
"sha1": "077a31ce116de322f089a627c5eb7ff5c2236a5d",
|
||||
"size": 256,
|
||||
"repo_path": "bios/Nintendo/Wii/setting.txt",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Load/WiiSD.raw",
|
||||
"sha1": "90d83d6c084deceeeb0466ac00723723ccfd0c6d",
|
||||
"size": 134217728,
|
||||
"repo_path": "bios/Nintendo/Wii/WiiSD.raw",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
],
|
||||
"storage": "release",
|
||||
"release_asset": "WiiSD.raw"
|
||||
},
|
||||
{
|
||||
"dest": "Wii/clientca.pem",
|
||||
"sha1": "4b937d4e81de11574b926386dd5f768aa23bf177",
|
||||
"size": 1005,
|
||||
"repo_path": "bios/Nintendo/Wii/clientca.pem",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Wii/clientcakey.pem",
|
||||
"sha1": "a2c11886e6d12a135e1d5eb50d5fe92e028577d9",
|
||||
"size": 609,
|
||||
"repo_path": "bios/Nintendo/Wii/clientcakey.pem",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Wii/rootca.pem",
|
||||
"sha1": "b5229455dd26c1f53c73060e9089b391389e1f75",
|
||||
"size": 897,
|
||||
"repo_path": "bios/Nintendo/Wii/rootca.pem",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "MT32_CONTROL.ROM",
|
||||
"sha1": "b083518fffb7f66b03c23b7eb4f868e62dc5a987",
|
||||
@@ -2694,6 +2795,33 @@
|
||||
"DOSBox Pure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/exdos14isdos10uk.rom",
|
||||
"sha1": "b82e21b6e3214432b6dc13f650e97de88fc90a72",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/exdos14isdos10uk.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/epdos16f.rom",
|
||||
"sha1": "9a9eac31e601a1ab0f7a5d2b13175d092fa84bd3",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/epdos16f.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/zx128.rom",
|
||||
"sha1": "16375d42ea109b47edded7a16028de7fdb3013a1",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/zx128.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "fbalpha2012/hiscore.dat",
|
||||
"sha1": "7381472bf046126257e51a0124e4553282f020e5",
|
||||
@@ -3063,6 +3191,53 @@
|
||||
"Handy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "hatari/BOOT.ST",
|
||||
"sha1": "5bcabba35bb8fbfe5a65b85efccf5ed657388308",
|
||||
"size": 737280,
|
||||
"repo_path": "bios/Atari/ST/hatari/BOOT.ST",
|
||||
"cores": [
|
||||
"Hatari"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Wii/sd.raw",
|
||||
"sha1": "8c8134f08b2e3baa603206ede30d3935365009b8",
|
||||
"size": 134217728,
|
||||
"repo_path": "bios/Nintendo/Wii/sd.raw",
|
||||
"cores": [
|
||||
"Ishiiruka"
|
||||
],
|
||||
"storage": "release",
|
||||
"release_asset": "sd.raw"
|
||||
},
|
||||
{
|
||||
"dest": "pcsx2/resources/GameIndex.yaml",
|
||||
"sha1": "b22389650f6c0a1e276b48213fed8a9c1c6476ce",
|
||||
"size": 2669341,
|
||||
"repo_path": "bios/Sony/PlayStation 2/GameIndex.yaml",
|
||||
"cores": [
|
||||
"LRPS2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "pcsx2/resources/cheats_ws.zip",
|
||||
"sha1": "773b4279d8c8e182dc57e444dbf448a1d115e7e0",
|
||||
"size": 1273296,
|
||||
"repo_path": "bios/Sony/PlayStation 2/cheats_ws.zip",
|
||||
"cores": [
|
||||
"LRPS2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "pcsx2/resources/cheats_ni.zip",
|
||||
"sha1": "dec5e4f137890c338babe13cfb9d41ff869fa721",
|
||||
"size": 42525,
|
||||
"repo_path": "bios/Sony/PlayStation 2/cheats_ni.zip",
|
||||
"cores": [
|
||||
"LRPS2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "mame2003/cheat.dat",
|
||||
"sha1": "32fc78415114a976af9c2ee53ea13de9f40b55ee",
|
||||
@@ -3234,17 +3409,6 @@
|
||||
"melonDS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "dsi_nand.bin",
|
||||
"sha1": "b48f44194fe918aaaec5298861479512b581d661",
|
||||
"size": 251658304,
|
||||
"repo_path": ".cache/large/dsi_nand.bin",
|
||||
"cores": [
|
||||
"melonDS"
|
||||
],
|
||||
"storage": "release",
|
||||
"release_asset": "dsi_nand.bin"
|
||||
},
|
||||
{
|
||||
"dest": "dsi_sd_card.bin",
|
||||
"sha1": "3b71f43ff30f4b15b5cd85dd9e95ebc7e84eb5a3",
|
||||
@@ -3254,6 +3418,15 @@
|
||||
"melonDS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "MacIIx.ROM",
|
||||
"sha1": "753b94351d94c369616c2c87b19d568dc5e2764e",
|
||||
"size": 262144,
|
||||
"repo_path": "bios/Apple/Macintosh II/MacIIx.ROM",
|
||||
"cores": [
|
||||
"Mini vMac"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "Mupen64plus/IPL.n64",
|
||||
"sha1": "bf861922dcb78c316360e3e742f4f70ff63c9bc3",
|
||||
@@ -3272,6 +3445,33 @@
|
||||
"Mupen64Plus-Next"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/bios9821.rom",
|
||||
"sha1": "2e92346727b0355bc1ec9a7ded1b444a4917f2b9",
|
||||
"size": 98304,
|
||||
"repo_path": "bios/NEC/PC-98/bios9821.rom",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/scsi.rom",
|
||||
"sha1": "3d7166f05daad1b022fa04c2569e788580158095",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/NEC/PC-98/scsi.rom",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/sasi.rom",
|
||||
"sha1": "b607707d74b5a7d3ba211825de31a8f32aec8146",
|
||||
"size": 4096,
|
||||
"repo_path": "bios/NEC/PC-98/sasi.rom",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "custom.pal",
|
||||
"sha1": "c7635019127aa28d84d28ae48304bfca6d5baefb",
|
||||
@@ -3281,6 +3481,33 @@
|
||||
"Nestopia UE"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/ide.rom",
|
||||
"sha1": "0877ffb4b4d1c18283468be3579b72ed8c22e3ac",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/NEC/PC-98/ide.rom",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/pci.rom",
|
||||
"sha1": "1246203bebf9f04e3bac2df7fc64719304f9f1bd",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/NEC/PC-98/pci.rom",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "eeprom.dat",
|
||||
"sha1": "ffc6261e487efa8c7442069f71acfc4aa826993d",
|
||||
"size": 64,
|
||||
"repo_path": "bios/Sony/PlayStation 2/eeprom.dat",
|
||||
"cores": [
|
||||
"PCSX2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "carthw.cfg",
|
||||
"sha1": "d3fe0b958705e4cb5eeb7822e60d474204df4bd3",
|
||||
@@ -3515,6 +3742,15 @@
|
||||
"PUAE (P-UAE)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "keropi/cgrom.tmp",
|
||||
"sha1": "8d72c5b4d63bb14c5dbdac495244d659aa1498b6",
|
||||
"size": 786432,
|
||||
"repo_path": "bios/Sharp/X68000/cgrom.dat",
|
||||
"cores": [
|
||||
"px68k"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "same_cdi/bios/cdimono1.zip",
|
||||
"sha1": "5d0b1b55b0d0958a5c9069c3219d4da5a87a6b93",
|
||||
@@ -3561,12 +3797,93 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "BIOS.col",
|
||||
"sha1": "45bedc4cbdeac66c7df59e9e599195c778d86a92",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/Coleco/ColecoVision/BIOS.col",
|
||||
"dest": "squirreljme-0.3.0-fast.jar",
|
||||
"sha1": "7c4cd0a5451eedeac9b328f48408dbc312198ccf",
|
||||
"size": 2367241,
|
||||
"repo_path": "bios/Other/SquirrelJME/squirreljme-0.3.0-fast.jar",
|
||||
"cores": [
|
||||
"SMS Plus GX"
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-0.3.0.jar",
|
||||
"sha1": "8a2ddc37df2366d4e205e9904cd8bac22db2afd5",
|
||||
"size": 15181774,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme.sqc",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-0.3.0-test.jar",
|
||||
"sha1": "600cf0440594b0c69339ba4b626a47288b342d77",
|
||||
"size": 3396757,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme-test.jar",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-0.3.0-slow.jar",
|
||||
"sha1": "3a23de6999b94fc088bdd9c9e4a73781a1d42233",
|
||||
"size": 3129653,
|
||||
"repo_path": "bios/Other/SquirrelJME/squirreljme-0.3.0-slow.jar",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-0.3.0-slow-test.jar",
|
||||
"sha1": "b411c128bfd8411f339a49bbcb1c9e97189c2161",
|
||||
"size": 3772058,
|
||||
"repo_path": "bios/Other/SquirrelJME/squirreljme-0.3.0-slow-test.jar",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-fast.jar",
|
||||
"sha1": "8a2ddc37df2366d4e205e9904cd8bac22db2afd5",
|
||||
"size": 15181774,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme.sqc",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme.jar",
|
||||
"sha1": "6933bbe1fc29f58581a7f64f859f756db9ef78a7",
|
||||
"size": 4700395,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme.jar",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-test.jar",
|
||||
"sha1": "600cf0440594b0c69339ba4b626a47288b342d77",
|
||||
"size": 3396757,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme-test.jar",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-slow.jar",
|
||||
"sha1": "8a2ddc37df2366d4e205e9904cd8bac22db2afd5",
|
||||
"size": 15181774,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme.sqc",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "squirreljme-slow-test.jar",
|
||||
"sha1": "8a2ddc37df2366d4e205e9904cd8bac22db2afd5",
|
||||
"size": 15181774,
|
||||
"repo_path": "bios/Java/J2ME/squirreljme.sqc",
|
||||
"cores": [
|
||||
"SquirrelJME"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -3722,6 +4039,51 @@
|
||||
"VICE xvic"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "xmil/IPLROM.X1",
|
||||
"sha1": "c4db9a6e99873808c8022afd1c50fef556a8b44d",
|
||||
"size": 4096,
|
||||
"repo_path": "bios/Sharp/X1/.variants/IPLROM.X1.c4db9a6e",
|
||||
"cores": [
|
||||
"X Millennium"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "xmil/IPLROM.X1T",
|
||||
"sha1": "44620f57a25f0bcac2b57ca2b0f1ebad3bf305d3",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Sharp/X1/.variants/IPLROM.X1T.44620f57",
|
||||
"cores": [
|
||||
"X Millennium"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "xmil/FNT0816.X1",
|
||||
"sha1": "4f06d20c997a79ee6af954b69498147789bf1847",
|
||||
"size": 4096,
|
||||
"repo_path": "bios/Sharp/X1/FNT0816.X1",
|
||||
"cores": [
|
||||
"X Millennium"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "xmil/FNT1616.X1",
|
||||
"sha1": "b9e6c320611f0842df6f45673c47c3e23bc14272",
|
||||
"size": 306176,
|
||||
"repo_path": "bios/Sharp/X1/FNT1616.X1",
|
||||
"cores": [
|
||||
"X Millennium"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "xbox_hdd.qcow2",
|
||||
"sha1": "9da5f9ecfb1c9c32efa616f0300d02e8f702244d",
|
||||
"size": 1638400,
|
||||
"repo_path": "bios/Microsoft/Xbox/xbox_hdd.qcow2",
|
||||
"cores": [
|
||||
"Xemu"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "channelf.zip",
|
||||
"sha1": "1cb23b462b990241013deb4b5e07ce741af28267",
|
||||
@@ -4127,6 +4489,213 @@
|
||||
"FinalBurn Neo"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "GC/USA/IPL.bin",
|
||||
"sha1": "a1837968288253ed541f2b11440b68f5a9b33875",
|
||||
"size": 2097152,
|
||||
"repo_path": "bios/Nintendo/GameCube/GC/JAP/IPL.bin",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "GC/EUR/IPL.bin",
|
||||
"sha1": "a1837968288253ed541f2b11440b68f5a9b33875",
|
||||
"size": 2097152,
|
||||
"repo_path": "bios/Nintendo/GameCube/GC/JAP/IPL.bin",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "GC/JAP/IPL.bin",
|
||||
"sha1": "a1837968288253ed541f2b11440b68f5a9b33875",
|
||||
"size": 2097152,
|
||||
"repo_path": "bios/Nintendo/GameCube/GC/JAP/IPL.bin",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "GBA/gba_bios.bin",
|
||||
"sha1": "300c20df6731a33952ded8c436f7f186d25d3492",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Nintendo/Game Boy Advance/GBA_bios.rom",
|
||||
"cores": [
|
||||
"Dolphin"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/exos21.rom",
|
||||
"sha1": "55315b20fecb4441a07ee4bc5dc7153f396e0a2e",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/exos21.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/exos20.rom",
|
||||
"sha1": "6033a0535136c40c47137e4d1cd9273c06d5fdff",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/exos20.bin",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/exos24uk.rom",
|
||||
"sha1": "cf12e971623a54bf8c4f891ca3a36d969f205c49",
|
||||
"size": 65536,
|
||||
"repo_path": "bios/Enterprise/64-128/exos24uk.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/basic21.rom",
|
||||
"sha1": "03bbb386cf530e804363acdfc1d13e64cf28af2e",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/basic21.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/basic20.rom",
|
||||
"sha1": "61d0987b906146e21b94f265d5b51b4938c986a9",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/basic20.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/exdos13.rom",
|
||||
"sha1": "cb43ab3676b93c279f1ed8ffcb0d4dcd4b34e631",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/exdos13.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/epfileio.rom",
|
||||
"sha1": "2f9077bcd89b1ec42dbdcd55d335bdbaf361eff3",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/epfileio.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/zt19uk.rom",
|
||||
"sha1": "b7af62f0bc95fdca4b31d236f8327dafc80f83b7",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Enterprise/64-128/zt19uk.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/hun.rom",
|
||||
"sha1": "325a5e28c2a0d896711f8829e7ff14fed5dd4103",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/hun.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/brd.rom",
|
||||
"sha1": "f34f0c330b44dbf2548329bea954d5991dec30ca",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/brd.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/tvc22_sys.rom",
|
||||
"sha1": "f2572ee83d09fc08f4de4a62f101c8bb301a9505",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/tvc22_sys.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/tvc22_ext.rom",
|
||||
"sha1": "abf119cf947ea32defd08b29a8a25d75f6bd4987",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/Enterprise/64-128/tvc22_ext.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/tvcfileio.rom",
|
||||
"sha1": "98889c3a56b11dedf077f866ed2e12d51b604113",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/Enterprise/64-128/tvcfileio.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/tvc_dos12d.rom",
|
||||
"sha1": "072c6160d4e7d406f5d8f5b1b66066c797d35561",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/tvc_dos12d.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/cpc464.rom",
|
||||
"sha1": "56d39c463da60968d93e58b4ba0e675829412a20",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Amstrad/CPC/cpc464.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/cpc664.rom",
|
||||
"sha1": "073a7665527b5bd8a148747a3947dbd3328682c8",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Amstrad/CPC/cpc664.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/cpc6128.rom",
|
||||
"sha1": "5977adbad3f7c1e0e082cd02fe76a700d9860c30",
|
||||
"size": 32768,
|
||||
"repo_path": "bios/Amstrad/CPC/cpc6128.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/cpc_amsdos.rom",
|
||||
"sha1": "39102c8e9cb55fcc0b9b62098780ed4a3cb6a4bb",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Amstrad/CPC/amsdos.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "ep128emu/roms/zx48.rom",
|
||||
"sha1": "5ea7c2b824672e914525d1d5c419d71b84a426a2",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/Enterprise/64-128/zx48.rom",
|
||||
"cores": [
|
||||
"ep128emu-core"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "fbneo/hiscore.dat",
|
||||
"sha1": "7381472bf046126257e51a0124e4553282f020e5",
|
||||
@@ -4180,6 +4749,276 @@
|
||||
"cores": [
|
||||
"MAME 2003-Plus"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/bios.rom",
|
||||
"sha1": "1f5f7013f18c08ff50d7942e76c4fbd782412414",
|
||||
"size": 65536,
|
||||
"repo_path": "bios/IBM/PC/ibmpcjr/bios.rom",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/font.bmp",
|
||||
"sha1": "b4f14e58030ed40fff2dc312b58ea4440bdf8cc5",
|
||||
"size": 524350,
|
||||
"repo_path": "bios/NEC/PC-98/font.bmp",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/sound.rom",
|
||||
"sha1": "d5dbc4fea3b8367024d363f5351baecd6adcd8ef",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/NEC/PC-98/sound.rom",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/2608_bd.wav",
|
||||
"sha1": "0a56c142ef40cec50f3ee56a6e42d0029c9e2818",
|
||||
"size": 19192,
|
||||
"repo_path": "bios/NEC/PC-98/2608_bd.wav",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/2608_sd.wav",
|
||||
"sha1": "3c79663ef74c0b0439d13351326eb1c52a657008",
|
||||
"size": 15558,
|
||||
"repo_path": "bios/NEC/PC-98/2608_sd.wav",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/2608_top.wav",
|
||||
"sha1": "aa4a8f766a86b830687d5083fd3b9db0652f46fc",
|
||||
"size": 57016,
|
||||
"repo_path": "bios/NEC/PC-98/2608_top.wav",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/2608_hh.wav",
|
||||
"sha1": "12f676cef249b82480b6f19c454e234b435ca7b6",
|
||||
"size": 36722,
|
||||
"repo_path": "bios/NEC/PC-98/2608_hh.wav",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/2608_tom.wav",
|
||||
"sha1": "9513fb4a3f41e75a972a273a5104cbd834c1e2c5",
|
||||
"size": 23092,
|
||||
"repo_path": "bios/NEC/PC-98/2608_tom.wav",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2/2608_rim.wav",
|
||||
"sha1": "c65592330c9dd84011151daed52f9aec926b7e56",
|
||||
"size": 5288,
|
||||
"repo_path": "bios/NEC/PC-98/2608_rim.wav",
|
||||
"cores": [
|
||||
"nekop2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/bios.rom",
|
||||
"sha1": "1f5f7013f18c08ff50d7942e76c4fbd782412414",
|
||||
"size": 65536,
|
||||
"repo_path": "bios/IBM/PC/ibmpcjr/bios.rom",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/font.bmp",
|
||||
"sha1": "b4f14e58030ed40fff2dc312b58ea4440bdf8cc5",
|
||||
"size": 524350,
|
||||
"repo_path": "bios/NEC/PC-98/font.bmp",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/sound.rom",
|
||||
"sha1": "d5dbc4fea3b8367024d363f5351baecd6adcd8ef",
|
||||
"size": 16384,
|
||||
"repo_path": "bios/NEC/PC-98/sound.rom",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/2608_bd.wav",
|
||||
"sha1": "0a56c142ef40cec50f3ee56a6e42d0029c9e2818",
|
||||
"size": 19192,
|
||||
"repo_path": "bios/NEC/PC-98/2608_bd.wav",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/2608_sd.wav",
|
||||
"sha1": "3c79663ef74c0b0439d13351326eb1c52a657008",
|
||||
"size": 15558,
|
||||
"repo_path": "bios/NEC/PC-98/2608_sd.wav",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/2608_top.wav",
|
||||
"sha1": "aa4a8f766a86b830687d5083fd3b9db0652f46fc",
|
||||
"size": 57016,
|
||||
"repo_path": "bios/NEC/PC-98/2608_top.wav",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/2608_hh.wav",
|
||||
"sha1": "12f676cef249b82480b6f19c454e234b435ca7b6",
|
||||
"size": 36722,
|
||||
"repo_path": "bios/NEC/PC-98/2608_hh.wav",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/2608_tom.wav",
|
||||
"sha1": "9513fb4a3f41e75a972a273a5104cbd834c1e2c5",
|
||||
"size": 23092,
|
||||
"repo_path": "bios/NEC/PC-98/2608_tom.wav",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/2608_rim.wav",
|
||||
"sha1": "c65592330c9dd84011151daed52f9aec926b7e56",
|
||||
"size": 5288,
|
||||
"repo_path": "bios/NEC/PC-98/2608_rim.wav",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/scsi.rom",
|
||||
"sha1": "3d7166f05daad1b022fa04c2569e788580158095",
|
||||
"size": 8192,
|
||||
"repo_path": "bios/NEC/PC-98/scsi.rom",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "np2kai/sasi.rom",
|
||||
"sha1": "b607707d74b5a7d3ba211825de31a8f32aec8146",
|
||||
"size": 4096,
|
||||
"repo_path": "bios/NEC/PC-98/sasi.rom",
|
||||
"cores": [
|
||||
"NP2kai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "keropi/iplrom.dat",
|
||||
"sha1": "0ed038ed2133b9f78c6e37256807424e0d927560",
|
||||
"size": 131072,
|
||||
"repo_path": "bios/Sharp/X68000/iplrom.dat",
|
||||
"cores": [
|
||||
"px68k"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "keropi/iplrom30.dat",
|
||||
"sha1": "239e9124568c862c31d9ec0605e32373ea74b86a",
|
||||
"size": 131072,
|
||||
"repo_path": "bios/Sharp/X68000/iplrom30.dat",
|
||||
"cores": [
|
||||
"px68k"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "keropi/iplromco.dat",
|
||||
"sha1": "77511fc58798404701f66b6bbc9cbde06596eba7",
|
||||
"size": 131072,
|
||||
"repo_path": "bios/Sharp/X68000/iplromco.dat",
|
||||
"cores": [
|
||||
"px68k"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "keropi/iplromxv.dat",
|
||||
"sha1": "e33cdcdb69cd257b0b211ef46e7a8b144637db57",
|
||||
"size": 131072,
|
||||
"repo_path": "bios/Sharp/X68000/iplromxv.dat",
|
||||
"cores": [
|
||||
"px68k"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "keropi/cgrom.dat",
|
||||
"sha1": "8d72c5b4d63bb14c5dbdac495244d659aa1498b6",
|
||||
"size": 786432,
|
||||
"repo_path": "bios/Sharp/X68000/cgrom.dat",
|
||||
"cores": [
|
||||
"px68k"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scummvm/extra/MT32_CONTROL.ROM",
|
||||
"sha1": "b083518fffb7f66b03c23b7eb4f868e62dc5a987",
|
||||
"size": 65536,
|
||||
"repo_path": "bios/Commodore/Amiga/mt32-roms/mt32_control.rom",
|
||||
"cores": [
|
||||
"ScummVM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scummvm/extra/MT32_PCM.ROM",
|
||||
"sha1": "f6b1eebc4b2d200ec6d3d21d51325d5b48c60252",
|
||||
"size": 524288,
|
||||
"repo_path": "bios/Commodore/Amiga/mt32-roms/pcm_mt32.rom",
|
||||
"cores": [
|
||||
"ScummVM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scummvm/extra/CM32L_CONTROL.ROM",
|
||||
"sha1": "a439fbb390da38cada95a7cbb1d6ca199cd66ef8",
|
||||
"size": 65536,
|
||||
"repo_path": "bios/Commodore/Amiga/mt32-roms/cm32l_control.rom",
|
||||
"cores": [
|
||||
"ScummVM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "scummvm/extra/CM32L_PCM.ROM",
|
||||
"sha1": "289cc298ad532b702461bfc738009d9ebe8025ea",
|
||||
"size": 1048576,
|
||||
"repo_path": "bios/Commodore/Amiga/mt32-roms/pcm_cm32l.rom",
|
||||
"cores": [
|
||||
"ScummVM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"dest": "xmil/FNT0808.X1",
|
||||
"sha1": "1c1a0d8c9f4c446ccd7470516b215ddca5052fb2",
|
||||
"size": 2048,
|
||||
"repo_path": "bios/Sharp/X1/FNT0808.X1",
|
||||
"cores": [
|
||||
"X Millennium"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
26
mkdocs.yml
26
mkdocs.yml
@@ -37,7 +37,11 @@ markdown_extensions:
|
||||
- toc:
|
||||
permalink: true
|
||||
- pymdownx.details
|
||||
- pymdownx.superfences
|
||||
- pymdownx.superfences:
|
||||
custom_fences:
|
||||
- name: mermaid
|
||||
class: mermaid
|
||||
format: !!python/name:pymdownx.superfences.fence_code_format
|
||||
- pymdownx.tabbed:
|
||||
alternate_style: true
|
||||
plugins:
|
||||
@@ -132,7 +136,7 @@ nav:
|
||||
- ZC: systems/zc.md
|
||||
- Emulators:
|
||||
- Overview: emulators/index.md
|
||||
- Official ports (61):
|
||||
- Official ports (63):
|
||||
- amiarcadia: emulators/amiarcadia.md
|
||||
- Amiberry: emulators/amiberry.md
|
||||
- Ardens: emulators/ardens.md
|
||||
@@ -176,9 +180,11 @@ nav:
|
||||
- mGBA: emulators/mgba.md
|
||||
- Mr.Boom: emulators/mrboom.md
|
||||
- Panda3DS: emulators/panda3ds.md
|
||||
- PCSX2: emulators/pcsx2.md
|
||||
- PicoDrive: emulators/picodrive.md
|
||||
- play: emulators/play.md
|
||||
- PPSSPP: emulators/ppsspp.md
|
||||
- RPCS3: emulators/rpcs3.md
|
||||
- Rustation: emulators/rustation.md
|
||||
- RVVM: emulators/rvvm.md
|
||||
- SameBoy: emulators/sameboy.md
|
||||
@@ -428,7 +434,7 @@ nav:
|
||||
- PCSX-ReARMed: emulators/pcsx_rearmed.md
|
||||
- Launchers (1):
|
||||
- Dolphin Launcher: emulators/dolphin_launcher.md
|
||||
- Other (24):
|
||||
- Other (23):
|
||||
- ares: emulators/ares.md
|
||||
- Beetle GBA (Mednafen): emulators/beetle_gba.md
|
||||
- BigPEmu: emulators/bigpemu.md
|
||||
@@ -440,12 +446,11 @@ nav:
|
||||
- Lexaloffle: emulators/lexaloffle.md
|
||||
- Model 2 Emulator: emulators/model2.md
|
||||
- openMSX: emulators/openmsx.md
|
||||
- PCSX2: emulators/pcsx2.md
|
||||
- Redream: emulators/redream.md
|
||||
- RPCS3: emulators/rpcs3.md
|
||||
- Ryujinx: emulators/ryujinx.md
|
||||
- shadps4: emulators/shadps4.md
|
||||
- Supermodel: emulators/supermodel.md
|
||||
- ti99sim: emulators/ti99sim.md
|
||||
- tsugaru: emulators/tsugaru.md
|
||||
- VBA-M: emulators/vba_m.md
|
||||
- VICE: emulators/vice.md
|
||||
@@ -457,8 +462,17 @@ nav:
|
||||
- Gap Analysis: gaps.md
|
||||
- Wiki:
|
||||
- Overview: wiki/index.md
|
||||
- Getting started: wiki/getting-started.md
|
||||
- FAQ: wiki/faq.md
|
||||
- Troubleshooting: wiki/troubleshooting.md
|
||||
- Architecture: wiki/architecture.md
|
||||
- Tools: wiki/tools.md
|
||||
- Profiling guide: wiki/profiling.md
|
||||
- Advanced usage: wiki/advanced-usage.md
|
||||
- Verification modes: wiki/verification-modes.md
|
||||
- Data model: wiki/data-model.md
|
||||
- Profiling guide: wiki/profiling.md
|
||||
- Adding a platform: wiki/adding-a-platform.md
|
||||
- Adding a scraper: wiki/adding-a-scraper.md
|
||||
- Testing guide: wiki/testing-guide.md
|
||||
- Release process: wiki/release-process.md
|
||||
- Contributing: contributing.md
|
||||
|
||||
@@ -1,231 +1,870 @@
|
||||
# Platform Registry
|
||||
# Central configuration for all supported platforms and their scraper sources.
|
||||
# Adding a new platform = adding an entry here + creating its YAML config.
|
||||
#
|
||||
# status: active | archived
|
||||
# active -- included in automated releases, scraped weekly/monthly
|
||||
# archived -- config preserved, user can generate pack manually, excluded from CI releases
|
||||
|
||||
platforms:
|
||||
retroarch:
|
||||
config: retroarch.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/libretro/RetroArch/master/media/retroarch-vector_invader-only.svg"
|
||||
logo: https://raw.githubusercontent.com/libretro/RetroArch/master/media/retroarch-vector_invader-only.svg
|
||||
scraper: libretro
|
||||
source_url: "https://raw.githubusercontent.com/libretro/libretro-database/master/dat/System.dat"
|
||||
source_url: https://raw.githubusercontent.com/libretro/libretro-database/master/dat/System.dat
|
||||
source_format: clrmamepro_dat
|
||||
hash_type: sha1
|
||||
verification_mode: existence
|
||||
base_destination: system
|
||||
case_insensitive_fs: true
|
||||
schedule: weekly
|
||||
cores: all_libretro
|
||||
target_scraper: retroarch_targets
|
||||
target_source: "https://buildbot.libretro.com/nightly/"
|
||||
target_source: https://buildbot.libretro.com/nightly/
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: "$HOME/.var/app/org.libretro.RetroArch/config/retroarch/retroarch.cfg"
|
||||
parse_key: system_directory
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: "$HOME/.config/retroarch/retroarch.cfg"
|
||||
parse_key: system_directory
|
||||
- os: darwin
|
||||
method: config_file
|
||||
config: "$HOME/Library/Application Support/RetroArch/retroarch.cfg"
|
||||
parse_key: system_directory
|
||||
- os: windows
|
||||
method: config_file
|
||||
config: "%APPDATA%\\RetroArch\\retroarch.cfg"
|
||||
parse_key: system_directory
|
||||
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: $HOME/.var/app/org.libretro.RetroArch/config/retroarch/retroarch.cfg
|
||||
parse_key: system_directory
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: $HOME/.config/retroarch/retroarch.cfg
|
||||
parse_key: system_directory
|
||||
- os: darwin
|
||||
method: config_file
|
||||
config: $HOME/Library/Application Support/RetroArch/retroarch.cfg
|
||||
parse_key: system_directory
|
||||
- os: windows
|
||||
method: config_file
|
||||
config: '%APPDATA%\RetroArch\retroarch.cfg'
|
||||
parse_key: system_directory
|
||||
batocera:
|
||||
config: batocera.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/batocera-linux/batocera-emulationstation/master/resources/splash_batocera.svg"
|
||||
logo: https://raw.githubusercontent.com/batocera-linux/batocera-emulationstation/master/resources/splash_batocera.svg
|
||||
scraper: batocera
|
||||
source_url: "https://raw.githubusercontent.com/batocera-linux/batocera.linux/master/package/batocera/core/batocera-scripts/scripts/batocera-systems"
|
||||
source_url: https://raw.githubusercontent.com/batocera-linux/batocera.linux/master/package/batocera/core/batocera-scripts/scripts/batocera-systems
|
||||
source_format: python_dict
|
||||
hash_type: md5
|
||||
schedule: weekly
|
||||
cores: [81, a5200, abuse, arduous, atari800, azahar, bennugd, bk, bluemsx, bsnes, bstone, cannonball, cap32, catacombgl, cdogs, cemu, cgenius, citron, clk, corsixth, demul, devilutionx, dhewm3, dice, dolphin, dosbox_pure, dxx-rebirth, easyrpg, ecwolf, eduke32, eka2l1, emuscv, etlegacy, fake08, fallout1-ce, fallout2-ce, fbneo, fceumm, flatpak, flycast, freechaf, freeintv, fury, fuse, gambatte, gearsystem, genesisplusgx, glide64mk2, gong, gsplus, gw, gzdoom, hatari, hcl, hurrican, hypseus-singe, ikemen, ioquake3, iortcw, jazz2-native, lindbergh-loader, lowresnx, lutro, mame, mame078plus, mednafen_lynx, mednafen_ngp, mednafen_supergrafx, mednafen_wswan, melonds, mgba, minivmac, model2emu, moonlight, mrboom, neocd, np2kai, nxengine, o2em, odcommander, openbor6412, openjazz, openjk, openjkdf2, openmohaa, opera, pce_fast, pcfx, pcsx2, pcsx_rearmed, pd777, picodrive, play, pokemini, potator, ppsspp, prboom, prosystem, puae, px68k, pygame, pyxel, quasi88, raze, reminiscence, rpcs3, ruffle, samcoupe, sameduck, scummvm, sdlpop, sh, shadps4, snes9x, solarus, sonic2013, sonic3-air, sonic-mania, steam, stella, superbroswar, supermodel, taradino, tgbdual, theforceengine, theodore, thextech, tic80, tr1x, tr2x, tsugaru, tyrian, tyrquake, uqm, uzem, vb, vecx, vice_x64, vircon32, virtualjaguar, vita3k, vox_official, vpinball, wasm4, wine-tkg, x1, x128, x16emu, xash3d_fwgs, xemu, xenia-canary, xpet, xplus4, xrick, xvic, yabasanshiro, yquake2, zc210]
|
||||
cores:
|
||||
- '81'
|
||||
- a5200
|
||||
- abuse
|
||||
- amiberry
|
||||
- applewin
|
||||
- arduous
|
||||
- atari800
|
||||
- azahar
|
||||
- beetle-saturn
|
||||
- bennugd
|
||||
- bigpemu
|
||||
- bk
|
||||
- blastem
|
||||
- bluemsx
|
||||
- boom3
|
||||
- bsnes
|
||||
- bstone
|
||||
- cannonball
|
||||
- cap32
|
||||
- catacombgl
|
||||
- cdogs
|
||||
- cemu
|
||||
- cgenius
|
||||
- citron
|
||||
- clk
|
||||
- corsixth
|
||||
- demul
|
||||
- desmume
|
||||
- devilutionx
|
||||
- dhewm3
|
||||
- dice
|
||||
- dolphin
|
||||
- dosbox
|
||||
- dosbox_pure
|
||||
- duckstation
|
||||
- dxx-rebirth
|
||||
- easyrpg
|
||||
- ecwolf
|
||||
- eduke32
|
||||
- eka2l1
|
||||
- emuscv
|
||||
- ep128emu-core
|
||||
- etlegacy
|
||||
- fake08
|
||||
- fallout1-ce
|
||||
- fallout2-ce
|
||||
- fbneo
|
||||
- fceumm
|
||||
- flatpak
|
||||
- flycast
|
||||
- fmsx
|
||||
- freechaf
|
||||
- freeintv
|
||||
- freej2me
|
||||
- fsuae
|
||||
- fury
|
||||
- fuse
|
||||
- gambatte
|
||||
- gearcoleco
|
||||
- gearsystem
|
||||
- genesisplusgx
|
||||
- glide64mk2
|
||||
- gong
|
||||
- gpsp
|
||||
- gsplus
|
||||
- gw
|
||||
- gzdoom
|
||||
- handy
|
||||
- hatari
|
||||
- hcl
|
||||
- holani
|
||||
- hurrican
|
||||
- hypseus-singe
|
||||
- ikemen
|
||||
- ioquake3
|
||||
- iortcw
|
||||
- jazz2-native
|
||||
- kronos
|
||||
- lindbergh-loader
|
||||
- lowresnx
|
||||
- lutro
|
||||
- mame
|
||||
- mame078plus
|
||||
- mamemess
|
||||
- mednafen_lynx
|
||||
- mednafen_ngp
|
||||
- mednafen_psx
|
||||
- mednafen_supergrafx
|
||||
- mednafen_wswan
|
||||
- melonds
|
||||
- mesen
|
||||
- mesen-s
|
||||
- mgba
|
||||
- minivmac
|
||||
- model2emu
|
||||
- moonlight
|
||||
- mrboom
|
||||
- mupen64plus-next
|
||||
- neocd
|
||||
- nestopia
|
||||
- np2kai
|
||||
- nxengine
|
||||
- o2em
|
||||
- odcommander
|
||||
- openbor6412
|
||||
- openjazz
|
||||
- openjk
|
||||
- openjkdf2
|
||||
- openmohaa
|
||||
- openmsx
|
||||
- opera
|
||||
- parallel_n64
|
||||
- pce_fast
|
||||
- pcfx
|
||||
- pcsx2
|
||||
- pcsx_rearmed
|
||||
- pd777
|
||||
- picodrive
|
||||
- play
|
||||
- pokemini
|
||||
- potator
|
||||
- ppsspp
|
||||
- prboom
|
||||
- prosystem
|
||||
- puae
|
||||
- puae2021
|
||||
- px68k
|
||||
- pygame
|
||||
- pyxel
|
||||
- quasi88
|
||||
- raze
|
||||
- redream
|
||||
- reminiscence
|
||||
- rpcs3
|
||||
- ruffle
|
||||
- ryujinx
|
||||
- samcoupe
|
||||
- same_cdi
|
||||
- sameduck
|
||||
- scummvm
|
||||
- sdlpop
|
||||
- sh
|
||||
- shadps4
|
||||
- smsplus
|
||||
- snes9x
|
||||
- solarus
|
||||
- sonic-mania
|
||||
- sonic2013
|
||||
- sonic3-air
|
||||
- squirreljme
|
||||
- steam
|
||||
- stella
|
||||
- stella2014
|
||||
- superbroswar
|
||||
- supermodel
|
||||
- swanstation
|
||||
- taradino
|
||||
- tgbdual
|
||||
- theforceengine
|
||||
- theodore
|
||||
- thextech
|
||||
- tic80
|
||||
- tr1x
|
||||
- tr2x
|
||||
- tsugaru
|
||||
- tyrian
|
||||
- tyrquake
|
||||
- uae4arm
|
||||
- uqm
|
||||
- uzem
|
||||
- vb
|
||||
- vba-m
|
||||
- vecx
|
||||
- vemulator
|
||||
- vice
|
||||
- vice_x128
|
||||
- vice_x64
|
||||
- vice_x64sc
|
||||
- vice_xpet
|
||||
- vice_xplus4
|
||||
- vice_xscpu64
|
||||
- vice_xvic
|
||||
- vircon32
|
||||
- virtualjaguar
|
||||
- vita3k
|
||||
- vitaquake2
|
||||
- vox_official
|
||||
- vpinball
|
||||
- wasm4
|
||||
- wine-tkg
|
||||
- x1
|
||||
- x128
|
||||
- x16emu
|
||||
- xash3d_fwgs
|
||||
- xemu
|
||||
- xenia
|
||||
- xenia-canary
|
||||
- xpet
|
||||
- xplus4
|
||||
- xrick
|
||||
- xroar
|
||||
- xvic
|
||||
- yabasanshiro
|
||||
- ymir
|
||||
- yquake2
|
||||
- zc210
|
||||
target_scraper: batocera_targets
|
||||
target_source: "https://github.com/batocera-linux/batocera.linux"
|
||||
target_source: https://github.com/batocera-linux/batocera.linux
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: file_exists
|
||||
file: /etc/batocera-version
|
||||
bios_path: /userdata/bios
|
||||
|
||||
- os: linux
|
||||
method: file_exists
|
||||
file: /etc/batocera-version
|
||||
bios_path: /userdata/bios
|
||||
recalbox:
|
||||
config: recalbox.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/homarr-labs/dashboard-icons/main/svg/recalbox.svg"
|
||||
logo: https://raw.githubusercontent.com/homarr-labs/dashboard-icons/main/svg/recalbox.svg
|
||||
scraper: recalbox
|
||||
source_url: "https://gitlab.com/recalbox/recalbox/-/raw/master/board/recalbox/fsoverlay/recalbox/share_init/system/.emulationstation/es_bios.xml"
|
||||
source_url: https://gitlab.com/recalbox/recalbox/-/raw/master/board/recalbox/fsoverlay/recalbox/share_init/system/.emulationstation/es_bios.xml
|
||||
source_format: xml
|
||||
hash_type: md5
|
||||
schedule: monthly
|
||||
target_scraper: null
|
||||
target_source: null
|
||||
cores: ["2048", 81, a5200, advancemame, amiberry, applewin, arduous, atari800, b2, beebem, bk, bluemsx, boom3, bsnes, bsneshd, cannonball, cap32, cdi2015, corsixth, craft, crocods, daphne, desmume, dice, dinothawr, dirksimple, dolphin, dolphin-gui, dosbox, dosbox_pure, duckstation, easyrpg, ecwolf, emuscv, fake08, fba2x, fbneo, fceumm, flycast, flycast-next, fmsx, freechaf, freeintv, frotz, fuse, gambatte, gearcoleco, geargrafx, gearsystem, genesisplusgx, genesisplusgx_ex, genesisplusgxwide, geolith, glide64mk2, gliden64, gliden64_20, gong, gpsp, gsplus, gw, handy, hatari, hatarib, holani, imageviewer, julius, kronos, lowresnx, lutro, mame0258, mame0278, mame2000, mame2003, mame2003_plus, mame2010, mame2015, mame2016, mednafen_lynx, mednafen_ngp, mednafen_pce_fast, mednafen_pcfx, mednafen_psx, mednafen_psx_hw, mednafen_saturn, mednafen_supafaust, mednafen_supergrafx, mednafen_vb, mednafen_wswan, melonds, mesen, mesen_s, meteor, mgba, minivmac, mojozork, moonlight, mrboom, mu, mupen64plus, mupen64plus_next, n64_gles2, neocd, nestopia, np2kai, nxengine, o2em, openbor, openlara, opera, oricutron, parallel_n64, pcsx2, pcsx_rearmed, pico8, picodrive, pisnes, pokemini, potator, ppsspp, prboom, prosystem, ps2, puae, px68k, quasi88, quicknes, race, rb5000, reicast, reminiscence, retro8, retrodream, rice, rice_gles2, sameboy, same_cdi, sameduck, scummvm, sdlpop, simcoupe, snes9x, snes9x2002, snes9x2005, snes9x2010, solarus, stella, stella2014, stonesoup, supermodel, swanstation, tamalibretro, tgbdual, theodore, thepowdertoy, ti99sim, tic80, tyrquake, uae4all, uae4arm, uzem, vecx, vice_x128, vice_x64, vice_x64sc, vice_xcbm2, vice_xcbm5x0, vice_xpet, vice_xplus4, vice_xscpu64, vice_xvic, virtualjaguar, vitaquake2, vitaquake3, vitavoyager, vpinball, vvvvvv, wasm4, x1, x128, x64, x64sx, xcbm2, xcbm5x0, xemu, xpet, xplus4, xrick, xroar, xscpu64, xvic, yabasanshiro, yabause]
|
||||
cores:
|
||||
- '2048'
|
||||
- '81'
|
||||
- a5200
|
||||
- advancemame
|
||||
- amiberry
|
||||
- applewin
|
||||
- arduous
|
||||
- atari800
|
||||
- b2
|
||||
- beebem
|
||||
- bk
|
||||
- bluemsx
|
||||
- boom3
|
||||
- bsnes
|
||||
- bsneshd
|
||||
- cannonball
|
||||
- cap32
|
||||
- cdi2015
|
||||
- corsixth
|
||||
- craft
|
||||
- crocods
|
||||
- daphne
|
||||
- desmume
|
||||
- dice
|
||||
- dinothawr
|
||||
- dirksimple
|
||||
- dolphin
|
||||
- dolphin-gui
|
||||
- dosbox
|
||||
- dosbox_pure
|
||||
- duckstation
|
||||
- easyrpg
|
||||
- ecwolf
|
||||
- emuscv
|
||||
- fake08
|
||||
- fba2x
|
||||
- fbneo
|
||||
- fceumm
|
||||
- flycast
|
||||
- flycast-next
|
||||
- fmsx
|
||||
- freechaf
|
||||
- freeintv
|
||||
- frotz
|
||||
- fuse
|
||||
- gambatte
|
||||
- gearcoleco
|
||||
- geargrafx
|
||||
- gearsystem
|
||||
- genesisplusgx
|
||||
- genesisplusgx_ex
|
||||
- genesisplusgxwide
|
||||
- geolith
|
||||
- glide64mk2
|
||||
- gliden64
|
||||
- gliden64_20
|
||||
- gong
|
||||
- gpsp
|
||||
- gsplus
|
||||
- gw
|
||||
- handy
|
||||
- hatari
|
||||
- hatarib
|
||||
- holani
|
||||
- imageviewer
|
||||
- julius
|
||||
- kronos
|
||||
- lowresnx
|
||||
- lutro
|
||||
- mame
|
||||
- mame0258
|
||||
- mame0278
|
||||
- mame2000
|
||||
- mame2003
|
||||
- mame2003_plus
|
||||
- mame2010
|
||||
- mame2015
|
||||
- mame2016
|
||||
- mamemess
|
||||
- mednafen_lynx
|
||||
- mednafen_ngp
|
||||
- mednafen_pce_fast
|
||||
- mednafen_pcfx
|
||||
- mednafen_psx
|
||||
- mednafen_psx_hw
|
||||
- mednafen_saturn
|
||||
- mednafen_supafaust
|
||||
- mednafen_supergrafx
|
||||
- mednafen_vb
|
||||
- mednafen_wswan
|
||||
- melonds
|
||||
- mesen
|
||||
- mesen_s
|
||||
- meteor
|
||||
- mgba
|
||||
- minivmac
|
||||
- mojozork
|
||||
- moonlight
|
||||
- mrboom
|
||||
- mu
|
||||
- mupen64plus
|
||||
- mupen64plus_next
|
||||
- n64_gles2
|
||||
- neocd
|
||||
- nestopia
|
||||
- np2kai
|
||||
- nxengine
|
||||
- o2em
|
||||
- openbor
|
||||
- openlara
|
||||
- opera
|
||||
- oricutron
|
||||
- parallel_n64
|
||||
- pcsx2
|
||||
- pcsx_rearmed
|
||||
- pico8
|
||||
- picodrive
|
||||
- pisnes
|
||||
- pokemini
|
||||
- potator
|
||||
- ppsspp
|
||||
- prboom
|
||||
- prosystem
|
||||
- ps2
|
||||
- puae
|
||||
- px68k
|
||||
- quasi88
|
||||
- quicknes
|
||||
- race
|
||||
- rb5000
|
||||
- reicast
|
||||
- reminiscence
|
||||
- retro8
|
||||
- retrodream
|
||||
- rice
|
||||
- rice_gles2
|
||||
- same_cdi
|
||||
- sameboy
|
||||
- sameduck
|
||||
- scummvm
|
||||
- sdlpop
|
||||
- simcoupe
|
||||
- snes9x
|
||||
- snes9x2002
|
||||
- snes9x2005
|
||||
- snes9x2010
|
||||
- solarus
|
||||
- stella
|
||||
- stella2014
|
||||
- stonesoup
|
||||
- supermodel
|
||||
- swanstation
|
||||
- tamalibretro
|
||||
- tgbdual
|
||||
- theodore
|
||||
- thepowdertoy
|
||||
- ti99sim
|
||||
- tic80
|
||||
- tyrquake
|
||||
- uae4all
|
||||
- uae4arm
|
||||
- uzem
|
||||
- vecx
|
||||
- vice_x128
|
||||
- vice_x64
|
||||
- vice_x64sc
|
||||
- vice_xcbm2
|
||||
- vice_xcbm5x0
|
||||
- vice_xpet
|
||||
- vice_xplus4
|
||||
- vice_xscpu64
|
||||
- vice_xvic
|
||||
- virtualjaguar
|
||||
- vitaquake2
|
||||
- vitaquake3
|
||||
- vitavoyager
|
||||
- vpinball
|
||||
- vvvvvv
|
||||
- wasm4
|
||||
- x1
|
||||
- x128
|
||||
- x64
|
||||
- x64sx
|
||||
- xcbm2
|
||||
- xcbm5x0
|
||||
- xemu
|
||||
- xpet
|
||||
- xplus4
|
||||
- xrick
|
||||
- xroar
|
||||
- xscpu64
|
||||
- xvic
|
||||
- yabasanshiro
|
||||
- yabause
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: file_exists
|
||||
file: /usr/bin/recalbox-settings
|
||||
bios_path: /recalbox/share/bios
|
||||
|
||||
- os: linux
|
||||
method: file_exists
|
||||
file: /usr/bin/recalbox-settings
|
||||
bios_path: /recalbox/share/bios
|
||||
retrobat:
|
||||
config: retrobat.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/RetroBat-Official/retrobat/main/system/resources/retrobat_logo_notext.png"
|
||||
logo: https://raw.githubusercontent.com/RetroBat-Official/retrobat/main/system/resources/retrobat_logo_notext.png
|
||||
scraper: retrobat
|
||||
source_url: "https://raw.githubusercontent.com/RetroBat-Official/emulatorlauncher/master/batocera-systems/Resources/batocera-systems.json"
|
||||
source_url: https://raw.githubusercontent.com/RetroBat-Official/emulatorlauncher/master/batocera-systems/Resources/batocera-systems.json
|
||||
source_format: json
|
||||
hash_type: md5
|
||||
schedule: weekly
|
||||
cores: [81, a5200, abuse, arduous, atari800, azahar, bennugd, bk, bluemsx, bsnes, bstone, cannonball, cap32, catacombgl, cdogs, cemu, cgenius, citron, clk, corsixth, demul, devilutionx, dhewm3, dice, dolphin, dosbox_pure, dxx-rebirth, easyrpg, ecwolf, eduke32, eka2l1, emuscv, etlegacy, fake08, fallout1-ce, fallout2-ce, fbneo, fceumm, flatpak, flycast, freechaf, freeintv, fury, fuse, gambatte, gearsystem, genesisplusgx, glide64mk2, gong, gsplus, gw, gzdoom, hatari, hcl, hurrican, hypseus-singe, ikemen, ioquake3, iortcw, jazz2-native, lindbergh-loader, lowresnx, lutro, mame, mame078plus, mednafen_lynx, mednafen_ngp, mednafen_supergrafx, mednafen_wswan, melonds, mgba, minivmac, model2emu, moonlight, mrboom, neocd, np2kai, nxengine, o2em, odcommander, openbor6412, openjazz, openjk, openjkdf2, openmohaa, opera, pce_fast, pcfx, pcsx2, pcsx_rearmed, pd777, picodrive, play, pokemini, potator, ppsspp, prboom, prosystem, puae, px68k, pygame, pyxel, quasi88, raze, reminiscence, rpcs3, ruffle, samcoupe, sameduck, scummvm, sdlpop, sh, shadps4, snes9x, solarus, sonic2013, sonic3-air, sonic-mania, steam, stella, superbroswar, supermodel, taradino, tgbdual, theforceengine, theodore, thextech, tic80, tr1x, tr2x, tsugaru, tyrian, tyrquake, uqm, uzem, vb, vecx, vice_x64, vircon32, virtualjaguar, vita3k, vox_official, vpinball, wasm4, wine-tkg, x1, x128, x16emu, xash3d_fwgs, xemu, xenia-canary, xpet, xplus4, xrick, xvic, yabasanshiro, yquake2, zc210]
|
||||
cores:
|
||||
- '81'
|
||||
- a5200
|
||||
- abuse
|
||||
- arduous
|
||||
- ares
|
||||
- atari800
|
||||
- azahar
|
||||
- bennugd
|
||||
- bk
|
||||
- bluemsx
|
||||
- bsnes
|
||||
- bstone
|
||||
- cannonball
|
||||
- cap32
|
||||
- catacombgl
|
||||
- cdogs
|
||||
- cemu
|
||||
- cgenius
|
||||
- citron
|
||||
- clk
|
||||
- corsixth
|
||||
- demul
|
||||
- devilutionx
|
||||
- dhewm3
|
||||
- dice
|
||||
- dolphin
|
||||
- dosbox_pure
|
||||
- dxx-rebirth
|
||||
- easyrpg
|
||||
- ecwolf
|
||||
- eduke32
|
||||
- eka2l1
|
||||
- emuscv
|
||||
- etlegacy
|
||||
- fake08
|
||||
- fallout1-ce
|
||||
- fallout2-ce
|
||||
- fbalpha2012
|
||||
- fbalpha2012_neogeo
|
||||
- fbneo
|
||||
- fceumm
|
||||
- flatpak
|
||||
- flycast
|
||||
- freechaf
|
||||
- freeintv
|
||||
- freej2me
|
||||
- fury
|
||||
- fuse
|
||||
- gambatte
|
||||
- geargrafx
|
||||
- gearsystem
|
||||
- genesisplusgx
|
||||
- glide64mk2
|
||||
- gong
|
||||
- gsplus
|
||||
- gw
|
||||
- gzdoom
|
||||
- hatari
|
||||
- hcl
|
||||
- hurrican
|
||||
- hypseus-singe
|
||||
- ikemen
|
||||
- ioquake3
|
||||
- iortcw
|
||||
- jazz2-native
|
||||
- lindbergh-loader
|
||||
- lowresnx
|
||||
- lutro
|
||||
- mame
|
||||
- mame078plus
|
||||
- mamemess
|
||||
- mednafen_lynx
|
||||
- mednafen_ngp
|
||||
- mednafen_supergrafx
|
||||
- mednafen_wswan
|
||||
- melonds
|
||||
- mgba
|
||||
- minivmac
|
||||
- model2emu
|
||||
- moonlight
|
||||
- mrboom
|
||||
- mupen64plus_next
|
||||
- neocd
|
||||
- np2kai
|
||||
- nxengine
|
||||
- o2em
|
||||
- odcommander
|
||||
- openbor6412
|
||||
- openjazz
|
||||
- openjk
|
||||
- openjkdf2
|
||||
- openmohaa
|
||||
- opera
|
||||
- parallel_n64
|
||||
- pce_fast
|
||||
- pcfx
|
||||
- pcsx2
|
||||
- pcsx_rearmed
|
||||
- pd777
|
||||
- picodrive
|
||||
- play
|
||||
- pokemini
|
||||
- potator
|
||||
- ppsspp
|
||||
- prboom
|
||||
- prosystem
|
||||
- puae
|
||||
- px68k
|
||||
- pygame
|
||||
- pyxel
|
||||
- quasi88
|
||||
- raze
|
||||
- reminiscence
|
||||
- rpcs3
|
||||
- ruffle
|
||||
- samcoupe
|
||||
- sameduck
|
||||
- scummvm
|
||||
- sdlpop
|
||||
- sh
|
||||
- shadps4
|
||||
- snes9x
|
||||
- solarus
|
||||
- sonic-mania
|
||||
- sonic2013
|
||||
- sonic3-air
|
||||
- squirreljme
|
||||
- steam
|
||||
- stella
|
||||
- superbroswar
|
||||
- supermodel
|
||||
- taradino
|
||||
- tgbdual
|
||||
- theforceengine
|
||||
- theodore
|
||||
- thextech
|
||||
- tic80
|
||||
- tr1x
|
||||
- tr2x
|
||||
- tsugaru
|
||||
- tyrian
|
||||
- tyrquake
|
||||
- uqm
|
||||
- uzem
|
||||
- vb
|
||||
- vecx
|
||||
- vice_x64
|
||||
- vircon32
|
||||
- virtualjaguar
|
||||
- vita3k
|
||||
- vox_official
|
||||
- vpinball
|
||||
- wasm4
|
||||
- wine-tkg
|
||||
- x1
|
||||
- x128
|
||||
- x16emu
|
||||
- xash3d_fwgs
|
||||
- xemu
|
||||
- xenia-canary
|
||||
- xpet
|
||||
- xplus4
|
||||
- xrick
|
||||
- xroar
|
||||
- xvic
|
||||
- yabasanshiro
|
||||
- yquake2
|
||||
- zc210
|
||||
target_scraper: null
|
||||
target_source: null
|
||||
install:
|
||||
detect:
|
||||
- os: windows
|
||||
method: path_exists
|
||||
path: "%USERPROFILE%\\RetroBat\\bios"
|
||||
|
||||
- os: windows
|
||||
method: path_exists
|
||||
path: '%USERPROFILE%\RetroBat\bios'
|
||||
emudeck:
|
||||
config: emudeck.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/dragoonDorise/EmuDeck/main/icons/EmuDeck.png"
|
||||
logo: https://raw.githubusercontent.com/dragoonDorise/EmuDeck/main/icons/EmuDeck.png
|
||||
scraper: emudeck
|
||||
source_url: "https://raw.githubusercontent.com/dragoonDorise/EmuDeck/main/functions/checkBIOS.sh"
|
||||
source_wiki: "https://raw.githubusercontent.com/EmuDeck/emudeck.github.io/main/docs/tables/"
|
||||
source_url: https://raw.githubusercontent.com/dragoonDorise/EmuDeck/main/functions/checkBIOS.sh
|
||||
source_wiki: https://raw.githubusercontent.com/EmuDeck/emudeck.github.io/main/docs/tables/
|
||||
source_format: bash_script+csv
|
||||
hash_type: md5
|
||||
schedule: weekly
|
||||
target_scraper: emudeck_targets
|
||||
target_source: "https://github.com/dragoonDorise/EmuDeck"
|
||||
# dragoonDorise/EmuDeck = official repo (creator's account, 3.4k stars)
|
||||
# EmuDeck/emudeck.github.io = official wiki (org account)
|
||||
target_source: https://github.com/dragoonDorise/EmuDeck
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: "$HOME/.config/EmuDeck/settings.sh"
|
||||
parse_key: emulationPath
|
||||
bios_subdir: bios
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: "$HOME/Emulation/bios"
|
||||
- os: windows
|
||||
method: config_file
|
||||
config: "%APPDATA%\\EmuDeck\\settings.ps1"
|
||||
parse_key: "$emulationPath"
|
||||
bios_subdir: bios
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: $HOME/.config/EmuDeck/settings.sh
|
||||
parse_key: emulationPath
|
||||
bios_subdir: bios
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: $HOME/Emulation/bios
|
||||
- os: windows
|
||||
method: config_file
|
||||
config: '%APPDATA%\EmuDeck\settings.ps1'
|
||||
parse_key: $emulationPath
|
||||
bios_subdir: bios
|
||||
standalone_copies:
|
||||
- file: prod.keys
|
||||
targets:
|
||||
linux:
|
||||
- "$HOME/.local/share/yuzu/keys"
|
||||
- "$HOME/.local/share/eden/keys"
|
||||
- "$HOME/.config/Ryujinx/system"
|
||||
windows:
|
||||
- "%APPDATA%\\yuzu\\keys"
|
||||
- "%APPDATA%\\eden\\keys"
|
||||
- file: aes_keys.txt
|
||||
targets:
|
||||
linux:
|
||||
- "$HOME/Emulation/bios/citra/keys"
|
||||
|
||||
- file: prod.keys
|
||||
targets:
|
||||
linux:
|
||||
- $HOME/.local/share/yuzu/keys
|
||||
- $HOME/.local/share/eden/keys
|
||||
- $HOME/.config/Ryujinx/system
|
||||
windows:
|
||||
- '%APPDATA%\yuzu\keys'
|
||||
- '%APPDATA%\eden\keys'
|
||||
- file: aes_keys.txt
|
||||
targets:
|
||||
linux:
|
||||
- $HOME/Emulation/bios/citra/keys
|
||||
lakka:
|
||||
config: lakka.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/libretro/retroarch-assets/master/src/xmb/flatui/lakka.svg"
|
||||
logo: https://raw.githubusercontent.com/libretro/retroarch-assets/master/src/xmb/flatui/lakka.svg
|
||||
scraper: libretro
|
||||
inherits_from: retroarch
|
||||
cores: all_libretro
|
||||
schedule: weekly
|
||||
target_scraper: lakka_targets
|
||||
target_source: "https://buildbot.libretro.com/nightly/"
|
||||
target_source: https://buildbot.libretro.com/nightly/
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: os_release
|
||||
id: lakka
|
||||
bios_path: /storage/system
|
||||
|
||||
- os: linux
|
||||
method: os_release
|
||||
id: lakka
|
||||
bios_path: /storage/system
|
||||
retrodeck:
|
||||
config: retrodeck.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/RetroDECK/RetroDECK/main/res/icon.svg"
|
||||
logo: https://raw.githubusercontent.com/RetroDECK/RetroDECK/main/res/icon.svg
|
||||
scraper: retrodeck
|
||||
source_url: "https://github.com/RetroDECK/components"
|
||||
source_url: https://github.com/RetroDECK/components
|
||||
source_format: github_component_manifests
|
||||
hash_type: md5
|
||||
schedule: monthly
|
||||
cores: [azahar, cemu, dolphin, duckstation, gzdoom, mame, melonds, openbor, pcsx2, pico-8, ppsspp, primehack, retroarch, rpcs3, ruffle, solarus, vita3k, xemu, xroar]
|
||||
cores:
|
||||
- azahar
|
||||
- cemu
|
||||
- clk
|
||||
- dolphin
|
||||
- duckstation
|
||||
- gsplus
|
||||
- gzdoom
|
||||
- mame
|
||||
- melonds
|
||||
- openbor
|
||||
- pcsx2
|
||||
- pico-8
|
||||
- ppsspp
|
||||
- primehack
|
||||
- retroarch
|
||||
- rpcs3
|
||||
- ruffle
|
||||
- solarus
|
||||
- vita3k
|
||||
- xemu
|
||||
- xroar
|
||||
target_scraper: null
|
||||
target_source: null
|
||||
# Each component/<name>/component_manifest.json declares BIOS requirements
|
||||
# Scraper enumerates top-level dirs via GitHub API, fetches each manifest directly
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: "$HOME/.var/app/net.retrodeck.retrodeck"
|
||||
bios_path: "$HOME/retrodeck/bios"
|
||||
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: $HOME/.var/app/net.retrodeck.retrodeck
|
||||
bios_path: $HOME/retrodeck/bios
|
||||
romm:
|
||||
config: romm.yml
|
||||
status: active
|
||||
logo: "https://avatars.githubusercontent.com/u/168586850"
|
||||
logo: https://avatars.githubusercontent.com/u/168586850
|
||||
scraper: romm
|
||||
source_url: "https://raw.githubusercontent.com/rommapp/romm/master/backend/models/fixtures/known_bios_files.json"
|
||||
source_url: https://raw.githubusercontent.com/rommapp/romm/master/backend/models/fixtures/known_bios_files.json
|
||||
source_format: json
|
||||
hash_type: sha1
|
||||
schedule: monthly
|
||||
inherits_from: emulatorjs # cores inherited from emulatorjs.yml
|
||||
inherits_from: emulatorjs
|
||||
target_scraper: null
|
||||
target_source: null
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: /romm/library/bios
|
||||
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: /romm/library/bios
|
||||
cores:
|
||||
- atari800
|
||||
- clk
|
||||
- directxbox
|
||||
- dolphin
|
||||
- dolphin_launcher
|
||||
- ecwolf
|
||||
- ep128emu
|
||||
- ep128emu_core
|
||||
- freej2me
|
||||
- hatari
|
||||
- ishiiruka
|
||||
- lrps2
|
||||
- minivmac
|
||||
- nekop2
|
||||
- np2kai
|
||||
- o2em
|
||||
- pcsx2
|
||||
- play
|
||||
- pokemini
|
||||
- primehack
|
||||
- px68k
|
||||
- scummvm
|
||||
- squirreljme
|
||||
- x1
|
||||
- xemu
|
||||
retropie:
|
||||
config: retropie.yml
|
||||
status: archived # Last release: v4.8 (March 2022) - no update in 4 years
|
||||
logo: "https://avatars.githubusercontent.com/u/11378204"
|
||||
status: archived
|
||||
logo: https://avatars.githubusercontent.com/u/11378204
|
||||
scraper: null
|
||||
cores: all_libretro
|
||||
schedule: null
|
||||
target_scraper: retropie_targets
|
||||
target_source: "https://retropie.org.uk/stats/pkgflags/"
|
||||
target_source: https://retropie.org.uk/stats/pkgflags/
|
||||
install:
|
||||
detect:
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: "$HOME/RetroPie/BIOS"
|
||||
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: $HOME/RetroPie/BIOS
|
||||
bizhawk:
|
||||
config: bizhawk.yml
|
||||
status: active
|
||||
logo: "https://raw.githubusercontent.com/TASEmulators/BizHawk/master/Assets/bizhawk.ico"
|
||||
logo: https://raw.githubusercontent.com/TASEmulators/BizHawk/master/Assets/bizhawk.ico
|
||||
scraper: bizhawk
|
||||
source_url: "https://raw.githubusercontent.com/TASEmulators/BizHawk/master/src/BizHawk.Emulation.Common/Database/FirmwareDatabase.cs"
|
||||
source_url: https://raw.githubusercontent.com/TASEmulators/BizHawk/master/src/BizHawk.Emulation.Common/Database/FirmwareDatabase.cs
|
||||
source_format: csharp_firmware_database
|
||||
hash_type: sha1
|
||||
schedule: monthly
|
||||
cores: [gambatte, mgba, sameboy, melonds, snes9x, bsnes, beetle_psx, beetle_saturn, beetle_pce, beetle_pcfx, beetle_wswan, beetle_vb, beetle_ngp, opera, stella, picodrive, ppsspp, handy, quicknes, genesis_plus_gx, ares, mupen64plus_next, puae, prboom, virtualjaguar, vice_x64, mame]
|
||||
cores:
|
||||
- applewin
|
||||
- ares
|
||||
- azahar
|
||||
- beetle_ngp
|
||||
- beetle_pce
|
||||
- beetle_pcfx
|
||||
- beetle_psx
|
||||
- beetle_saturn
|
||||
- beetle_vb
|
||||
- beetle_wswan
|
||||
- bsnes
|
||||
- citra
|
||||
- citra2018
|
||||
- citra_canary
|
||||
- clk
|
||||
- fbneo
|
||||
- freechaf
|
||||
- freeintv
|
||||
- gambatte
|
||||
- genesis_plus_gx
|
||||
- handy
|
||||
- mame
|
||||
- melonds
|
||||
- mgba
|
||||
- mupen64plus_next
|
||||
- numero
|
||||
- o2em
|
||||
- opera
|
||||
- panda3ds
|
||||
- picodrive
|
||||
- ppsspp
|
||||
- prboom
|
||||
- prosystem
|
||||
- puae
|
||||
- quicknes
|
||||
- sameboy
|
||||
- snes9x
|
||||
- stella
|
||||
- trident
|
||||
- vecx
|
||||
- vice_x64
|
||||
- virtualjaguar
|
||||
target_scraper: null
|
||||
target_source: null
|
||||
install:
|
||||
detect:
|
||||
- os: windows
|
||||
method: path_exists
|
||||
path: "%USERPROFILE%\\BizHawk\\Firmware"
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: "$HOME/.config/BizHawk/Firmware"
|
||||
- os: windows
|
||||
method: path_exists
|
||||
path: '%USERPROFILE%\BizHawk\Firmware'
|
||||
- os: linux
|
||||
method: path_exists
|
||||
path: $HOME/.config/BizHawk/Firmware
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -4,10 +4,8 @@ dat_version: v1.19.0
|
||||
homepage: https://www.retroarch.com
|
||||
source: https://github.com/libretro/libretro-database/blob/master/dat/System.dat
|
||||
base_destination: system
|
||||
cores: all_libretro
|
||||
hash_type: sha1
|
||||
verification_mode: existence
|
||||
case_insensitive_fs: true
|
||||
systems:
|
||||
3do:
|
||||
files:
|
||||
@@ -102,6 +100,7 @@ systems:
|
||||
md5: 35fa1a1ebaaeea286dc5cd15487c13ea
|
||||
crc32: d5cbc509
|
||||
size: 1048576
|
||||
native_id: 3DO Company, The - 3DO
|
||||
core: opera
|
||||
manufacturer: Panasonic|GoldStar|Sanyo
|
||||
docs: https://docs.libretro.com/library/opera/
|
||||
@@ -135,6 +134,7 @@ systems:
|
||||
md5: 25629dfe870d097469c217b95fdc1c95
|
||||
crc32: 1fe22ecd
|
||||
size: 16384
|
||||
native_id: Amstrad - CPC
|
||||
arcade:
|
||||
files:
|
||||
- name: bubsys.zip
|
||||
@@ -249,6 +249,10 @@ systems:
|
||||
- name: aes.zip
|
||||
destination: aes.zip
|
||||
required: true
|
||||
native_id: Arcade
|
||||
core: fbneo
|
||||
manufacturer: Various
|
||||
docs: https://docs.libretro.com/library/fbneo/
|
||||
data_directories:
|
||||
- ref: fbneo-hiscore
|
||||
destination: ''
|
||||
@@ -256,9 +260,6 @@ systems:
|
||||
destination: fbneo
|
||||
- ref: fbneo-samples
|
||||
destination: fbneo
|
||||
core: fbneo
|
||||
manufacturer: Various
|
||||
docs: https://docs.libretro.com/library/fbneo/
|
||||
atari-400-800:
|
||||
files:
|
||||
- name: ATARIBAS.ROM
|
||||
@@ -303,6 +304,7 @@ systems:
|
||||
md5: d7eb37aec6960cba36bc500e0e5d00bc
|
||||
crc32: bdca01fb
|
||||
size: 8192
|
||||
native_id: Atari - 400-800
|
||||
atari-5200:
|
||||
files:
|
||||
- name: 5200.rom
|
||||
@@ -312,6 +314,7 @@ systems:
|
||||
md5: 281f20ea4320404ec820fb7ec0693b38
|
||||
crc32: 4248d3e3
|
||||
size: 2048
|
||||
native_id: Atari - 5200
|
||||
core: a5200
|
||||
manufacturer: Atari
|
||||
docs: https://docs.libretro.com/library/a5200/
|
||||
@@ -331,6 +334,7 @@ systems:
|
||||
md5: 0763f1ffb006ddbe32e52d497ee848ae
|
||||
crc32: 5d13730c
|
||||
size: 4096
|
||||
native_id: Atari - 7800
|
||||
core: prosystem
|
||||
manufacturer: Atari
|
||||
docs: https://docs.libretro.com/library/prosystem/
|
||||
@@ -343,6 +347,7 @@ systems:
|
||||
md5: fcd403db69f54290b51035d82f835e7b
|
||||
crc32: 0d973c9d
|
||||
size: 512
|
||||
native_id: Atari - Lynx
|
||||
core: handy
|
||||
manufacturer: Atari
|
||||
docs: https://docs.libretro.com/library/handy/
|
||||
@@ -355,6 +360,7 @@ systems:
|
||||
md5: c1c57ce48e8ee4135885cee9e63a68a2
|
||||
crc32: d3c32283
|
||||
size: 196608
|
||||
native_id: Atari - ST
|
||||
core: hatari
|
||||
manufacturer: Atari
|
||||
docs: https://docs.libretro.com/library/hatari/
|
||||
@@ -376,6 +382,7 @@ systems:
|
||||
- name: bioscv.rom
|
||||
destination: bioscv.rom
|
||||
required: true
|
||||
native_id: Coleco - ColecoVision
|
||||
commodore-amiga:
|
||||
files:
|
||||
- name: kick33180.A500
|
||||
@@ -462,6 +469,7 @@ systems:
|
||||
md5: bb72565701b1b6faece07d68ea5da639
|
||||
crc32: 87746be2
|
||||
size: 524288
|
||||
native_id: Commodore - Amiga
|
||||
core: puae
|
||||
manufacturer: Commodore
|
||||
docs: https://docs.libretro.com/library/puae/
|
||||
@@ -510,6 +518,7 @@ systems:
|
||||
destination: scpu-dos-2.04.bin
|
||||
required: true
|
||||
md5: b2869f8678b8b274227f35aad26ba509
|
||||
native_id: Commodore - C128
|
||||
core: vice_x128
|
||||
manufacturer: Commodore
|
||||
docs: https://docs.libretro.com/library/vice_x128/
|
||||
@@ -522,6 +531,7 @@ systems:
|
||||
md5: a2e891e330d146c4046c2b622fc31462
|
||||
crc32: 683ed4ad
|
||||
size: 5763199
|
||||
native_id: Dinothawr
|
||||
dos:
|
||||
files:
|
||||
- name: MT32_CONTROL.ROM
|
||||
@@ -552,6 +562,7 @@ systems:
|
||||
md5: 08cdcfa0ed93e9cb16afa76e6ac5f0a4
|
||||
crc32: 4b961eba
|
||||
size: 1048576
|
||||
native_id: DOS
|
||||
elektronika-bk:
|
||||
files:
|
||||
- name: B11M_BOS.ROM
|
||||
@@ -610,6 +621,7 @@ systems:
|
||||
md5: 95f8c41c6abf7640e35a6a03cecebd01
|
||||
crc32: 26c6e8a0
|
||||
size: 8192
|
||||
native_id: Elektronika - BK-0010/BK-0011(M)
|
||||
enterprise-64-128:
|
||||
files:
|
||||
- name: hun.rom
|
||||
@@ -696,6 +708,7 @@ systems:
|
||||
md5: 55af78f877a21ca45eb2df68a74fcc60
|
||||
crc32: c099a5e3
|
||||
size: 65536
|
||||
native_id: Enterprise - 64/128
|
||||
includes:
|
||||
- ep128emu
|
||||
epoch-scv:
|
||||
@@ -707,6 +720,7 @@ systems:
|
||||
md5: 635a978fd40db9a18ee44eff449fc126
|
||||
crc32: 7ac06182
|
||||
size: 4096
|
||||
native_id: EPOCH/YENO Super Cassette Vision
|
||||
fairchild-channel-f:
|
||||
files:
|
||||
- name: sl31253.bin
|
||||
@@ -730,6 +744,7 @@ systems:
|
||||
md5: 95d339631d867c8f1d15a5f2ec26069d
|
||||
crc32: 015c1e38
|
||||
size: 1024
|
||||
native_id: Fairchild Channel F
|
||||
doom:
|
||||
files:
|
||||
- name: prboom.wad
|
||||
@@ -739,6 +754,7 @@ systems:
|
||||
md5: 72ae1b47820fcc93cc0df9c428d0face
|
||||
crc32: a5751b99
|
||||
size: 143312
|
||||
native_id: Id Software - Doom
|
||||
j2me:
|
||||
files:
|
||||
- name: freej2me-lr.jar
|
||||
@@ -762,6 +778,7 @@ systems:
|
||||
md5: 29a92d0867da2917275b7c6c805d256f
|
||||
crc32: ffb98ffa
|
||||
size: 552039
|
||||
native_id: J2ME
|
||||
core: freej2me
|
||||
manufacturer: Java
|
||||
docs: https://docs.libretro.com/library/freej2me/
|
||||
@@ -774,6 +791,7 @@ systems:
|
||||
md5: 66223be1497460f1e60885eeb35e03cc
|
||||
crc32: 4df6d054
|
||||
size: 262144
|
||||
native_id: MacII
|
||||
magnavox-odyssey2:
|
||||
files:
|
||||
- name: o2rom.bin
|
||||
@@ -804,6 +822,7 @@ systems:
|
||||
md5: 279008e4a0db2dc5f1c048853b033828
|
||||
crc32: 11647ca5
|
||||
size: 1024
|
||||
native_id: Magnavox - Odyssey2
|
||||
core: o2em
|
||||
manufacturer: Magnavox|Philips
|
||||
docs: https://docs.libretro.com/library/o2em/
|
||||
@@ -823,6 +842,7 @@ systems:
|
||||
md5: 0cd5946c6473e42e8e4c2137785e427f
|
||||
crc32: 683a4158
|
||||
size: 2048
|
||||
native_id: Mattel - Intellivision
|
||||
data_directories:
|
||||
- ref: freeintv-overlays
|
||||
destination: freeintv_overlays
|
||||
@@ -933,6 +953,7 @@ systems:
|
||||
md5: 279efd1eae0d358eecd4edc7d9adedf3
|
||||
crc32: ab6874f8
|
||||
size: 16640
|
||||
native_id: Microsoft - MSX
|
||||
core: bluemsx
|
||||
manufacturer: Spectravideo|Philips|Al Alamiah|Sony|Sanyo|Mitsubishi|Toshiba|Hitachi|Panasonic|Canon|Casio|Pioneer|Fujitsu|Yamaha|JVC|Kyocera|GoldStar|Samsung|Daewoo|Gradiente|Sharp|Talent|NTT|ACVS/CIEL|DDX|AGE
|
||||
Labs
|
||||
@@ -991,6 +1012,7 @@ systems:
|
||||
md5: 0754f903b52e3b3342202bdafb13efa5
|
||||
crc32: 2b5b75fe
|
||||
size: 262144
|
||||
native_id: NEC - PC Engine - TurboGrafx 16 - SuperGrafx
|
||||
core: mednafen_pce_fast
|
||||
manufacturer: NEC
|
||||
docs: https://docs.libretro.com/library/mednafen_pce_fast/
|
||||
@@ -1073,6 +1095,7 @@ systems:
|
||||
md5: 524473c1a5a03b17e21d86a0408ff827
|
||||
crc32: fe9f57f2
|
||||
size: 16384
|
||||
native_id: NEC - PC-98
|
||||
core: np2kai
|
||||
manufacturer: NEC
|
||||
docs: https://docs.libretro.com/library/np2kai/
|
||||
@@ -1115,6 +1138,7 @@ systems:
|
||||
md5: e2fb7c7220e3a7838c2dd7e401a7f3d8
|
||||
crc32: 236102c9
|
||||
size: 1048576
|
||||
native_id: NEC - PC-FX
|
||||
core: mednafen_pcfx
|
||||
manufacturer: NEC
|
||||
docs: https://docs.libretro.com/library/mednafen_pcfx/
|
||||
@@ -1134,6 +1158,7 @@ systems:
|
||||
md5: 7f98d77d7a094ad7d069b74bd553ec98
|
||||
crc32: 4c514089
|
||||
size: 24592
|
||||
native_id: Nintendo - Famicom Disk System
|
||||
nintendo-gb:
|
||||
files:
|
||||
- name: dmg_boot.bin
|
||||
@@ -1150,6 +1175,7 @@ systems:
|
||||
md5: 32fbbd84168d3482956eb3c5051637f5
|
||||
crc32: 59c8598e
|
||||
size: 256
|
||||
native_id: Nintendo - Gameboy
|
||||
core: gambatte
|
||||
manufacturer: Nintendo
|
||||
docs: https://docs.libretro.com/library/gambatte/
|
||||
@@ -1162,6 +1188,7 @@ systems:
|
||||
md5: a860e8c0b6d573d191e4ec7db1b1e4f6
|
||||
crc32: '81977335'
|
||||
size: 16384
|
||||
native_id: Nintendo - Game Boy Advance
|
||||
core: gpsp
|
||||
manufacturer: Nintendo
|
||||
docs: https://docs.libretro.com/library/gpsp/
|
||||
@@ -1181,6 +1208,7 @@ systems:
|
||||
md5: dbfce9db9deaa2567f6a84fde55f9680
|
||||
crc32: 41884e46
|
||||
size: 2304
|
||||
native_id: Nintendo - Gameboy Color
|
||||
nintendo-gamecube:
|
||||
files:
|
||||
- name: gc-dvd-20010608.bin
|
||||
@@ -1274,6 +1302,7 @@ systems:
|
||||
- name: font_japanese.bin
|
||||
destination: dolphin-emu/Sys/GC/font_japanese.bin
|
||||
required: false
|
||||
native_id: Nintendo - GameCube
|
||||
core: dolphin
|
||||
manufacturer: Nintendo
|
||||
docs: https://docs.libretro.com/library/dolphin/
|
||||
@@ -1289,6 +1318,7 @@ systems:
|
||||
md5: 8d3d9f294b6e174bc7b1d2fd1c727530
|
||||
crc32: 7f933ce2
|
||||
size: 4194304
|
||||
native_id: Nintendo - Nintendo 64DD
|
||||
nintendo-ds:
|
||||
files:
|
||||
- name: bios7.bin
|
||||
@@ -1324,6 +1354,7 @@ systems:
|
||||
- name: dsi_nand.bin
|
||||
destination: dsi_nand.bin
|
||||
required: true
|
||||
native_id: Nintendo - Nintendo DS
|
||||
core: desmume
|
||||
manufacturer: Nintendo
|
||||
docs: https://docs.libretro.com/library/desmume/
|
||||
@@ -1332,10 +1363,11 @@ systems:
|
||||
- name: NstDatabase.xml
|
||||
destination: NstDatabase.xml
|
||||
required: true
|
||||
sha1: f92312bae56e29c5bf00a5103105fce78472bf5c
|
||||
md5: 0ee6cbdc6f5c96ce9c8aa5edb59066f4
|
||||
crc32: 0e4d552b
|
||||
sha1: 26322f182540211e9b5e3647675b7c593706ae2b
|
||||
md5: 7bfe8c0540ed4bd6a0f1e2a0f0118ced
|
||||
crc32: ebb2196c
|
||||
size: 1009534
|
||||
native_id: Nintendo - Nintendo Entertainment System
|
||||
core: fceumm
|
||||
manufacturer: Nintendo
|
||||
docs: https://docs.libretro.com/library/fceumm/
|
||||
@@ -1348,6 +1380,7 @@ systems:
|
||||
md5: 1e4fb124a3a886865acb574f388c803d
|
||||
crc32: aed3c14d
|
||||
size: 4096
|
||||
native_id: Nintendo - Pokemon Mini
|
||||
nintendo-satellaview:
|
||||
files:
|
||||
- name: BS-X.bin
|
||||
@@ -1371,6 +1404,7 @@ systems:
|
||||
md5: 4ed9648505ab33a4daec93707b16caba
|
||||
crc32: 8c573c7e
|
||||
size: 1048576
|
||||
native_id: Nintendo - Satellaview
|
||||
nintendo-sufami-turbo:
|
||||
files:
|
||||
- name: STBIOS.bin
|
||||
@@ -1380,6 +1414,7 @@ systems:
|
||||
md5: d3a44ba7d42a74d3ac58cb9c14c6a5ca
|
||||
crc32: 9b4ca911
|
||||
size: 262144
|
||||
native_id: Nintendo - SuFami Turbo
|
||||
nintendo-sgb:
|
||||
files:
|
||||
- name: SGB1.sfc
|
||||
@@ -1441,6 +1476,7 @@ systems:
|
||||
- name: sgb.boot.rom
|
||||
destination: sgb.boot.rom
|
||||
required: false
|
||||
native_id: Nintendo - Super Game Boy
|
||||
nintendo-snes:
|
||||
files:
|
||||
- name: cx4.data.rom
|
||||
@@ -1562,6 +1598,7 @@ systems:
|
||||
md5: dda40ccd57390c96e49d30a041f9a9e7
|
||||
crc32: f73d5e10
|
||||
size: 131072
|
||||
native_id: Nintendo - Super Nintendo Entertainment System
|
||||
core: bsnes
|
||||
manufacturer: Nintendo
|
||||
docs: https://docs.libretro.com/library/bsnes/
|
||||
@@ -1588,6 +1625,7 @@ systems:
|
||||
md5: 279008e4a0db2dc5f1c048853b033828
|
||||
crc32: 11647ca5
|
||||
size: 1024
|
||||
native_id: Phillips - Videopac+
|
||||
sega-dreamcast:
|
||||
files:
|
||||
- name: dc_boot.bin
|
||||
@@ -1611,6 +1649,7 @@ systems:
|
||||
md5: 0a93f7940c455905bea6e392dfde92a4
|
||||
crc32: c611b498
|
||||
size: 131072
|
||||
native_id: Sega - Dreamcast
|
||||
core: flycast
|
||||
manufacturer: Sega
|
||||
docs: https://docs.libretro.com/library/flycast/
|
||||
@@ -1668,6 +1707,7 @@ systems:
|
||||
- name: segasp.zip
|
||||
destination: dc/segasp.zip
|
||||
required: true
|
||||
native_id: Sega - Dreamcast-based Arcade
|
||||
sega-game-gear:
|
||||
files:
|
||||
- name: bios.gg
|
||||
@@ -1677,6 +1717,7 @@ systems:
|
||||
md5: 672e104c3be3a238301aceffc3b23fd6
|
||||
crc32: 0ebea9d4
|
||||
size: 1024
|
||||
native_id: Sega - Game Gear
|
||||
sega-master-system:
|
||||
files:
|
||||
- name: bios.sms
|
||||
@@ -1707,6 +1748,7 @@ systems:
|
||||
md5: 840481177270d5642a14ca71ee72844c
|
||||
crc32: 0072ed54
|
||||
size: 8192
|
||||
native_id: Sega - Master System - Mark III
|
||||
sega-mega-cd:
|
||||
files:
|
||||
- name: bios_CD_E.bin
|
||||
@@ -1730,6 +1772,7 @@ systems:
|
||||
md5: 2efd74e3232ff260e371b99f84024f7f
|
||||
crc32: c6d10268
|
||||
size: 131072
|
||||
native_id: Sega - Mega CD - Sega CD
|
||||
sega-mega-drive:
|
||||
files:
|
||||
- name: areplay.bin
|
||||
@@ -1774,6 +1817,7 @@ systems:
|
||||
md5: b4e76e416b887f4e7413ba76fa735f16
|
||||
crc32: 4dcfd55c
|
||||
size: 262144
|
||||
native_id: Sega - Mega Drive - Genesis
|
||||
core: genesis_plus_gx
|
||||
manufacturer: Sega
|
||||
docs: https://docs.libretro.com/library/genesis_plus_gx/
|
||||
@@ -1859,6 +1903,7 @@ systems:
|
||||
- name: stvbios.zip
|
||||
destination: kronos/stvbios.zip
|
||||
required: true
|
||||
native_id: Sega - Saturn
|
||||
core: kronos
|
||||
manufacturer: Sega
|
||||
docs: https://docs.libretro.com/library/kronos/
|
||||
@@ -1880,6 +1925,7 @@ systems:
|
||||
md5: 851e4a5936f17d13f8c39a980cf00d77
|
||||
crc32: e3995a57
|
||||
size: 2048
|
||||
native_id: Sharp - X1
|
||||
core: x1
|
||||
manufacturer: Sharp
|
||||
docs: https://docs.libretro.com/library/x1/
|
||||
@@ -1920,6 +1966,7 @@ systems:
|
||||
md5: 0617321daa182c3f3d6f41fd02fb3275
|
||||
crc32: 00eeb408
|
||||
size: 131072
|
||||
native_id: Sharp - X68000
|
||||
core: px68k
|
||||
manufacturer: Sharp
|
||||
docs: https://docs.libretro.com/library/px68k/
|
||||
@@ -2270,6 +2317,7 @@ systems:
|
||||
md5: 85fede415f4294cc777517d7eada482e
|
||||
crc32: 2cbe8995
|
||||
size: 32768
|
||||
native_id: Sinclair - ZX Spectrum
|
||||
core: fuse
|
||||
manufacturer: Sinclair|Amstrad
|
||||
docs: https://docs.libretro.com/library/fuse/
|
||||
@@ -2352,6 +2400,7 @@ systems:
|
||||
md5: 08ca8b2dba6662e8024f9e789711c6fc
|
||||
crc32: ff3abc59
|
||||
size: 524288
|
||||
native_id: SNK - NeoGeo CD
|
||||
core: neocd
|
||||
manufacturer: SNK
|
||||
docs: https://docs.libretro.com/library/neocd/
|
||||
@@ -2511,6 +2560,7 @@ systems:
|
||||
md5: 81bbe60ba7a3d1cea1d48c14cbcc647b
|
||||
crc32: 2f53b852
|
||||
size: 524288
|
||||
native_id: Sony - PlayStation
|
||||
core: duckstation
|
||||
manufacturer: Sony
|
||||
docs: https://docs.libretro.com/library/duckstation/
|
||||
@@ -3027,6 +3077,7 @@ systems:
|
||||
md5: d3e81e95db25f5a86a7b7474550a2155
|
||||
crc32: 4e8c160c
|
||||
size: 4194304
|
||||
native_id: Sony - PlayStation 2
|
||||
sony-psp:
|
||||
files:
|
||||
- name: ppge_atlas.zim
|
||||
@@ -3036,6 +3087,7 @@ systems:
|
||||
md5: 866855cc330b9b95cc69135fb7b41d38
|
||||
crc32: 7b57fa78
|
||||
size: 666530
|
||||
native_id: Sony - PlayStation Portable
|
||||
core: ppsspp
|
||||
manufacturer: Sony
|
||||
docs: https://docs.libretro.com/library/ppsspp/
|
||||
@@ -3065,6 +3117,7 @@ systems:
|
||||
md5: d4448d09bbfde687c04f9e3310e023ab
|
||||
crc32: 4bf05697
|
||||
size: 262144
|
||||
native_id: Texas Instruments TI-83
|
||||
core: numero
|
||||
manufacturer: Texas Instruments
|
||||
docs: https://docs.libretro.com/library/numero/
|
||||
@@ -3098,6 +3151,7 @@ systems:
|
||||
md5: 88dc7876d584f90e4106f91444ab23b7
|
||||
crc32: 1466aed4
|
||||
size: 16384
|
||||
native_id: Videoton - TV Computer
|
||||
wolfenstein-3d:
|
||||
files:
|
||||
- name: ecwolf.pk3
|
||||
@@ -3107,6 +3161,7 @@ systems:
|
||||
md5: c011b428819eea4a80b455c245a5a04d
|
||||
crc32: 26dc3fba
|
||||
size: 178755
|
||||
native_id: Wolfenstein 3D
|
||||
scummvm:
|
||||
files:
|
||||
- name: scummvm.zip
|
||||
@@ -3116,6 +3171,7 @@ systems:
|
||||
md5: a17e0e0150155400d8cced329563d9c8
|
||||
crc32: a93f1c4b
|
||||
size: 9523360
|
||||
native_id: ScummVM
|
||||
core: scummvm
|
||||
manufacturer: Various
|
||||
docs: https://docs.libretro.com/library/scummvm/
|
||||
@@ -3141,3 +3197,5 @@ systems:
|
||||
core: xrick
|
||||
manufacturer: Other
|
||||
docs: https://docs.libretro.com/library/xrick/
|
||||
case_insensitive_fs: true
|
||||
cores: all_libretro
|
||||
|
||||
@@ -1,170 +1,10 @@
|
||||
platform: RetroBat
|
||||
version: 7.5.3
|
||||
homepage: "https://www.retrobat.org"
|
||||
source: "https://raw.githubusercontent.com/RetroBat-Official/emulatorlauncher/master/batocera-systems/Resources/batocera-systems.json"
|
||||
homepage: https://www.retrobat.org
|
||||
source: https://raw.githubusercontent.com/RetroBat-Official/emulatorlauncher/master/batocera-systems/Resources/batocera-systems.json
|
||||
base_destination: bios
|
||||
hash_type: md5
|
||||
verification_mode: md5
|
||||
case_insensitive_fs: true
|
||||
cores:
|
||||
- 81
|
||||
- a5200
|
||||
- abuse
|
||||
- arduous
|
||||
- atari800
|
||||
- azahar
|
||||
- bennugd
|
||||
- bk
|
||||
- bluemsx
|
||||
- bsnes
|
||||
- bstone
|
||||
- cannonball
|
||||
- cap32
|
||||
- catacombgl
|
||||
- cdogs
|
||||
- cemu
|
||||
- cgenius
|
||||
- citron
|
||||
- clk
|
||||
- corsixth
|
||||
- demul
|
||||
- devilutionx
|
||||
- dhewm3
|
||||
- dice
|
||||
- dolphin
|
||||
- dosbox_pure
|
||||
- dxx-rebirth
|
||||
- easyrpg
|
||||
- ecwolf
|
||||
- eduke32
|
||||
- eka2l1
|
||||
- emuscv
|
||||
- etlegacy
|
||||
- fake08
|
||||
- fallout1-ce
|
||||
- fallout2-ce
|
||||
- fbneo
|
||||
- fceumm
|
||||
- flatpak
|
||||
- flycast
|
||||
- freechaf
|
||||
- freeintv
|
||||
- fury
|
||||
- fuse
|
||||
- gambatte
|
||||
- gearsystem
|
||||
- genesisplusgx
|
||||
- glide64mk2
|
||||
- gong
|
||||
- gsplus
|
||||
- gw
|
||||
- gzdoom
|
||||
- hatari
|
||||
- hcl
|
||||
- hurrican
|
||||
- hypseus-singe
|
||||
- ikemen
|
||||
- ioquake3
|
||||
- iortcw
|
||||
- jazz2-native
|
||||
- lindbergh-loader
|
||||
- lowresnx
|
||||
- lutro
|
||||
- mame
|
||||
- mame078plus
|
||||
- mednafen_lynx
|
||||
- mednafen_ngp
|
||||
- mednafen_supergrafx
|
||||
- mednafen_wswan
|
||||
- melonds
|
||||
- mgba
|
||||
- minivmac
|
||||
- model2emu
|
||||
- moonlight
|
||||
- mrboom
|
||||
- neocd
|
||||
- np2kai
|
||||
- nxengine
|
||||
- o2em
|
||||
- odcommander
|
||||
- openbor6412
|
||||
- openjazz
|
||||
- openjk
|
||||
- openjkdf2
|
||||
- openmohaa
|
||||
- opera
|
||||
- pce_fast
|
||||
- pcfx
|
||||
- pcsx2
|
||||
- pcsx_rearmed
|
||||
- pd777
|
||||
- picodrive
|
||||
- play
|
||||
- pokemini
|
||||
- potator
|
||||
- ppsspp
|
||||
- prboom
|
||||
- prosystem
|
||||
- puae
|
||||
- px68k
|
||||
- pygame
|
||||
- pyxel
|
||||
- quasi88
|
||||
- raze
|
||||
- reminiscence
|
||||
- rpcs3
|
||||
- ruffle
|
||||
- samcoupe
|
||||
- sameduck
|
||||
- scummvm
|
||||
- sdlpop
|
||||
- sh
|
||||
- shadps4
|
||||
- snes9x
|
||||
- solarus
|
||||
- sonic2013
|
||||
- sonic3-air
|
||||
- sonic-mania
|
||||
- steam
|
||||
- stella
|
||||
- superbroswar
|
||||
- supermodel
|
||||
- taradino
|
||||
- tgbdual
|
||||
- theforceengine
|
||||
- theodore
|
||||
- thextech
|
||||
- tic80
|
||||
- tr1x
|
||||
- tr2x
|
||||
- tsugaru
|
||||
- tyrian
|
||||
- tyrquake
|
||||
- uqm
|
||||
- uzem
|
||||
- vb
|
||||
- vecx
|
||||
- vice_x64
|
||||
- vircon32
|
||||
- virtualjaguar
|
||||
- vita3k
|
||||
- vox_official
|
||||
- vpinball
|
||||
- wasm4
|
||||
- wine-tkg
|
||||
- x1
|
||||
- x128
|
||||
- x16emu
|
||||
- xash3d_fwgs
|
||||
- xemu
|
||||
- xenia-canary
|
||||
- xpet
|
||||
- xplus4
|
||||
- xrick
|
||||
- xvic
|
||||
- yabasanshiro
|
||||
- yquake2
|
||||
- zc210
|
||||
systems:
|
||||
3do:
|
||||
files:
|
||||
@@ -425,12 +265,12 @@ systems:
|
||||
md5: 281f20ea4320404ec820fb7ec0693b38
|
||||
atari7800:
|
||||
files:
|
||||
- name: "7800 BIOS (E).rom"
|
||||
destination: "7800 BIOS (E).rom"
|
||||
- name: 7800 BIOS (E).rom
|
||||
destination: 7800 BIOS (E).rom
|
||||
required: true
|
||||
md5: 397bb566584be7b9764e7a68974c4263
|
||||
- name: "7800 BIOS (U).rom"
|
||||
destination: "7800 BIOS (U).rom"
|
||||
- name: 7800 BIOS (U).rom
|
||||
destination: 7800 BIOS (U).rom
|
||||
required: true
|
||||
md5: 0763f1ffb006ddbe32e52d497ee848ae
|
||||
- name: ProSystem.dat
|
||||
@@ -548,6 +388,40 @@ systems:
|
||||
- name: bbcmc_flop.xml
|
||||
destination: mame/hash/bbcmc_flop.xml
|
||||
required: true
|
||||
bk:
|
||||
files:
|
||||
- name: B11M_BOS.ROM
|
||||
destination: bk/B11M_BOS.ROM
|
||||
required: true
|
||||
md5: fe4627d1e3a1535874085050733263e7
|
||||
- name: B11M_EXT.ROM
|
||||
destination: bk/B11M_EXT.ROM
|
||||
required: true
|
||||
md5: dc52f365d56fa1951f5d35b1101b9e3f
|
||||
- name: BAS11M_0.ROM
|
||||
destination: bk/BAS11M_0.ROM
|
||||
required: true
|
||||
md5: 946f6f23ded03c0e26187f0b3ca75993
|
||||
- name: BAS11M_1.ROM
|
||||
destination: bk/BAS11M_1.ROM
|
||||
required: true
|
||||
md5: 1e6637f32aa7d1de03510030cac40bcf
|
||||
- name: DISK_327.ROM
|
||||
destination: bk/DISK_327.ROM
|
||||
required: true
|
||||
md5: 5015228eeeb238e65da8edcd1b6dfac7
|
||||
- name: BASIC10.ROM
|
||||
destination: bk/BASIC10.ROM
|
||||
required: true
|
||||
md5: 3fa774326d75410a065659aea80252f0
|
||||
- name: FOCAL10.ROM
|
||||
destination: bk/FOCAL10.ROM
|
||||
required: true
|
||||
md5: 5737f972e8638831ab71e9139abae052
|
||||
- name: MONIT10.ROM
|
||||
destination: bk/MONIT10.ROM
|
||||
required: true
|
||||
md5: 95f8c41c6abf7640e35a6a03cecebd01
|
||||
loopy:
|
||||
files:
|
||||
- name: casloopy.zip
|
||||
@@ -845,34 +719,30 @@ systems:
|
||||
md5: a6f31483d1da4558cc19025e21f95c1d
|
||||
jaguar:
|
||||
files:
|
||||
- name: "[BIOS] Atari Jaguar (World).j64"
|
||||
destination: "[BIOS] Atari Jaguar (World).j64"
|
||||
- name: '[BIOS] Atari Jaguar (World).j64'
|
||||
destination: '[BIOS] Atari Jaguar (World).j64'
|
||||
required: true
|
||||
md5: bcfe348c565d9dedb173822ee6850dea
|
||||
laseractive:
|
||||
files:
|
||||
- name: "[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin"
|
||||
destination: "laseractive/[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin"
|
||||
- name: '[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin'
|
||||
destination: laseractive/[BIOS] LaserActive PAC-N1 (Japan) (v1.02).bin
|
||||
required: true
|
||||
md5: f69f173b251d8bf7649b10a9167a10bf
|
||||
- name: "[BIOS] LaserActive PAC-N10 (US) (v1.02).bin"
|
||||
destination: "laseractive/[BIOS] LaserActive PAC-N10 (US) (v1.02).bin"
|
||||
- name: '[BIOS] LaserActive PAC-N10 (US) (v1.02).bin'
|
||||
destination: laseractive/[BIOS] LaserActive PAC-N10 (US) (v1.02).bin
|
||||
required: true
|
||||
md5: f0fb8a4605ac7eefbafd4f2d5a793cc8
|
||||
- name: "[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin"
|
||||
destination: "laseractive/[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin"
|
||||
- name: '[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin'
|
||||
destination: laseractive/[BIOS] LaserActive PCE-LP1 (Japan) (v1.02).bin
|
||||
required: true
|
||||
md5: 761fea207d0eafd4cfd78da7c44cac88
|
||||
- name: "Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer - Sega)(JP)(en-ja).bin"
|
||||
destination: "laseractive/Pioneer LaserActive Sega PAC Boot ROM v1.02 (1993)(Pioneer\
|
||||
\ - Sega)(JP)(en-ja).bin"
|
||||
- name: '[BIOS] LaserActive PAC-S10 (US) (v1.04).bin'
|
||||
destination: laseractive/[BIOS] LaserActive PAC-S10 (US) (v1.04).bin
|
||||
required: true
|
||||
md5: a5a2f9aae57d464bc66b80ee79c3da6e
|
||||
- name: "Pioneer LaserActive Sega PAC Boot ROM v1.04 (1993)(Pioneer - Sega)(US).bin"
|
||||
destination: "laseractive/Pioneer LaserActive Sega PAC Boot ROM v1.04 (1993)(Pioneer\
|
||||
\ - Sega)(US).bin"
|
||||
- name: '[BIOS] LaserActive PAC-S1 (Japan) (v1.05).bin'
|
||||
destination: laseractive/[BIOS] LaserActive PAC-S1 (Japan) (v1.05).bin
|
||||
required: true
|
||||
md5: 0e7393cd0951d6dde818fcd4cd819466
|
||||
lynx:
|
||||
files:
|
||||
- name: lynxboot.img
|
||||
@@ -916,11 +786,11 @@ systems:
|
||||
required: true
|
||||
mastersystem:
|
||||
files:
|
||||
- name: "[BIOS] Sega Master System (USA, Europe) (v1.3).sms"
|
||||
destination: "[BIOS] Sega Master System (USA, Europe) (v1.3).sms"
|
||||
- name: '[BIOS] Sega Master System (USA, Europe) (v1.3).sms'
|
||||
destination: '[BIOS] Sega Master System (USA, Europe) (v1.3).sms'
|
||||
required: true
|
||||
- name: "[BIOS] Sega Master System (Japan) (v2.1).sms"
|
||||
destination: "[BIOS] Sega Master System (Japan) (v2.1).sms"
|
||||
- name: '[BIOS] Sega Master System (Japan) (v2.1).sms'
|
||||
destination: '[BIOS] Sega Master System (Japan) (v2.1).sms'
|
||||
required: true
|
||||
msx:
|
||||
files:
|
||||
@@ -1205,13 +1075,13 @@ systems:
|
||||
md5: 64a95a4a884cf4cc15a566b856603193
|
||||
ngp:
|
||||
files:
|
||||
- name: "[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp"
|
||||
destination: "[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp"
|
||||
- name: '[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp'
|
||||
destination: '[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp'
|
||||
required: true
|
||||
ngpc:
|
||||
files:
|
||||
- name: "[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp"
|
||||
destination: "[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp"
|
||||
- name: '[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp'
|
||||
destination: '[BIOS] SNK Neo Geo Pocket (Japan, Europe) (En,Ja).ngp'
|
||||
required: true
|
||||
odyssey2:
|
||||
files:
|
||||
@@ -1319,7 +1189,7 @@ systems:
|
||||
destination: bios.min
|
||||
required: true
|
||||
md5: 1e4fb124a3a886865acb574f388c803d
|
||||
ps2:
|
||||
sony-playstation-2:
|
||||
files:
|
||||
- name: ps2-0230a-20080220.bin
|
||||
destination: pcsx2/bios/ps2-0230a-20080220.bin
|
||||
|
||||
@@ -191,6 +191,37 @@ def load_platform_config(platform_name: str, platforms_dir: str = "platforms") -
|
||||
system.setdefault("files", []).append(gf)
|
||||
existing.add(key)
|
||||
|
||||
# Merge metadata from _registry.yml. The registry is our curated source;
|
||||
# the scraped YAML may be incomplete (missing cores, metadata fields).
|
||||
# Registry fields supplement (not replace) the scraped config.
|
||||
registry_path = os.path.join(platforms_dir, "_registry.yml")
|
||||
if os.path.exists(registry_path):
|
||||
reg_real = os.path.realpath(registry_path)
|
||||
if reg_real not in _shared_yml_cache:
|
||||
with open(registry_path) as f:
|
||||
_shared_yml_cache[reg_real] = yaml.safe_load(f) or {}
|
||||
reg = _shared_yml_cache[reg_real]
|
||||
reg_entry = reg.get("platforms", {}).get(platform_name, {})
|
||||
|
||||
# Merge cores (union for lists, override for all_libretro)
|
||||
reg_cores = reg_entry.get("cores")
|
||||
if reg_cores is not None:
|
||||
cfg_cores = config.get("cores")
|
||||
if reg_cores == "all_libretro":
|
||||
config["cores"] = "all_libretro"
|
||||
elif isinstance(reg_cores, list) and isinstance(cfg_cores, list):
|
||||
merged_set = {str(c) for c in cfg_cores} | {str(c) for c in reg_cores}
|
||||
config["cores"] = sorted(merged_set)
|
||||
elif isinstance(reg_cores, list) and cfg_cores is None:
|
||||
config["cores"] = reg_cores
|
||||
|
||||
# Merge all registry fields absent from config (except cores,
|
||||
# handled above with union logic). No hardcoded list — any field
|
||||
# added to the registry is automatically available in the config.
|
||||
for key, val in reg_entry.items():
|
||||
if key != "cores" and key not in config:
|
||||
config[key] = val
|
||||
|
||||
_platform_config_cache[cache_key] = config
|
||||
return config
|
||||
|
||||
@@ -522,6 +553,25 @@ def resolve_local_file(
|
||||
if fn.casefold() in basename_targets:
|
||||
return os.path.join(root, fn), "data_dir"
|
||||
|
||||
# Agnostic fallback: for filename-agnostic files, find any DB file
|
||||
# matching the system path prefix and size criteria
|
||||
if file_entry.get("agnostic"):
|
||||
agnostic_prefix = file_entry.get("agnostic_path_prefix", "")
|
||||
min_size = file_entry.get("min_size", 0)
|
||||
max_size = file_entry.get("max_size", float("inf"))
|
||||
exact_size = file_entry.get("size")
|
||||
if exact_size and not min_size:
|
||||
min_size = exact_size
|
||||
max_size = exact_size
|
||||
if agnostic_prefix:
|
||||
for _sha1, entry in files_db.items():
|
||||
path = entry.get("path", "")
|
||||
if not path.startswith(agnostic_prefix):
|
||||
continue
|
||||
size = entry.get("size", 0)
|
||||
if min_size <= size <= max_size and os.path.exists(path):
|
||||
return path, "agnostic_fallback"
|
||||
|
||||
return None, "not_found"
|
||||
|
||||
|
||||
@@ -633,6 +683,8 @@ def load_emulator_profiles(
|
||||
if not emu_path.exists():
|
||||
return profiles
|
||||
for f in sorted(emu_path.glob("*.yml")):
|
||||
if f.name.endswith(".old.yml"):
|
||||
continue
|
||||
with open(f) as fh:
|
||||
profile = yaml.safe_load(fh) or {}
|
||||
if "emulator" not in profile:
|
||||
@@ -735,6 +787,15 @@ def resolve_platform_cores(
|
||||
for c in core_set
|
||||
if c in core_to_profile
|
||||
}
|
||||
# Support "all_libretro" as a list element: combines all libretro
|
||||
# profiles with explicitly listed standalone cores (e.g. RetroDECK
|
||||
# ships RetroArch + standalone emulators)
|
||||
if "all_libretro" in core_set or "retroarch" in core_set:
|
||||
result |= {
|
||||
name for name, p in profiles.items()
|
||||
if "libretro" in p.get("type", "")
|
||||
and p.get("type") != "alias"
|
||||
}
|
||||
else:
|
||||
# Fallback: system ID intersection with normalization
|
||||
norm_plat_systems = {_norm_system_id(s) for s in config.get("systems", {})}
|
||||
@@ -882,6 +943,72 @@ def filter_systems_by_target(
|
||||
return filtered
|
||||
|
||||
|
||||
def expand_platform_declared_names(config: dict, db: dict) -> set[str]:
|
||||
"""Build set of file names declared by a platform config.
|
||||
|
||||
Enriches the set with canonical names and aliases from the database
|
||||
by resolving each platform file's MD5 through by_md5. This handles
|
||||
cases where a platform declares a file under a different name than
|
||||
the emulator profile (e.g. Batocera ROM1 vs gsplus ROM).
|
||||
"""
|
||||
declared: set[str] = set()
|
||||
by_md5 = db.get("indexes", {}).get("by_md5", {})
|
||||
files_db = db.get("files", {})
|
||||
for system in config.get("systems", {}).values():
|
||||
for fe in system.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name:
|
||||
declared.add(name)
|
||||
md5 = fe.get("md5", "")
|
||||
if not md5:
|
||||
continue
|
||||
# Skip multi-hash and zippedFile entries (inner ROM MD5, not file MD5)
|
||||
if "," in md5 or fe.get("zippedFile"):
|
||||
continue
|
||||
sha1 = by_md5.get(md5.lower())
|
||||
if not sha1:
|
||||
continue
|
||||
entry = files_db.get(sha1, {})
|
||||
db_name = entry.get("name", "")
|
||||
if db_name:
|
||||
declared.add(db_name)
|
||||
for alias in entry.get("aliases", []):
|
||||
declared.add(alias)
|
||||
return declared
|
||||
|
||||
|
||||
import re
|
||||
|
||||
_TIMESTAMP_PATTERNS = [
|
||||
re.compile(r'"generated_at":\s*"[^"]*"'), # database.json
|
||||
re.compile(r'\*Auto-generated on [^*]*\*'), # README.md
|
||||
re.compile(r'\*Generated on [^*]*\*'), # docs site pages
|
||||
]
|
||||
|
||||
|
||||
def write_if_changed(path: str, content: str) -> bool:
|
||||
"""Write content to path only if the non-timestamp content differs.
|
||||
|
||||
Compares new and existing content after stripping timestamp lines.
|
||||
Returns True if the file was written, False if skipped (unchanged).
|
||||
"""
|
||||
if os.path.exists(path):
|
||||
with open(path) as f:
|
||||
existing = f.read()
|
||||
if _strip_timestamps(existing) == _strip_timestamps(content):
|
||||
return False
|
||||
with open(path, "w") as f:
|
||||
f.write(content)
|
||||
return True
|
||||
|
||||
|
||||
def _strip_timestamps(text: str) -> str:
|
||||
"""Remove known timestamp patterns for content comparison."""
|
||||
result = text
|
||||
for pattern in _TIMESTAMP_PATTERNS:
|
||||
result = pattern.sub("", result)
|
||||
return result
|
||||
|
||||
|
||||
# Validation and mode filtering -extracted to validation.py for SoC.
|
||||
# Re-exported below for backward compatibility.
|
||||
|
||||
@@ -16,15 +16,27 @@ from exporter import discover_exporters
|
||||
|
||||
OUTPUT_FILENAMES: dict[str, str] = {
|
||||
"retroarch": "System.dat",
|
||||
"lakka": "System.dat",
|
||||
"retropie": "System.dat",
|
||||
"batocera": "batocera-systems",
|
||||
"recalbox": "es_bios.xml",
|
||||
"retrobat": "batocera-systems.json",
|
||||
"emudeck": "checkBIOS.sh",
|
||||
"retrodeck": "component_manifest.json",
|
||||
"romm": "known_bios_files.json",
|
||||
}
|
||||
|
||||
|
||||
def output_filename(platform: str) -> str:
|
||||
"""Return the native output filename for a platform."""
|
||||
return OUTPUT_FILENAMES.get(platform, f"{platform}_bios.dat")
|
||||
def output_path(platform: str, output_dir: str) -> str:
|
||||
"""Return the full output path for a platform's native export.
|
||||
|
||||
Each platform gets its own subdirectory to avoid filename collisions
|
||||
(e.g. retroarch, lakka, retropie all produce System.dat).
|
||||
"""
|
||||
filename = OUTPUT_FILENAMES.get(platform, f"{platform}_bios.dat")
|
||||
plat_dir = Path(output_dir) / platform
|
||||
plat_dir.mkdir(parents=True, exist_ok=True)
|
||||
return str(plat_dir / filename)
|
||||
|
||||
|
||||
def run(
|
||||
@@ -35,8 +47,6 @@ def run(
|
||||
) -> int:
|
||||
"""Export truth to native formats, return exit code."""
|
||||
exporters = discover_exporters()
|
||||
output_path = Path(output_dir)
|
||||
output_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
errors = 0
|
||||
|
||||
@@ -60,7 +70,7 @@ def run(
|
||||
except (FileNotFoundError, OSError):
|
||||
pass
|
||||
|
||||
dest = str(output_path / output_filename(platform))
|
||||
dest = output_path(platform, output_dir)
|
||||
exporter = exporter_cls()
|
||||
exporter.export(truth_data, dest, scraped_data=scraped)
|
||||
|
||||
|
||||
@@ -25,3 +25,37 @@ class BaseExporter(ABC):
|
||||
@abstractmethod
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
"""Validate exported file against truth data, return list of issues."""
|
||||
|
||||
@staticmethod
|
||||
def _is_pattern(name: str) -> bool:
|
||||
"""Check if a filename is a placeholder pattern (not a real file)."""
|
||||
return "<" in name or ">" in name or "*" in name
|
||||
|
||||
@staticmethod
|
||||
def _dest(fe: dict) -> str:
|
||||
"""Get destination path for a file entry, falling back to name."""
|
||||
return fe.get("path") or fe.get("destination") or fe.get("name", "")
|
||||
|
||||
@staticmethod
|
||||
def _display_name(
|
||||
sys_id: str, scraped_sys: dict | None = None,
|
||||
) -> str:
|
||||
"""Get display name for a system from scraped data or slug."""
|
||||
if scraped_sys:
|
||||
name = scraped_sys.get("name")
|
||||
if name:
|
||||
return name
|
||||
# Fallback: convert slug to display name with acronym handling
|
||||
_UPPER = {
|
||||
"3do", "cdi", "cpc", "cps1", "cps2", "cps3", "dos", "gba",
|
||||
"gbc", "hle", "msx", "nes", "nds", "ngp", "psp", "psx",
|
||||
"sms", "snes", "stv", "tvc", "vb", "zx",
|
||||
}
|
||||
parts = sys_id.replace("-", " ").split()
|
||||
result = []
|
||||
for p in parts:
|
||||
if p.lower() in _UPPER:
|
||||
result.append(p.upper())
|
||||
else:
|
||||
result.append(p.capitalize())
|
||||
return " ".join(result)
|
||||
|
||||
111
scripts/exporter/batocera_exporter.py
Normal file
111
scripts/exporter/batocera_exporter.py
Normal file
@@ -0,0 +1,111 @@
|
||||
"""Exporter for Batocera batocera-systems format.
|
||||
|
||||
Produces a Python dict matching the exact format of
|
||||
batocera-linux/batocera-scripts/scripts/batocera-systems.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .base_exporter import BaseExporter
|
||||
|
||||
|
||||
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
"""Export truth data to Batocera batocera-systems format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "batocera"
|
||||
|
||||
def export(
|
||||
self,
|
||||
truth_data: dict,
|
||||
output_path: str,
|
||||
scraped_data: dict | None = None,
|
||||
) -> None:
|
||||
# Build native_id and display name maps from scraped data
|
||||
native_map: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
for sys_id, sys_data in scraped_data.get("systems", {}).items():
|
||||
nid = sys_data.get("native_id")
|
||||
if nid:
|
||||
native_map[sys_id] = nid
|
||||
|
||||
lines: list[str] = ["systems = {", ""]
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id in sorted(systems):
|
||||
sys_data = systems[sys_id]
|
||||
files = sys_data.get("files", [])
|
||||
if not files:
|
||||
continue
|
||||
|
||||
native_id = native_map.get(sys_id, sys_id)
|
||||
scraped_sys = scraped_data.get("systems", {}).get(sys_id) if scraped_data else None
|
||||
display_name = self._display_name(sys_id, scraped_sys)
|
||||
|
||||
# Build md5 lookup from scraped data for this system
|
||||
scraped_md5: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
s_sys = scraped_data.get("systems", {}).get(sys_id, {})
|
||||
for sf in s_sys.get("files", []):
|
||||
sname = sf.get("name", "").lower()
|
||||
smd5 = sf.get("md5", "")
|
||||
if sname and smd5:
|
||||
scraped_md5[sname] = smd5
|
||||
|
||||
# Build biosFiles entries as compact single-line dicts
|
||||
# Original format ALWAYS has md5 — use scraped md5 as fallback
|
||||
bios_parts: list[str] = []
|
||||
for fe in files:
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
dest = self._dest(fe)
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
if not md5:
|
||||
md5 = scraped_md5.get(name.lower(), "")
|
||||
|
||||
# Original format requires md5 for every entry — skip without
|
||||
if not md5:
|
||||
continue
|
||||
bios_parts.append(
|
||||
f'{{ "md5": "{md5}", "file": "bios/{dest}" }}'
|
||||
)
|
||||
|
||||
bios_str = ", ".join(bios_parts)
|
||||
line = (
|
||||
f' "{native_id}": '
|
||||
f'{{ "name": "{display_name}", '
|
||||
f'"biosFiles": [ {bios_str} ] }},'
|
||||
)
|
||||
lines.append(line)
|
||||
|
||||
lines.append("")
|
||||
lines.append("}")
|
||||
lines.append("")
|
||||
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
content = Path(output_path).read_text(encoding="utf-8")
|
||||
issues: list[str] = []
|
||||
for sys_data in truth_data.get("systems", {}).values():
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
# Skip entries without md5 (not exportable in this format)
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
if not md5:
|
||||
continue
|
||||
dest = self._dest(fe)
|
||||
if dest not in content and name not in content:
|
||||
issues.append(f"missing: {name}")
|
||||
return issues
|
||||
207
scripts/exporter/emudeck_exporter.py
Normal file
207
scripts/exporter/emudeck_exporter.py
Normal file
@@ -0,0 +1,207 @@
|
||||
"""Exporter for EmuDeck checkBIOS.sh format.
|
||||
|
||||
Produces a bash script matching the exact pattern of EmuDeck's
|
||||
functions/checkBIOS.sh: per-system check functions with MD5 arrays
|
||||
inside the function body, iterating over $biosPath/* files.
|
||||
|
||||
Two patterns:
|
||||
- MD5 pattern: systems with known hashes, loop $biosPath/*, md5sum each, match
|
||||
- File-exists pattern: systems with specific paths, check -f
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from .base_exporter import BaseExporter
|
||||
|
||||
# Map our system IDs to EmuDeck function naming conventions
|
||||
_SYSTEM_CONFIG: dict[str, dict] = {
|
||||
"sony-playstation": {
|
||||
"func": "checkPS1BIOS",
|
||||
"var": "PSXBIOS",
|
||||
"array": "PSBios",
|
||||
"pattern": "md5",
|
||||
},
|
||||
"sony-playstation-2": {
|
||||
"func": "checkPS2BIOS",
|
||||
"var": "PS2BIOS",
|
||||
"array": "PS2Bios",
|
||||
"pattern": "md5",
|
||||
},
|
||||
"sega-mega-cd": {
|
||||
"func": "checkSegaCDBios",
|
||||
"var": "SEGACDBIOS",
|
||||
"array": "CDBios",
|
||||
"pattern": "md5",
|
||||
},
|
||||
"sega-saturn": {
|
||||
"func": "checkSaturnBios",
|
||||
"var": "SATURNBIOS",
|
||||
"array": "SaturnBios",
|
||||
"pattern": "md5",
|
||||
},
|
||||
"sega-dreamcast": {
|
||||
"func": "checkDreamcastBios",
|
||||
"var": "BIOS",
|
||||
"array": "hashes",
|
||||
"pattern": "md5",
|
||||
},
|
||||
"nintendo-ds": {
|
||||
"func": "checkDSBios",
|
||||
"var": "BIOS",
|
||||
"array": "hashes",
|
||||
"pattern": "md5",
|
||||
},
|
||||
"nintendo-switch": {
|
||||
"func": "checkCitronBios",
|
||||
"pattern": "file-exists",
|
||||
"firmware_path": "$biosPath/citron/firmware",
|
||||
"keys_path": "$biosPath/citron/keys/prod.keys",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _make_md5_function(cfg: dict, md5s: list[str]) -> list[str]:
|
||||
"""Generate a MD5-checking function matching EmuDeck's exact pattern."""
|
||||
func = cfg["func"]
|
||||
var = cfg["var"]
|
||||
array = cfg["array"]
|
||||
md5_str = " ".join(md5s)
|
||||
|
||||
return [
|
||||
f"{func}(){{",
|
||||
"",
|
||||
f'\t{var}="NULL"',
|
||||
"",
|
||||
'\tfor entry in "$biosPath/"*',
|
||||
"\tdo",
|
||||
'\t\tif [ -f "$entry" ]; then',
|
||||
'\t\t\tmd5=($(md5sum "$entry"))',
|
||||
f'\t\t\tif [[ "${var}" != true ]]; then',
|
||||
f"\t\t\t\t{array}=({md5_str})",
|
||||
f'\t\t\t\tfor i in "${{{array}[@]}}"',
|
||||
"\t\t\t\tdo",
|
||||
'\t\t\t\tif [[ "$md5" == *"${i}"* ]]; then',
|
||||
f"\t\t\t\t\t{var}=true",
|
||||
"\t\t\t\t\tbreak",
|
||||
"\t\t\t\telse",
|
||||
f"\t\t\t\t\t{var}=false",
|
||||
"\t\t\t\tfi",
|
||||
"\t\t\t\tdone",
|
||||
"\t\t\tfi",
|
||||
"\t\tfi",
|
||||
"\tdone",
|
||||
"",
|
||||
"",
|
||||
f"\tif [ ${var} == true ]; then",
|
||||
'\t\techo "$entry true";',
|
||||
"\telse",
|
||||
'\t\techo "false";',
|
||||
"\tfi",
|
||||
"}",
|
||||
]
|
||||
|
||||
|
||||
def _make_file_exists_function(cfg: dict) -> list[str]:
|
||||
"""Generate a file-exists function matching EmuDeck's pattern."""
|
||||
func = cfg["func"]
|
||||
firmware = cfg.get("firmware_path", "")
|
||||
keys = cfg.get("keys_path", "")
|
||||
|
||||
return [
|
||||
f"{func}(){{",
|
||||
"",
|
||||
f'\tlocal FIRMWARE="{firmware}"',
|
||||
f'\tlocal KEYS="{keys}"',
|
||||
'\tif [[ -f "$KEYS" ]] && [[ "$( ls -A "$FIRMWARE")" ]]; then',
|
||||
'\t\t\techo "true";',
|
||||
"\telse",
|
||||
'\t\t\techo "false";',
|
||||
"\tfi",
|
||||
"}",
|
||||
]
|
||||
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
"""Export truth data to EmuDeck checkBIOS.sh format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "emudeck"
|
||||
|
||||
def export(
|
||||
self,
|
||||
truth_data: dict,
|
||||
output_path: str,
|
||||
scraped_data: dict | None = None,
|
||||
) -> None:
|
||||
lines: list[str] = ["#!/bin/bash"]
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
|
||||
for sys_id, cfg in sorted(_SYSTEM_CONFIG.items(), key=lambda x: x[1]["func"]):
|
||||
sys_data = systems.get(sys_id)
|
||||
if not sys_data:
|
||||
continue
|
||||
|
||||
lines.append("")
|
||||
|
||||
if cfg["pattern"] == "md5":
|
||||
md5s: list[str] = []
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if self._is_pattern(name) or name.startswith("_"):
|
||||
continue
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5s.extend(m for m in md5 if m and re.fullmatch(r"[a-f0-9]{32}", m))
|
||||
elif md5 and re.fullmatch(r"[a-f0-9]{32}", md5):
|
||||
md5s.append(md5)
|
||||
if md5s:
|
||||
lines.extend(_make_md5_function(cfg, md5s))
|
||||
elif cfg["pattern"] == "file-exists":
|
||||
lines.extend(_make_file_exists_function(cfg))
|
||||
|
||||
lines.append("")
|
||||
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
content = Path(output_path).read_text(encoding="utf-8")
|
||||
issues: list[str] = []
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id, cfg in _SYSTEM_CONFIG.items():
|
||||
if cfg["pattern"] != "md5":
|
||||
continue
|
||||
sys_data = systems.get(sys_id)
|
||||
if not sys_data:
|
||||
continue
|
||||
for fe in sys_data.get("files", []):
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
if md5 and re.fullmatch(r"[a-f0-9]{32}", md5) and md5 not in content:
|
||||
issues.append(f"missing md5: {md5} ({fe.get('name', '')})")
|
||||
|
||||
for sys_id, cfg in _SYSTEM_CONFIG.items():
|
||||
func = cfg["func"]
|
||||
if func in content:
|
||||
continue
|
||||
sys_data = systems.get(sys_id)
|
||||
if not sys_data or not sys_data.get("files"):
|
||||
continue
|
||||
# Only flag if the system has usable data for the function type
|
||||
if cfg["pattern"] == "md5":
|
||||
has_md5 = any(
|
||||
fe.get("md5") and isinstance(fe.get("md5"), str)
|
||||
and re.fullmatch(r"[a-f0-9]{32}", fe["md5"])
|
||||
for fe in sys_data["files"]
|
||||
)
|
||||
if has_md5:
|
||||
issues.append(f"missing function: {func}")
|
||||
elif cfg["pattern"] == "file-exists":
|
||||
issues.append(f"missing function: {func}")
|
||||
|
||||
return issues
|
||||
17
scripts/exporter/lakka_exporter.py
Normal file
17
scripts/exporter/lakka_exporter.py
Normal file
@@ -0,0 +1,17 @@
|
||||
"""Exporter for Lakka (System.dat format, same as RetroArch).
|
||||
|
||||
Lakka inherits RetroArch cores and uses the same System.dat format.
|
||||
Delegates to systemdat_exporter for export and validation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .systemdat_exporter import Exporter as SystemDatExporter
|
||||
|
||||
|
||||
class Exporter(SystemDatExporter):
|
||||
"""Export truth data to Lakka System.dat format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "lakka"
|
||||
130
scripts/exporter/recalbox_exporter.py
Normal file
130
scripts/exporter/recalbox_exporter.py
Normal file
@@ -0,0 +1,130 @@
|
||||
"""Exporter for Recalbox es_bios.xml format.
|
||||
|
||||
Produces XML matching the exact format of recalbox's es_bios.xml:
|
||||
- XML namespace declaration
|
||||
- <system fullname="..." platform="...">
|
||||
- <bios path="system/file" md5="..." core="..." /> with optional mandatory, hashMatchMandatory, note
|
||||
- mandatory absent = true (only explicit when false)
|
||||
- 2-space indentation
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .base_exporter import BaseExporter
|
||||
|
||||
|
||||
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
"""Export truth data to Recalbox es_bios.xml format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "recalbox"
|
||||
|
||||
def export(
|
||||
self,
|
||||
truth_data: dict,
|
||||
output_path: str,
|
||||
scraped_data: dict | None = None,
|
||||
) -> None:
|
||||
native_map: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
for sys_id, sys_data in scraped_data.get("systems", {}).items():
|
||||
nid = sys_data.get("native_id")
|
||||
if nid:
|
||||
native_map[sys_id] = nid
|
||||
|
||||
lines: list[str] = [
|
||||
'<?xml version="1.0" encoding="UTF-8"?>',
|
||||
'<biosList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"'
|
||||
' xsi:noNamespaceSchemaLocation="es_bios.xsd">',
|
||||
]
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id in sorted(systems):
|
||||
sys_data = systems[sys_id]
|
||||
files = sys_data.get("files", [])
|
||||
if not files:
|
||||
continue
|
||||
|
||||
native_id = native_map.get(sys_id, sys_id)
|
||||
scraped_sys = scraped_data.get("systems", {}).get(sys_id) if scraped_data else None
|
||||
display_name = self._display_name(sys_id, scraped_sys)
|
||||
|
||||
lines.append(f' <system fullname="{display_name}" platform="{native_id}">')
|
||||
|
||||
# Build path lookup from scraped data for this system
|
||||
scraped_paths: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
s_sys = scraped_data.get("systems", {}).get(sys_id, {})
|
||||
for sf in s_sys.get("files", []):
|
||||
sname = sf.get("name", "").lower()
|
||||
spath = sf.get("destination", sf.get("name", ""))
|
||||
if sname and spath:
|
||||
scraped_paths[sname] = spath
|
||||
|
||||
for fe in files:
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
|
||||
# Use scraped path when available (preserves original format)
|
||||
path = scraped_paths.get(name.lower())
|
||||
if not path:
|
||||
dest = self._dest(fe)
|
||||
path = f"{native_id}/{dest}" if "/" not in dest else dest
|
||||
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = ",".join(md5)
|
||||
|
||||
required = fe.get("required", True)
|
||||
|
||||
# Build cores string from _cores
|
||||
cores_list = fe.get("_cores", [])
|
||||
core_str = ",".join(f"libretro/{c}" for c in cores_list) if cores_list else ""
|
||||
|
||||
attrs = [f'path="{path}"']
|
||||
if md5:
|
||||
attrs.append(f'md5="{md5}"')
|
||||
if not required:
|
||||
attrs.append('mandatory="false"')
|
||||
if not required:
|
||||
attrs.append('hashMatchMandatory="true"')
|
||||
if core_str:
|
||||
attrs.append(f'core="{core_str}"')
|
||||
|
||||
lines.append(f' <bios {" ".join(attrs)} />')
|
||||
|
||||
lines.append(" </system>")
|
||||
|
||||
lines.append("</biosList>")
|
||||
lines.append("")
|
||||
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
from xml.etree.ElementTree import parse as xml_parse
|
||||
|
||||
tree = xml_parse(output_path)
|
||||
root = tree.getroot()
|
||||
|
||||
exported_paths: set[str] = set()
|
||||
for bios_el in root.iter("bios"):
|
||||
path = bios_el.get("path", "")
|
||||
if path:
|
||||
exported_paths.add(path.lower())
|
||||
exported_paths.add(path.split("/")[-1].lower())
|
||||
|
||||
issues: list[str] = []
|
||||
for sys_data in truth_data.get("systems", {}).values():
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
dest = self._dest(fe)
|
||||
if name.lower() not in exported_paths and dest.lower() not in exported_paths:
|
||||
issues.append(f"missing: {name}")
|
||||
return issues
|
||||
114
scripts/exporter/retrobat_exporter.py
Normal file
114
scripts/exporter/retrobat_exporter.py
Normal file
@@ -0,0 +1,114 @@
|
||||
"""Exporter for RetroBat batocera-systems.json format.
|
||||
|
||||
Produces JSON matching the exact format of
|
||||
RetroBat-Official/emulatorlauncher/batocera-systems/Resources/batocera-systems.json:
|
||||
- System keys with "name" and "biosFiles" fields
|
||||
- Each biosFile has "md5" before "file" (matching original key order)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
|
||||
from .base_exporter import BaseExporter
|
||||
|
||||
|
||||
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
"""Export truth data to RetroBat batocera-systems.json format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "retrobat"
|
||||
|
||||
def export(
|
||||
self,
|
||||
truth_data: dict,
|
||||
output_path: str,
|
||||
scraped_data: dict | None = None,
|
||||
) -> None:
|
||||
native_map: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
for sys_id, sys_data in scraped_data.get("systems", {}).items():
|
||||
nid = sys_data.get("native_id")
|
||||
if nid:
|
||||
native_map[sys_id] = nid
|
||||
|
||||
output: OrderedDict[str, dict] = OrderedDict()
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id in sorted(systems):
|
||||
sys_data = systems[sys_id]
|
||||
files = sys_data.get("files", [])
|
||||
if not files:
|
||||
continue
|
||||
|
||||
native_id = native_map.get(sys_id, sys_id)
|
||||
scraped_sys = scraped_data.get("systems", {}).get(sys_id) if scraped_data else None
|
||||
display_name = self._display_name(sys_id, scraped_sys)
|
||||
bios_files: list[OrderedDict] = []
|
||||
|
||||
for fe in files:
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
dest = self._dest(fe)
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
|
||||
# Original format requires md5 for every entry
|
||||
if not md5:
|
||||
continue
|
||||
entry: OrderedDict[str, str] = OrderedDict()
|
||||
entry["md5"] = md5
|
||||
entry["file"] = f"bios/{dest}"
|
||||
bios_files.append(entry)
|
||||
|
||||
if bios_files:
|
||||
if native_id in output:
|
||||
existing_files = {e.get("file") for e in output[native_id]["biosFiles"]}
|
||||
for entry in bios_files:
|
||||
if entry.get("file") not in existing_files:
|
||||
output[native_id]["biosFiles"].append(entry)
|
||||
else:
|
||||
sys_entry: OrderedDict[str, object] = OrderedDict()
|
||||
sys_entry["name"] = display_name
|
||||
sys_entry["biosFiles"] = bios_files
|
||||
output[native_id] = sys_entry
|
||||
|
||||
Path(output_path).write_text(
|
||||
json.dumps(output, indent=2, ensure_ascii=False) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
data = json.loads(Path(output_path).read_text(encoding="utf-8"))
|
||||
|
||||
exported_files: set[str] = set()
|
||||
for sys_data in data.values():
|
||||
for bf in sys_data.get("biosFiles", []):
|
||||
path = bf.get("file", "")
|
||||
stripped = path.removeprefix("bios/")
|
||||
exported_files.add(stripped)
|
||||
basename = path.split("/")[-1] if "/" in path else path
|
||||
exported_files.add(basename)
|
||||
|
||||
issues: list[str] = []
|
||||
for sys_data in truth_data.get("systems", {}).values():
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
if not md5:
|
||||
continue
|
||||
dest = self._dest(fe)
|
||||
if name not in exported_files and dest not in exported_files:
|
||||
issues.append(f"missing: {name}")
|
||||
return issues
|
||||
208
scripts/exporter/retrodeck_exporter.py
Normal file
208
scripts/exporter/retrodeck_exporter.py
Normal file
@@ -0,0 +1,208 @@
|
||||
"""Exporter for RetroDECK component_manifest.json format.
|
||||
|
||||
Produces a JSON file compatible with RetroDECK's component manifests.
|
||||
Each system maps to a component with BIOS entries containing filename,
|
||||
md5 (comma-separated if multiple), paths ($bios_path default), and
|
||||
required status.
|
||||
|
||||
Path tokens: $bios_path for bios/, $roms_path for roms/.
|
||||
Entries without an explicit path default to $bios_path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
|
||||
from .base_exporter import BaseExporter
|
||||
|
||||
# retrobios slug -> RetroDECK system ID (reverse of scraper SYSTEM_SLUG_MAP)
|
||||
_REVERSE_SLUG: dict[str, str] = {
|
||||
"nintendo-nes": "nes",
|
||||
"nintendo-snes": "snes",
|
||||
"nintendo-64": "n64",
|
||||
"nintendo-64dd": "n64dd",
|
||||
"nintendo-gamecube": "gc",
|
||||
"nintendo-wii": "wii",
|
||||
"nintendo-wii-u": "wiiu",
|
||||
"nintendo-switch": "switch",
|
||||
"nintendo-gb": "gb",
|
||||
"nintendo-gbc": "gbc",
|
||||
"nintendo-gba": "gba",
|
||||
"nintendo-ds": "nds",
|
||||
"nintendo-3ds": "3ds",
|
||||
"nintendo-fds": "fds",
|
||||
"nintendo-sgb": "sgb",
|
||||
"nintendo-virtual-boy": "virtualboy",
|
||||
"nintendo-pokemon-mini": "pokemini",
|
||||
"sony-playstation": "psx",
|
||||
"sony-playstation-2": "ps2",
|
||||
"sony-playstation-3": "ps3",
|
||||
"sony-psp": "psp",
|
||||
"sony-psvita": "psvita",
|
||||
"sega-mega-drive": "megadrive",
|
||||
"sega-mega-cd": "megacd",
|
||||
"sega-saturn": "saturn",
|
||||
"sega-dreamcast": "dreamcast",
|
||||
"sega-dreamcast-arcade": "naomi",
|
||||
"sega-game-gear": "gamegear",
|
||||
"sega-master-system": "mastersystem",
|
||||
"nec-pc-engine": "pcengine",
|
||||
"nec-pc-fx": "pcfx",
|
||||
"nec-pc-98": "pc98",
|
||||
"nec-pc-88": "pc88",
|
||||
"3do": "3do",
|
||||
"amstrad-cpc": "amstradcpc",
|
||||
"arcade": "arcade",
|
||||
"atari-400-800": "atari800",
|
||||
"atari-5200": "atari5200",
|
||||
"atari-7800": "atari7800",
|
||||
"atari-jaguar": "atarijaguar",
|
||||
"atari-lynx": "atarilynx",
|
||||
"atari-st": "atarist",
|
||||
"commodore-c64": "c64",
|
||||
"commodore-amiga": "amiga",
|
||||
"philips-cdi": "cdimono1",
|
||||
"fairchild-channel-f": "channelf",
|
||||
"coleco-colecovision": "colecovision",
|
||||
"mattel-intellivision": "intellivision",
|
||||
"microsoft-msx": "msx",
|
||||
"microsoft-xbox": "xbox",
|
||||
"doom": "doom",
|
||||
"j2me": "j2me",
|
||||
"apple-macintosh-ii": "macintosh",
|
||||
"apple-ii": "apple2",
|
||||
"apple-iigs": "apple2gs",
|
||||
"enterprise-64-128": "enterprise",
|
||||
"tiger-game-com": "gamecom",
|
||||
"hartung-game-master": "gmaster",
|
||||
"epoch-scv": "scv",
|
||||
"watara-supervision": "supervision",
|
||||
"bandai-wonderswan": "wonderswan",
|
||||
"snk-neogeo-cd": "neogeocd",
|
||||
"tandy-coco": "coco",
|
||||
"tandy-trs-80": "trs80",
|
||||
"dragon-32-64": "dragon",
|
||||
"pico8": "pico8",
|
||||
"wolfenstein-3d": "wolfenstein",
|
||||
"sinclair-zx-spectrum": "zxspectrum",
|
||||
}
|
||||
|
||||
|
||||
def _dest_to_path_token(destination: str) -> str:
|
||||
"""Convert a truth destination path to a RetroDECK path token."""
|
||||
if destination.startswith("roms/"):
|
||||
return "$roms_path/" + destination.removeprefix("roms/")
|
||||
if destination.startswith("bios/"):
|
||||
return "$bios_path/" + destination.removeprefix("bios/")
|
||||
# Default: bios path
|
||||
return "$bios_path/" + destination
|
||||
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
"""Export truth data to RetroDECK component_manifest.json format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "retrodeck"
|
||||
|
||||
def export(
|
||||
self,
|
||||
truth_data: dict,
|
||||
output_path: str,
|
||||
scraped_data: dict | None = None,
|
||||
) -> None:
|
||||
native_map: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
for sys_id, sys_data in scraped_data.get("systems", {}).items():
|
||||
nid = sys_data.get("native_id")
|
||||
if nid:
|
||||
native_map[sys_id] = nid
|
||||
|
||||
manifest: OrderedDict[str, dict] = OrderedDict()
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id in sorted(systems):
|
||||
sys_data = systems[sys_id]
|
||||
files = sys_data.get("files", [])
|
||||
if not files:
|
||||
continue
|
||||
|
||||
native_id = native_map.get(sys_id, _REVERSE_SLUG.get(sys_id, sys_id))
|
||||
|
||||
bios_entries: list[OrderedDict] = []
|
||||
for fe in files:
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
|
||||
dest = self._dest(fe)
|
||||
path_token = _dest_to_path_token(dest)
|
||||
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = ",".join(m for m in md5 if m)
|
||||
|
||||
required = fe.get("required", True)
|
||||
|
||||
entry: OrderedDict[str, object] = OrderedDict()
|
||||
entry["filename"] = name
|
||||
if md5:
|
||||
# Validate MD5 entries
|
||||
parts = [
|
||||
m.strip().lower()
|
||||
for m in str(md5).split(",")
|
||||
if re.fullmatch(r"[0-9a-f]{32}", m.strip())
|
||||
]
|
||||
if parts:
|
||||
entry["md5"] = ",".join(parts) if len(parts) > 1 else parts[0]
|
||||
entry["paths"] = path_token
|
||||
entry["required"] = required
|
||||
|
||||
system_val = native_id
|
||||
entry["system"] = system_val
|
||||
|
||||
bios_entries.append(entry)
|
||||
|
||||
if bios_entries:
|
||||
if native_id in manifest:
|
||||
# Merge into existing component (multiple truth systems
|
||||
# may map to the same native ID)
|
||||
existing_names = {e["filename"] for e in manifest[native_id]["bios"]}
|
||||
for entry in bios_entries:
|
||||
if entry["filename"] not in existing_names:
|
||||
manifest[native_id]["bios"].append(entry)
|
||||
else:
|
||||
component = OrderedDict()
|
||||
component["system"] = native_id
|
||||
component["bios"] = bios_entries
|
||||
manifest[native_id] = component
|
||||
|
||||
Path(output_path).write_text(
|
||||
json.dumps(manifest, indent=2, ensure_ascii=False) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
data = json.loads(Path(output_path).read_text(encoding="utf-8"))
|
||||
|
||||
exported_names: set[str] = set()
|
||||
for comp_data in data.values():
|
||||
bios = comp_data.get("bios", [])
|
||||
if isinstance(bios, list):
|
||||
for entry in bios:
|
||||
fn = entry.get("filename", "")
|
||||
if fn:
|
||||
exported_names.add(fn)
|
||||
|
||||
issues: list[str] = []
|
||||
for sys_data in truth_data.get("systems", {}).values():
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
if name not in exported_names:
|
||||
issues.append(f"missing: {name}")
|
||||
return issues
|
||||
17
scripts/exporter/retropie_exporter.py
Normal file
17
scripts/exporter/retropie_exporter.py
Normal file
@@ -0,0 +1,17 @@
|
||||
"""Exporter for RetroPie (System.dat format, same as RetroArch).
|
||||
|
||||
RetroPie inherits RetroArch cores and uses the same System.dat format.
|
||||
Delegates to systemdat_exporter for export and validation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .systemdat_exporter import Exporter as SystemDatExporter
|
||||
|
||||
|
||||
class Exporter(SystemDatExporter):
|
||||
"""Export truth data to RetroPie System.dat format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "retropie"
|
||||
160
scripts/exporter/romm_exporter.py
Normal file
160
scripts/exporter/romm_exporter.py
Normal file
@@ -0,0 +1,160 @@
|
||||
"""Exporter for RomM known_bios_files.json format.
|
||||
|
||||
Produces JSON matching the exact format of
|
||||
rommapp/romm/backend/models/fixtures/known_bios_files.json:
|
||||
- Keys are "igdb_slug:filename"
|
||||
- Values contain size, crc, md5, sha1 (all optional but at least one hash)
|
||||
- Hashes are lowercase hex strings
|
||||
- Size is an integer
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
|
||||
from .base_exporter import BaseExporter
|
||||
|
||||
# retrobios slug -> IGDB slug (reverse of scraper SLUG_MAP)
|
||||
_REVERSE_SLUG: dict[str, str] = {
|
||||
"3do": "3do",
|
||||
"nintendo-64dd": "64dd",
|
||||
"amstrad-cpc": "acpc",
|
||||
"commodore-amiga": "amiga",
|
||||
"arcade": "arcade",
|
||||
"atari-st": "atari-st",
|
||||
"atari-5200": "atari5200",
|
||||
"atari-7800": "atari7800",
|
||||
"atari-400-800": "atari8bit",
|
||||
"coleco-colecovision": "colecovision",
|
||||
"sega-dreamcast": "dc",
|
||||
"doom": "doom",
|
||||
"enterprise-64-128": "enterprise",
|
||||
"fairchild-channel-f": "fairchild-channel-f",
|
||||
"nintendo-fds": "fds",
|
||||
"sega-game-gear": "gamegear",
|
||||
"nintendo-gb": "gb",
|
||||
"nintendo-gba": "gba",
|
||||
"nintendo-gbc": "gbc",
|
||||
"sega-mega-drive": "genesis",
|
||||
"mattel-intellivision": "intellivision",
|
||||
"j2me": "j2me",
|
||||
"atari-lynx": "lynx",
|
||||
"apple-macintosh-ii": "mac",
|
||||
"microsoft-msx": "msx",
|
||||
"nintendo-ds": "nds",
|
||||
"snk-neogeo-cd": "neo-geo-cd",
|
||||
"nintendo-nes": "nes",
|
||||
"nintendo-gamecube": "ngc",
|
||||
"magnavox-odyssey2": "odyssey-2-slash-videopac-g7000",
|
||||
"nec-pc-98": "pc-9800-series",
|
||||
"nec-pc-fx": "pc-fx",
|
||||
"nintendo-pokemon-mini": "pokemon-mini",
|
||||
"sony-playstation-2": "ps2",
|
||||
"sony-psp": "psp",
|
||||
"sony-playstation": "psx",
|
||||
"nintendo-satellaview": "satellaview",
|
||||
"sega-saturn": "saturn",
|
||||
"scummvm": "scummvm",
|
||||
"sega-mega-cd": "segacd",
|
||||
"sharp-x68000": "sharp-x68000",
|
||||
"sega-master-system": "sms",
|
||||
"nintendo-snes": "snes",
|
||||
"nintendo-sufami-turbo": "sufami-turbo",
|
||||
"nintendo-sgb": "super-gb",
|
||||
"nec-pc-engine": "tg16",
|
||||
"videoton-tvc": "tvc",
|
||||
"philips-videopac": "videopac-g7400",
|
||||
"wolfenstein-3d": "wolfenstein",
|
||||
"sharp-x1": "x1",
|
||||
"microsoft-xbox": "xbox",
|
||||
"sinclair-zx-spectrum": "zxs",
|
||||
}
|
||||
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
"""Export truth data to RomM known_bios_files.json format."""
|
||||
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "romm"
|
||||
|
||||
def export(
|
||||
self,
|
||||
truth_data: dict,
|
||||
output_path: str,
|
||||
scraped_data: dict | None = None,
|
||||
) -> None:
|
||||
native_map: dict[str, str] = {}
|
||||
if scraped_data:
|
||||
for sys_id, sys_data in scraped_data.get("systems", {}).items():
|
||||
nid = sys_data.get("native_id")
|
||||
if nid:
|
||||
native_map[sys_id] = nid
|
||||
|
||||
output: OrderedDict[str, dict] = OrderedDict()
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id in sorted(systems):
|
||||
sys_data = systems[sys_id]
|
||||
files = sys_data.get("files", [])
|
||||
if not files:
|
||||
continue
|
||||
|
||||
igdb_slug = native_map.get(sys_id, _REVERSE_SLUG.get(sys_id, sys_id))
|
||||
|
||||
for fe in files:
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
|
||||
key = f"{igdb_slug}:{name}"
|
||||
|
||||
entry: OrderedDict[str, object] = OrderedDict()
|
||||
|
||||
size = fe.get("size")
|
||||
if size is not None:
|
||||
entry["size"] = int(size)
|
||||
|
||||
crc = fe.get("crc32", "")
|
||||
if crc:
|
||||
entry["crc"] = str(crc).strip().lower()
|
||||
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
if md5:
|
||||
entry["md5"] = str(md5).strip().lower()
|
||||
|
||||
sha1 = fe.get("sha1", "")
|
||||
if isinstance(sha1, list):
|
||||
sha1 = sha1[0] if sha1 else ""
|
||||
if sha1:
|
||||
entry["sha1"] = str(sha1).strip().lower()
|
||||
|
||||
output[key] = entry
|
||||
|
||||
Path(output_path).write_text(
|
||||
json.dumps(output, indent=2, ensure_ascii=False) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
data = json.loads(Path(output_path).read_text(encoding="utf-8"))
|
||||
|
||||
exported_names: set[str] = set()
|
||||
for key in data:
|
||||
if ":" in key:
|
||||
_, filename = key.split(":", 1)
|
||||
exported_names.add(filename)
|
||||
|
||||
issues: list[str] = []
|
||||
for sys_data in truth_data.get("systems", {}).values():
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
if name not in exported_names:
|
||||
issues.append(f"missing: {name}")
|
||||
return issues
|
||||
@@ -1,4 +1,8 @@
|
||||
"""Exporter for libretro System.dat (clrmamepro DAT format)."""
|
||||
"""Exporter for libretro System.dat (clrmamepro DAT format).
|
||||
|
||||
Produces a single 'game' block with all ROMs grouped by system,
|
||||
matching the exact format of libretro-database/dat/System.dat.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -13,7 +17,7 @@ from .base_exporter import BaseExporter
|
||||
|
||||
|
||||
def _slug_to_native(slug: str) -> str:
|
||||
"""Convert a system slug to a native 'Manufacturer - Console' name."""
|
||||
"""Convert a system slug to 'Manufacturer - Console' format."""
|
||||
parts = slug.split("-", 1)
|
||||
if len(parts) == 1:
|
||||
return parts[0].title()
|
||||
@@ -42,45 +46,69 @@ class Exporter(BaseExporter):
|
||||
if nid:
|
||||
native_map[sys_id] = nid
|
||||
|
||||
lines: list[str] = []
|
||||
lines.append('clrmamepro (')
|
||||
lines.append('\tname "System.dat"')
|
||||
lines.append(')')
|
||||
# Match exact header format of libretro-database/dat/System.dat
|
||||
version = ""
|
||||
if scraped_data:
|
||||
version = scraped_data.get("dat_version", scraped_data.get("version", ""))
|
||||
lines: list[str] = [
|
||||
"clrmamepro (",
|
||||
'\tname "System"',
|
||||
'\tdescription "System"',
|
||||
'\tcomment "System, firmware, and BIOS files used by libretro cores."',
|
||||
]
|
||||
if version:
|
||||
lines.append(f"\tversion {version}")
|
||||
lines.extend([
|
||||
'\tauthor "libretro"',
|
||||
'\thomepage "https://github.com/libretro/libretro-database/blob/master/dat/System.dat"',
|
||||
'\turl "https://raw.githubusercontent.com/libretro/libretro-database/master/dat/System.dat"',
|
||||
")",
|
||||
"",
|
||||
"game (",
|
||||
'\tname "System"',
|
||||
'\tcomment "System"',
|
||||
])
|
||||
|
||||
systems = truth_data.get("systems", {})
|
||||
for sys_id in sorted(systems):
|
||||
sys_data = systems[sys_id]
|
||||
native_name = native_map.get(sys_id, _slug_to_native(sys_id))
|
||||
files = sys_data.get("files", [])
|
||||
if not files:
|
||||
continue
|
||||
|
||||
for fe in sys_data.get("files", []):
|
||||
native_name = native_map.get(sys_id, _slug_to_native(sys_id))
|
||||
lines.append("")
|
||||
lines.append(f'\tcomment "{native_name}"')
|
||||
|
||||
for fe in files:
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_"):
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
|
||||
dest = fe.get("path", name)
|
||||
size = fe.get("size", 0)
|
||||
# Quote names with spaces or special chars (matching original format)
|
||||
needs_quote = " " in name or "(" in name or ")" in name
|
||||
name_str = f'"{name}"' if needs_quote else name
|
||||
rom_parts = [f"name {name_str}"]
|
||||
size = fe.get("size")
|
||||
if size:
|
||||
rom_parts.append(f"size {size}")
|
||||
crc = fe.get("crc32", "")
|
||||
md5 = fe.get("md5", "")
|
||||
sha1 = fe.get("sha1", "")
|
||||
|
||||
rom_parts = [f'name "{name}"']
|
||||
rom_parts.append(f"size {size}")
|
||||
if crc:
|
||||
rom_parts.append(f"crc {crc}")
|
||||
rom_parts.append(f"crc {crc.upper()}")
|
||||
md5 = fe.get("md5", "")
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
if md5:
|
||||
rom_parts.append(f"md5 {md5}")
|
||||
sha1 = fe.get("sha1", "")
|
||||
if isinstance(sha1, list):
|
||||
sha1 = sha1[0] if sha1 else ""
|
||||
if sha1:
|
||||
rom_parts.append(f"sha1 {sha1}")
|
||||
rom_str = " ".join(rom_parts)
|
||||
|
||||
game_name = f"{native_name}/{dest}"
|
||||
lines.append("")
|
||||
lines.append("game (")
|
||||
lines.append(f'\tname "{game_name}"')
|
||||
lines.append(f'\tdescription "{name}"')
|
||||
lines.append(f"\trom ( {rom_str} )")
|
||||
lines.append(")")
|
||||
lines.append(f"\trom ( {' '.join(rom_parts)} )")
|
||||
|
||||
lines.append(")")
|
||||
lines.append("")
|
||||
Path(output_path).write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
@@ -93,12 +121,11 @@ class Exporter(BaseExporter):
|
||||
exported_names.add(rom.name)
|
||||
|
||||
issues: list[str] = []
|
||||
for sys_id, sys_data in truth_data.get("systems", {}).items():
|
||||
for sys_data in truth_data.get("systems", {}).values():
|
||||
for fe in sys_data.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name.startswith("_"):
|
||||
if name.startswith("_") or self._is_pattern(name):
|
||||
continue
|
||||
if name not in exported_names:
|
||||
issues.append(f"missing: {name} (system {sys_id})")
|
||||
|
||||
issues.append(f"missing: {name}")
|
||||
return issues
|
||||
|
||||
@@ -18,7 +18,7 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import compute_hashes, list_registered_platforms
|
||||
from common import compute_hashes, list_registered_platforms, write_if_changed
|
||||
|
||||
CACHE_DIR = ".cache"
|
||||
CACHE_FILE = os.path.join(CACHE_DIR, "db_cache.json")
|
||||
@@ -315,14 +315,15 @@ def main():
|
||||
"indexes": indexes,
|
||||
}
|
||||
|
||||
with open(args.output, "w") as f:
|
||||
json.dump(database, f, indent=2)
|
||||
new_content = json.dumps(database, indent=2)
|
||||
written = write_if_changed(args.output, new_content)
|
||||
|
||||
save_cache(CACHE_FILE, new_cache)
|
||||
|
||||
alias_count = sum(len(v) for v in aliases.values())
|
||||
name_count = len(indexes["by_name"])
|
||||
print(f"Generated {args.output}: {len(files)} files, {total_size:,} bytes total")
|
||||
status = "Generated" if written else "Unchanged"
|
||||
print(f"{status} {args.output}: {len(files)} files, {total_size:,} bytes total")
|
||||
print(f" Name index: {name_count} names ({alias_count} aliases)")
|
||||
return 0
|
||||
|
||||
@@ -400,6 +401,8 @@ def _collect_all_aliases(files: dict) -> dict:
|
||||
try:
|
||||
import yaml
|
||||
for emu_file in emulators_dir.glob("*.yml"):
|
||||
if emu_file.name.endswith(".old.yml"):
|
||||
continue
|
||||
try:
|
||||
with open(emu_file) as f:
|
||||
emu_config = yaml.safe_load(f) or {}
|
||||
|
||||
@@ -28,7 +28,7 @@ sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import (
|
||||
MANUFACTURER_PREFIXES,
|
||||
build_target_cores_cache, build_zip_contents_index, check_inside_zip,
|
||||
compute_hashes, fetch_large_file, group_identical_platforms,
|
||||
compute_hashes, expand_platform_declared_names, fetch_large_file, group_identical_platforms,
|
||||
list_emulator_profiles, list_platform_system_ids, list_registered_platforms,
|
||||
filter_systems_by_target, list_system_ids, load_database,
|
||||
load_data_dir_registry, load_emulator_profiles, load_platform_config,
|
||||
@@ -371,12 +371,8 @@ def _collect_emulator_extras(
|
||||
by_path_suffix = db.get("indexes", {}).get("by_path_suffix", {})
|
||||
|
||||
# Build set of filenames already covered (platform baseline + first pass extras)
|
||||
covered_names: set[str] = set()
|
||||
for sys_id, system in config.get("systems", {}).items():
|
||||
for fe in system.get("files", []):
|
||||
n = fe.get("name", "")
|
||||
if n:
|
||||
covered_names.add(n)
|
||||
# Enriched with canonical names from DB via MD5 (handles platform renaming)
|
||||
covered_names = expand_platform_declared_names(config, db)
|
||||
for e in extras:
|
||||
covered_names.add(e["name"])
|
||||
|
||||
@@ -426,6 +422,91 @@ def _collect_emulator_extras(
|
||||
"source_emulator": profile.get("emulator", emu_name),
|
||||
})
|
||||
|
||||
# Third pass: agnostic scan — for filename-agnostic cores, include all
|
||||
# DB files matching the system path prefix and size criteria.
|
||||
files_db = db.get("files", {})
|
||||
for emu_name, profile in sorted(profiles.items()):
|
||||
if profile.get("type") in ("launcher", "alias"):
|
||||
continue
|
||||
if emu_name not in relevant:
|
||||
continue
|
||||
is_profile_agnostic = profile.get("bios_mode") == "agnostic"
|
||||
if not is_profile_agnostic:
|
||||
if not any(f.get("agnostic") for f in profile.get("files", [])):
|
||||
continue
|
||||
|
||||
for f in profile.get("files", []):
|
||||
if not is_profile_agnostic and not f.get("agnostic"):
|
||||
continue
|
||||
fname = f.get("name", "")
|
||||
if not fname:
|
||||
continue
|
||||
|
||||
# Derive path prefix from the representative file in the DB
|
||||
path_prefix = None
|
||||
sha1_list = by_name.get(fname, [])
|
||||
for sha1 in sha1_list:
|
||||
entry = files_db.get(sha1, {})
|
||||
path = entry.get("path", "")
|
||||
if path:
|
||||
parts = path.rsplit("/", 1)
|
||||
if len(parts) == 2:
|
||||
path_prefix = parts[0] + "/"
|
||||
break
|
||||
|
||||
if not path_prefix:
|
||||
# Fallback: try other files in the profile for the same system
|
||||
for other_f in profile.get("files", []):
|
||||
if other_f is f:
|
||||
continue
|
||||
other_name = other_f.get("name", "")
|
||||
for sha1 in by_name.get(other_name, []):
|
||||
entry = files_db.get(sha1, {})
|
||||
path = entry.get("path", "")
|
||||
if path:
|
||||
parts = path.rsplit("/", 1)
|
||||
if len(parts) == 2:
|
||||
path_prefix = parts[0] + "/"
|
||||
break
|
||||
if path_prefix:
|
||||
break
|
||||
|
||||
if not path_prefix:
|
||||
continue
|
||||
|
||||
# Size criteria from the file entry
|
||||
min_size = f.get("min_size", 0)
|
||||
max_size = f.get("max_size", float("inf"))
|
||||
exact_size = f.get("size")
|
||||
if exact_size and not min_size:
|
||||
min_size = exact_size
|
||||
max_size = exact_size
|
||||
|
||||
# Scan DB for all files under this prefix matching size
|
||||
for sha1, entry in files_db.items():
|
||||
path = entry.get("path", "")
|
||||
if not path.startswith(path_prefix):
|
||||
continue
|
||||
size = entry.get("size", 0)
|
||||
if not (min_size <= size <= max_size):
|
||||
continue
|
||||
scan_name = entry.get("name", "")
|
||||
if not scan_name:
|
||||
continue
|
||||
dest = scan_name
|
||||
full_dest = f"{base_dest}/{dest}" if base_dest else dest
|
||||
if full_dest in seen_dests:
|
||||
continue
|
||||
seen_dests.add(full_dest)
|
||||
extras.append({
|
||||
"name": scan_name,
|
||||
"destination": dest,
|
||||
"required": False,
|
||||
"hle_fallback": False,
|
||||
"source_emulator": profile.get("emulator", emu_name),
|
||||
"agnostic_scan": True,
|
||||
})
|
||||
|
||||
return extras
|
||||
|
||||
|
||||
@@ -625,6 +706,24 @@ def _build_readme(platform_name: str, platform_display: str,
|
||||
return header + guide + footer
|
||||
|
||||
|
||||
def _build_agnostic_rename_readme(
|
||||
destination: str, original: str, alternatives: list[str],
|
||||
) -> str:
|
||||
"""Build a README explaining an agnostic file rename."""
|
||||
lines = [
|
||||
"This file was renamed for compatibility:",
|
||||
f" {destination} <- {original}",
|
||||
"",
|
||||
]
|
||||
if alternatives:
|
||||
lines.append("All variants included in this pack:")
|
||||
for alt in sorted(alternatives):
|
||||
lines.append(f" {alt}")
|
||||
lines.append("")
|
||||
lines.append(f"To use a different variant, rename it to: {destination}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def generate_pack(
|
||||
platform_name: str,
|
||||
platforms_dir: str,
|
||||
@@ -792,10 +891,71 @@ def generate_pack(
|
||||
continue
|
||||
|
||||
if status == "not_found":
|
||||
if not already_packed:
|
||||
missing_files.append(file_entry["name"])
|
||||
file_status[dedup_key] = "missing"
|
||||
continue
|
||||
# Agnostic fallback: if an agnostic core covers this system,
|
||||
# find any matching file in the DB
|
||||
by_name = db.get("indexes", {}).get("by_name", {})
|
||||
files_db = db.get("files", {})
|
||||
agnostic_path = None
|
||||
agnostic_resolved = False
|
||||
if emu_profiles:
|
||||
for _emu_key, _emu_prof in emu_profiles.items():
|
||||
if _emu_prof.get("bios_mode") != "agnostic":
|
||||
continue
|
||||
if sys_id not in set(_emu_prof.get("systems", [])):
|
||||
continue
|
||||
for _ef in _emu_prof.get("files", []):
|
||||
ef_name = _ef.get("name", "")
|
||||
for _sha1 in by_name.get(ef_name, []):
|
||||
_entry = files_db.get(_sha1, {})
|
||||
_path = _entry.get("path", "")
|
||||
if _path:
|
||||
_prefix = _path.rsplit("/", 1)[0] + "/"
|
||||
_min = _ef.get("min_size", 0)
|
||||
_max = _ef.get("max_size", float("inf"))
|
||||
if _ef.get("size") and not _min:
|
||||
_min = _ef["size"]
|
||||
_max = _ef["size"]
|
||||
for _s, _e in files_db.items():
|
||||
if _e.get("path", "").startswith(_prefix):
|
||||
if _min <= _e.get("size", 0) <= _max:
|
||||
if os.path.exists(_e["path"]):
|
||||
local_path = _e["path"]
|
||||
agnostic_path = _prefix
|
||||
agnostic_resolved = True
|
||||
break
|
||||
break
|
||||
if agnostic_resolved:
|
||||
break
|
||||
if agnostic_resolved:
|
||||
break
|
||||
|
||||
if agnostic_resolved and local_path:
|
||||
# Write rename README
|
||||
original_name = os.path.basename(local_path)
|
||||
dest_name = file_entry.get("name", "")
|
||||
if original_name != dest_name and agnostic_path:
|
||||
alt_names = []
|
||||
for _s, _e in files_db.items():
|
||||
_p = _e.get("path", "")
|
||||
if _p.startswith(agnostic_path):
|
||||
_n = _e.get("name", "")
|
||||
if _n and _n != original_name:
|
||||
alt_names.append(_n)
|
||||
readme_text = _build_agnostic_rename_readme(
|
||||
dest_name, original_name, alt_names,
|
||||
)
|
||||
readme_name = f"RENAMED_{dest_name}.txt"
|
||||
readme_full = f"{base_dest}/{readme_name}" if base_dest else readme_name
|
||||
if readme_full not in seen_destinations:
|
||||
zf.writestr(readme_full, readme_text)
|
||||
seen_destinations.add(readme_full)
|
||||
status = "agnostic_fallback"
|
||||
# Fall through to normal packing below
|
||||
else:
|
||||
if not already_packed:
|
||||
missing_files.append(file_entry["name"])
|
||||
file_status[dedup_key] = "missing"
|
||||
continue
|
||||
|
||||
if status == "hash_mismatch" and verification_mode != "existence":
|
||||
zf_name = file_entry.get("zipped_file")
|
||||
|
||||
@@ -18,7 +18,7 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import list_registered_platforms, load_database, load_platform_config
|
||||
from common import list_registered_platforms, load_database, load_platform_config, write_if_changed
|
||||
from verify import verify_platform
|
||||
|
||||
def compute_coverage(platform_name: str, platforms_dir: str, db: dict) -> dict:
|
||||
@@ -91,6 +91,7 @@ def generate_readme(db: dict, platforms_dir: str) -> str:
|
||||
|
||||
emulator_count = sum(
|
||||
1 for f in Path("emulators").glob("*.yml")
|
||||
if not f.name.endswith(".old.yml")
|
||||
) if Path("emulators").exists() else 0
|
||||
|
||||
# Count systems from emulator profiles
|
||||
@@ -100,6 +101,8 @@ def generate_readme(db: dict, platforms_dir: str) -> str:
|
||||
try:
|
||||
import yaml
|
||||
for f in emu_dir.glob("*.yml"):
|
||||
if f.name.endswith(".old.yml"):
|
||||
continue
|
||||
with open(f) as fh:
|
||||
p = yaml.safe_load(fh) or {}
|
||||
system_ids.update(p.get("systems", []))
|
||||
@@ -316,14 +319,12 @@ def main():
|
||||
db = load_database(args.db)
|
||||
|
||||
readme = generate_readme(db, args.platforms_dir)
|
||||
with open("README.md", "w") as f:
|
||||
f.write(readme)
|
||||
print(f"Generated ./README.md")
|
||||
status = "Generated" if write_if_changed("README.md", readme) else "Unchanged"
|
||||
print(f"{status} ./README.md")
|
||||
|
||||
contributing = generate_contributing()
|
||||
with open("CONTRIBUTING.md", "w") as f:
|
||||
f.write(contributing)
|
||||
print(f"Generated ./CONTRIBUTING.md")
|
||||
status = "Generated" if write_if_changed("CONTRIBUTING.md", contributing) else "Unchanged"
|
||||
print(f"{status} ./CONTRIBUTING.md")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -20,7 +20,7 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import list_registered_platforms, load_database, load_emulator_profiles, load_platform_config, require_yaml
|
||||
from common import list_registered_platforms, load_database, load_emulator_profiles, load_platform_config, require_yaml, write_if_changed
|
||||
|
||||
yaml = require_yaml()
|
||||
from generate_readme import compute_coverage
|
||||
@@ -1358,6 +1358,8 @@ The CI automatically:
|
||||
|
||||
|
||||
# Wiki pages
|
||||
# index, architecture, tools, profiling are maintained as wiki/ sources
|
||||
# and copied verbatim by main(). Only data-model is generated dynamically.
|
||||
|
||||
def generate_wiki_index() -> str:
|
||||
"""Generate wiki landing page."""
|
||||
@@ -1994,10 +1996,19 @@ def generate_mkdocs_nav(
|
||||
|
||||
wiki_nav = [
|
||||
{"Overview": "wiki/index.md"},
|
||||
{"Getting started": "wiki/getting-started.md"},
|
||||
{"FAQ": "wiki/faq.md"},
|
||||
{"Troubleshooting": "wiki/troubleshooting.md"},
|
||||
{"Architecture": "wiki/architecture.md"},
|
||||
{"Tools": "wiki/tools.md"},
|
||||
{"Profiling guide": "wiki/profiling.md"},
|
||||
{"Advanced usage": "wiki/advanced-usage.md"},
|
||||
{"Verification modes": "wiki/verification-modes.md"},
|
||||
{"Data model": "wiki/data-model.md"},
|
||||
{"Profiling guide": "wiki/profiling.md"},
|
||||
{"Adding a platform": "wiki/adding-a-platform.md"},
|
||||
{"Adding a scraper": "wiki/adding-a-scraper.md"},
|
||||
{"Testing guide": "wiki/testing-guide.md"},
|
||||
{"Release process": "wiki/release-process.md"},
|
||||
]
|
||||
|
||||
return [
|
||||
@@ -2064,7 +2075,7 @@ def main():
|
||||
|
||||
# Generate home
|
||||
print("Generating home page...")
|
||||
(docs / "index.md").write_text(generate_home(db, coverages, profiles, registry))
|
||||
write_if_changed(str(docs / "index.md"), generate_home(db, coverages, profiles, registry))
|
||||
|
||||
# Build system_id -> manufacturer page map (needed by all generators)
|
||||
print("Building system cross-reference map...")
|
||||
@@ -2074,37 +2085,35 @@ def main():
|
||||
|
||||
# Generate platform pages
|
||||
print("Generating platform pages...")
|
||||
(docs / "platforms" / "index.md").write_text(generate_platform_index(coverages))
|
||||
write_if_changed(str(docs / "platforms" / "index.md"), generate_platform_index(coverages))
|
||||
for name, cov in coverages.items():
|
||||
(docs / "platforms" / f"{name}.md").write_text(generate_platform_page(name, cov, registry, emulator_files))
|
||||
write_if_changed(str(docs / "platforms" / f"{name}.md"), generate_platform_page(name, cov, registry, emulator_files))
|
||||
|
||||
# Generate system pages
|
||||
print("Generating system pages...")
|
||||
|
||||
(docs / "systems" / "index.md").write_text(generate_systems_index(manufacturers))
|
||||
write_if_changed(str(docs / "systems" / "index.md"), generate_systems_index(manufacturers))
|
||||
for mfr, consoles in manufacturers.items():
|
||||
slug = mfr.lower().replace(" ", "-")
|
||||
page = generate_system_page(mfr, consoles, platform_files, emulator_files)
|
||||
(docs / "systems" / f"{slug}.md").write_text(page)
|
||||
write_if_changed(str(docs / "systems" / f"{slug}.md"), page)
|
||||
|
||||
# Generate emulator pages
|
||||
print("Generating emulator pages...")
|
||||
(docs / "emulators" / "index.md").write_text(generate_emulators_index(profiles))
|
||||
write_if_changed(str(docs / "emulators" / "index.md"), generate_emulators_index(profiles))
|
||||
for name, profile in profiles.items():
|
||||
page = generate_emulator_page(name, profile, db, platform_files)
|
||||
(docs / "emulators" / f"{name}.md").write_text(page)
|
||||
write_if_changed(str(docs / "emulators" / f"{name}.md"), page)
|
||||
|
||||
# Generate cross-reference page
|
||||
print("Generating cross-reference page...")
|
||||
(docs / "cross-reference.md").write_text(
|
||||
generate_cross_reference(coverages, profiles)
|
||||
)
|
||||
write_if_changed(str(docs / "cross-reference.md"),
|
||||
generate_cross_reference(coverages, profiles))
|
||||
|
||||
# Generate gap analysis page
|
||||
print("Generating gap analysis page...")
|
||||
(docs / "gaps.md").write_text(
|
||||
generate_gap_analysis(profiles, coverages, db)
|
||||
)
|
||||
write_if_changed(str(docs / "gaps.md"),
|
||||
generate_gap_analysis(profiles, coverages, db))
|
||||
|
||||
# Wiki pages: copy manually maintained sources + generate dynamic ones
|
||||
print("Generating wiki pages...")
|
||||
@@ -2115,11 +2124,11 @@ def main():
|
||||
for src_file in wiki_src.glob("*.md"):
|
||||
shutil.copy2(src_file, wiki_dest / src_file.name)
|
||||
# data-model.md is generated (contains live DB stats)
|
||||
(wiki_dest / "data-model.md").write_text(generate_wiki_data_model(db, profiles))
|
||||
write_if_changed(str(wiki_dest / "data-model.md"), generate_wiki_data_model(db, profiles))
|
||||
|
||||
# Generate contributing
|
||||
print("Generating contributing page...")
|
||||
(docs / "contributing.md").write_text(generate_contributing())
|
||||
write_if_changed(str(docs / "contributing.md"), generate_contributing())
|
||||
|
||||
# Update mkdocs.yml nav section only (avoid yaml.dump round-trip mangling quotes)
|
||||
print("Updating mkdocs.yml nav...")
|
||||
@@ -2167,15 +2176,17 @@ markdown_extensions:
|
||||
- toc:
|
||||
permalink: true
|
||||
- pymdownx.details
|
||||
- pymdownx.superfences
|
||||
- pymdownx.superfences:
|
||||
custom_fences:
|
||||
- name: mermaid
|
||||
class: mermaid
|
||||
format: !!python/name:pymdownx.superfences.fence_code_format
|
||||
- pymdownx.tabbed:
|
||||
alternate_style: true
|
||||
plugins:
|
||||
- search
|
||||
"""
|
||||
with open("mkdocs.yml", "w") as f:
|
||||
f.write(mkdocs_static)
|
||||
f.write(nav_yaml)
|
||||
write_if_changed("mkdocs.yml", mkdocs_static + nav_yaml)
|
||||
|
||||
total_pages = (
|
||||
1 # home
|
||||
@@ -2184,7 +2195,7 @@ plugins:
|
||||
+ 1 # cross-reference
|
||||
+ 1 + len(profiles) # emulator index + detail
|
||||
+ 1 # gap analysis
|
||||
+ 5 # wiki (index, architecture, tools, profiling, data model)
|
||||
+ 14 # wiki pages (copied from wiki/ + generated data-model)
|
||||
+ 1 # contributing
|
||||
)
|
||||
print(f"\nGenerated {total_pages} pages in {args.docs_dir}/")
|
||||
|
||||
@@ -177,6 +177,28 @@ def main():
|
||||
print("\n--- 2/9 refresh data directories: SKIPPED (--offline) ---")
|
||||
results["refresh_data"] = True
|
||||
|
||||
# Step 2a: Refresh MAME BIOS hashes
|
||||
if not args.offline:
|
||||
ok, _ = run(
|
||||
[sys.executable, "-m", "scripts.scraper.mame_hash_scraper"],
|
||||
"2a refresh MAME hashes",
|
||||
)
|
||||
results["mame_hashes"] = ok
|
||||
else:
|
||||
print("\n--- 2a refresh MAME hashes: SKIPPED (--offline) ---")
|
||||
results["mame_hashes"] = True
|
||||
|
||||
# Step 2a2: Refresh FBNeo BIOS hashes
|
||||
if not args.offline:
|
||||
ok, _ = run(
|
||||
[sys.executable, "-m", "scripts.scraper.fbneo_hash_scraper"],
|
||||
"2a2 refresh FBNeo hashes",
|
||||
)
|
||||
results["fbneo_hashes"] = ok
|
||||
else:
|
||||
print("\n--- 2a2 refresh FBNeo hashes: SKIPPED (--offline) ---")
|
||||
results["fbneo_hashes"] = True
|
||||
|
||||
# Step 2b: Check buildbot system directory (non-blocking)
|
||||
if args.check_buildbot and not args.offline:
|
||||
ok, _ = run(
|
||||
|
||||
565
scripts/scraper/_hash_merge.py
Normal file
565
scripts/scraper/_hash_merge.py
Normal file
@@ -0,0 +1,565 @@
|
||||
"""Merge fetched hash data into emulator YAML profiles.
|
||||
|
||||
Supports two strategies:
|
||||
- MAME: bios_zip entries with contents lists
|
||||
- FBNeo: individual ROM entries grouped by archive field
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
def merge_mame_profile(
|
||||
profile_path: str,
|
||||
hashes_path: str,
|
||||
write: bool = False,
|
||||
add_new: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
"""Merge MAME bios_zip entries from upstream hash data.
|
||||
|
||||
Preserves system, note, required per entry. Updates contents and
|
||||
source_ref from the hashes JSON. New sets are only added when
|
||||
add_new=True (main profile). Entries not in the hash data are
|
||||
left untouched (the scraper only covers MACHINE_IS_BIOS_ROOT sets,
|
||||
not all machine ROM sets).
|
||||
|
||||
If write=True, backs up existing profile to .old.yml before writing.
|
||||
"""
|
||||
profile = _load_yaml(profile_path)
|
||||
hashes = _load_json(hashes_path)
|
||||
|
||||
profile['core_version'] = hashes.get('version', profile.get('core_version'))
|
||||
|
||||
files = profile.get('files', [])
|
||||
bios_zip, non_bios = _split_files(files, lambda f: f.get('category') == 'bios_zip')
|
||||
|
||||
existing_by_name: dict[str, dict] = {}
|
||||
for entry in bios_zip:
|
||||
key = _zip_name_to_set(entry['name'])
|
||||
existing_by_name[key] = entry
|
||||
|
||||
updated_bios: list[dict] = []
|
||||
matched_names: set[str] = set()
|
||||
|
||||
for set_name, set_data in hashes.get('bios_sets', {}).items():
|
||||
contents = _build_contents(set_data.get('roms', []))
|
||||
source_ref = _build_source_ref(set_data)
|
||||
|
||||
if set_name in existing_by_name:
|
||||
# Update existing entry: preserve manual fields, update contents
|
||||
entry = existing_by_name[set_name].copy()
|
||||
entry['contents'] = contents
|
||||
if source_ref:
|
||||
entry['source_ref'] = source_ref
|
||||
updated_bios.append(entry)
|
||||
matched_names.add(set_name)
|
||||
elif add_new:
|
||||
# New BIOS set — only added to the main profile
|
||||
entry = {
|
||||
'name': f'{set_name}.zip',
|
||||
'required': True,
|
||||
'category': 'bios_zip',
|
||||
'system': None,
|
||||
'source_ref': source_ref,
|
||||
'contents': contents,
|
||||
}
|
||||
updated_bios.append(entry)
|
||||
|
||||
# Entries not matched by the scraper stay untouched
|
||||
# (computer ROMs, device ROMs, etc. — outside BIOS root set scope)
|
||||
for set_name, entry in existing_by_name.items():
|
||||
if set_name not in matched_names:
|
||||
updated_bios.append(entry)
|
||||
|
||||
profile['files'] = non_bios + updated_bios
|
||||
|
||||
if write:
|
||||
_backup_and_write(profile_path, profile)
|
||||
|
||||
return profile
|
||||
|
||||
|
||||
def merge_fbneo_profile(
|
||||
profile_path: str,
|
||||
hashes_path: str,
|
||||
write: bool = False,
|
||||
add_new: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
"""Merge FBNeo individual ROM entries from upstream hash data.
|
||||
|
||||
Preserves system, required per entry. Updates crc32, size, and
|
||||
source_ref. New ROMs are only added when add_new=True (main profile).
|
||||
Entries not in the hash data are left untouched.
|
||||
|
||||
If write=True, backs up existing profile to .old.yml before writing.
|
||||
"""
|
||||
profile = _load_yaml(profile_path)
|
||||
hashes = _load_json(hashes_path)
|
||||
|
||||
profile['core_version'] = hashes.get('version', profile.get('core_version'))
|
||||
|
||||
files = profile.get('files', [])
|
||||
archive_files, non_archive = _split_files(files, lambda f: 'archive' in f)
|
||||
|
||||
existing_by_key: dict[tuple[str, str], dict] = {}
|
||||
for entry in archive_files:
|
||||
key = (entry['archive'], entry['name'])
|
||||
existing_by_key[key] = entry
|
||||
|
||||
merged: list[dict] = []
|
||||
matched_keys: set[tuple[str, str]] = set()
|
||||
|
||||
for set_name, set_data in hashes.get('bios_sets', {}).items():
|
||||
archive_name = f'{set_name}.zip'
|
||||
source_ref = _build_source_ref(set_data)
|
||||
|
||||
for rom in set_data.get('roms', []):
|
||||
rom_name = rom['name']
|
||||
key = (archive_name, rom_name)
|
||||
|
||||
if key in existing_by_key:
|
||||
entry = existing_by_key[key].copy()
|
||||
entry['size'] = rom['size']
|
||||
entry['crc32'] = rom['crc32']
|
||||
if rom.get('sha1'):
|
||||
entry['sha1'] = rom['sha1']
|
||||
if source_ref:
|
||||
entry['source_ref'] = source_ref
|
||||
merged.append(entry)
|
||||
matched_keys.add(key)
|
||||
elif add_new:
|
||||
entry = {
|
||||
'name': rom_name,
|
||||
'archive': archive_name,
|
||||
'required': True,
|
||||
'size': rom['size'],
|
||||
'crc32': rom['crc32'],
|
||||
}
|
||||
if rom.get('sha1'):
|
||||
entry['sha1'] = rom['sha1']
|
||||
if source_ref:
|
||||
entry['source_ref'] = source_ref
|
||||
merged.append(entry)
|
||||
|
||||
# Entries not matched stay untouched
|
||||
for key, entry in existing_by_key.items():
|
||||
if key not in matched_keys:
|
||||
merged.append(entry)
|
||||
|
||||
profile['files'] = non_archive + merged
|
||||
|
||||
if write:
|
||||
_backup_and_write_fbneo(profile_path, profile, hashes)
|
||||
|
||||
return profile
|
||||
|
||||
|
||||
def compute_diff(
|
||||
profile_path: str,
|
||||
hashes_path: str,
|
||||
mode: str = 'mame',
|
||||
) -> dict[str, Any]:
|
||||
"""Compute diff between profile and hashes without writing.
|
||||
|
||||
Returns counts of added, updated, removed, and unchanged entries.
|
||||
"""
|
||||
profile = _load_yaml(profile_path)
|
||||
hashes = _load_json(hashes_path)
|
||||
|
||||
if mode == 'mame':
|
||||
return _diff_mame(profile, hashes)
|
||||
return _diff_fbneo(profile, hashes)
|
||||
|
||||
|
||||
def _diff_mame(
|
||||
profile: dict[str, Any],
|
||||
hashes: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
files = profile.get('files', [])
|
||||
bios_zip, _ = _split_files(files, lambda f: f.get('category') == 'bios_zip')
|
||||
|
||||
existing_by_name: dict[str, dict] = {}
|
||||
for entry in bios_zip:
|
||||
existing_by_name[_zip_name_to_set(entry['name'])] = entry
|
||||
|
||||
added: list[str] = []
|
||||
updated: list[str] = []
|
||||
unchanged = 0
|
||||
|
||||
bios_sets = hashes.get('bios_sets', {})
|
||||
for set_name, set_data in bios_sets.items():
|
||||
if set_name not in existing_by_name:
|
||||
added.append(set_name)
|
||||
continue
|
||||
|
||||
old_entry = existing_by_name[set_name]
|
||||
new_contents = _build_contents(set_data.get('roms', []))
|
||||
old_contents = old_entry.get('contents', [])
|
||||
|
||||
if _contents_differ(old_contents, new_contents):
|
||||
updated.append(set_name)
|
||||
else:
|
||||
unchanged += 1
|
||||
|
||||
# Items in profile but not in scraper output = out of scope (not removed)
|
||||
out_of_scope = len(existing_by_name) - sum(
|
||||
1 for s in existing_by_name if s in bios_sets
|
||||
)
|
||||
|
||||
return {
|
||||
'added': added,
|
||||
'updated': updated,
|
||||
'removed': [],
|
||||
'unchanged': unchanged,
|
||||
'out_of_scope': out_of_scope,
|
||||
}
|
||||
|
||||
|
||||
def _diff_fbneo(
|
||||
profile: dict[str, Any],
|
||||
hashes: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
files = profile.get('files', [])
|
||||
archive_files, _ = _split_files(files, lambda f: 'archive' in f)
|
||||
|
||||
existing_by_key: dict[tuple[str, str], dict] = {}
|
||||
for entry in archive_files:
|
||||
existing_by_key[(entry['archive'], entry['name'])] = entry
|
||||
|
||||
added: list[str] = []
|
||||
updated: list[str] = []
|
||||
unchanged = 0
|
||||
|
||||
seen_keys: set[tuple[str, str]] = set()
|
||||
bios_sets = hashes.get('bios_sets', {})
|
||||
|
||||
for set_name, set_data in bios_sets.items():
|
||||
archive_name = f'{set_name}.zip'
|
||||
for rom in set_data.get('roms', []):
|
||||
key = (archive_name, rom['name'])
|
||||
seen_keys.add(key)
|
||||
label = f"{archive_name}:{rom['name']}"
|
||||
|
||||
if key not in existing_by_key:
|
||||
added.append(label)
|
||||
continue
|
||||
|
||||
old = existing_by_key[key]
|
||||
if old.get('crc32') != rom.get('crc32') or old.get('size') != rom.get('size'):
|
||||
updated.append(label)
|
||||
else:
|
||||
unchanged += 1
|
||||
|
||||
out_of_scope = sum(1 for k in existing_by_key if k not in seen_keys)
|
||||
|
||||
return {
|
||||
'added': added,
|
||||
'updated': updated,
|
||||
'removed': [],
|
||||
'unchanged': unchanged,
|
||||
'out_of_scope': out_of_scope,
|
||||
}
|
||||
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _load_yaml(path: str) -> dict[str, Any]:
|
||||
with open(path, encoding='utf-8') as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
|
||||
|
||||
def _load_json(path: str) -> dict[str, Any]:
|
||||
with open(path, encoding='utf-8') as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def _split_files(
|
||||
files: list[dict],
|
||||
predicate: Any,
|
||||
) -> tuple[list[dict], list[dict]]:
|
||||
matching: list[dict] = []
|
||||
rest: list[dict] = []
|
||||
for f in files:
|
||||
if predicate(f):
|
||||
matching.append(f)
|
||||
else:
|
||||
rest.append(f)
|
||||
return matching, rest
|
||||
|
||||
|
||||
def _zip_name_to_set(name: str) -> str:
|
||||
if name.endswith('.zip'):
|
||||
return name[:-4]
|
||||
return name
|
||||
|
||||
|
||||
def _build_contents(roms: list[dict]) -> list[dict]:
|
||||
contents: list[dict] = []
|
||||
for rom in roms:
|
||||
entry: dict[str, Any] = {
|
||||
'name': rom['name'],
|
||||
'size': rom['size'],
|
||||
'crc32': rom['crc32'],
|
||||
}
|
||||
if rom.get('sha1'):
|
||||
entry['sha1'] = rom['sha1']
|
||||
desc = rom.get('bios_description') or rom.get('bios_label') or ''
|
||||
if desc:
|
||||
entry['description'] = desc
|
||||
if rom.get('bad_dump'):
|
||||
entry['bad_dump'] = True
|
||||
contents.append(entry)
|
||||
return contents
|
||||
|
||||
|
||||
def _build_source_ref(set_data: dict) -> str:
|
||||
source_file = set_data.get('source_file', '')
|
||||
source_line = set_data.get('source_line')
|
||||
if source_file and source_line is not None:
|
||||
return f'{source_file}:{source_line}'
|
||||
return source_file
|
||||
|
||||
|
||||
def _contents_differ(old: list[dict], new: list[dict]) -> bool:
|
||||
if len(old) != len(new):
|
||||
return True
|
||||
old_by_name = {c['name']: c for c in old}
|
||||
for entry in new:
|
||||
prev = old_by_name.get(entry['name'])
|
||||
if prev is None:
|
||||
return True
|
||||
if prev.get('crc32') != entry.get('crc32'):
|
||||
return True
|
||||
if prev.get('size') != entry.get('size'):
|
||||
return True
|
||||
if prev.get('sha1') != entry.get('sha1'):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _backup_and_write(path: str, data: dict) -> None:
|
||||
"""Write merged profile using text-based patching to preserve formatting.
|
||||
|
||||
Instead of yaml.dump (which destroys comments, quoting, indentation),
|
||||
this reads the original file as text, patches specific fields
|
||||
(core_version, contents, source_ref), and appends new entries.
|
||||
"""
|
||||
p = Path(path)
|
||||
backup = p.with_suffix('.old.yml')
|
||||
shutil.copy2(p, backup)
|
||||
|
||||
original = p.read_text(encoding='utf-8')
|
||||
patched = _patch_core_version(original, data.get('core_version', ''))
|
||||
patched = _patch_bios_entries(patched, data.get('files', []))
|
||||
patched = _append_new_entries(patched, data.get('files', []), original)
|
||||
|
||||
p.write_text(patched, encoding='utf-8')
|
||||
|
||||
|
||||
def _patch_core_version(text: str, version: str) -> str:
|
||||
"""Replace core_version value in-place."""
|
||||
if not version:
|
||||
return text
|
||||
import re
|
||||
return re.sub(
|
||||
r'^(core_version:\s*).*$',
|
||||
rf'\g<1>"{version}"',
|
||||
text,
|
||||
count=1,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
|
||||
|
||||
def _patch_bios_entries(text: str, files: list[dict]) -> str:
|
||||
"""Patch contents and source_ref for existing bios_zip entries in-place.
|
||||
|
||||
Processes entries in reverse order to preserve line offsets.
|
||||
Each entry's "owned" lines are: the `- name:` line plus all indented
|
||||
lines that follow (4+ spaces), stopping at blank lines, comments,
|
||||
or the next `- name:`.
|
||||
"""
|
||||
import re
|
||||
|
||||
# Build a lookup of what to patch
|
||||
patches: dict[str, dict] = {}
|
||||
for fe in files:
|
||||
if fe.get('category') != 'bios_zip':
|
||||
continue
|
||||
patches[fe['name']] = fe
|
||||
|
||||
if not patches:
|
||||
return text
|
||||
|
||||
lines = text.split('\n')
|
||||
# Find all entry start positions (line indices)
|
||||
entry_starts: list[tuple[int, str]] = []
|
||||
for i, line in enumerate(lines):
|
||||
m = re.match(r'^ - name:\s*(.+?)\s*$', line)
|
||||
if m:
|
||||
entry_starts.append((i, m.group(1).strip('"').strip("'")))
|
||||
|
||||
# Process in reverse so line insertions don't shift indices
|
||||
for idx in range(len(entry_starts) - 1, -1, -1):
|
||||
start_line, entry_name = entry_starts[idx]
|
||||
if entry_name not in patches:
|
||||
continue
|
||||
|
||||
fe = patches[entry_name]
|
||||
contents = fe.get('contents', [])
|
||||
source_ref = fe.get('source_ref', '')
|
||||
|
||||
# Find the last "owned" line of this entry
|
||||
# Owned = indented with 4+ spaces (field lines of this entry)
|
||||
last_owned = start_line
|
||||
for j in range(start_line + 1, len(lines)):
|
||||
stripped = lines[j].strip()
|
||||
if not stripped:
|
||||
break # blank line = end of entry
|
||||
if stripped.startswith('#'):
|
||||
break # comment = belongs to next entry
|
||||
if re.match(r'^ - ', lines[j]):
|
||||
break # next list item
|
||||
if re.match(r'^ ', lines[j]) or re.match(r'^ \w', lines[j]):
|
||||
last_owned = j
|
||||
else:
|
||||
break
|
||||
|
||||
# Patch source_ref in-place
|
||||
if source_ref:
|
||||
found_sr = False
|
||||
for j in range(start_line + 1, last_owned + 1):
|
||||
if re.match(r'^ source_ref:', lines[j]):
|
||||
lines[j] = f' source_ref: "{source_ref}"'
|
||||
found_sr = True
|
||||
break
|
||||
if not found_sr:
|
||||
lines.insert(last_owned + 1, f' source_ref: "{source_ref}"')
|
||||
last_owned += 1
|
||||
|
||||
# Remove existing contents block if present
|
||||
contents_start = None
|
||||
contents_end = None
|
||||
for j in range(start_line + 1, last_owned + 1):
|
||||
if re.match(r'^ contents:', lines[j]):
|
||||
contents_start = j
|
||||
elif contents_start is not None:
|
||||
if re.match(r'^ ', lines[j]):
|
||||
contents_end = j
|
||||
else:
|
||||
break
|
||||
if contents_end is None and contents_start is not None:
|
||||
contents_end = contents_start
|
||||
|
||||
if contents_start is not None:
|
||||
del lines[contents_start:contents_end + 1]
|
||||
last_owned -= (contents_end - contents_start + 1)
|
||||
|
||||
# Insert new contents after last owned line
|
||||
if contents:
|
||||
new_lines = _format_contents(contents).split('\n')
|
||||
for k, cl in enumerate(new_lines):
|
||||
lines.insert(last_owned + 1 + k, cl)
|
||||
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
def _append_new_entries(text: str, files: list[dict], original: str) -> str:
|
||||
"""Append new bios_zip entries (system=None) that aren't in the original."""
|
||||
# Parse original to get existing entry names (more reliable than text search)
|
||||
existing_data = yaml.safe_load(original) or {}
|
||||
existing_names = {f['name'] for f in existing_data.get('files', [])}
|
||||
|
||||
new_entries = []
|
||||
for fe in files:
|
||||
if fe.get('category') != 'bios_zip' or fe.get('system') is not None:
|
||||
continue
|
||||
if fe['name'] in existing_names:
|
||||
continue
|
||||
new_entries.append(fe)
|
||||
|
||||
if not new_entries:
|
||||
return text
|
||||
|
||||
lines = []
|
||||
for fe in new_entries:
|
||||
lines.append(f'\n - name: {fe["name"]}')
|
||||
lines.append(f' required: {str(fe["required"]).lower()}')
|
||||
lines.append(f' category: bios_zip')
|
||||
if fe.get('source_ref'):
|
||||
lines.append(f' source_ref: "{fe["source_ref"]}"')
|
||||
if fe.get('contents'):
|
||||
lines.append(_format_contents(fe['contents']))
|
||||
|
||||
if lines:
|
||||
text = text.rstrip('\n') + '\n' + '\n'.join(lines) + '\n'
|
||||
|
||||
return text
|
||||
|
||||
|
||||
def _format_contents(contents: list[dict]) -> str:
|
||||
"""Format a contents list as YAML text."""
|
||||
lines = [' contents:']
|
||||
for rom in contents:
|
||||
lines.append(f' - name: {rom["name"]}')
|
||||
if rom.get('description'):
|
||||
lines.append(f' description: {rom["description"]}')
|
||||
if rom.get('size'):
|
||||
lines.append(f' size: {rom["size"]}')
|
||||
if rom.get('crc32'):
|
||||
lines.append(f' crc32: "{rom["crc32"]}"')
|
||||
if rom.get('sha1'):
|
||||
lines.append(f' sha1: "{rom["sha1"]}"')
|
||||
if rom.get('bad_dump'):
|
||||
lines.append(f' bad_dump: true')
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
def _backup_and_write_fbneo(path: str, data: dict, hashes: dict) -> None:
|
||||
"""Write merged FBNeo profile using text-based patching.
|
||||
|
||||
FBNeo profiles have individual ROM entries with archive: field.
|
||||
Only patches core_version and appends new ROM entries.
|
||||
Existing entries are left untouched (CRC32 changes are rare).
|
||||
"""
|
||||
p = Path(path)
|
||||
backup = p.with_suffix('.old.yml')
|
||||
shutil.copy2(p, backup)
|
||||
|
||||
original = p.read_text(encoding='utf-8')
|
||||
patched = _patch_core_version(original, data.get('core_version', ''))
|
||||
|
||||
# Identify new ROM entries by comparing parsed data keys, not text search
|
||||
existing_data = yaml.safe_load(original) or {}
|
||||
existing_keys = {
|
||||
(f['archive'], f['name'])
|
||||
for f in existing_data.get('files', [])
|
||||
if f.get('archive')
|
||||
}
|
||||
new_roms = [
|
||||
f for f in data.get('files', [])
|
||||
if f.get('archive') and (f['archive'], f['name']) not in existing_keys
|
||||
]
|
||||
|
||||
if new_roms:
|
||||
lines = []
|
||||
for fe in new_roms:
|
||||
lines.append(f' - name: "{fe["name"]}"')
|
||||
lines.append(f' archive: {fe["archive"]}')
|
||||
lines.append(f' required: {str(fe.get("required", True)).lower()}')
|
||||
if fe.get('size'):
|
||||
lines.append(f' size: {fe["size"]}')
|
||||
if fe.get('crc32'):
|
||||
lines.append(f' crc32: "{fe["crc32"]}"')
|
||||
if fe.get('source_ref'):
|
||||
lines.append(f' source_ref: "{fe["source_ref"]}"')
|
||||
lines.append('')
|
||||
patched = patched.rstrip('\n') + '\n\n' + '\n'.join(lines)
|
||||
|
||||
p.write_text(patched, encoding='utf-8')
|
||||
@@ -8,6 +8,7 @@ import urllib.request
|
||||
import urllib.error
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -231,7 +232,28 @@ def scraper_cli(scraper_class: type, description: str = "Scrape BIOS requirement
|
||||
if req.zipped_file:
|
||||
entry["zipped_file"] = req.zipped_file
|
||||
config["systems"][sys_id]["files"].append(entry)
|
||||
with open(args.output, "w") as f:
|
||||
# Merge into existing YAML: preserve fields the scraper doesn't generate
|
||||
# (data_directories, case_insensitive_fs, manually added metadata).
|
||||
# The scraper replaces systems + files; everything else is preserved.
|
||||
output_path = Path(args.output)
|
||||
if output_path.exists():
|
||||
with open(output_path) as f:
|
||||
existing = yaml.safe_load(f) or {}
|
||||
# Preserve existing keys not generated by the scraper.
|
||||
# Only keys present in the NEW config are considered scraper-generated.
|
||||
# Everything else in the existing file is preserved.
|
||||
for key, val in existing.items():
|
||||
if key not in config:
|
||||
config[key] = val
|
||||
# Preserve per-system fields not generated by the scraper
|
||||
# (data_directories, native_id from manual additions, etc.)
|
||||
existing_systems = existing.get("systems", {})
|
||||
for sys_id, sys_data in config.get("systems", {}).items():
|
||||
old_sys = existing_systems.get(sys_id, {})
|
||||
for field in ("data_directories",):
|
||||
if field in old_sys and field not in sys_data:
|
||||
sys_data[field] = old_sys[field]
|
||||
with open(output_path, "w") as f:
|
||||
yaml.dump(config, f, default_flow_style=False, sort_keys=False)
|
||||
print(f"Written {len(reqs)} entries to {args.output}")
|
||||
return
|
||||
|
||||
@@ -92,6 +92,27 @@ SYSTEM_SLUG_MAP = {
|
||||
"dos": "dos",
|
||||
"videopac": "philips-videopac",
|
||||
"pokemini": "nintendo-pokemon-mini",
|
||||
"gsplus": "apple-iigs",
|
||||
"apple2": "apple-ii",
|
||||
"apple2gs": "apple-iigs",
|
||||
"ps3": "sony-playstation-3",
|
||||
"psvita": "sony-playstation-vita",
|
||||
"coco": "coco",
|
||||
"dragon32": "dragon32",
|
||||
"dragon64": "dragon64",
|
||||
"mc10": "mc10",
|
||||
"msx2+": "microsoft-msx",
|
||||
"msxturbor": "microsoft-msx",
|
||||
"spectravideo": "spectravideo",
|
||||
"tvc": "videoton-tvc",
|
||||
"enterprise": "enterprise-64-128",
|
||||
"vis": "tandy-vis",
|
||||
"supracan": "supracan",
|
||||
"jaguar": "atari-jaguar",
|
||||
"jaguarcd": "atari-jaguar",
|
||||
"switch": "nintendo-switch",
|
||||
"wii": "nintendo-wii",
|
||||
"xbox360": "microsoft-xbox-360",
|
||||
}
|
||||
|
||||
|
||||
@@ -282,12 +303,25 @@ class Scraper(BaseScraper):
|
||||
"""Generate a platform YAML config dict from scraped data."""
|
||||
requirements = self.fetch_requirements()
|
||||
|
||||
# Parse source to extract display names per system
|
||||
raw = self._fetch_raw()
|
||||
source_dict = self._extract_systems_dict(raw)
|
||||
display_names: dict[str, str] = {}
|
||||
for sys_key, sys_data in source_dict.items():
|
||||
dname = sys_data.get("name", "")
|
||||
if dname:
|
||||
slug = SYSTEM_SLUG_MAP.get(sys_key, sys_key)
|
||||
display_names[slug] = dname
|
||||
|
||||
systems = {}
|
||||
for req in requirements:
|
||||
if req.system not in systems:
|
||||
sys_entry: dict = {"files": []}
|
||||
if req.native_id:
|
||||
sys_entry["native_id"] = req.native_id
|
||||
dname = display_names.get(req.system)
|
||||
if dname:
|
||||
sys_entry["name"] = dname
|
||||
systems[req.system] = sys_entry
|
||||
|
||||
entry = {
|
||||
@@ -308,6 +342,13 @@ class Scraper(BaseScraper):
|
||||
num = tag.removeprefix("batocera-")
|
||||
if num.isdigit():
|
||||
batocera_version = num
|
||||
if not batocera_version:
|
||||
# Preserve existing version when fetch fails (offline mode)
|
||||
existing = Path(__file__).resolve().parents[2] / "platforms" / "batocera.yml"
|
||||
if existing.exists():
|
||||
with open(existing) as f:
|
||||
old = yaml.safe_load(f) or {}
|
||||
batocera_version = str(old.get("version", ""))
|
||||
|
||||
cores, standalone = self._fetch_cores()
|
||||
result = {
|
||||
|
||||
315
scripts/scraper/fbneo_hash_scraper.py
Normal file
315
scripts/scraper/fbneo_hash_scraper.py
Normal file
@@ -0,0 +1,315 @@
|
||||
"""Scrape FBNeo BIOS set hashes from upstream source via sparse clone.
|
||||
|
||||
Does NOT inherit BaseScraper (uses git sparse clone, not URL fetch).
|
||||
Parses BDF_BOARDROM drivers from src/burn/drv/ to extract CRC32/size
|
||||
for all BIOS ROM sets, then optionally merges into emulator profiles.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from scripts.scraper.fbneo_parser import parse_fbneo_source_tree
|
||||
from scripts.scraper._hash_merge import compute_diff, merge_fbneo_profile
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
REPO_URL = 'https://github.com/finalburnneo/FBNeo.git'
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
CLONE_DIR = REPO_ROOT / 'tmp' / 'fbneo'
|
||||
CACHE_PATH = REPO_ROOT / 'data' / 'fbneo-hashes.json'
|
||||
EMULATORS_DIR = REPO_ROOT / 'emulators'
|
||||
STALE_HOURS = 24
|
||||
|
||||
|
||||
def _is_cache_fresh() -> bool:
|
||||
"""Check if the JSON cache exists and is less than 24 hours old."""
|
||||
if not CACHE_PATH.exists():
|
||||
return False
|
||||
try:
|
||||
data = json.loads(CACHE_PATH.read_text(encoding='utf-8'))
|
||||
fetched_at = datetime.fromisoformat(data['fetched_at'])
|
||||
return datetime.now(timezone.utc) - fetched_at < timedelta(hours=STALE_HOURS)
|
||||
except (json.JSONDecodeError, KeyError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
def _sparse_clone() -> None:
|
||||
"""Sparse clone FBNeo repo, checking out only src/burn/drv."""
|
||||
if CLONE_DIR.exists():
|
||||
shutil.rmtree(CLONE_DIR)
|
||||
|
||||
CLONE_DIR.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
subprocess.run(
|
||||
[
|
||||
'git', 'clone', '--depth', '1', '--filter=blob:none',
|
||||
'--sparse', REPO_URL, str(CLONE_DIR),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
subprocess.run(
|
||||
['git', 'sparse-checkout', 'set', 'src/burn/drv', 'src/burner/resource.h'],
|
||||
cwd=CLONE_DIR,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
|
||||
def _extract_version() -> tuple[str, str]:
|
||||
"""Extract version tag and commit SHA from the cloned repo.
|
||||
|
||||
Returns (version, commit_sha). Falls back to resource.h if no tag.
|
||||
"""
|
||||
result = subprocess.run(
|
||||
['git', 'describe', '--tags', '--abbrev=0'],
|
||||
cwd=CLONE_DIR,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
# Prefer real version tags over pseudo-tags like "latest"
|
||||
version = 'unknown'
|
||||
if result.returncode == 0:
|
||||
tag = result.stdout.strip()
|
||||
if tag and tag != 'latest':
|
||||
version = tag
|
||||
# Fallback: resource.h
|
||||
if version == 'unknown':
|
||||
version = _version_from_resource_h()
|
||||
# Last resort: use GitHub API for latest real release tag
|
||||
if version == 'unknown':
|
||||
try:
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
req = urllib.request.Request(
|
||||
'https://api.github.com/repos/finalburnneo/FBNeo/tags?per_page=10',
|
||||
headers={'User-Agent': 'retrobios-scraper/1.0'},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
import json as json_mod
|
||||
tags = json_mod.loads(resp.read())
|
||||
for t in tags:
|
||||
if t['name'] != 'latest' and t['name'].startswith('v'):
|
||||
version = t['name']
|
||||
break
|
||||
except (urllib.error.URLError, OSError):
|
||||
pass
|
||||
|
||||
sha_result = subprocess.run(
|
||||
['git', 'rev-parse', 'HEAD'],
|
||||
cwd=CLONE_DIR,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
commit = sha_result.stdout.strip()
|
||||
|
||||
return version, commit
|
||||
|
||||
|
||||
def _version_from_resource_h() -> str:
|
||||
"""Fallback: parse VER_FULL_VERSION_STR from resource.h."""
|
||||
resource_h = CLONE_DIR / 'src' / 'burner' / 'resource.h'
|
||||
if not resource_h.exists():
|
||||
return 'unknown'
|
||||
|
||||
text = resource_h.read_text(encoding='utf-8', errors='replace')
|
||||
for line in text.splitlines():
|
||||
if 'VER_FULL_VERSION_STR' in line:
|
||||
parts = line.split('"')
|
||||
if len(parts) >= 2:
|
||||
return parts[1]
|
||||
return 'unknown'
|
||||
|
||||
|
||||
def _cleanup() -> None:
|
||||
"""Remove the sparse clone directory."""
|
||||
if CLONE_DIR.exists():
|
||||
shutil.rmtree(CLONE_DIR)
|
||||
|
||||
|
||||
def fetch_and_cache(force: bool = False) -> dict[str, Any]:
|
||||
"""Clone, parse, and write JSON cache. Returns the cache dict."""
|
||||
if not force and _is_cache_fresh():
|
||||
log.info('cache fresh, skipping clone (use --force to override)')
|
||||
return json.loads(CACHE_PATH.read_text(encoding='utf-8'))
|
||||
|
||||
try:
|
||||
log.info('sparse cloning %s', REPO_URL)
|
||||
_sparse_clone()
|
||||
|
||||
log.info('extracting version')
|
||||
version, commit = _extract_version()
|
||||
|
||||
log.info('parsing source tree')
|
||||
bios_sets = parse_fbneo_source_tree(str(CLONE_DIR))
|
||||
|
||||
cache: dict[str, Any] = {
|
||||
'source': 'finalburnneo/FBNeo',
|
||||
'version': version,
|
||||
'commit': commit,
|
||||
'fetched_at': datetime.now(timezone.utc).isoformat(),
|
||||
'bios_sets': bios_sets,
|
||||
}
|
||||
|
||||
CACHE_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
CACHE_PATH.write_text(
|
||||
json.dumps(cache, indent=2, ensure_ascii=False) + '\n',
|
||||
encoding='utf-8',
|
||||
)
|
||||
log.info('wrote %d BIOS sets to %s', len(bios_sets), CACHE_PATH)
|
||||
|
||||
return cache
|
||||
finally:
|
||||
_cleanup()
|
||||
|
||||
|
||||
def _find_fbneo_profiles() -> list[Path]:
|
||||
"""Find emulator profiles whose upstream references finalburnneo/FBNeo."""
|
||||
profiles: list[Path] = []
|
||||
for path in sorted(EMULATORS_DIR.glob('*.yml')):
|
||||
if path.name.endswith('.old.yml'):
|
||||
continue
|
||||
try:
|
||||
data = yaml.safe_load(path.read_text(encoding='utf-8'))
|
||||
except (yaml.YAMLError, OSError):
|
||||
continue
|
||||
if not data or not isinstance(data, dict):
|
||||
continue
|
||||
upstream = data.get('upstream', '')
|
||||
if isinstance(upstream, str) and 'finalburnneo/fbneo' in upstream.lower():
|
||||
profiles.append(path)
|
||||
return profiles
|
||||
|
||||
|
||||
def _format_diff(profile_name: str, diff: dict[str, Any], show_added: bool = True) -> str:
|
||||
"""Format diff for a single profile."""
|
||||
lines: list[str] = []
|
||||
lines.append(f' {profile_name}:')
|
||||
|
||||
added = diff.get('added', [])
|
||||
updated = diff.get('updated', [])
|
||||
oos = diff.get('out_of_scope', 0)
|
||||
|
||||
if not added and not updated:
|
||||
lines.append(' no changes')
|
||||
if oos:
|
||||
lines.append(f' . {oos} out of scope')
|
||||
return '\n'.join(lines)
|
||||
|
||||
if show_added:
|
||||
for label in added:
|
||||
lines.append(f' + {label}')
|
||||
elif added:
|
||||
lines.append(f' + {len(added)} new ROMs available (main profile only)')
|
||||
for label in updated:
|
||||
lines.append(f' ~ {label}')
|
||||
lines.append(f' = {diff["unchanged"]} unchanged')
|
||||
if oos:
|
||||
lines.append(f' . {oos} out of scope')
|
||||
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
def run(
|
||||
dry_run: bool = False,
|
||||
force: bool = False,
|
||||
json_output: bool = False,
|
||||
) -> int:
|
||||
"""Main entry point for the scraper."""
|
||||
cache = fetch_and_cache(force=force)
|
||||
|
||||
version = cache.get('version', 'unknown')
|
||||
commit = cache.get('commit', '?')[:12]
|
||||
bios_sets = cache.get('bios_sets', {})
|
||||
profiles = _find_fbneo_profiles()
|
||||
|
||||
if json_output:
|
||||
result: dict[str, Any] = {
|
||||
'source': cache.get('source'),
|
||||
'version': version,
|
||||
'commit': cache.get('commit'),
|
||||
'bios_set_count': len(bios_sets),
|
||||
'profiles': {},
|
||||
}
|
||||
for path in profiles:
|
||||
diff = compute_diff(str(path), str(CACHE_PATH), mode='fbneo')
|
||||
result['profiles'][path.stem] = diff
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
|
||||
header = (
|
||||
f'fbneo-hashes: {len(bios_sets)} BIOS sets '
|
||||
f'from finalburnneo/FBNeo @ {version} ({commit})'
|
||||
)
|
||||
print(header)
|
||||
print()
|
||||
|
||||
if not profiles:
|
||||
print(' no matching emulator profiles found')
|
||||
return 0
|
||||
|
||||
for path in profiles:
|
||||
is_main = path.name == 'fbneo.yml'
|
||||
diff = compute_diff(str(path), str(CACHE_PATH), mode='fbneo')
|
||||
print(_format_diff(path.stem, diff, show_added=is_main))
|
||||
|
||||
effective_added = diff['added'] if is_main else []
|
||||
if not dry_run and (effective_added or diff['updated']):
|
||||
merge_fbneo_profile(str(path), str(CACHE_PATH), write=True, add_new=is_main)
|
||||
log.info('merged changes into %s', path.name)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Scrape FBNeo BIOS set hashes from upstream source',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--dry-run',
|
||||
action='store_true',
|
||||
help='show diff without writing changes',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--force',
|
||||
action='store_true',
|
||||
help='force re-clone even if cache is fresh',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--json',
|
||||
action='store_true',
|
||||
dest='json_output',
|
||||
help='output diff as JSON',
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(name)s: %(message)s',
|
||||
)
|
||||
|
||||
sys.exit(run(
|
||||
dry_run=args.dry_run,
|
||||
force=args.force,
|
||||
json_output=args.json_output,
|
||||
))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
127
scripts/scraper/fbneo_parser.py
Normal file
127
scripts/scraper/fbneo_parser.py
Normal file
@@ -0,0 +1,127 @@
|
||||
"""Parser for FBNeo source files to extract BIOS sets and ROM definitions.
|
||||
|
||||
Parses BurnRomInfo structs (static ROM arrays) and BurnDriver structs
|
||||
(driver registration) from FBNeo C source files. BIOS sets are identified
|
||||
by the BDF_BOARDROM flag in BurnDriver definitions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
_ROM_ENTRY_RE = re.compile(
|
||||
r'\{\s*"([^"]+)"\s*,\s*(0x[\da-fA-F]+)\s*,\s*(0x[\da-fA-F]+)\s*,\s*([^}]+)\}',
|
||||
)
|
||||
|
||||
_BURN_DRIVER_RE = re.compile(
|
||||
r'struct\s+BurnDriver\s+BurnDrv(\w+)\s*=\s*\{(.*?)\};',
|
||||
re.DOTALL,
|
||||
)
|
||||
|
||||
_ROM_DESC_RE = re.compile(
|
||||
r'static\s+struct\s+BurnRomInfo\s+(\w+)RomDesc\s*\[\s*\]\s*=\s*\{(.*?)\};',
|
||||
re.DOTALL,
|
||||
)
|
||||
|
||||
|
||||
def find_bios_sets(source: str, filename: str) -> dict[str, dict]:
|
||||
"""Find BDF_BOARDROM drivers in source code.
|
||||
|
||||
Returns a dict mapping set name to metadata:
|
||||
{set_name: {"source_file": str, "source_line": int}}
|
||||
"""
|
||||
results: dict[str, dict] = {}
|
||||
|
||||
for match in _BURN_DRIVER_RE.finditer(source):
|
||||
body = match.group(2)
|
||||
if 'BDF_BOARDROM' not in body:
|
||||
continue
|
||||
|
||||
# Set name is the first quoted string in the struct body
|
||||
name_match = re.search(r'"([^"]+)"', body)
|
||||
if not name_match:
|
||||
continue
|
||||
|
||||
set_name = name_match.group(1)
|
||||
line_num = source[:match.start()].count('\n') + 1
|
||||
|
||||
results[set_name] = {
|
||||
'source_file': filename,
|
||||
'source_line': line_num,
|
||||
}
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def parse_rom_info(source: str, set_name: str) -> list[dict]:
|
||||
"""Parse a BurnRomInfo array for the given set name.
|
||||
|
||||
Returns a list of dicts with keys: name, size, crc32.
|
||||
Sentinel entries (empty name) are skipped.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'static\s+struct\s+BurnRomInfo\s+'
|
||||
+ re.escape(set_name)
|
||||
+ r'RomDesc\s*\[\s*\]\s*=\s*\{(.*?)\};',
|
||||
re.DOTALL,
|
||||
)
|
||||
match = pattern.search(source)
|
||||
if not match:
|
||||
return []
|
||||
|
||||
body = match.group(1)
|
||||
roms: list[dict] = []
|
||||
|
||||
for entry in _ROM_ENTRY_RE.finditer(body):
|
||||
name = entry.group(1)
|
||||
if not name:
|
||||
continue
|
||||
size = int(entry.group(2), 16)
|
||||
crc32 = format(int(entry.group(3), 16), '08x')
|
||||
|
||||
roms.append({
|
||||
'name': name,
|
||||
'size': size,
|
||||
'crc32': crc32,
|
||||
})
|
||||
|
||||
return roms
|
||||
|
||||
|
||||
def parse_fbneo_source_tree(base_path: str) -> dict[str, dict]:
|
||||
"""Walk the FBNeo driver source tree and extract all BIOS sets.
|
||||
|
||||
Scans .cpp files under src/burn/drv/ for BDF_BOARDROM drivers,
|
||||
then parses their associated BurnRomInfo arrays.
|
||||
|
||||
Returns a dict mapping set name to:
|
||||
{source_file, source_line, roms: [{name, size, crc32}, ...]}
|
||||
"""
|
||||
drv_path = Path(base_path) / 'src' / 'burn' / 'drv'
|
||||
if not drv_path.is_dir():
|
||||
return {}
|
||||
|
||||
results: dict[str, dict] = {}
|
||||
|
||||
for root, _dirs, files in os.walk(drv_path):
|
||||
for fname in files:
|
||||
if not fname.endswith('.cpp'):
|
||||
continue
|
||||
|
||||
filepath = Path(root) / fname
|
||||
source = filepath.read_text(encoding='utf-8', errors='replace')
|
||||
rel_path = str(filepath.relative_to(base_path))
|
||||
|
||||
bios_sets = find_bios_sets(source, rel_path)
|
||||
for set_name, meta in bios_sets.items():
|
||||
roms = parse_rom_info(source, set_name)
|
||||
results[set_name] = {
|
||||
'source_file': meta['source_file'],
|
||||
'source_line': meta['source_line'],
|
||||
'roms': roms,
|
||||
}
|
||||
|
||||
return results
|
||||
322
scripts/scraper/mame_hash_scraper.py
Normal file
322
scripts/scraper/mame_hash_scraper.py
Normal file
@@ -0,0 +1,322 @@
|
||||
"""Fetch MAME BIOS hashes from mamedev/mame source and merge into profiles.
|
||||
|
||||
Sparse clones the MAME repo, parses the source tree for BIOS root sets,
|
||||
caches results to data/mame-hashes.json, and optionally merges into
|
||||
emulator profiles that reference mamedev/mame upstream.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from .mame_parser import parse_mame_source_tree
|
||||
from ._hash_merge import compute_diff, merge_mame_profile
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_ROOT = Path(__file__).resolve().parents[2]
|
||||
_CACHE_PATH = _ROOT / 'data' / 'mame-hashes.json'
|
||||
_CLONE_DIR = _ROOT / 'tmp' / 'mame'
|
||||
_EMULATORS_DIR = _ROOT / 'emulators'
|
||||
_REPO_URL = 'https://github.com/mamedev/mame.git'
|
||||
_STALE_HOURS = 24
|
||||
|
||||
|
||||
# ── Cache ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _load_cache() -> dict[str, Any] | None:
|
||||
if not _CACHE_PATH.exists():
|
||||
return None
|
||||
try:
|
||||
with open(_CACHE_PATH, encoding='utf-8') as f:
|
||||
return json.load(f)
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return None
|
||||
|
||||
|
||||
def _is_stale(cache: dict[str, Any] | None) -> bool:
|
||||
if cache is None:
|
||||
return True
|
||||
fetched_at = cache.get('fetched_at')
|
||||
if not fetched_at:
|
||||
return True
|
||||
try:
|
||||
ts = datetime.fromisoformat(fetched_at)
|
||||
age = datetime.now(timezone.utc) - ts
|
||||
return age.total_seconds() > _STALE_HOURS * 3600
|
||||
except (ValueError, TypeError):
|
||||
return True
|
||||
|
||||
|
||||
def _write_cache(data: dict[str, Any]) -> None:
|
||||
_CACHE_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(_CACHE_PATH, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
log.info('cache written to %s', _CACHE_PATH)
|
||||
|
||||
|
||||
# ── Git operations ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _run_git(args: list[str], cwd: Path | None = None) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
['git', *args],
|
||||
cwd=cwd,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
|
||||
def _sparse_clone() -> None:
|
||||
if _CLONE_DIR.exists():
|
||||
shutil.rmtree(_CLONE_DIR)
|
||||
_CLONE_DIR.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
log.info('sparse cloning mamedev/mame into %s', _CLONE_DIR)
|
||||
_run_git([
|
||||
'clone',
|
||||
'--depth', '1',
|
||||
'--filter=blob:none',
|
||||
'--sparse',
|
||||
_REPO_URL,
|
||||
str(_CLONE_DIR),
|
||||
])
|
||||
_run_git(
|
||||
['sparse-checkout', 'set', 'src/mame', 'src/devices'],
|
||||
cwd=_CLONE_DIR,
|
||||
)
|
||||
|
||||
|
||||
def _get_version() -> str:
|
||||
# version.cpp is generated at build time, not in the repo.
|
||||
# Use GitHub API to get the latest release tag.
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
'https://api.github.com/repos/mamedev/mame/releases/latest',
|
||||
headers={'User-Agent': 'retrobios-scraper/1.0',
|
||||
'Accept': 'application/vnd.github.v3+json'},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
data = json.loads(resp.read())
|
||||
tag = data.get('tag_name', '')
|
||||
if tag:
|
||||
return _parse_version_tag(tag)
|
||||
except (urllib.error.URLError, json.JSONDecodeError, OSError):
|
||||
pass
|
||||
return 'unknown'
|
||||
|
||||
|
||||
def _parse_version_tag(tag: str) -> str:
|
||||
prefix = 'mame'
|
||||
raw = tag.removeprefix(prefix) if tag.startswith(prefix) else tag
|
||||
if raw.isdigit() and len(raw) >= 4:
|
||||
return f'{raw[0]}.{raw[1:]}'
|
||||
return raw
|
||||
|
||||
|
||||
|
||||
|
||||
def _get_commit() -> str:
|
||||
try:
|
||||
result = _run_git(['rev-parse', 'HEAD'], cwd=_CLONE_DIR)
|
||||
return result.stdout.strip()
|
||||
except subprocess.CalledProcessError:
|
||||
return ''
|
||||
|
||||
|
||||
def _cleanup() -> None:
|
||||
if _CLONE_DIR.exists():
|
||||
log.info('cleaning up %s', _CLONE_DIR)
|
||||
shutil.rmtree(_CLONE_DIR)
|
||||
|
||||
|
||||
# ── Profile discovery ────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _find_mame_profiles() -> list[Path]:
|
||||
profiles: list[Path] = []
|
||||
for path in sorted(_EMULATORS_DIR.glob('*.yml')):
|
||||
if path.name.endswith('.old.yml'):
|
||||
continue
|
||||
try:
|
||||
with open(path, encoding='utf-8') as f:
|
||||
data = yaml.safe_load(f)
|
||||
if not isinstance(data, dict):
|
||||
continue
|
||||
upstream = data.get('upstream', '')
|
||||
# Only match profiles tracking current MAME (not frozen snapshots
|
||||
# which have upstream like "mamedev/mame/tree/mame0139")
|
||||
if isinstance(upstream, str) and upstream.rstrip('/') == 'https://github.com/mamedev/mame':
|
||||
profiles.append(path)
|
||||
except (yaml.YAMLError, OSError):
|
||||
continue
|
||||
return profiles
|
||||
|
||||
|
||||
# ── Diff formatting ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _format_diff(
|
||||
profile_path: Path,
|
||||
diff: dict[str, Any],
|
||||
hashes: dict[str, Any],
|
||||
show_added: bool = True,
|
||||
) -> list[str]:
|
||||
lines: list[str] = []
|
||||
name = profile_path.stem
|
||||
|
||||
added = diff.get('added', [])
|
||||
updated = diff.get('updated', [])
|
||||
removed = diff.get('removed', [])
|
||||
unchanged = diff.get('unchanged', 0)
|
||||
|
||||
if not added and not updated and not removed:
|
||||
lines.append(f' {name}:')
|
||||
lines.append(' no changes')
|
||||
return lines
|
||||
|
||||
lines.append(f' {name}:')
|
||||
|
||||
if show_added:
|
||||
bios_sets = hashes.get('bios_sets', {})
|
||||
for set_name in added:
|
||||
rom_count = len(bios_sets.get(set_name, {}).get('roms', []))
|
||||
source_file = bios_sets.get(set_name, {}).get('source_file', '')
|
||||
source_line = bios_sets.get(set_name, {}).get('source_line', '')
|
||||
ref = f'{source_file}:{source_line}' if source_file else ''
|
||||
lines.append(f' + {set_name}.zip ({ref}, {rom_count} ROMs)')
|
||||
elif added:
|
||||
lines.append(f' + {len(added)} new sets available (main profile only)')
|
||||
|
||||
for set_name in updated:
|
||||
lines.append(f' ~ {set_name}.zip (contents changed)')
|
||||
|
||||
oos = diff.get('out_of_scope', 0)
|
||||
lines.append(f' = {unchanged} unchanged')
|
||||
if oos:
|
||||
lines.append(f' . {oos} out of scope (not BIOS root sets)')
|
||||
return lines
|
||||
|
||||
|
||||
# ── Main ─────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _fetch_hashes(force: bool) -> dict[str, Any]:
|
||||
cache = _load_cache()
|
||||
if not force and not _is_stale(cache):
|
||||
log.info('using cached data from %s', cache.get('fetched_at', ''))
|
||||
return cache # type: ignore[return-value]
|
||||
|
||||
try:
|
||||
_sparse_clone()
|
||||
bios_sets = parse_mame_source_tree(str(_CLONE_DIR))
|
||||
version = _get_version()
|
||||
commit = _get_commit()
|
||||
|
||||
data: dict[str, Any] = {
|
||||
'source': 'mamedev/mame',
|
||||
'version': version,
|
||||
'commit': commit,
|
||||
'fetched_at': datetime.now(timezone.utc).isoformat(),
|
||||
'bios_sets': bios_sets,
|
||||
}
|
||||
_write_cache(data)
|
||||
return data
|
||||
finally:
|
||||
_cleanup()
|
||||
|
||||
|
||||
def _run(args: argparse.Namespace) -> None:
|
||||
hashes = _fetch_hashes(args.force)
|
||||
|
||||
total_sets = len(hashes.get('bios_sets', {}))
|
||||
version = hashes.get('version', 'unknown')
|
||||
commit = hashes.get('commit', '')[:12]
|
||||
|
||||
if args.json:
|
||||
json.dump(hashes, sys.stdout, indent=2, ensure_ascii=False)
|
||||
sys.stdout.write('\n')
|
||||
return
|
||||
|
||||
print(f'mame-hashes: {total_sets} BIOS root sets from mamedev/mame'
|
||||
f' @ {version} ({commit})')
|
||||
print()
|
||||
|
||||
profiles = _find_mame_profiles()
|
||||
if not profiles:
|
||||
print(' no profiles with mamedev/mame upstream found')
|
||||
return
|
||||
|
||||
for profile_path in profiles:
|
||||
is_main = profile_path.name == 'mame.yml'
|
||||
diff = compute_diff(str(profile_path), str(_CACHE_PATH), mode='mame')
|
||||
lines = _format_diff(profile_path, diff, hashes, show_added=is_main)
|
||||
for line in lines:
|
||||
print(line)
|
||||
|
||||
if not args.dry_run:
|
||||
updated = diff.get('updated', [])
|
||||
added = diff.get('added', []) if is_main else []
|
||||
if added or updated:
|
||||
merge_mame_profile(
|
||||
str(profile_path),
|
||||
str(_CACHE_PATH),
|
||||
write=True,
|
||||
add_new=is_main,
|
||||
)
|
||||
log.info('merged into %s', profile_path.name)
|
||||
|
||||
print()
|
||||
if args.dry_run:
|
||||
print('(dry run, no files modified)')
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog='mame_hash_scraper',
|
||||
description='Fetch MAME BIOS hashes from source and merge into profiles.',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--dry-run',
|
||||
action='store_true',
|
||||
help='show diff only, do not modify profiles',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--json',
|
||||
action='store_true',
|
||||
help='output raw JSON to stdout',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--force',
|
||||
action='store_true',
|
||||
help='re-fetch even if cache is fresh',
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
def main() -> None:
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(levelname)s: %(message)s',
|
||||
)
|
||||
parser = build_parser()
|
||||
args = parser.parse_args()
|
||||
_run(args)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
389
scripts/scraper/mame_parser.py
Normal file
389
scripts/scraper/mame_parser.py
Normal file
@@ -0,0 +1,389 @@
|
||||
"""Parser for MAME C source files.
|
||||
|
||||
Extracts BIOS root sets and ROM definitions from MAME driver sources.
|
||||
Handles GAME/SYST/COMP/CONS macros with MACHINE_IS_BIOS_ROOT flag,
|
||||
ROM_START/ROM_END blocks, ROM_LOAD variants, ROM_REGION, ROM_SYSTEM_BIOS,
|
||||
NO_DUMP filtering, and BAD_DUMP flagging.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
# Macros that declare a machine entry
|
||||
_MACHINE_MACROS = re.compile(
|
||||
r'\b(GAME|SYST|COMP|CONS)\s*\(',
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
# ROM block boundaries
|
||||
_ROM_START = re.compile(r'ROM_START\s*\(\s*(\w+)\s*\)')
|
||||
_ROM_END = re.compile(r'ROM_END')
|
||||
|
||||
# ROM_REGION variants: ROM_REGION, ROM_REGION16_BE, ROM_REGION16_LE, ROM_REGION32_LE, etc.
|
||||
_ROM_REGION = re.compile(
|
||||
r'ROM_REGION\w*\s*\('
|
||||
r'\s*(0x[\da-fA-F]+|\d+)\s*,' # size
|
||||
r'\s*"([^"]+)"\s*,', # tag
|
||||
)
|
||||
|
||||
# ROM_SYSTEM_BIOS( index, label, description )
|
||||
_ROM_SYSTEM_BIOS = re.compile(
|
||||
r'ROM_SYSTEM_BIOS\s*\('
|
||||
r'\s*(\d+)\s*,' # index
|
||||
r'\s*"([^"]+)"\s*,' # label
|
||||
r'\s*"([^"]+)"\s*\)', # description
|
||||
)
|
||||
|
||||
# All ROM_LOAD variants including custom BIOS macros.
|
||||
# Standard: ROM_LOAD("name", offset, size, hash)
|
||||
# BIOS variant: ROM_LOAD_BIOS(biosidx, "name", offset, size, hash)
|
||||
# ROM_LOAD16_WORD_SWAP_BIOS(biosidx, "name", offset, size, hash)
|
||||
# The key pattern: any macro containing "ROM_LOAD" or "ROMX_LOAD" in its name,
|
||||
# with the first quoted string being the ROM filename.
|
||||
_ROM_LOAD = re.compile(
|
||||
r'\b\w*ROMX?_LOAD\w*\s*\('
|
||||
r'[^"]*' # skip any args before the filename (e.g., bios index)
|
||||
r'"([^"]+)"\s*,' # name (first quoted string)
|
||||
r'\s*(0x[\da-fA-F]+|\d+)\s*,' # offset
|
||||
r'\s*(0x[\da-fA-F]+|\d+)\s*,', # size
|
||||
)
|
||||
|
||||
# CRC32 and SHA1 within a ROM_LOAD line
|
||||
_CRC_SHA = re.compile(
|
||||
r'CRC\s*\(\s*([0-9a-fA-F]+)\s*\)'
|
||||
r'\s+'
|
||||
r'SHA1\s*\(\s*([0-9a-fA-F]+)\s*\)',
|
||||
)
|
||||
|
||||
_NO_DUMP = re.compile(r'\bNO_DUMP\b')
|
||||
_BAD_DUMP = re.compile(r'\bBAD_DUMP\b')
|
||||
_ROM_BIOS = re.compile(r'ROM_BIOS\s*\(\s*(\d+)\s*\)')
|
||||
|
||||
|
||||
def find_bios_root_sets(source: str, filename: str) -> dict[str, dict]:
|
||||
"""Find machine entries flagged as BIOS root sets.
|
||||
|
||||
Scans for GAME/SYST/COMP/CONS macros where the args include
|
||||
MACHINE_IS_BIOS_ROOT, returns set names with source location.
|
||||
"""
|
||||
results: dict[str, dict] = {}
|
||||
|
||||
for match in _MACHINE_MACROS.finditer(source):
|
||||
start = match.end() - 1 # position of opening paren
|
||||
block_end = _find_closing_paren(source, start)
|
||||
if block_end == -1:
|
||||
continue
|
||||
|
||||
block = source[start:block_end + 1]
|
||||
if 'MACHINE_IS_BIOS_ROOT' not in block:
|
||||
continue
|
||||
|
||||
# Extract set name: first arg after the opening paren
|
||||
inner = block[1:] # skip opening paren
|
||||
args = _split_macro_args(inner)
|
||||
if not args:
|
||||
continue
|
||||
|
||||
# The set name position varies by macro type
|
||||
# GAME(year, setname, parent, machine, input, init, monitor, company, fullname, flags)
|
||||
# CONS(year, setname, parent, compat, machine, input, init, company, fullname, flags)
|
||||
# COMP(year, setname, parent, compat, machine, input, init, company, fullname, flags)
|
||||
# SYST(year, setname, parent, compat, machine, input, init, company, fullname, flags)
|
||||
# In all cases, setname is the second arg (index 1)
|
||||
if len(args) < 2:
|
||||
continue
|
||||
|
||||
set_name = args[1].strip()
|
||||
line_no = source[:match.start()].count('\n') + 1
|
||||
|
||||
results[set_name] = {
|
||||
'source_file': filename,
|
||||
'source_line': line_no,
|
||||
}
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def parse_rom_block(source: str, set_name: str) -> list[dict]:
|
||||
"""Parse ROM definitions for a given set name.
|
||||
|
||||
Finds the ROM_START(set_name)...ROM_END block, expands local
|
||||
#define macros that contain ROM_LOAD/ROM_REGION calls, then
|
||||
extracts all ROM entries. Skips NO_DUMP, flags BAD_DUMP.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'ROM_START\s*\(\s*' + re.escape(set_name) + r'\s*\)',
|
||||
)
|
||||
start_match = pattern.search(source)
|
||||
if not start_match:
|
||||
return []
|
||||
|
||||
end_match = _ROM_END.search(source, start_match.end())
|
||||
if not end_match:
|
||||
return []
|
||||
|
||||
block = source[start_match.end():end_match.start()]
|
||||
|
||||
# Pre-expand macros: find #define macros in the file that contain
|
||||
# ROM_LOAD/ROM_REGION/ROM_SYSTEM_BIOS calls, then expand their
|
||||
# invocations within the ROM block.
|
||||
macros = _collect_rom_macros(source)
|
||||
block = _expand_macros(block, macros, depth=5)
|
||||
|
||||
return _parse_rom_entries(block)
|
||||
|
||||
|
||||
def parse_mame_source_tree(base_path: str) -> dict[str, dict]:
|
||||
"""Walk MAME source tree and extract all BIOS root sets with ROMs.
|
||||
|
||||
Scans src/mame/ and src/devices/ for C/C++ source files.
|
||||
"""
|
||||
results: dict[str, dict] = {}
|
||||
root = Path(base_path)
|
||||
|
||||
search_dirs = [root / 'src' / 'mame', root / 'src' / 'devices']
|
||||
|
||||
for search_dir in search_dirs:
|
||||
if not search_dir.is_dir():
|
||||
continue
|
||||
for dirpath, _dirnames, filenames in os.walk(search_dir):
|
||||
for fname in filenames:
|
||||
if not fname.endswith(('.cpp', '.c', '.h', '.hxx')):
|
||||
continue
|
||||
filepath = Path(dirpath) / fname
|
||||
rel_path = str(filepath.relative_to(root))
|
||||
content = filepath.read_text(encoding='utf-8', errors='replace')
|
||||
|
||||
bios_sets = find_bios_root_sets(content, rel_path)
|
||||
for set_name, info in bios_sets.items():
|
||||
roms = parse_rom_block(content, set_name)
|
||||
results[set_name] = {
|
||||
'source_file': info['source_file'],
|
||||
'source_line': info['source_line'],
|
||||
'roms': roms,
|
||||
}
|
||||
|
||||
return results
|
||||
|
||||
|
||||
# Regex for #define macros that span multiple lines (backslash continuation)
|
||||
_DEFINE_RE = re.compile(
|
||||
r'^\s*#\s*define\s+(\w+)(?:\([^)]*\))?\s*((?:.*\\\n)*.*)',
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
# ROM-related tokens that indicate a macro is relevant for expansion
|
||||
_ROM_TOKENS = {'ROM_LOAD', 'ROMX_LOAD', 'ROM_REGION', 'ROM_SYSTEM_BIOS',
|
||||
'ROM_FILL', 'ROM_COPY', 'ROM_RELOAD'}
|
||||
|
||||
|
||||
def _collect_rom_macros(source: str) -> dict[str, str]:
|
||||
"""Collect #define macros that contain ROM-related calls.
|
||||
|
||||
Returns {macro_name: expanded_body} with backslash continuations joined.
|
||||
Only collects macros that contain actual ROM data (quoted filenames),
|
||||
not wrapper macros like ROM_LOAD16_WORD_SWAP_BIOS that just redirect
|
||||
to ROMX_LOAD with formal parameters.
|
||||
"""
|
||||
macros: dict[str, str] = {}
|
||||
for m in _DEFINE_RE.finditer(source):
|
||||
name = m.group(1)
|
||||
body = m.group(2)
|
||||
# Join backslash-continued lines
|
||||
body = body.replace('\\\n', ' ')
|
||||
# Only keep macros that contain ROM-related tokens
|
||||
if not any(tok in body for tok in _ROM_TOKENS):
|
||||
continue
|
||||
# Skip wrapper macros: if the body contains ROMX_LOAD/ROM_LOAD
|
||||
# with unquoted args (formal parameters), it's a wrapper.
|
||||
# These are already recognized by the _ROM_LOAD regex directly.
|
||||
if re.search(r'ROMX?_LOAD\s*\(\s*\w+\s*,\s*\w+\s*,', body):
|
||||
continue
|
||||
macros[name] = body
|
||||
return macros
|
||||
|
||||
|
||||
def _expand_macros(block: str, macros: dict[str, str], depth: int = 5) -> str:
|
||||
"""Expand macro invocations in a ROM block.
|
||||
|
||||
Handles both simple macros (NEOGEO_BIOS) and parameterized ones
|
||||
(NEOGEO_UNIBIOS_2_2_AND_NEWER(16)). Recurses up to `depth` levels
|
||||
for nested macros.
|
||||
"""
|
||||
if depth <= 0 or not macros:
|
||||
return block
|
||||
|
||||
changed = True
|
||||
iterations = 0
|
||||
while changed and iterations < depth:
|
||||
changed = False
|
||||
iterations += 1
|
||||
for name, body in macros.items():
|
||||
# Match macro invocation: NAME or NAME(args)
|
||||
pattern = re.compile(r'\b' + re.escape(name) + r'(?:\s*\([^)]*\))?')
|
||||
if pattern.search(block):
|
||||
block = pattern.sub(body, block)
|
||||
changed = True
|
||||
|
||||
return block
|
||||
|
||||
|
||||
def _find_closing_paren(source: str, start: int) -> int:
|
||||
"""Find the matching closing paren for source[start] which must be '('."""
|
||||
depth = 0
|
||||
i = start
|
||||
while i < len(source):
|
||||
ch = source[i]
|
||||
if ch == '(':
|
||||
depth += 1
|
||||
elif ch == ')':
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return i
|
||||
elif ch == '"':
|
||||
i += 1
|
||||
while i < len(source) and source[i] != '"':
|
||||
i += 1
|
||||
i += 1
|
||||
return -1
|
||||
|
||||
|
||||
def _split_macro_args(inner: str) -> list[str]:
|
||||
"""Split macro arguments respecting nested parens and strings."""
|
||||
args: list[str] = []
|
||||
depth = 0
|
||||
current: list[str] = []
|
||||
|
||||
i = 0
|
||||
while i < len(inner):
|
||||
ch = inner[i]
|
||||
if ch == '"':
|
||||
current.append(ch)
|
||||
i += 1
|
||||
while i < len(inner) and inner[i] != '"':
|
||||
current.append(inner[i])
|
||||
i += 1
|
||||
if i < len(inner):
|
||||
current.append(inner[i])
|
||||
elif ch == '(':
|
||||
depth += 1
|
||||
current.append(ch)
|
||||
elif ch == ')':
|
||||
if depth == 0:
|
||||
args.append(''.join(current))
|
||||
break
|
||||
depth -= 1
|
||||
current.append(ch)
|
||||
elif ch == ',' and depth == 0:
|
||||
args.append(''.join(current))
|
||||
current = []
|
||||
else:
|
||||
current.append(ch)
|
||||
i += 1
|
||||
|
||||
if current:
|
||||
remaining = ''.join(current).strip()
|
||||
if remaining:
|
||||
args.append(remaining)
|
||||
|
||||
return args
|
||||
|
||||
|
||||
def _parse_rom_entries(block: str) -> list[dict]:
|
||||
"""Parse ROM entries from a ROM block (content between ROM_START and ROM_END).
|
||||
|
||||
Uses regex scanning over the entire block (not line-by-line) to handle
|
||||
macro-expanded content where multiple statements may be on one line.
|
||||
Processes matches in order of appearance to track region and BIOS context.
|
||||
"""
|
||||
roms: list[dict] = []
|
||||
current_region = ''
|
||||
bios_labels: dict[int, tuple[str, str]] = {}
|
||||
|
||||
# Build a combined pattern that matches all interesting tokens
|
||||
# and process them in order of occurrence
|
||||
token_patterns = [
|
||||
('region', _ROM_REGION),
|
||||
('bios_label', _ROM_SYSTEM_BIOS),
|
||||
('rom_load', _ROM_LOAD),
|
||||
]
|
||||
|
||||
# Collect all matches with their positions
|
||||
events: list[tuple[int, str, re.Match]] = []
|
||||
for tag, pat in token_patterns:
|
||||
for m in pat.finditer(block):
|
||||
events.append((m.start(), tag, m))
|
||||
|
||||
# Sort by position in block
|
||||
events.sort(key=lambda e: e[0])
|
||||
|
||||
for _pos, tag, m in events:
|
||||
if tag == 'region':
|
||||
current_region = m.group(2)
|
||||
elif tag == 'bios_label':
|
||||
idx = int(m.group(1))
|
||||
bios_labels[idx] = (m.group(2), m.group(3))
|
||||
elif tag == 'rom_load':
|
||||
# Get the full macro call as context (find closing paren)
|
||||
context_start = m.start()
|
||||
# Find the opening paren of the ROM_LOAD macro
|
||||
paren_pos = block.find('(', context_start)
|
||||
if paren_pos != -1:
|
||||
close_pos = _find_closing_paren(block, paren_pos)
|
||||
context_end = close_pos + 1 if close_pos != -1 else m.end() + 200
|
||||
else:
|
||||
context_end = m.end() + 200
|
||||
context = block[context_start:min(context_end, len(block))]
|
||||
|
||||
if _NO_DUMP.search(context):
|
||||
continue
|
||||
|
||||
rom_name = m.group(1)
|
||||
rom_size = _parse_int(m.group(3))
|
||||
|
||||
crc_sha_match = _CRC_SHA.search(context)
|
||||
crc32 = ''
|
||||
sha1 = ''
|
||||
if crc_sha_match:
|
||||
crc32 = crc_sha_match.group(1).lower()
|
||||
sha1 = crc_sha_match.group(2).lower()
|
||||
|
||||
bad_dump = bool(_BAD_DUMP.search(context))
|
||||
|
||||
bios_index = None
|
||||
bios_label = ''
|
||||
bios_description = ''
|
||||
bios_ref = _ROM_BIOS.search(context)
|
||||
if bios_ref:
|
||||
bios_index = int(bios_ref.group(1))
|
||||
if bios_index in bios_labels:
|
||||
bios_label, bios_description = bios_labels[bios_index]
|
||||
|
||||
entry: dict = {
|
||||
'name': rom_name,
|
||||
'size': rom_size,
|
||||
'crc32': crc32,
|
||||
'sha1': sha1,
|
||||
'region': current_region,
|
||||
'bad_dump': bad_dump,
|
||||
}
|
||||
|
||||
if bios_index is not None:
|
||||
entry['bios_index'] = bios_index
|
||||
entry['bios_label'] = bios_label
|
||||
entry['bios_description'] = bios_description
|
||||
|
||||
roms.append(entry)
|
||||
|
||||
return roms
|
||||
|
||||
|
||||
def _parse_int(value: str) -> int:
|
||||
"""Parse an integer that may be hex (0x...) or decimal."""
|
||||
value = value.strip()
|
||||
if value.startswith('0x') or value.startswith('0X'):
|
||||
return int(value, 16)
|
||||
return int(value)
|
||||
@@ -27,6 +27,14 @@ SOURCE_URL = (
|
||||
|
||||
GITHUB_REPO = "RetroBat-Official/retrobat"
|
||||
|
||||
# Map RetroBat system keys to our normalized system IDs
|
||||
SYSTEM_SLUG_MAP = {
|
||||
"ps2": "sony-playstation-2",
|
||||
"ps3": "sony-playstation-3",
|
||||
"psvita": "sony-playstation-vita",
|
||||
"gsplus": "apple-iigs",
|
||||
}
|
||||
|
||||
|
||||
class Scraper(BaseScraper):
|
||||
"""Scraper for RetroBat batocera-systems.json."""
|
||||
@@ -83,7 +91,7 @@ class Scraper(BaseScraper):
|
||||
|
||||
requirements.append(BiosRequirement(
|
||||
name=name,
|
||||
system=sys_key,
|
||||
system=SYSTEM_SLUG_MAP.get(sys_key, sys_key),
|
||||
md5=md5 or None,
|
||||
destination=file_path,
|
||||
required=True,
|
||||
@@ -113,10 +121,25 @@ class Scraper(BaseScraper):
|
||||
"""Generate a platform YAML config dict from scraped data."""
|
||||
requirements = self.fetch_requirements()
|
||||
|
||||
# Parse source to extract display names per system
|
||||
raw = self._fetch_raw()
|
||||
source_data = json.loads(raw)
|
||||
display_names: dict[str, str] = {}
|
||||
for sys_key, sys_data in source_data.items():
|
||||
if isinstance(sys_data, dict):
|
||||
dname = sys_data.get("name", "")
|
||||
if dname:
|
||||
slug = SYSTEM_SLUG_MAP.get(sys_key, sys_key)
|
||||
display_names[slug] = dname
|
||||
|
||||
systems = {}
|
||||
for req in requirements:
|
||||
if req.system not in systems:
|
||||
systems[req.system] = {"files": []}
|
||||
sys_entry: dict = {"files": []}
|
||||
dname = display_names.get(req.system)
|
||||
if dname:
|
||||
sys_entry["name"] = dname
|
||||
systems[req.system] = sys_entry
|
||||
|
||||
entry = {
|
||||
"name": req.name,
|
||||
|
||||
@@ -38,8 +38,14 @@ def _enrich_hashes(entry: dict, db: dict) -> None:
|
||||
sha1 = entry.get("sha1", "")
|
||||
md5 = entry.get("md5", "")
|
||||
|
||||
# Hashes can be lists (multi-hash) — use first string value
|
||||
if isinstance(sha1, list):
|
||||
sha1 = sha1[0] if sha1 else ""
|
||||
if isinstance(md5, list):
|
||||
md5 = md5[0] if md5 else ""
|
||||
|
||||
record = None
|
||||
if sha1 and db.get("files"):
|
||||
if sha1 and isinstance(sha1, str) and db.get("files"):
|
||||
record = db["files"].get(sha1)
|
||||
if record is None and md5:
|
||||
by_md5 = db.get("by_md5", {})
|
||||
@@ -323,9 +329,46 @@ def _diff_system(truth_sys: dict, scraped_sys: dict) -> dict:
|
||||
"scraped_required": s_req,
|
||||
})
|
||||
|
||||
# Truth files not matched -> missing
|
||||
for fe in truth_sys.get("files", []):
|
||||
if fe["name"].lower() not in matched_truth_names:
|
||||
# Collect unmatched files from both sides
|
||||
unmatched_truth = [
|
||||
fe for fe in truth_sys.get("files", [])
|
||||
if fe["name"].lower() not in matched_truth_names
|
||||
]
|
||||
unmatched_scraped = {
|
||||
s_key: s_entry for s_key, s_entry in scraped_index.items()
|
||||
if s_key not in truth_index
|
||||
}
|
||||
|
||||
# Hash-based fallback: detect platform renames (e.g. Batocera ROM → ROM1)
|
||||
# If an unmatched scraped file shares a hash with an unmatched truth file,
|
||||
# it's the same file under a different name — a platform rename, not a gap.
|
||||
rename_matched_truth: set[str] = set()
|
||||
rename_matched_scraped: set[str] = set()
|
||||
|
||||
if unmatched_truth and unmatched_scraped:
|
||||
# Build hash → truth file index for unmatched truth files
|
||||
truth_hash_index: dict[str, dict] = {}
|
||||
for fe in unmatched_truth:
|
||||
for h in ("sha1", "md5", "crc32"):
|
||||
val = fe.get(h)
|
||||
if val and isinstance(val, str):
|
||||
truth_hash_index[val.lower()] = fe
|
||||
|
||||
for s_key, s_entry in unmatched_scraped.items():
|
||||
for h in ("sha1", "md5", "crc32"):
|
||||
s_val = s_entry.get(h)
|
||||
if not s_val or not isinstance(s_val, str):
|
||||
continue
|
||||
t_entry = truth_hash_index.get(s_val.lower())
|
||||
if t_entry is not None:
|
||||
# Rename detected — count as matched
|
||||
rename_matched_truth.add(t_entry["name"].lower())
|
||||
rename_matched_scraped.add(s_key)
|
||||
break
|
||||
|
||||
# Truth files not matched (by name, alias, or hash) -> missing
|
||||
for fe in unmatched_truth:
|
||||
if fe["name"].lower() not in rename_matched_truth:
|
||||
missing.append({
|
||||
"name": fe["name"],
|
||||
"cores": list(fe.get("_cores", [])),
|
||||
@@ -335,13 +378,14 @@ def _diff_system(truth_sys: dict, scraped_sys: dict) -> dict:
|
||||
# Scraped files not in truth -> extra
|
||||
coverage = truth_sys.get("_coverage", {})
|
||||
has_unprofiled = bool(coverage.get("cores_unprofiled"))
|
||||
for s_key, s_entry in scraped_index.items():
|
||||
if s_key not in truth_index:
|
||||
entry = {"name": s_entry["name"]}
|
||||
if has_unprofiled:
|
||||
extra_unprofiled.append(entry)
|
||||
else:
|
||||
extra_phantom.append(entry)
|
||||
for s_key, s_entry in unmatched_scraped.items():
|
||||
if s_key in rename_matched_scraped:
|
||||
continue
|
||||
entry = {"name": s_entry["name"]}
|
||||
if has_unprofiled:
|
||||
extra_unprofiled.append(entry)
|
||||
else:
|
||||
extra_phantom.append(entry)
|
||||
|
||||
result: dict = {}
|
||||
if missing:
|
||||
|
||||
@@ -31,10 +31,11 @@ from pathlib import Path
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import (
|
||||
build_target_cores_cache, build_zip_contents_index, check_inside_zip,
|
||||
compute_hashes, filter_systems_by_target, group_identical_platforms,
|
||||
list_emulator_profiles, list_system_ids, load_data_dir_registry,
|
||||
load_emulator_profiles, load_platform_config, md5sum, md5_composite,
|
||||
require_yaml, resolve_local_file, resolve_platform_cores,
|
||||
compute_hashes, expand_platform_declared_names, filter_systems_by_target,
|
||||
group_identical_platforms, list_emulator_profiles, list_system_ids,
|
||||
load_data_dir_registry, load_emulator_profiles, load_platform_config,
|
||||
md5sum, md5_composite, require_yaml, resolve_local_file,
|
||||
resolve_platform_cores,
|
||||
)
|
||||
|
||||
yaml = require_yaml()
|
||||
@@ -261,13 +262,9 @@ def find_undeclared_files(
|
||||
data_names: set[str] | None = None,
|
||||
) -> list[dict]:
|
||||
"""Find files needed by cores but not declared in platform config."""
|
||||
# Collect all filenames declared by this platform
|
||||
declared_names: set[str] = set()
|
||||
for sys_id, system in config.get("systems", {}).items():
|
||||
for fe in system.get("files", []):
|
||||
name = fe.get("name", "")
|
||||
if name:
|
||||
declared_names.add(name)
|
||||
# Collect all filenames declared by this platform, enriched with
|
||||
# canonical names from DB via MD5 (handles platform renaming)
|
||||
declared_names = expand_platform_declared_names(config, db)
|
||||
|
||||
# Collect data_directory refs
|
||||
declared_dd: set[str] = set()
|
||||
@@ -294,6 +291,10 @@ def find_undeclared_files(
|
||||
if emu_name not in relevant:
|
||||
continue
|
||||
|
||||
# Skip agnostic profiles entirely (filename-agnostic BIOS detection)
|
||||
if profile.get("bios_mode") == "agnostic":
|
||||
continue
|
||||
|
||||
# Check if this profile is standalone: match profile name or any cores: alias
|
||||
is_standalone = emu_name in standalone_set or bool(
|
||||
standalone_set & {str(c) for c in profile.get("cores", [])}
|
||||
@@ -320,6 +321,10 @@ def find_undeclared_files(
|
||||
if load_from and load_from != "system_dir":
|
||||
continue
|
||||
|
||||
# Skip agnostic files (filename-agnostic, handled by agnostic scan)
|
||||
if f.get("agnostic"):
|
||||
continue
|
||||
|
||||
archive = f.get("archive")
|
||||
|
||||
# Skip files declared by the platform (by name or archive)
|
||||
|
||||
@@ -31,6 +31,7 @@ sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "scripts"))
|
||||
import yaml
|
||||
from common import (
|
||||
build_zip_contents_index, check_inside_zip, compute_hashes,
|
||||
expand_platform_declared_names,
|
||||
group_identical_platforms, load_emulator_profiles, load_platform_config,
|
||||
md5_composite, md5sum, parse_md5_list, resolve_local_file,
|
||||
resolve_platform_cores, safe_extract_zip,
|
||||
@@ -169,6 +170,7 @@ class TestE2E(unittest.TestCase):
|
||||
"md5": info["md5"],
|
||||
"name": name,
|
||||
"crc32": info.get("crc32", ""),
|
||||
"size": len(info["data"]),
|
||||
}
|
||||
by_md5[info["md5"]] = sha1
|
||||
by_name.setdefault(name, []).append(sha1)
|
||||
@@ -269,6 +271,12 @@ class TestE2E(unittest.TestCase):
|
||||
{"ref": "test-data-dir", "destination": "TestData"},
|
||||
],
|
||||
},
|
||||
"sys-renamed": {
|
||||
"files": [
|
||||
{"name": "renamed_file.bin", "destination": "renamed_file.bin",
|
||||
"md5": f["correct_hash.bin"]["md5"], "required": True},
|
||||
],
|
||||
},
|
||||
},
|
||||
}
|
||||
with open(os.path.join(self.platforms_dir, "test_md5.yml"), "w") as fh:
|
||||
@@ -490,6 +498,46 @@ class TestE2E(unittest.TestCase):
|
||||
with open(os.path.join(self.emulators_dir, "test_subdir_core.yml"), "w") as fh:
|
||||
yaml.dump(emu_subdir, fh)
|
||||
|
||||
# Emulator whose file is declared by platform under a different name
|
||||
# (e.g. gsplus ROM vs Batocera ROM1) — hash-based matching should resolve
|
||||
emu_renamed = {
|
||||
"emulator": "TestRenamed",
|
||||
"type": "standalone",
|
||||
"systems": ["sys-renamed"],
|
||||
"files": [
|
||||
{"name": "correct_hash.bin", "required": True},
|
||||
],
|
||||
}
|
||||
with open(os.path.join(self.emulators_dir, "test_renamed.yml"), "w") as fh:
|
||||
yaml.dump(emu_renamed, fh)
|
||||
|
||||
# Agnostic profile (bios_mode: agnostic) — skipped by find_undeclared_files
|
||||
emu_agnostic = {
|
||||
"emulator": "TestAgnostic",
|
||||
"type": "standalone",
|
||||
"bios_mode": "agnostic",
|
||||
"systems": ["console-a"],
|
||||
"files": [
|
||||
{"name": "correct_hash.bin", "required": True,
|
||||
"min_size": 1, "max_size": 999999},
|
||||
],
|
||||
}
|
||||
with open(os.path.join(self.emulators_dir, "test_agnostic.yml"), "w") as fh:
|
||||
yaml.dump(emu_agnostic, fh)
|
||||
|
||||
# Mixed profile with per-file agnostic
|
||||
emu_mixed_agnostic = {
|
||||
"emulator": "TestMixedAgnostic",
|
||||
"type": "libretro",
|
||||
"systems": ["console-a"],
|
||||
"files": [
|
||||
{"name": "undeclared_req.bin", "required": True},
|
||||
{"name": "agnostic_file.bin", "required": True, "agnostic": True},
|
||||
],
|
||||
}
|
||||
with open(os.path.join(self.emulators_dir, "test_mixed_agnostic.yml"), "w") as fh:
|
||||
yaml.dump(emu_mixed_agnostic, fh)
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# THE TEST -one method per feature area, all using same fixtures
|
||||
# ---------------------------------------------------------------
|
||||
@@ -3122,7 +3170,8 @@ class TestE2E(unittest.TestCase):
|
||||
self.assertIn("Sony - PlayStation", content)
|
||||
self.assertIn("scph5501.bin", content)
|
||||
self.assertIn("b056ee5a4d65937e1a3a17e1e78f3258ea49c38e", content)
|
||||
self.assertIn('name "System.dat"', content)
|
||||
self.assertIn('name "System"', content)
|
||||
self.assertIn("71AF80B4", content) # CRC uppercase
|
||||
|
||||
issues = exporter.validate(truth, out_path)
|
||||
self.assertEqual(issues, [])
|
||||
@@ -3251,6 +3300,433 @@ class TestE2E(unittest.TestCase):
|
||||
self.assertEqual(div["extra_unprofiled"][0]["name"], "phantom.bin")
|
||||
self.assertNotIn("extra_phantom", div)
|
||||
|
||||
def test_173_cross_ref_hash_matching(self):
|
||||
"""Platform file under different name matched by MD5 is not undeclared."""
|
||||
config = load_platform_config("test_md5", self.platforms_dir)
|
||||
profiles = load_emulator_profiles(self.emulators_dir)
|
||||
undeclared = find_undeclared_files(config, self.emulators_dir, self.db, profiles)
|
||||
names = {u["name"] for u in undeclared}
|
||||
# correct_hash.bin is declared by platform as renamed_file.bin with same MD5
|
||||
# hash-based matching should suppress it from undeclared
|
||||
self.assertNotIn("correct_hash.bin", names)
|
||||
|
||||
def test_174_expand_platform_declared_names(self):
|
||||
"""expand_platform_declared_names enriches with DB canonical names."""
|
||||
config = load_platform_config("test_md5", self.platforms_dir)
|
||||
result = expand_platform_declared_names(config, self.db)
|
||||
# renamed_file.bin is declared directly
|
||||
self.assertIn("renamed_file.bin", result)
|
||||
# correct_hash.bin is the DB canonical name for the same MD5
|
||||
self.assertIn("correct_hash.bin", result)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Registry merge + all_libretro expansion + diff hash fallback
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def test_175_registry_merge_cores(self):
|
||||
"""load_platform_config merges cores from _registry.yml."""
|
||||
from common import _platform_config_cache
|
||||
|
||||
# Platform YAML with 1 core
|
||||
config = {
|
||||
"platform": "TestMerge",
|
||||
"cores": ["core_a"],
|
||||
"systems": {"test-system": {"files": []}},
|
||||
}
|
||||
with open(os.path.join(self.platforms_dir, "testmerge.yml"), "w") as f:
|
||||
yaml.dump(config, f)
|
||||
|
||||
# Registry with 2 cores (superset)
|
||||
registry = {
|
||||
"platforms": {
|
||||
"testmerge": {
|
||||
"config": "testmerge.yml",
|
||||
"status": "active",
|
||||
"cores": ["core_a", "core_b"],
|
||||
}
|
||||
}
|
||||
}
|
||||
with open(os.path.join(self.platforms_dir, "_registry.yml"), "w") as f:
|
||||
yaml.dump(registry, f)
|
||||
|
||||
_platform_config_cache.clear()
|
||||
loaded = load_platform_config("testmerge", self.platforms_dir)
|
||||
cores = [str(c) for c in loaded["cores"]]
|
||||
self.assertIn("core_a", cores)
|
||||
self.assertIn("core_b", cores)
|
||||
|
||||
def test_176_all_libretro_in_list(self):
|
||||
"""resolve_platform_cores expands all_libretro/retroarch in a list."""
|
||||
from common import resolve_platform_cores, load_emulator_profiles
|
||||
|
||||
# Create a libretro profile and a standalone profile
|
||||
for name, ptype in [("lr_core", "libretro"), ("sa_core", "standalone")]:
|
||||
profile = {
|
||||
"emulator": name,
|
||||
"type": ptype,
|
||||
"cores": [name],
|
||||
"systems": ["test-system"],
|
||||
"files": [],
|
||||
}
|
||||
with open(os.path.join(self.emulators_dir, f"{name}.yml"), "w") as f:
|
||||
yaml.dump(profile, f)
|
||||
|
||||
profiles = load_emulator_profiles(self.emulators_dir)
|
||||
|
||||
# Config with retroarch + sa_core in cores list
|
||||
config = {"cores": ["retroarch", "sa_core"]}
|
||||
resolved = resolve_platform_cores(config, profiles)
|
||||
self.assertIn("lr_core", resolved) # expanded via retroarch
|
||||
self.assertIn("sa_core", resolved) # explicit
|
||||
|
||||
def test_177_diff_hash_fallback_rename(self):
|
||||
"""Diff detects platform renames via hash fallback."""
|
||||
from truth import diff_platform_truth
|
||||
|
||||
truth = {
|
||||
"systems": {
|
||||
"test-system": {
|
||||
"_coverage": {"cores_profiled": ["c"], "cores_unprofiled": []},
|
||||
"files": [
|
||||
{"name": "ROM", "required": True, "md5": "abcd1234" * 4,
|
||||
"_cores": ["c"], "_source_refs": []},
|
||||
],
|
||||
}
|
||||
}
|
||||
}
|
||||
scraped = {
|
||||
"systems": {
|
||||
"test-system": {
|
||||
"files": [
|
||||
{"name": "ROM1", "required": True, "md5": "abcd1234" * 4},
|
||||
],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result = diff_platform_truth(truth, scraped)
|
||||
# ROM and ROM1 share the same hash — rename, not missing+phantom
|
||||
self.assertEqual(result["summary"]["total_missing"], 0)
|
||||
self.assertEqual(result["summary"]["total_extra_phantom"], 0)
|
||||
|
||||
def test_178_diff_system_normalization(self):
|
||||
"""Diff matches systems with different IDs via normalization."""
|
||||
from truth import diff_platform_truth
|
||||
|
||||
truth = {
|
||||
"systems": {
|
||||
"sega-gamegear": {
|
||||
"_coverage": {"cores_profiled": ["c"], "cores_unprofiled": []},
|
||||
"files": [
|
||||
{"name": "bios.gg", "required": True, "md5": "a" * 32,
|
||||
"_cores": ["c"], "_source_refs": []},
|
||||
],
|
||||
},
|
||||
}
|
||||
}
|
||||
scraped = {
|
||||
"systems": {
|
||||
"sega-game-gear": {
|
||||
"files": [
|
||||
{"name": "bios.gg", "required": True, "md5": "a" * 32},
|
||||
],
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
result = diff_platform_truth(truth, scraped)
|
||||
self.assertEqual(result["summary"]["systems_uncovered"], 0)
|
||||
self.assertEqual(result["summary"]["total_missing"], 0)
|
||||
self.assertEqual(result["summary"]["systems_compared"], 1)
|
||||
|
||||
def test_179_agnostic_profile_skipped_in_undeclared(self):
|
||||
"""bios_mode: agnostic profiles are skipped entirely by find_undeclared_files."""
|
||||
config = load_platform_config("test_existence", self.platforms_dir)
|
||||
profiles = load_emulator_profiles(self.emulators_dir)
|
||||
undeclared = find_undeclared_files(config, self.emulators_dir, self.db, profiles)
|
||||
emulators = {u["emulator"] for u in undeclared}
|
||||
# TestAgnostic should NOT appear in undeclared (bios_mode: agnostic)
|
||||
self.assertNotIn("TestAgnostic", emulators)
|
||||
|
||||
def test_180_agnostic_file_skipped_in_undeclared(self):
|
||||
"""Files with agnostic: true are skipped, others in same profile are not."""
|
||||
config = load_platform_config("test_existence", self.platforms_dir)
|
||||
profiles = load_emulator_profiles(self.emulators_dir)
|
||||
undeclared = find_undeclared_files(config, self.emulators_dir, self.db, profiles)
|
||||
names = {u["name"] for u in undeclared}
|
||||
# agnostic_file.bin should NOT be in undeclared (agnostic: true)
|
||||
self.assertNotIn("agnostic_file.bin", names)
|
||||
# undeclared_req.bin should still be in undeclared (not agnostic)
|
||||
self.assertIn("undeclared_req.bin", names)
|
||||
|
||||
def test_181_agnostic_extras_scan(self):
|
||||
"""Agnostic profiles add all matching DB files as extras."""
|
||||
from generate_pack import _collect_emulator_extras
|
||||
config = load_platform_config("test_existence", self.platforms_dir)
|
||||
profiles = load_emulator_profiles(self.emulators_dir)
|
||||
extras = _collect_emulator_extras(
|
||||
config, self.emulators_dir, self.db, set(), "system", profiles,
|
||||
)
|
||||
agnostic_extras = [e for e in extras if e.get("source_emulator") == "TestAgnostic"]
|
||||
# Agnostic scan should find files in the same directory as correct_hash.bin
|
||||
self.assertTrue(len(agnostic_extras) > 0, "Agnostic scan should produce extras")
|
||||
# All agnostic extras should have agnostic_scan flag
|
||||
for e in agnostic_extras:
|
||||
self.assertTrue(e.get("agnostic_scan", False))
|
||||
|
||||
def test_182_agnostic_rename_readme(self):
|
||||
"""_build_agnostic_rename_readme generates correct text."""
|
||||
from generate_pack import _build_agnostic_rename_readme
|
||||
result = _build_agnostic_rename_readme(
|
||||
"dsi_nand.bin", "DSi_Nand_AUS.bin",
|
||||
["DSi_Nand_EUR.bin", "DSi_Nand_USA.bin"],
|
||||
)
|
||||
self.assertIn("dsi_nand.bin <- DSi_Nand_AUS.bin", result)
|
||||
self.assertIn("DSi_Nand_EUR.bin", result)
|
||||
self.assertIn("DSi_Nand_USA.bin", result)
|
||||
self.assertIn("rename it to: dsi_nand.bin", result)
|
||||
|
||||
def test_183_agnostic_resolve_fallback(self):
|
||||
"""resolve_local_file with agnostic fallback finds a system file."""
|
||||
file_entry = {
|
||||
"name": "nonexistent_agnostic.bin",
|
||||
"agnostic": True,
|
||||
"min_size": 1,
|
||||
"max_size": 999999,
|
||||
"agnostic_path_prefix": self.bios_dir + "/",
|
||||
}
|
||||
path, status = resolve_local_file(file_entry, self.db)
|
||||
self.assertIsNotNone(path)
|
||||
self.assertEqual(status, "agnostic_fallback")
|
||||
|
||||
|
||||
def test_179_batocera_exporter_round_trip(self):
|
||||
"""Batocera exporter produces valid Python dict format."""
|
||||
from exporter.batocera_exporter import Exporter
|
||||
|
||||
truth = {
|
||||
"systems": {
|
||||
"sony-playstation": {
|
||||
"_coverage": {"cores_profiled": ["c"]},
|
||||
"files": [
|
||||
{"name": "scph5501.bin", "destination": "scph5501.bin",
|
||||
"required": True, "md5": "b" * 32,
|
||||
"_cores": ["c"], "_source_refs": []},
|
||||
],
|
||||
}
|
||||
}
|
||||
}
|
||||
scraped = {
|
||||
"systems": {
|
||||
"sony-playstation": {"native_id": "psx", "files": []},
|
||||
}
|
||||
}
|
||||
out = os.path.join(self.root, "batocera-systems")
|
||||
exp = Exporter()
|
||||
exp.export(truth, out, scraped_data=scraped)
|
||||
|
||||
content = open(out).read()
|
||||
self.assertIn('"psx"', content)
|
||||
self.assertIn("scph5501.bin", content)
|
||||
self.assertIn("b" * 32, content)
|
||||
self.assertEqual(exp.validate(truth, out), [])
|
||||
|
||||
def test_180_recalbox_exporter_round_trip(self):
|
||||
"""Recalbox exporter produces valid es_bios.xml."""
|
||||
from exporter.recalbox_exporter import Exporter
|
||||
|
||||
truth = {
|
||||
"systems": {
|
||||
"sony-playstation": {
|
||||
"_coverage": {"cores_profiled": ["c"]},
|
||||
"files": [
|
||||
{"name": "scph5501.bin", "destination": "scph5501.bin",
|
||||
"required": True, "md5": "b" * 32,
|
||||
"_cores": ["c"], "_source_refs": []},
|
||||
],
|
||||
}
|
||||
}
|
||||
}
|
||||
scraped = {
|
||||
"systems": {
|
||||
"sony-playstation": {"native_id": "psx", "files": []},
|
||||
}
|
||||
}
|
||||
out = os.path.join(self.root, "es_bios.xml")
|
||||
exp = Exporter()
|
||||
exp.export(truth, out, scraped_data=scraped)
|
||||
|
||||
content = open(out).read()
|
||||
self.assertIn("<biosList", content)
|
||||
self.assertIn('platform="psx"', content)
|
||||
self.assertIn('fullname=', content)
|
||||
self.assertIn("scph5501.bin", content)
|
||||
# mandatory="true" is the default, not emitted (matching Recalbox format)
|
||||
self.assertNotIn('mandatory="false"', content)
|
||||
self.assertIn('core="libretro/c"', content)
|
||||
self.assertEqual(exp.validate(truth, out), [])
|
||||
|
||||
def test_181_retrobat_exporter_round_trip(self):
|
||||
"""RetroBat exporter produces valid JSON."""
|
||||
import json as _json
|
||||
from exporter.retrobat_exporter import Exporter
|
||||
|
||||
truth = {
|
||||
"systems": {
|
||||
"sony-playstation": {
|
||||
"_coverage": {"cores_profiled": ["c"]},
|
||||
"files": [
|
||||
{"name": "scph5501.bin", "destination": "scph5501.bin",
|
||||
"required": True, "md5": "b" * 32,
|
||||
"_cores": ["c"], "_source_refs": []},
|
||||
],
|
||||
}
|
||||
}
|
||||
}
|
||||
scraped = {
|
||||
"systems": {
|
||||
"sony-playstation": {"native_id": "psx", "files": []},
|
||||
}
|
||||
}
|
||||
out = os.path.join(self.root, "batocera-systems.json")
|
||||
exp = Exporter()
|
||||
exp.export(truth, out, scraped_data=scraped)
|
||||
|
||||
data = _json.loads(open(out).read())
|
||||
self.assertIn("psx", data)
|
||||
self.assertTrue(any("scph5501" in bf["file"] for bf in data["psx"]["biosFiles"]))
|
||||
self.assertEqual(exp.validate(truth, out), [])
|
||||
|
||||
def test_182_exporter_discovery(self):
|
||||
"""All exporters are discovered by the plugin system."""
|
||||
from exporter import discover_exporters
|
||||
exporters = discover_exporters()
|
||||
self.assertIn("retroarch", exporters)
|
||||
self.assertIn("batocera", exporters)
|
||||
self.assertIn("recalbox", exporters)
|
||||
self.assertIn("retrobat", exporters)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Hash scraper: parsers + merge
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
def test_mame_parser_finds_bios_root_sets(self):
|
||||
from scripts.scraper.mame_parser import find_bios_root_sets, parse_rom_block
|
||||
source = '''
|
||||
ROM_START( neogeo )
|
||||
ROM_REGION( 0x020000, "mainbios", 0 )
|
||||
ROM_LOAD( "sp-s2.sp1", 0x00000, 0x020000, CRC(9036d879) SHA1(4f834c580f3471ce40c3210ef5e7491df38d8851) )
|
||||
ROM_END
|
||||
GAME( 1990, neogeo, 0, ng, neogeo, ng_state, empty_init, ROT0, "SNK", "Neo Geo", MACHINE_IS_BIOS_ROOT )
|
||||
ROM_START( pacman )
|
||||
ROM_REGION( 0x10000, "maincpu", 0 )
|
||||
ROM_LOAD( "pacman.6e", 0x0000, 0x1000, CRC(c1e6ab10) SHA1(e87e059c5be45753f7e9f33dff851f16d6751181) )
|
||||
ROM_END
|
||||
GAME( 1980, pacman, 0, pacman, pacman, pacman_state, empty_init, ROT90, "Namco", "Pac-Man", 0 )
|
||||
'''
|
||||
sets = find_bios_root_sets(source, "neogeo.cpp")
|
||||
self.assertIn("neogeo", sets)
|
||||
self.assertNotIn("pacman", sets)
|
||||
roms = parse_rom_block(source, "neogeo")
|
||||
self.assertEqual(len(roms), 1)
|
||||
self.assertEqual(roms[0]["crc32"], "9036d879")
|
||||
|
||||
def test_fbneo_parser_finds_bios_sets(self):
|
||||
from scripts.scraper.fbneo_parser import find_bios_sets, parse_rom_info
|
||||
source = '''
|
||||
static struct BurnRomInfo neogeoRomDesc[] = {
|
||||
{ "sp-s2.sp1", 0x020000, 0x9036d879, BRF_ESS | BRF_BIOS },
|
||||
{ "", 0, 0, 0 }
|
||||
};
|
||||
STD_ROM_PICK(neogeo)
|
||||
STD_ROM_FN(neogeo)
|
||||
struct BurnDriver BurnDrvneogeo = {
|
||||
"neogeo", NULL, NULL, NULL, "1990",
|
||||
"Neo Geo\\0", "BIOS only", "SNK", "Neo Geo MVS",
|
||||
NULL, NULL, NULL, NULL, BDF_BOARDROM, 0, 0,
|
||||
0, 0, 0, NULL, neogeoRomInfo, neogeoRomName, NULL, NULL,
|
||||
NULL, NULL, NULL, NULL, 0
|
||||
};
|
||||
'''
|
||||
sets = find_bios_sets(source, "d_neogeo.cpp")
|
||||
self.assertIn("neogeo", sets)
|
||||
roms = parse_rom_info(source, "neogeo")
|
||||
self.assertEqual(len(roms), 1)
|
||||
self.assertEqual(roms[0]["crc32"], "9036d879")
|
||||
|
||||
def test_mame_merge_preserves_manual_fields(self):
|
||||
import json as json_mod
|
||||
from scripts.scraper._hash_merge import merge_mame_profile
|
||||
merge_dir = os.path.join(self.root, "merge_mame")
|
||||
os.makedirs(merge_dir)
|
||||
profile = {
|
||||
"emulator": "Test", "type": "libretro",
|
||||
"upstream": "https://github.com/mamedev/mame",
|
||||
"core_version": "0.285",
|
||||
"files": [{
|
||||
"name": "neogeo.zip", "required": True, "category": "bios_zip",
|
||||
"system": "snk-neogeo-mvs", "note": "MVS BIOS",
|
||||
"source_ref": "old.cpp:1",
|
||||
"contents": [{"name": "sp-s2.sp1", "size": 131072, "crc32": "oldcrc"}],
|
||||
}],
|
||||
}
|
||||
profile_path = os.path.join(merge_dir, "test.yml")
|
||||
with open(profile_path, "w") as f:
|
||||
yaml.dump(profile, f, sort_keys=False)
|
||||
hashes = {
|
||||
"source": "mamedev/mame", "version": "0.286", "commit": "abc",
|
||||
"fetched_at": "2026-03-30T00:00:00Z",
|
||||
"bios_sets": {"neogeo": {
|
||||
"source_file": "neo.cpp", "source_line": 42,
|
||||
"roms": [{"name": "sp-s2.sp1", "size": 131072, "crc32": "newcrc", "sha1": "abc123"}],
|
||||
}},
|
||||
}
|
||||
hashes_path = os.path.join(merge_dir, "hashes.json")
|
||||
with open(hashes_path, "w") as f:
|
||||
json_mod.dump(hashes, f)
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
neo = next(f for f in result["files"] if f["name"] == "neogeo.zip")
|
||||
self.assertEqual(neo["contents"][0]["crc32"], "newcrc")
|
||||
self.assertEqual(neo["system"], "snk-neogeo-mvs")
|
||||
self.assertEqual(neo["note"], "MVS BIOS")
|
||||
self.assertEqual(neo["source_ref"], "neo.cpp:42")
|
||||
self.assertEqual(result["core_version"], "0.286")
|
||||
|
||||
def test_fbneo_merge_updates_individual_roms(self):
|
||||
import json as json_mod
|
||||
from scripts.scraper._hash_merge import merge_fbneo_profile
|
||||
merge_dir = os.path.join(self.root, "merge_fbneo")
|
||||
os.makedirs(merge_dir)
|
||||
profile = {
|
||||
"emulator": "FBNeo", "type": "libretro",
|
||||
"upstream": "https://github.com/finalburnneo/FBNeo",
|
||||
"core_version": "v1.0.0.02",
|
||||
"files": [{"name": "sp-s2.sp1", "archive": "neogeo.zip",
|
||||
"system": "snk-neogeo-mvs", "required": True,
|
||||
"size": 131072, "crc32": "oldcrc"}],
|
||||
}
|
||||
profile_path = os.path.join(merge_dir, "fbneo.yml")
|
||||
with open(profile_path, "w") as f:
|
||||
yaml.dump(profile, f, sort_keys=False)
|
||||
hashes = {
|
||||
"source": "finalburnneo/FBNeo", "version": "v1.0.0.03", "commit": "def",
|
||||
"fetched_at": "2026-03-30T00:00:00Z",
|
||||
"bios_sets": {"neogeo": {
|
||||
"source_file": "neo.cpp", "source_line": 10,
|
||||
"roms": [{"name": "sp-s2.sp1", "size": 131072, "crc32": "newcrc"}],
|
||||
}},
|
||||
}
|
||||
hashes_path = os.path.join(merge_dir, "hashes.json")
|
||||
with open(hashes_path, "w") as f:
|
||||
json_mod.dump(hashes, f)
|
||||
result = merge_fbneo_profile(profile_path, hashes_path)
|
||||
rom = next(f for f in result["files"] if f["name"] == "sp-s2.sp1")
|
||||
self.assertEqual(rom["crc32"], "newcrc")
|
||||
self.assertEqual(rom["system"], "snk-neogeo-mvs")
|
||||
self.assertEqual(result["core_version"], "v1.0.0.03")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
190
tests/test_fbneo_parser.py
Normal file
190
tests/test_fbneo_parser.py
Normal file
@@ -0,0 +1,190 @@
|
||||
"""Tests for the FBNeo source parser."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.scraper.fbneo_parser import (
|
||||
find_bios_sets,
|
||||
parse_fbneo_source_tree,
|
||||
parse_rom_info,
|
||||
)
|
||||
|
||||
NEOGEO_FIXTURE = """\
|
||||
static struct BurnRomInfo neogeoRomDesc[] = {
|
||||
{ "sp-s2.sp1", 0x020000, 0x9036d879, BRF_ESS | BRF_BIOS },
|
||||
{ "sp-s.sp1", 0x020000, 0xc7f2fa45, BRF_ESS | BRF_BIOS },
|
||||
{ "asia-s3.rom", 0x020000, 0x91b64be3, BRF_ESS | BRF_BIOS },
|
||||
{ "vs-bios.rom", 0x020000, 0xf0e8f27d, BRF_ESS | BRF_BIOS },
|
||||
{ "uni-bios.rom", 0x020000, 0x2d50996a, BRF_ESS | BRF_BIOS },
|
||||
{ "", 0, 0, 0 }
|
||||
};
|
||||
|
||||
STD_ROM_FN(neogeo)
|
||||
|
||||
struct BurnDriver BurnDrvneogeo = {
|
||||
"neogeo", NULL, NULL, NULL, "1990",
|
||||
"Neo Geo\\0", "BIOS only", "SNK", "Neo Geo MVS",
|
||||
NULL, NULL, NULL, NULL,
|
||||
BDF_BOARDROM, 0, HARDWARE_PREFIX_CARTRIDGE | HARDWARE_SNK_NEOGEO,
|
||||
GBF_BIOS, 0,
|
||||
NULL, neogeoRomInfo, neogeoRomName, NULL, NULL, NULL, NULL,
|
||||
neogeoInputInfo, neogeoDIPInfo,
|
||||
NULL, NULL, NULL, NULL, 0x1000,
|
||||
304, 224, 4, 3
|
||||
};
|
||||
"""
|
||||
|
||||
PGM_FIXTURE = """\
|
||||
static struct BurnRomInfo pgmRomDesc[] = {
|
||||
{ "pgm_t01s.rom", 0x200000, 0x1a7123a0, BRF_GRA },
|
||||
{ "pgm_m01s.rom", 0x200000, 0x45ae7159, BRF_SND },
|
||||
{ "pgm_p01s.rom", 0x020000, 0xe42b166e, BRF_ESS | BRF_BIOS },
|
||||
{ "", 0, 0, 0 }
|
||||
};
|
||||
|
||||
STD_ROM_FN(pgm)
|
||||
|
||||
struct BurnDriver BurnDrvpgm = {
|
||||
"pgm", NULL, NULL, NULL, "1997",
|
||||
"PGM (Polygame Master)\\0", "BIOS only", "IGS", "PGM",
|
||||
NULL, NULL, NULL, NULL,
|
||||
BDF_BOARDROM, 0, HARDWARE_IGS_PGM,
|
||||
GBF_BIOS, 0,
|
||||
NULL, pgmRomInfo, pgmRomName, NULL, NULL, NULL, NULL,
|
||||
pgmInputInfo, pgmDIPInfo,
|
||||
NULL, NULL, NULL, NULL, 0x900,
|
||||
448, 224, 4, 3
|
||||
};
|
||||
"""
|
||||
|
||||
NON_BIOS_FIXTURE = """\
|
||||
static struct BurnRomInfo mslugRomDesc[] = {
|
||||
{ "201-p1.p1", 0x100000, 0x08d8daa5, BRF_ESS | BRF_PRG },
|
||||
{ "", 0, 0, 0 }
|
||||
};
|
||||
|
||||
STD_ROM_FN(mslug)
|
||||
|
||||
struct BurnDriver BurnDrvmslug = {
|
||||
"mslug", NULL, "neogeo", NULL, "1996",
|
||||
"Metal Slug\\0", NULL, "Nazca", "Neo Geo MVS",
|
||||
NULL, NULL, NULL, NULL,
|
||||
BDF_GAME_WORKING, 2, HARDWARE_PREFIX_CARTRIDGE | HARDWARE_SNK_NEOGEO,
|
||||
GBF_PLATFORM | GBF_HORSHOOT, 0,
|
||||
NULL, mslugRomInfo, mslugRomName, NULL, NULL, NULL, NULL,
|
||||
neogeoInputInfo, neogeoDIPInfo,
|
||||
NULL, NULL, NULL, NULL, 0x1000,
|
||||
304, 224, 4, 3
|
||||
};
|
||||
"""
|
||||
|
||||
|
||||
class TestFindBiosSets(unittest.TestCase):
|
||||
|
||||
def test_detects_neogeo(self) -> None:
|
||||
result = find_bios_sets(NEOGEO_FIXTURE, 'd_neogeo.cpp')
|
||||
self.assertIn('neogeo', result)
|
||||
self.assertEqual(result['neogeo']['source_file'], 'd_neogeo.cpp')
|
||||
|
||||
def test_detects_pgm(self) -> None:
|
||||
result = find_bios_sets(PGM_FIXTURE, 'd_pgm.cpp')
|
||||
self.assertIn('pgm', result)
|
||||
self.assertEqual(result['pgm']['source_file'], 'd_pgm.cpp')
|
||||
|
||||
def test_ignores_non_bios(self) -> None:
|
||||
result = find_bios_sets(NON_BIOS_FIXTURE, 'd_neogeo.cpp')
|
||||
self.assertEqual(result, {})
|
||||
|
||||
def test_source_line_positive(self) -> None:
|
||||
result = find_bios_sets(NEOGEO_FIXTURE, 'd_neogeo.cpp')
|
||||
self.assertGreater(result['neogeo']['source_line'], 0)
|
||||
|
||||
|
||||
class TestParseRomInfo(unittest.TestCase):
|
||||
|
||||
def test_neogeo_rom_count(self) -> None:
|
||||
roms = parse_rom_info(NEOGEO_FIXTURE, 'neogeo')
|
||||
self.assertEqual(len(roms), 5)
|
||||
|
||||
def test_sentinel_skipped(self) -> None:
|
||||
roms = parse_rom_info(NEOGEO_FIXTURE, 'neogeo')
|
||||
names = [r['name'] for r in roms]
|
||||
self.assertNotIn('', names)
|
||||
|
||||
def test_crc32_lowercase_hex(self) -> None:
|
||||
roms = parse_rom_info(NEOGEO_FIXTURE, 'neogeo')
|
||||
first = roms[0]
|
||||
self.assertEqual(first['crc32'], '9036d879')
|
||||
self.assertRegex(first['crc32'], r'^[0-9a-f]{8}$')
|
||||
|
||||
def test_no_sha1(self) -> None:
|
||||
roms = parse_rom_info(NEOGEO_FIXTURE, 'neogeo')
|
||||
for rom in roms:
|
||||
self.assertNotIn('sha1', rom)
|
||||
|
||||
def test_neogeo_first_rom(self) -> None:
|
||||
roms = parse_rom_info(NEOGEO_FIXTURE, 'neogeo')
|
||||
first = roms[0]
|
||||
self.assertEqual(first['name'], 'sp-s2.sp1')
|
||||
self.assertEqual(first['size'], 0x020000)
|
||||
self.assertEqual(first['crc32'], '9036d879')
|
||||
|
||||
def test_pgm_rom_count(self) -> None:
|
||||
roms = parse_rom_info(PGM_FIXTURE, 'pgm')
|
||||
self.assertEqual(len(roms), 3)
|
||||
|
||||
def test_pgm_bios_entry(self) -> None:
|
||||
roms = parse_rom_info(PGM_FIXTURE, 'pgm')
|
||||
bios = roms[2]
|
||||
self.assertEqual(bios['name'], 'pgm_p01s.rom')
|
||||
self.assertEqual(bios['crc32'], 'e42b166e')
|
||||
|
||||
def test_unknown_set_returns_empty(self) -> None:
|
||||
roms = parse_rom_info(NEOGEO_FIXTURE, 'nonexistent')
|
||||
self.assertEqual(roms, [])
|
||||
|
||||
|
||||
class TestParseSourceTree(unittest.TestCase):
|
||||
|
||||
def test_walks_drv_directory(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
drv_dir = Path(tmpdir) / 'src' / 'burn' / 'drv' / 'neogeo'
|
||||
drv_dir.mkdir(parents=True)
|
||||
(drv_dir / 'd_neogeo.cpp').write_text(NEOGEO_FIXTURE)
|
||||
|
||||
result = parse_fbneo_source_tree(tmpdir)
|
||||
self.assertIn('neogeo', result)
|
||||
self.assertEqual(len(result['neogeo']['roms']), 5)
|
||||
|
||||
def test_skips_non_cpp(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
drv_dir = Path(tmpdir) / 'src' / 'burn' / 'drv'
|
||||
drv_dir.mkdir(parents=True)
|
||||
(drv_dir / 'd_neogeo.h').write_text(NEOGEO_FIXTURE)
|
||||
|
||||
result = parse_fbneo_source_tree(tmpdir)
|
||||
self.assertEqual(result, {})
|
||||
|
||||
def test_missing_directory_returns_empty(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
result = parse_fbneo_source_tree(tmpdir)
|
||||
self.assertEqual(result, {})
|
||||
|
||||
def test_multiple_sets(self) -> None:
|
||||
combined = NEOGEO_FIXTURE + '\n' + PGM_FIXTURE
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
drv_dir = Path(tmpdir) / 'src' / 'burn' / 'drv'
|
||||
drv_dir.mkdir(parents=True)
|
||||
(drv_dir / 'd_combined.cpp').write_text(combined)
|
||||
|
||||
result = parse_fbneo_source_tree(tmpdir)
|
||||
self.assertIn('neogeo', result)
|
||||
self.assertIn('pgm', result)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
423
tests/test_hash_merge.py
Normal file
423
tests/test_hash_merge.py
Normal file
@@ -0,0 +1,423 @@
|
||||
"""Tests for the hash merge module."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from scripts.scraper._hash_merge import (
|
||||
compute_diff,
|
||||
merge_fbneo_profile,
|
||||
merge_mame_profile,
|
||||
)
|
||||
|
||||
|
||||
def _write_yaml(path: Path, data: dict) -> str:
|
||||
p = str(path)
|
||||
with open(p, 'w', encoding='utf-8') as f:
|
||||
yaml.dump(data, f, default_flow_style=False, sort_keys=False)
|
||||
return p
|
||||
|
||||
|
||||
def _write_json(path: Path, data: dict) -> str:
|
||||
p = str(path)
|
||||
with open(p, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f)
|
||||
return p
|
||||
|
||||
|
||||
def _make_mame_profile(**overrides: object) -> dict:
|
||||
base = {
|
||||
'emulator': 'MAME',
|
||||
'core_version': '0.285',
|
||||
'files': [
|
||||
{
|
||||
'name': 'neogeo.zip',
|
||||
'required': True,
|
||||
'category': 'bios_zip',
|
||||
'system': 'snk-neogeo-mvs',
|
||||
'source_ref': 'src/mame/neogeo/neogeo.cpp:2400',
|
||||
'contents': [
|
||||
{
|
||||
'name': 'sp-s2.sp1',
|
||||
'size': 131072,
|
||||
'crc32': 'oldcrc32',
|
||||
'description': 'Europe MVS (Ver. 2)',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
def _make_mame_hashes(**overrides: object) -> dict:
|
||||
base = {
|
||||
'source': 'mamedev/mame',
|
||||
'version': '0.286',
|
||||
'commit': 'abc123',
|
||||
'fetched_at': '2026-03-30T12:00:00Z',
|
||||
'bios_sets': {
|
||||
'neogeo': {
|
||||
'source_file': 'src/mame/neogeo/neogeo.cpp',
|
||||
'source_line': 2432,
|
||||
'roms': [
|
||||
{
|
||||
'name': 'sp-s2.sp1',
|
||||
'size': 131072,
|
||||
'crc32': '9036d879',
|
||||
'sha1': '4f834c55',
|
||||
'region': 'mainbios',
|
||||
'bios_label': 'euro',
|
||||
'bios_description': 'Europe MVS (Ver. 2)',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
}
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
def _make_fbneo_profile(**overrides: object) -> dict:
|
||||
base = {
|
||||
'emulator': 'FinalBurn Neo',
|
||||
'core_version': 'v1.0.0.02',
|
||||
'files': [
|
||||
{
|
||||
'name': 'sp-s2.sp1',
|
||||
'archive': 'neogeo.zip',
|
||||
'system': 'snk-neogeo-mvs',
|
||||
'required': True,
|
||||
'size': 131072,
|
||||
'crc32': 'oldcrc32',
|
||||
'source_ref': 'src/burn/drv/neogeo/d_neogeo.cpp:1605',
|
||||
},
|
||||
{
|
||||
'name': 'hiscore.dat',
|
||||
'required': False,
|
||||
},
|
||||
],
|
||||
}
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
def _make_fbneo_hashes(**overrides: object) -> dict:
|
||||
base = {
|
||||
'source': 'finalburnneo/FBNeo',
|
||||
'version': 'v1.0.0.03',
|
||||
'commit': 'def456',
|
||||
'fetched_at': '2026-03-30T12:00:00Z',
|
||||
'bios_sets': {
|
||||
'neogeo': {
|
||||
'source_file': 'src/burn/drv/neogeo/d_neogeo.cpp',
|
||||
'source_line': 1604,
|
||||
'roms': [
|
||||
{
|
||||
'name': 'sp-s2.sp1',
|
||||
'size': 131072,
|
||||
'crc32': '9036d879',
|
||||
'sha1': 'aabbccdd',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
}
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
class TestMameMerge(unittest.TestCase):
|
||||
"""Tests for merge_mame_profile."""
|
||||
|
||||
def test_merge_updates_contents(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_mame_hashes())
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
bios_files = [f for f in result['files'] if f.get('category') == 'bios_zip']
|
||||
self.assertEqual(len(bios_files), 1)
|
||||
contents = bios_files[0]['contents']
|
||||
self.assertEqual(contents[0]['crc32'], '9036d879')
|
||||
self.assertEqual(contents[0]['sha1'], '4f834c55')
|
||||
self.assertEqual(contents[0]['description'], 'Europe MVS (Ver. 2)')
|
||||
|
||||
def test_merge_preserves_manual_fields(self) -> None:
|
||||
profile = _make_mame_profile()
|
||||
profile['files'][0]['note'] = 'manually curated note'
|
||||
profile['files'][0]['system'] = 'snk-neogeo-mvs'
|
||||
profile['files'][0]['required'] = False
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', profile)
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_mame_hashes())
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
entry = [f for f in result['files'] if f.get('category') == 'bios_zip'][0]
|
||||
self.assertEqual(entry['note'], 'manually curated note')
|
||||
self.assertEqual(entry['system'], 'snk-neogeo-mvs')
|
||||
self.assertFalse(entry['required'])
|
||||
|
||||
def test_merge_adds_new_bios_set(self) -> None:
|
||||
hashes = _make_mame_hashes()
|
||||
hashes['bios_sets']['pgm'] = {
|
||||
'source_file': 'src/mame/igs/pgm.cpp',
|
||||
'source_line': 5515,
|
||||
'roms': [
|
||||
{'name': 'pgm_t01s.rom', 'size': 2097152, 'crc32': '1a7123a0'},
|
||||
],
|
||||
}
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
bios_files = [f for f in result['files'] if f.get('category') == 'bios_zip']
|
||||
names = {f['name'] for f in bios_files}
|
||||
self.assertIn('pgm.zip', names)
|
||||
|
||||
pgm = next(f for f in bios_files if f['name'] == 'pgm.zip')
|
||||
self.assertIsNone(pgm['system'])
|
||||
self.assertTrue(pgm['required'])
|
||||
self.assertEqual(pgm['category'], 'bios_zip')
|
||||
|
||||
def test_merge_preserves_non_bios_files(self) -> None:
|
||||
profile = _make_mame_profile()
|
||||
profile['files'].append({'name': 'hiscore.dat', 'required': False})
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', profile)
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_mame_hashes())
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
non_bios = [f for f in result['files'] if f.get('category') != 'bios_zip']
|
||||
self.assertEqual(len(non_bios), 1)
|
||||
self.assertEqual(non_bios[0]['name'], 'hiscore.dat')
|
||||
|
||||
def test_merge_keeps_unmatched_bios_set(self) -> None:
|
||||
"""Entries not in scraper scope stay untouched (no _upstream_removed)."""
|
||||
hashes = _make_mame_hashes()
|
||||
hashes['bios_sets'] = {} # nothing from scraper
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
bios_files = [f for f in result['files'] if f.get('category') == 'bios_zip']
|
||||
self.assertEqual(len(bios_files), 1)
|
||||
self.assertNotIn('_upstream_removed', bios_files[0])
|
||||
self.assertEqual(bios_files[0]['name'], 'neogeo.zip')
|
||||
|
||||
def test_merge_updates_core_version(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_mame_hashes())
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
self.assertEqual(result['core_version'], '0.286')
|
||||
|
||||
def test_merge_backup_created(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_mame_hashes())
|
||||
|
||||
merge_mame_profile(profile_path, hashes_path, write=True)
|
||||
|
||||
backup = p / 'mame.old.yml'
|
||||
self.assertTrue(backup.exists())
|
||||
|
||||
with open(backup, encoding='utf-8') as f:
|
||||
old = yaml.safe_load(f)
|
||||
self.assertEqual(old['core_version'], '0.285')
|
||||
|
||||
def test_merge_updates_source_ref(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_mame_hashes())
|
||||
|
||||
result = merge_mame_profile(profile_path, hashes_path)
|
||||
|
||||
entry = [f for f in result['files'] if f.get('category') == 'bios_zip'][0]
|
||||
self.assertEqual(entry['source_ref'], 'src/mame/neogeo/neogeo.cpp:2432')
|
||||
|
||||
|
||||
class TestFbneoMerge(unittest.TestCase):
|
||||
"""Tests for merge_fbneo_profile."""
|
||||
|
||||
def test_merge_updates_rom_entries(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', _make_fbneo_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_fbneo_hashes())
|
||||
|
||||
result = merge_fbneo_profile(profile_path, hashes_path)
|
||||
|
||||
archive_files = [f for f in result['files'] if 'archive' in f]
|
||||
self.assertEqual(len(archive_files), 1)
|
||||
self.assertEqual(archive_files[0]['crc32'], '9036d879')
|
||||
self.assertEqual(archive_files[0]['system'], 'snk-neogeo-mvs')
|
||||
|
||||
def test_merge_adds_new_roms(self) -> None:
|
||||
hashes = _make_fbneo_hashes()
|
||||
hashes['bios_sets']['neogeo']['roms'].append({
|
||||
'name': 'sp-s3.sp1',
|
||||
'size': 131072,
|
||||
'crc32': '91b64be3',
|
||||
})
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', _make_fbneo_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
result = merge_fbneo_profile(profile_path, hashes_path)
|
||||
|
||||
archive_files = [f for f in result['files'] if 'archive' in f]
|
||||
self.assertEqual(len(archive_files), 2)
|
||||
new_rom = next(f for f in archive_files if f['name'] == 'sp-s3.sp1')
|
||||
self.assertEqual(new_rom['archive'], 'neogeo.zip')
|
||||
self.assertTrue(new_rom['required'])
|
||||
|
||||
def test_merge_preserves_non_archive_files(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', _make_fbneo_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_fbneo_hashes())
|
||||
|
||||
result = merge_fbneo_profile(profile_path, hashes_path)
|
||||
|
||||
non_archive = [f for f in result['files'] if 'archive' not in f]
|
||||
self.assertEqual(len(non_archive), 1)
|
||||
self.assertEqual(non_archive[0]['name'], 'hiscore.dat')
|
||||
|
||||
def test_merge_keeps_unmatched_roms(self) -> None:
|
||||
"""Entries not in scraper scope stay untouched (no _upstream_removed)."""
|
||||
hashes = _make_fbneo_hashes()
|
||||
hashes['bios_sets'] = {}
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', _make_fbneo_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
result = merge_fbneo_profile(profile_path, hashes_path)
|
||||
|
||||
archive_files = [f for f in result['files'] if 'archive' in f]
|
||||
self.assertEqual(len(archive_files), 1)
|
||||
self.assertNotIn('_upstream_removed', archive_files[0])
|
||||
|
||||
def test_merge_updates_core_version(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', _make_fbneo_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', _make_fbneo_hashes())
|
||||
|
||||
result = merge_fbneo_profile(profile_path, hashes_path)
|
||||
|
||||
self.assertEqual(result['core_version'], 'v1.0.0.03')
|
||||
|
||||
|
||||
class TestDiff(unittest.TestCase):
|
||||
"""Tests for compute_diff."""
|
||||
|
||||
def test_diff_mame_detects_changes(self) -> None:
|
||||
hashes = _make_mame_hashes()
|
||||
hashes['bios_sets']['pgm'] = {
|
||||
'source_file': 'src/mame/igs/pgm.cpp',
|
||||
'source_line': 5515,
|
||||
'roms': [
|
||||
{'name': 'pgm_t01s.rom', 'size': 2097152, 'crc32': '1a7123a0'},
|
||||
],
|
||||
}
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
diff = compute_diff(profile_path, hashes_path, mode='mame')
|
||||
|
||||
self.assertIn('pgm', diff['added'])
|
||||
self.assertIn('neogeo', diff['updated'])
|
||||
self.assertEqual(len(diff['removed']), 0)
|
||||
self.assertEqual(diff['unchanged'], 0)
|
||||
|
||||
def test_diff_mame_out_of_scope(self) -> None:
|
||||
"""Items in profile but not in scraper output = out of scope, not removed."""
|
||||
hashes = _make_mame_hashes()
|
||||
hashes['bios_sets'] = {}
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'mame.yml', _make_mame_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
diff = compute_diff(profile_path, hashes_path, mode='mame')
|
||||
|
||||
self.assertEqual(diff['removed'], [])
|
||||
self.assertEqual(diff['out_of_scope'], 1)
|
||||
self.assertEqual(len(diff['added']), 0)
|
||||
|
||||
def test_diff_fbneo_detects_changes(self) -> None:
|
||||
hashes = _make_fbneo_hashes()
|
||||
hashes['bios_sets']['neogeo']['roms'].append({
|
||||
'name': 'sp-s3.sp1',
|
||||
'size': 131072,
|
||||
'crc32': '91b64be3',
|
||||
})
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', _make_fbneo_profile())
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
diff = compute_diff(profile_path, hashes_path, mode='fbneo')
|
||||
|
||||
self.assertIn('neogeo.zip:sp-s3.sp1', diff['added'])
|
||||
self.assertIn('neogeo.zip:sp-s2.sp1', diff['updated'])
|
||||
self.assertEqual(len(diff['removed']), 0)
|
||||
|
||||
def test_diff_fbneo_unchanged(self) -> None:
|
||||
profile = _make_fbneo_profile()
|
||||
profile['files'][0]['crc32'] = '9036d879'
|
||||
profile['files'][0]['size'] = 131072
|
||||
|
||||
hashes = _make_fbneo_hashes()
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td)
|
||||
profile_path = _write_yaml(p / 'fbneo.yml', profile)
|
||||
hashes_path = _write_json(p / 'hashes.json', hashes)
|
||||
|
||||
diff = compute_diff(profile_path, hashes_path, mode='fbneo')
|
||||
|
||||
self.assertEqual(diff['unchanged'], 1)
|
||||
self.assertEqual(len(diff['added']), 0)
|
||||
self.assertEqual(len(diff['updated']), 0)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
244
tests/test_mame_parser.py
Normal file
244
tests/test_mame_parser.py
Normal file
@@ -0,0 +1,244 @@
|
||||
"""Tests for MAME source code parser."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from scripts.scraper.mame_parser import (
|
||||
find_bios_root_sets,
|
||||
parse_mame_source_tree,
|
||||
parse_rom_block,
|
||||
)
|
||||
|
||||
# Standard GAME macro with MACHINE_IS_BIOS_ROOT, multiple ROM entries, BIOS variants
|
||||
NEOGEO_FIXTURE = """\
|
||||
ROM_START( neogeo )
|
||||
ROM_REGION( 0x100000, "mainbios", 0 )
|
||||
|
||||
ROM_SYSTEM_BIOS( 0, "euro", "Europe MVS (Ver. 2)" )
|
||||
ROMX_LOAD( "sp-s2.sp1", 0x00000, 0x020000, CRC(9036d879) SHA1(4f5ed7105b7128794654ce82b51723e16e389543), ROM_BIOS(0) )
|
||||
|
||||
ROM_SYSTEM_BIOS( 1, "japan", "Japan MVS (Ver. 3)" )
|
||||
ROMX_LOAD( "vs-bios.rom", 0x00000, 0x020000, CRC(f0e8f27d) SHA1(ecf01bf6b3d6c7e4e0aae01e51e3ed4c0e1d5c2e), ROM_BIOS(1) )
|
||||
|
||||
ROM_REGION( 0x10000, "audiocpu", 0 )
|
||||
ROM_LOAD( "sm1.sm1", 0x00000, 0x20000, CRC(94416d67) SHA1(42f9d7ddd6c0931fd64226a60dc73602b2819571) )
|
||||
ROM_END
|
||||
|
||||
GAME( 1990, neogeo, 0, neogeo_noslot, neogeo, neogeo_state, init_neogeo, ROT0, "SNK", "Neo Geo", MACHINE_IS_BIOS_ROOT )
|
||||
"""
|
||||
|
||||
# COMP macro with MACHINE_IS_BIOS_ROOT
|
||||
DEVICE_FIXTURE = """\
|
||||
ROM_START( bbcb )
|
||||
ROM_REGION( 0x40000, "maincpu", 0 )
|
||||
ROM_LOAD( "basic2.rom", 0x00000, 0x4000, CRC(a1b6a0e9) SHA1(6a0b9b8b7c3b3b9e6b7e8d0f2e7a6e7b8c9a0b1c) )
|
||||
ROM_END
|
||||
|
||||
COMP( 1981, bbcb, 0, 0, bbcb, bbcb, bbc_state, init_bbc, "Acorn", "BBC Micro Model B", MACHINE_IS_BIOS_ROOT )
|
||||
"""
|
||||
|
||||
# ROM_LOAD with NO_DUMP (should be skipped)
|
||||
NODUMP_FIXTURE = """\
|
||||
ROM_START( testnd )
|
||||
ROM_REGION( 0x10000, "maincpu", 0 )
|
||||
ROM_LOAD( "good.rom", 0x00000, 0x4000, CRC(aabbccdd) SHA1(1122334455667788990011223344556677889900) )
|
||||
ROM_LOAD( "missing.rom", 0x04000, 0x4000, NO_DUMP )
|
||||
ROM_END
|
||||
|
||||
GAME( 2000, testnd, 0, testnd, testnd, test_state, init_test, ROT0, "Test", "Test ND", MACHINE_IS_BIOS_ROOT )
|
||||
"""
|
||||
|
||||
# ROM_LOAD with BAD_DUMP
|
||||
BADDUMP_FIXTURE = """\
|
||||
ROM_START( testbd )
|
||||
ROM_REGION( 0x10000, "maincpu", 0 )
|
||||
ROM_LOAD( "badrom.bin", 0x00000, 0x4000, BAD_DUMP CRC(deadbeef) SHA1(0123456789abcdef0123456789abcdef01234567) )
|
||||
ROM_END
|
||||
|
||||
GAME( 2000, testbd, 0, testbd, testbd, test_state, init_test, ROT0, "Test", "Test BD", MACHINE_IS_BIOS_ROOT )
|
||||
"""
|
||||
|
||||
# CONS macro with ROM_LOAD16_WORD
|
||||
CONS_FIXTURE = """\
|
||||
ROM_START( megadriv )
|
||||
ROM_REGION( 0x400000, "maincpu", 0 )
|
||||
ROM_LOAD16_WORD( "epr-6209.ic7", 0x000000, 0x004000, CRC(cafebabe) SHA1(abcdef0123456789abcdef0123456789abcdef01) )
|
||||
ROM_END
|
||||
|
||||
CONS( 1988, megadriv, 0, 0, megadriv, megadriv, md_state, init_megadriv, "Sega", "Mega Drive", MACHINE_IS_BIOS_ROOT )
|
||||
"""
|
||||
|
||||
# GAME macro WITHOUT MACHINE_IS_BIOS_ROOT (should NOT be detected)
|
||||
NON_BIOS_FIXTURE = """\
|
||||
ROM_START( pacman )
|
||||
ROM_REGION( 0x10000, "maincpu", 0 )
|
||||
ROM_LOAD( "pacman.6e", 0x0000, 0x1000, CRC(c1e6ab10) SHA1(e87e059c5be45753f7e9f33dff851f16d6751181) )
|
||||
ROM_END
|
||||
|
||||
GAME( 1980, pacman, 0, pacman, pacman, pacman_state, init_pacman, ROT90, "Namco", "Pac-Man", MACHINE_SUPPORTS_SAVE )
|
||||
"""
|
||||
|
||||
|
||||
class TestFindBiosRootSets(unittest.TestCase):
|
||||
"""Tests for find_bios_root_sets."""
|
||||
|
||||
def test_detects_neogeo_from_game_macro(self) -> None:
|
||||
result = find_bios_root_sets(NEOGEO_FIXTURE, 'src/mame/snk/neogeo.cpp')
|
||||
self.assertIn('neogeo', result)
|
||||
self.assertEqual(result['neogeo']['source_file'], 'src/mame/snk/neogeo.cpp')
|
||||
self.assertIsInstance(result['neogeo']['source_line'], int)
|
||||
|
||||
def test_detects_from_comp_macro(self) -> None:
|
||||
result = find_bios_root_sets(DEVICE_FIXTURE, 'src/mame/acorn/bbc.cpp')
|
||||
self.assertIn('bbcb', result)
|
||||
|
||||
def test_detects_from_cons_macro(self) -> None:
|
||||
result = find_bios_root_sets(CONS_FIXTURE, 'src/mame/sega/megadriv.cpp')
|
||||
self.assertIn('megadriv', result)
|
||||
|
||||
def test_ignores_non_bios_games(self) -> None:
|
||||
result = find_bios_root_sets(NON_BIOS_FIXTURE, 'src/mame/pacman/pacman.cpp')
|
||||
self.assertEqual(result, {})
|
||||
|
||||
def test_detects_from_nodump_fixture(self) -> None:
|
||||
result = find_bios_root_sets(NODUMP_FIXTURE, 'test.cpp')
|
||||
self.assertIn('testnd', result)
|
||||
|
||||
def test_detects_from_baddump_fixture(self) -> None:
|
||||
result = find_bios_root_sets(BADDUMP_FIXTURE, 'test.cpp')
|
||||
self.assertIn('testbd', result)
|
||||
|
||||
|
||||
class TestParseRomBlock(unittest.TestCase):
|
||||
"""Tests for parse_rom_block."""
|
||||
|
||||
def test_extracts_rom_names(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'neogeo')
|
||||
names = [r['name'] for r in roms]
|
||||
self.assertIn('sp-s2.sp1', names)
|
||||
self.assertIn('vs-bios.rom', names)
|
||||
self.assertIn('sm1.sm1', names)
|
||||
|
||||
def test_extracts_crc32_and_sha1(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'neogeo')
|
||||
sp_s2 = next(r for r in roms if r['name'] == 'sp-s2.sp1')
|
||||
self.assertEqual(sp_s2['crc32'], '9036d879')
|
||||
self.assertEqual(sp_s2['sha1'], '4f5ed7105b7128794654ce82b51723e16e389543')
|
||||
|
||||
def test_extracts_size(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'neogeo')
|
||||
sp_s2 = next(r for r in roms if r['name'] == 'sp-s2.sp1')
|
||||
self.assertEqual(sp_s2['size'], 0x020000)
|
||||
|
||||
def test_extracts_bios_metadata(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'neogeo')
|
||||
sp_s2 = next(r for r in roms if r['name'] == 'sp-s2.sp1')
|
||||
self.assertEqual(sp_s2['bios_index'], 0)
|
||||
self.assertEqual(sp_s2['bios_label'], 'euro')
|
||||
self.assertEqual(sp_s2['bios_description'], 'Europe MVS (Ver. 2)')
|
||||
|
||||
def test_non_bios_rom_has_no_bios_fields(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'neogeo')
|
||||
sm1 = next(r for r in roms if r['name'] == 'sm1.sm1')
|
||||
self.assertNotIn('bios_index', sm1)
|
||||
self.assertNotIn('bios_label', sm1)
|
||||
|
||||
def test_skips_no_dump(self) -> None:
|
||||
roms = parse_rom_block(NODUMP_FIXTURE, 'testnd')
|
||||
names = [r['name'] for r in roms]
|
||||
self.assertIn('good.rom', names)
|
||||
self.assertNotIn('missing.rom', names)
|
||||
|
||||
def test_includes_bad_dump_with_flag(self) -> None:
|
||||
roms = parse_rom_block(BADDUMP_FIXTURE, 'testbd')
|
||||
self.assertEqual(len(roms), 1)
|
||||
self.assertEqual(roms[0]['name'], 'badrom.bin')
|
||||
self.assertTrue(roms[0]['bad_dump'])
|
||||
self.assertEqual(roms[0]['crc32'], 'deadbeef')
|
||||
self.assertEqual(roms[0]['sha1'], '0123456789abcdef0123456789abcdef01234567')
|
||||
|
||||
def test_handles_rom_load16_word(self) -> None:
|
||||
roms = parse_rom_block(CONS_FIXTURE, 'megadriv')
|
||||
self.assertEqual(len(roms), 1)
|
||||
self.assertEqual(roms[0]['name'], 'epr-6209.ic7')
|
||||
self.assertEqual(roms[0]['crc32'], 'cafebabe')
|
||||
|
||||
def test_tracks_rom_region(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'neogeo')
|
||||
sp_s2 = next(r for r in roms if r['name'] == 'sp-s2.sp1')
|
||||
sm1 = next(r for r in roms if r['name'] == 'sm1.sm1')
|
||||
self.assertEqual(sp_s2['region'], 'mainbios')
|
||||
self.assertEqual(sm1['region'], 'audiocpu')
|
||||
|
||||
def test_returns_empty_for_unknown_set(self) -> None:
|
||||
roms = parse_rom_block(NEOGEO_FIXTURE, 'nonexistent')
|
||||
self.assertEqual(roms, [])
|
||||
|
||||
def test_good_rom_not_flagged_bad_dump(self) -> None:
|
||||
roms = parse_rom_block(NODUMP_FIXTURE, 'testnd')
|
||||
good = next(r for r in roms if r['name'] == 'good.rom')
|
||||
self.assertFalse(good['bad_dump'])
|
||||
|
||||
def test_crc32_sha1_lowercase(self) -> None:
|
||||
fixture = """\
|
||||
ROM_START( upper )
|
||||
ROM_REGION( 0x10000, "maincpu", 0 )
|
||||
ROM_LOAD( "test.rom", 0x00000, 0x4000, CRC(AABBCCDD) SHA1(AABBCCDDEEFF00112233AABBCCDDEEFF00112233) )
|
||||
ROM_END
|
||||
"""
|
||||
roms = parse_rom_block(fixture, 'upper')
|
||||
self.assertEqual(roms[0]['crc32'], 'aabbccdd')
|
||||
self.assertEqual(roms[0]['sha1'], 'aabbccddeeff00112233aabbccddeeff00112233')
|
||||
|
||||
|
||||
class TestParseMameSourceTree(unittest.TestCase):
|
||||
"""Tests for parse_mame_source_tree."""
|
||||
|
||||
def test_walks_source_tree(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
mame_dir = os.path.join(tmpdir, 'src', 'mame', 'snk')
|
||||
os.makedirs(mame_dir)
|
||||
filepath = os.path.join(mame_dir, 'neogeo.cpp')
|
||||
with open(filepath, 'w') as f:
|
||||
f.write(NEOGEO_FIXTURE)
|
||||
|
||||
results = parse_mame_source_tree(tmpdir)
|
||||
self.assertIn('neogeo', results)
|
||||
self.assertEqual(len(results['neogeo']['roms']), 3)
|
||||
self.assertEqual(
|
||||
results['neogeo']['source_file'],
|
||||
'src/mame/snk/neogeo.cpp',
|
||||
)
|
||||
|
||||
def test_ignores_non_source_files(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
mame_dir = os.path.join(tmpdir, 'src', 'mame')
|
||||
os.makedirs(mame_dir)
|
||||
# Write a .txt file that should be ignored
|
||||
with open(os.path.join(mame_dir, 'notes.txt'), 'w') as f:
|
||||
f.write(NEOGEO_FIXTURE)
|
||||
|
||||
results = parse_mame_source_tree(tmpdir)
|
||||
self.assertEqual(results, {})
|
||||
|
||||
def test_scans_devices_dir(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
dev_dir = os.path.join(tmpdir, 'src', 'devices', 'bus')
|
||||
os.makedirs(dev_dir)
|
||||
with open(os.path.join(dev_dir, 'test.cpp'), 'w') as f:
|
||||
f.write(DEVICE_FIXTURE)
|
||||
|
||||
results = parse_mame_source_tree(tmpdir)
|
||||
self.assertIn('bbcb', results)
|
||||
|
||||
def test_empty_tree(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
results = parse_mame_source_tree(tmpdir)
|
||||
self.assertEqual(results, {})
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
352
wiki/adding-a-platform.md
Normal file
352
wiki/adding-a-platform.md
Normal file
@@ -0,0 +1,352 @@
|
||||
# Adding a platform
|
||||
|
||||
How to add support for a new retrogaming platform (e.g. a frontend like Batocera,
|
||||
a manager like EmuDeck, or a firmware database like BizHawk).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before starting, gather the following from the upstream project:
|
||||
|
||||
- **Where does it define BIOS requirements?** Each platform has a canonical source:
|
||||
a DAT file, a JSON fixture, an XML manifest, a Bash script, a C# database, etc.
|
||||
- **What verification mode does it use?** Read the platform source code to determine
|
||||
how it checks BIOS files at runtime: file existence only (`existence`), MD5 hash
|
||||
matching (`md5`), SHA1 matching (`sha1`), or a combination of size and hash.
|
||||
- **What is the base destination?** The directory name where BIOS files are placed
|
||||
on disk (e.g. `system` for RetroArch, `bios` for Batocera, `Firmware` for BizHawk).
|
||||
- **What hash type does it store?** The primary hash format used in the platform's
|
||||
own data files (SHA1 for RetroArch/BizHawk, MD5 for Batocera/Recalbox/EmuDeck).
|
||||
|
||||
## Step 1: Create the scraper
|
||||
|
||||
Scrapers live in `scripts/scraper/` and are auto-discovered by the plugin system.
|
||||
Any file matching `*_scraper.py` in that directory is loaded at import time via
|
||||
`pkgutil.iter_modules`. No registration step is needed beyond placing the file.
|
||||
|
||||
### Module contract
|
||||
|
||||
The module must export two names:
|
||||
|
||||
```python
|
||||
PLATFORM_NAME = "myplatform" # matches the key in _registry.yml
|
||||
|
||||
class Scraper(BaseScraper):
|
||||
...
|
||||
```
|
||||
|
||||
### Inheriting BaseScraper
|
||||
|
||||
`BaseScraper` provides:
|
||||
|
||||
- `_fetch_raw() -> str` - HTTP GET with 50 MB response limit, cached after first call.
|
||||
Uses `urllib.request` with a `retrobios-scraper/1.0` user-agent and 30s timeout.
|
||||
- `compare_with_config(config) -> ChangeSet` - diffs scraped requirements against
|
||||
an existing platform YAML, returning added/removed/modified entries.
|
||||
- `test_connection() -> bool` - checks if the source URL is reachable.
|
||||
|
||||
Two abstract methods must be implemented:
|
||||
|
||||
```python
|
||||
def fetch_requirements(self) -> list[BiosRequirement]:
|
||||
"""Parse the upstream source and return one BiosRequirement per file."""
|
||||
|
||||
def validate_format(self, raw_data: str) -> bool:
|
||||
"""Return False if the upstream format has changed unexpectedly."""
|
||||
```
|
||||
|
||||
### BiosRequirement fields
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `name` | `str` | Filename as the platform expects it |
|
||||
| `system` | `str` | Retrobios system ID (e.g. `sony-playstation`) |
|
||||
| `sha1` | `str \| None` | SHA1 hash if available |
|
||||
| `md5` | `str \| None` | MD5 hash if available |
|
||||
| `crc32` | `str \| None` | CRC32 if available |
|
||||
| `size` | `int \| None` | Expected file size in bytes |
|
||||
| `destination` | `str` | Relative path within the BIOS directory |
|
||||
| `required` | `bool` | Whether the platform considers this file mandatory |
|
||||
| `zipped_file` | `str \| None` | If set, the hash refers to a ROM inside a ZIP |
|
||||
| `native_id` | `str \| None` | Original system name before normalization |
|
||||
|
||||
### System ID mapping
|
||||
|
||||
Every scraper needs a mapping from the platform's native system identifiers to
|
||||
retrobios system IDs. Define this as a module-level dict:
|
||||
|
||||
```python
|
||||
SLUG_MAP: dict[str, str] = {
|
||||
"psx": "sony-playstation",
|
||||
"saturn": "sega-saturn",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Warn on unmapped slugs so new systems are surfaced during scraping.
|
||||
|
||||
### generate_platform_yaml (optional)
|
||||
|
||||
If the scraper defines a `generate_platform_yaml() -> dict` method, the shared
|
||||
CLI will use it instead of the generic YAML builder. This allows the scraper to
|
||||
include platform metadata (homepage, version, inherits, cores list) in the output.
|
||||
|
||||
### CLI entry point
|
||||
|
||||
Add a `main()` function and `__main__` guard:
|
||||
|
||||
```python
|
||||
def main():
|
||||
from scripts.scraper.base_scraper import scraper_cli
|
||||
scraper_cli(Scraper, "Scrape MyPlatform BIOS requirements")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
`scraper_cli` provides `--dry-run`, `--json`, and `--output` flags automatically.
|
||||
|
||||
### Test the scraper
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.myplatform_scraper --dry-run
|
||||
```
|
||||
|
||||
This fetches from upstream and prints a summary without writing anything.
|
||||
|
||||
## Step 2: Register the platform
|
||||
|
||||
Add an entry to `platforms/_registry.yml` under the `platforms:` key.
|
||||
|
||||
### Required fields
|
||||
|
||||
```yaml
|
||||
platforms:
|
||||
myplatform:
|
||||
config: myplatform.yml # platform YAML filename in platforms/
|
||||
status: active # active or archived
|
||||
scraper: myplatform # matches PLATFORM_NAME in the scraper
|
||||
source_url: https://... # upstream data URL
|
||||
source_format: json # json, xml, clrmamepro_dat, python_dict, bash_script+csv, csharp_firmware_database, github_component_manifests
|
||||
hash_type: md5 # primary hash in the upstream data
|
||||
verification_mode: md5 # how the platform checks files: existence, md5, sha1
|
||||
base_destination: bios # where files go on disk
|
||||
cores: # which emulator profiles apply
|
||||
- core_a
|
||||
- core_b
|
||||
```
|
||||
|
||||
The `cores` field determines which emulator profiles are resolved for this platform.
|
||||
Three strategies exist:
|
||||
|
||||
- **Explicit list**: `cores: [beetle_psx, dolphin, ...]` - match by profile key name.
|
||||
Used by Batocera, Recalbox, RetroBat, RomM.
|
||||
- **all_libretro**: `cores: all_libretro` - include every profile with `type: libretro`
|
||||
or `type: standalone + libretro`. Used by RetroArch, Lakka, RetroPie.
|
||||
- **Omitted**: fallback to system ID intersection. Used by EmuDeck.
|
||||
|
||||
### Optional fields
|
||||
|
||||
```yaml
|
||||
logo: https://... # SVG or PNG for UI/docs
|
||||
schedule: weekly # scrape frequency: weekly, monthly, or null
|
||||
inherits_from: retroarch # inherit systems/cores from another platform
|
||||
case_insensitive_fs: true # if the platform runs on case-insensitive filesystems
|
||||
target_scraper: myplatform_targets # hardware target scraper name
|
||||
target_source: https://... # target data source URL
|
||||
install:
|
||||
detect: # auto-detection for install.py
|
||||
- os: linux
|
||||
method: config_file
|
||||
config: $HOME/.config/myplatform/config.ini
|
||||
parse_key: bios_directory
|
||||
```
|
||||
|
||||
### Inheritance
|
||||
|
||||
If the new platform inherits from an existing one (e.g. Lakka inherits RetroArch),
|
||||
set `inherits_from` in the registry AND add `inherits: retroarch` in the platform
|
||||
YAML itself. `load_platform_config()` reads the `inherits:` field from the YAML to
|
||||
merge parent systems and shared groups into the child. The child YAML only needs to
|
||||
declare overrides.
|
||||
|
||||
## Step 3: Generate the platform YAML
|
||||
|
||||
Run the scraper with `--output` to produce the initial platform configuration:
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.myplatform_scraper --output platforms/myplatform.yml
|
||||
```
|
||||
|
||||
If a file already exists at the output path, the CLI preserves fields that the
|
||||
scraper does not generate (e.g. `data_directories`, manually added metadata).
|
||||
Only the `systems` section is replaced.
|
||||
|
||||
Verify the result:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --platform myplatform
|
||||
python scripts/verify.py --platform myplatform --verbose
|
||||
```
|
||||
|
||||
## Step 4: Add verification logic
|
||||
|
||||
Check how the platform verifies BIOS files by reading its source code.
|
||||
The `verification_mode` in the registry tells `verify.py` which strategy to use:
|
||||
|
||||
| Mode | Behavior | Example platforms |
|
||||
|------|----------|-------------------|
|
||||
| `existence` | File must exist, no hash check | RetroArch, Lakka, RetroPie |
|
||||
| `md5` | MD5 must match the declared hash | Batocera, Recalbox, RetroBat, EmuDeck, RetroDECK |
|
||||
| `sha1` | SHA1 must match | BizHawk |
|
||||
|
||||
If the platform has unique verification behavior (e.g. Batocera's `checkInsideZip`,
|
||||
Recalbox's multi-hash comma-separated MD5, RomM's size + any-hash), add the logic
|
||||
to `verify.py` in the platform-specific verification path.
|
||||
|
||||
Read the platform's source code to understand its exact verification behavior before writing any logic. Batocera's `checkInsideZip` uses `casefold()` for case-insensitive matching. Recalbox supports comma-separated MD5 lists. RomM checks file size before hashing. These details matter: the project replicates native behavior, not an approximation of it.
|
||||
|
||||
## Step 5: Create an exporter (optional)
|
||||
|
||||
Exporters convert truth data back to the platform's native format. They live in
|
||||
`scripts/exporter/` and follow the same auto-discovery pattern (`*_exporter.py`).
|
||||
|
||||
### Module contract
|
||||
|
||||
The module must export an `Exporter` class inheriting `BaseExporter`:
|
||||
|
||||
```python
|
||||
from scripts.exporter.base_exporter import BaseExporter
|
||||
|
||||
class Exporter(BaseExporter):
|
||||
@staticmethod
|
||||
def platform_name() -> str:
|
||||
return "myplatform"
|
||||
|
||||
def export(self, truth_data: dict, output_path: str, scraped_data: dict | None = None) -> None:
|
||||
# Write truth_data in the platform's native format to output_path
|
||||
...
|
||||
|
||||
def validate(self, truth_data: dict, output_path: str) -> list[str]:
|
||||
# Return a list of issues (empty = valid)
|
||||
...
|
||||
```
|
||||
|
||||
`BaseExporter` provides helper methods:
|
||||
|
||||
- `_is_pattern(name)` - True if the filename contains wildcards or placeholders.
|
||||
- `_dest(fe)` - resolve destination path from a file entry dict.
|
||||
- `_display_name(sys_id, scraped_sys)` - convert a system slug to a display name.
|
||||
|
||||
### Round-trip validation
|
||||
|
||||
The exporter enables a scrape-export-compare workflow:
|
||||
|
||||
```bash
|
||||
# Scrape upstream
|
||||
python -m scripts.scraper.myplatform_scraper --output /tmp/scraped.yml
|
||||
# Export truth data
|
||||
python scripts/export_native.py --platform myplatform --output /tmp/exported.json
|
||||
# Compare exported file with upstream
|
||||
diff /tmp/scraped.yml /tmp/exported.json
|
||||
```
|
||||
|
||||
## Step 6: Create a target scraper (optional)
|
||||
|
||||
Target scrapers determine which emulator cores are available on each hardware
|
||||
target (e.g. which RetroArch cores exist for Switch, RPi4, or x86_64).
|
||||
They live in `scripts/scraper/targets/` and are auto-discovered by filename
|
||||
(`*_targets_scraper.py`).
|
||||
|
||||
### Module contract
|
||||
|
||||
```python
|
||||
from scripts.scraper.targets import BaseTargetScraper
|
||||
|
||||
PLATFORM_NAME = "myplatform_targets"
|
||||
|
||||
class Scraper(BaseTargetScraper):
|
||||
def fetch_targets(self) -> dict:
|
||||
return {
|
||||
"platform": "myplatform",
|
||||
"source": "https://...",
|
||||
"scraped_at": "2026-03-30T00:00:00Z",
|
||||
"targets": {
|
||||
"x86_64": {
|
||||
"architecture": "x86_64",
|
||||
"cores": ["beetle_psx", "dolphin", "..."],
|
||||
},
|
||||
"rpi4": {
|
||||
"architecture": "aarch64",
|
||||
"cores": ["pcsx_rearmed", "mgba", "..."],
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Add `target_scraper` and `target_source` to the platform's registry entry.
|
||||
|
||||
### Overrides
|
||||
|
||||
Hardware-specific overrides go in `platforms/targets/_overrides.yml`. This file
|
||||
defines aliases (e.g. `arm64` maps to `aarch64`) and per-platform core
|
||||
additions/removals that the scraper cannot determine automatically.
|
||||
|
||||
### Single-target platforms
|
||||
|
||||
For platforms that only run on one target (e.g. RetroBat on Windows, RomM in the
|
||||
browser), create a static YAML file in `platforms/targets/` instead of a scraper.
|
||||
Set `target_scraper: null` in the registry.
|
||||
|
||||
## Step 7: Add install detection (optional)
|
||||
|
||||
The `install` section in `_registry.yml` tells `install.py` how to detect
|
||||
the platform on the user's machine and locate its BIOS directory.
|
||||
|
||||
Three detection methods are available:
|
||||
|
||||
| Method | Description | Fields |
|
||||
|--------|-------------|--------|
|
||||
| `config_file` | Parse a key from a config file | `config`, `parse_key`, optionally `bios_subdir` |
|
||||
| `path_exists` | Check if a directory exists | `path`, optionally `bios_path` |
|
||||
| `file_exists` | Check if a file exists | `file`, optionally `bios_path` |
|
||||
|
||||
Each entry is scoped to an OS (`linux`, `darwin`, `windows`). Multiple entries
|
||||
per OS are tried in order.
|
||||
|
||||
## Step 8: Validate the full pipeline
|
||||
|
||||
After all pieces are in place, run the full pipeline:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline
|
||||
```
|
||||
|
||||
This executes in sequence:
|
||||
|
||||
1. `generate_db.py` - rebuild `database.json` from `bios/`
|
||||
2. `refresh_data_dirs.py` - update data directories
|
||||
3. `verify.py --all` - verify all platforms including the new one
|
||||
4. `generate_pack.py --all` - build ZIP packs
|
||||
5. Consistency check - verify counts match between verify and pack
|
||||
|
||||
Check the output for:
|
||||
|
||||
- The new platform appears in verify results
|
||||
- No unexpected CRITICAL or WARNING entries
|
||||
- Pack generation succeeds and includes the expected files
|
||||
- Consistency check passes (verify file counts match pack file counts)
|
||||
|
||||
Verification is not optional. A platform that passes `pipeline.py` today may break tomorrow if upstream changes its data format. Run the full pipeline on every change, even if the modification seems trivial. The consistency check (verify counts must match pack counts) catches subtle issues where files resolve during verification but fail during pack generation, or vice versa.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Scraper file in `scripts/scraper/<name>_scraper.py`
|
||||
- [ ] `PLATFORM_NAME` and `Scraper` class exported
|
||||
- [ ] `fetch_requirements()` and `validate_format()` implemented
|
||||
- [ ] System ID mapping covers all upstream systems
|
||||
- [ ] Entry added to `platforms/_registry.yml`
|
||||
- [ ] Platform YAML generated and verified
|
||||
- [ ] `python scripts/pipeline.py --offline` passes
|
||||
- [ ] Exporter in `scripts/exporter/<name>_exporter.py` (if applicable)
|
||||
- [ ] Target scraper in `scripts/scraper/targets/<name>_targets_scraper.py` (if applicable)
|
||||
- [ ] Install detection entries in `_registry.yml` (if applicable)
|
||||
423
wiki/adding-a-scraper.md
Normal file
423
wiki/adding-a-scraper.md
Normal file
@@ -0,0 +1,423 @@
|
||||
# Adding a scraper
|
||||
|
||||
How to create or modify a scraper for fetching BIOS requirements from upstream
|
||||
platform sources.
|
||||
|
||||
## Scraper architecture
|
||||
|
||||
### Plugin discovery
|
||||
|
||||
Scrapers are discovered automatically at import time. The `scripts/scraper/__init__.py`
|
||||
module uses `pkgutil.iter_modules` to scan for files matching `*_scraper.py` in
|
||||
the scraper directory. Each module must export:
|
||||
|
||||
- `PLATFORM_NAME: str` - the platform identifier (matches `_registry.yml`)
|
||||
- `Scraper: class` - a subclass of `BaseScraper`
|
||||
|
||||
No registration code is needed. Drop a file, export the two names, and it works.
|
||||
|
||||
```python
|
||||
# scripts/scraper/__init__.py (simplified)
|
||||
for finder, name, ispkg in pkgutil.iter_modules([package_dir]):
|
||||
if not name.endswith("_scraper"):
|
||||
continue
|
||||
module = importlib.import_module(f".{name}", package=__package__)
|
||||
# looks for PLATFORM_NAME and Scraper attributes
|
||||
```
|
||||
|
||||
### BaseScraper ABC
|
||||
|
||||
`BaseScraper` (`scripts/scraper/base_scraper.py`) provides the foundation:
|
||||
|
||||
```
|
||||
BaseScraper
|
||||
__init__(url: str)
|
||||
_fetch_raw() -> str # HTTP GET, cached, 50 MB limit
|
||||
fetch_requirements() -> list # abstract: parse upstream data
|
||||
validate_format(raw_data) -> bool # abstract: detect format changes
|
||||
compare_with_config(config) -> ChangeSet # diff against existing YAML
|
||||
test_connection() -> bool # reachability check
|
||||
```
|
||||
|
||||
`_fetch_raw()` handles HTTP with `urllib.request`, sets a `retrobios-scraper/1.0`
|
||||
user-agent, enforces a 30-second timeout, and reads the response in 64 KB chunks
|
||||
with a 50 MB hard limit to prevent memory exhaustion. The result is cached on the
|
||||
instance after the first call.
|
||||
|
||||
### BiosRequirement
|
||||
|
||||
A dataclass representing a single BIOS file entry:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class BiosRequirement:
|
||||
name: str # filename
|
||||
system: str # retrobios system ID
|
||||
sha1: str | None = None
|
||||
md5: str | None = None
|
||||
crc32: str | None = None
|
||||
size: int | None = None
|
||||
destination: str = "" # relative path in BIOS dir
|
||||
required: bool = True
|
||||
zipped_file: str | None = None # ROM name inside a ZIP
|
||||
native_id: str | None = None # original system name
|
||||
```
|
||||
|
||||
### ChangeSet
|
||||
|
||||
Returned by `compare_with_config()`. Contains:
|
||||
|
||||
- `added: list[BiosRequirement]` - new files not in the existing config
|
||||
- `removed: list[BiosRequirement]` - files present in config but gone upstream
|
||||
- `modified: list[tuple[BiosRequirement, BiosRequirement]]` - hash changes
|
||||
- `has_changes: bool` - True if any of the above are non-empty
|
||||
- `summary() -> str` - human-readable summary (e.g. `+3 added, ~1 modified`)
|
||||
|
||||
### scraper_cli
|
||||
|
||||
`scraper_cli(scraper_class, description)` provides a shared CLI with three modes:
|
||||
|
||||
| Flag | Behavior |
|
||||
|------|----------|
|
||||
| `--dry-run` | Fetch and print a summary grouped by system |
|
||||
| `--json` | Output all requirements as JSON |
|
||||
| `--output FILE` | Write platform YAML to FILE |
|
||||
|
||||
When `--output` targets an existing file, the CLI preserves keys not generated by
|
||||
the scraper (e.g. `data_directories`, manual additions). Only the `systems` section
|
||||
is replaced. If the scraper defines `generate_platform_yaml()`, that method is used
|
||||
instead of the generic YAML builder.
|
||||
|
||||
### Helper functions
|
||||
|
||||
Two additional functions in `base_scraper.py`:
|
||||
|
||||
- `fetch_github_latest_version(repo)` - fetches the latest release tag via GitHub API.
|
||||
- `fetch_github_latest_tag(repo, prefix)` - fetches the most recent tag matching
|
||||
an optional prefix.
|
||||
|
||||
## Creating a BIOS scraper
|
||||
|
||||
### Minimal example
|
||||
|
||||
Based on the RomM scraper pattern (JSON source, flat structure):
|
||||
|
||||
```python
|
||||
"""Scraper for MyPlatform BIOS requirements."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
|
||||
try:
|
||||
from .base_scraper import BaseScraper, BiosRequirement
|
||||
except ImportError:
|
||||
from base_scraper import BaseScraper, BiosRequirement
|
||||
|
||||
PLATFORM_NAME = "myplatform"
|
||||
|
||||
SOURCE_URL = "https://raw.githubusercontent.com/org/repo/main/bios_list.json"
|
||||
|
||||
SLUG_MAP: dict[str, str] = {
|
||||
"psx": "sony-playstation",
|
||||
"saturn": "sega-saturn",
|
||||
}
|
||||
|
||||
|
||||
class Scraper(BaseScraper):
|
||||
def __init__(self, url: str = SOURCE_URL):
|
||||
super().__init__(url=url)
|
||||
|
||||
def fetch_requirements(self) -> list[BiosRequirement]:
|
||||
raw = self._fetch_raw()
|
||||
if not self.validate_format(raw):
|
||||
raise ValueError("Format validation failed")
|
||||
|
||||
data = json.loads(raw)
|
||||
requirements = []
|
||||
for entry in data:
|
||||
system = SLUG_MAP.get(entry["platform"])
|
||||
if not system:
|
||||
print(f"Warning: unmapped '{entry['platform']}'", file=sys.stderr)
|
||||
continue
|
||||
requirements.append(BiosRequirement(
|
||||
name=entry["filename"],
|
||||
system=system,
|
||||
md5=entry.get("md5"),
|
||||
sha1=entry.get("sha1"),
|
||||
size=entry.get("size"),
|
||||
destination=entry["filename"],
|
||||
required=entry.get("required", True),
|
||||
))
|
||||
return requirements
|
||||
|
||||
def validate_format(self, raw_data: str) -> bool:
|
||||
try:
|
||||
data = json.loads(raw_data)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return False
|
||||
return isinstance(data, list) and len(data) > 0
|
||||
|
||||
|
||||
def main():
|
||||
from scripts.scraper.base_scraper import scraper_cli
|
||||
scraper_cli(Scraper, "Scrape MyPlatform BIOS requirements")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
### Parsing different upstream formats
|
||||
|
||||
Each platform stores its BIOS requirements differently. The scraper's job is to
|
||||
normalize them into `BiosRequirement` entries.
|
||||
|
||||
| Format | Example | Parsing approach |
|
||||
|--------|---------|-----------------|
|
||||
| JSON | RomM `known_bios_files.json` | `json.loads()`, iterate keys |
|
||||
| XML | Recalbox `es_bios.xml` | `xml.etree.ElementTree`, xpath or iter |
|
||||
| clrmamepro DAT | RetroArch `System.dat` | Use `dat_parser` module (see below) |
|
||||
| Python dict | Batocera `batocera-systems` | `ast.literal_eval` or regex extraction |
|
||||
| Bash script | EmuDeck `checkBIOS.sh` | Line-by-line regex parsing |
|
||||
| C# source | BizHawk `FirmwareDatabase.cs` | Regex for method calls and string literals |
|
||||
| C source | MAME/FBNeo drivers | Use `mame_parser` or `fbneo_parser` (see below) |
|
||||
| JSON (GitHub API) | RetroDECK component manifests | `json.loads()` per manifest file |
|
||||
|
||||
### System ID mapping
|
||||
|
||||
Every scraper maintains a `SLUG_MAP` (or equivalent) that translates the platform's
|
||||
native system identifiers to retrobios system IDs. The retrobios system ID format
|
||||
is `manufacturer-console` in lowercase with hyphens (e.g. `sony-playstation`,
|
||||
`sega-mega-drive`, `nintendo-gba`).
|
||||
|
||||
When a native slug has no mapping, print a warning to stderr. This surfaces new
|
||||
systems added upstream that need to be mapped.
|
||||
|
||||
System ID consistency matters for cross-platform operations. The same console must use the same ID across all scrapers and platforms. Before inventing a new ID, check existing profiles and platform YAMLs for precedent. The canonical format is `manufacturer-console` in lowercase with hyphens (e.g., `sony-playstation`, `sega-mega-drive`). The `SYSTEM_ALIASES` dict in `common.py` maps common variations to canonical IDs.
|
||||
|
||||
### Hash normalization
|
||||
|
||||
- Normalize all hashes to lowercase hex strings.
|
||||
- Handle missing hashes gracefully (set to `None`, not empty string).
|
||||
- Some platforms provide multiple hash types per entry. Populate whichever fields
|
||||
are available.
|
||||
- Batocera uses 29-character truncated MD5 hashes in some entries. The resolution
|
||||
layer handles prefix matching, but the scraper should store the hash as-is.
|
||||
|
||||
Scraped data reflects what the upstream declares, which may not match reality. The scraper's job is faithful transcription of upstream data, not correction. Corrections happen in the emulator profiles (source-verified) and in `_shared.yml` (curated). If a scraper detects an obviously wrong hash or filename, log a warning but still include the upstream value. The divergence will surface during truth diffing.
|
||||
|
||||
## Creating a target scraper
|
||||
|
||||
Target scrapers determine which emulator cores are available on each hardware
|
||||
target. They live in `scripts/scraper/targets/` and follow the same auto-discovery
|
||||
pattern (`*_targets_scraper.py`).
|
||||
|
||||
### BaseTargetScraper ABC
|
||||
|
||||
```
|
||||
BaseTargetScraper
|
||||
__init__(url: str)
|
||||
fetch_targets() -> dict # abstract: return target data
|
||||
write_output(data, path) # write YAML to disk
|
||||
```
|
||||
|
||||
### Output format
|
||||
|
||||
`fetch_targets()` must return a dict with this structure:
|
||||
|
||||
```python
|
||||
{
|
||||
"platform": "myplatform",
|
||||
"source": "https://...",
|
||||
"scraped_at": "2026-03-30T12:00:00Z",
|
||||
"targets": {
|
||||
"x86_64": {
|
||||
"architecture": "x86_64",
|
||||
"cores": ["beetle_psx", "dolphin", "snes9x"],
|
||||
},
|
||||
"rpi4": {
|
||||
"architecture": "aarch64",
|
||||
"cores": ["pcsx_rearmed", "mgba"],
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
The `targets` dict maps target names to their available cores. Core names must
|
||||
match the names used in emulator profile `cores:` fields for the target filtering
|
||||
pipeline to work correctly.
|
||||
|
||||
### Overrides
|
||||
|
||||
`platforms/targets/_overrides.yml` provides post-scrape adjustments:
|
||||
|
||||
- **aliases**: map alternate target names to canonical ones (e.g. `arm64` -> `aarch64`)
|
||||
- **add_cores**: cores present on a target but not detected by the scraper
|
||||
- **remove_cores**: cores detected by the scraper but not actually functional
|
||||
|
||||
Overrides are applied by `load_target_config()` in `common.py` after loading
|
||||
the scraped data. The scraper itself does not need to handle overrides.
|
||||
|
||||
### Module contract
|
||||
|
||||
```python
|
||||
from scripts.scraper.targets import BaseTargetScraper
|
||||
|
||||
PLATFORM_NAME = "myplatform_targets"
|
||||
|
||||
class Scraper(BaseTargetScraper):
|
||||
def __init__(self):
|
||||
super().__init__(url="https://...")
|
||||
|
||||
def fetch_targets(self) -> dict:
|
||||
# Fetch and parse target data
|
||||
...
|
||||
```
|
||||
|
||||
Register the target scraper in `_registry.yml`:
|
||||
|
||||
```yaml
|
||||
myplatform:
|
||||
target_scraper: myplatform_targets
|
||||
target_source: https://...
|
||||
```
|
||||
|
||||
### Existing target scrapers
|
||||
|
||||
| Scraper | Source | Approach |
|
||||
|---------|--------|----------|
|
||||
| `retroarch_targets` | libretro buildbot nightly | Scrape directory listings for each target arch |
|
||||
| `batocera_targets` | Config.in + es_systems.yml | Cross-reference kernel config with system definitions |
|
||||
| `emudeck_targets` | EmuScripts + RetroArch cores | GitHub API for script availability per OS |
|
||||
| `retropie_targets` | scriptmodules + rp_module_flags | Parse Bash scriptmodules for platform flags |
|
||||
|
||||
## Parser modules
|
||||
|
||||
Shared parsers in `scripts/scraper/` handle formats used by multiple scrapers
|
||||
or formats complex enough to warrant dedicated parsing logic.
|
||||
|
||||
### dat_parser
|
||||
|
||||
Parses clrmamepro DAT format as used in RetroArch's `System.dat`:
|
||||
|
||||
```
|
||||
game (
|
||||
name "System"
|
||||
comment "Platform Name"
|
||||
rom ( name filename size 12345 crc ABCD1234 md5 ... sha1 ... )
|
||||
)
|
||||
```
|
||||
|
||||
Produces `DatRom` dataclass instances with `name`, `size`, `crc32`, `md5`, `sha1`,
|
||||
and `system` fields. The `libretro_scraper` uses this parser.
|
||||
|
||||
### mame_parser
|
||||
|
||||
Parses MAME C source files to extract BIOS root sets. Handles:
|
||||
|
||||
- Machine declaration macros: `GAME`, `SYST`, `COMP`, `CONS`
|
||||
- `MACHINE_IS_BIOS_ROOT` flag detection
|
||||
- `ROM_START`/`ROM_END` blocks
|
||||
- `ROM_LOAD` variants and `ROM_REGION` declarations
|
||||
- `ROM_SYSTEM_BIOS` entries
|
||||
- `NO_DUMP` filtering and `BAD_DUMP` flagging
|
||||
|
||||
Used by `mame_hash_scraper` to auto-fetch BIOS hashes from MAME driver sources
|
||||
for each tagged MAME version.
|
||||
|
||||
### fbneo_parser
|
||||
|
||||
Parses FBNeo C source files:
|
||||
|
||||
- `BurnRomInfo` structs (static ROM arrays with name, size, CRC)
|
||||
- `BurnDriver` structs (driver registration with `BDF_BOARDROM` flag)
|
||||
- BIOS set identification via the boardrom flag
|
||||
|
||||
Used by `fbneo_hash_scraper` to extract BIOS ROM definitions.
|
||||
|
||||
### _hash_merge
|
||||
|
||||
Text-based YAML patching that merges fetched hash data into emulator profiles
|
||||
while preserving formatting. Two strategies:
|
||||
|
||||
- **MAME**: updates `bios_zip` entries with `contents` lists (name, size, CRC32)
|
||||
- **FBNeo**: updates individual ROM entries grouped by `archive` field
|
||||
|
||||
The merge preserves fields the hash data does not generate (system, note, required)
|
||||
and leaves entries not present in the hash data untouched. Uses text-level YAML
|
||||
manipulation rather than load-dump to maintain human-readable formatting.
|
||||
|
||||
## Testing
|
||||
|
||||
### Development workflow
|
||||
|
||||
1. **Start with --dry-run**. It's helpful to preview before writing output:
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.myplatform_scraper --dry-run
|
||||
```
|
||||
|
||||
2. **Check JSON output** for data quality:
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.myplatform_scraper --json | python -m json.tool | head -50
|
||||
```
|
||||
|
||||
3. **Compare with existing YAML** if updating a scraper:
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.myplatform_scraper --output /tmp/test.yml
|
||||
diff platforms/myplatform.yml /tmp/test.yml
|
||||
```
|
||||
|
||||
4. **Run verification** after generating:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --platform myplatform
|
||||
python scripts/verify.py --platform myplatform --verbose
|
||||
```
|
||||
|
||||
5. **Run the full pipeline** before committing:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline
|
||||
```
|
||||
|
||||
### Round-trip testing
|
||||
|
||||
If an exporter exists for the platform, validate the scrape-export-compare cycle:
|
||||
|
||||
```bash
|
||||
# Scrape upstream -> platform YAML
|
||||
python -m scripts.scraper.myplatform_scraper --output /tmp/scraped.yml
|
||||
|
||||
# Export truth data -> native format
|
||||
python scripts/export_native.py --platform myplatform --output /tmp/exported.json
|
||||
|
||||
# Compare
|
||||
diff <(python -m scripts.scraper.myplatform_scraper --json | python -m json.tool) \
|
||||
/tmp/exported.json
|
||||
```
|
||||
|
||||
### Common issues
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Unmapped slug warnings | New system added upstream | Add mapping to `SLUG_MAP` |
|
||||
| Empty requirements list | Upstream format changed | Check `validate_format()`, update parser |
|
||||
| Hash mismatch in verify | Upstream updated hashes | Re-scrape and regenerate platform YAML |
|
||||
| Scraper hangs | URL unreachable, no timeout | `_fetch_raw()` has 30s timeout; check URL |
|
||||
| `Response exceeds 50 MB` | Upstream file grew | Investigate; may need chunked parsing |
|
||||
| `validate_format` fails | Upstream restructured | Update both `validate_format` and `fetch_requirements` |
|
||||
|
||||
### E2E tests
|
||||
|
||||
The project's test suite (`tests/test_e2e.py`) covers scraper integration at the
|
||||
pipeline level. When adding a new scraper, verify that the full pipeline passes:
|
||||
|
||||
```bash
|
||||
python -m unittest tests.test_e2e
|
||||
python scripts/pipeline.py --offline
|
||||
```
|
||||
|
||||
Both must pass before the scraper is considered complete.
|
||||
280
wiki/advanced-usage.md
Normal file
280
wiki/advanced-usage.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# Advanced Usage
|
||||
|
||||
Fine-grained control over pack generation, hardware filtering, truth analysis, and verification.
|
||||
|
||||
## Custom Packs
|
||||
|
||||
### Build from hash
|
||||
|
||||
Look up a single MD5 in the database:
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --from-md5 d8f1206299c48946e6ec5ef96d014eaa
|
||||
```
|
||||
|
||||
Build a pack containing only files matching hashes from a list (one MD5 per line, `#` for comments):
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --platform batocera --from-md5-file missing.txt
|
||||
```
|
||||
|
||||
This is useful when a platform reports missing files and you want to generate a targeted pack
|
||||
rather than re-downloading the full archive.
|
||||
|
||||
### Split packs
|
||||
|
||||
Generate one ZIP per system instead of a single monolithic pack:
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --platform retroarch --split
|
||||
```
|
||||
|
||||
Group the split ZIPs by manufacturer (Sony, Nintendo, Sega, etc.):
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --platform retroarch --split --group-by manufacturer
|
||||
```
|
||||
|
||||
### System-specific packs
|
||||
|
||||
Extract only the files for a single system within a platform:
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --platform retroarch --system sony-playstation
|
||||
```
|
||||
|
||||
### Required only
|
||||
|
||||
Exclude optional files from the pack:
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --platform batocera --required-only
|
||||
```
|
||||
|
||||
What counts as "required" depends on the platform YAML. For existence-mode platforms
|
||||
(RetroArch), the distinction comes from the `.info` file's `required` field.
|
||||
For MD5-mode platforms (Batocera), all declared files are treated as required unless
|
||||
explicitly marked optional.
|
||||
|
||||
|
||||
## Hardware Target Filtering
|
||||
|
||||
### What targets are
|
||||
|
||||
A target represents a hardware architecture where a platform runs. Each architecture
|
||||
has a different set of available cores. For example, the RetroArch Switch target
|
||||
has fewer cores than the x86_64 target because some cores are not ported to ARM.
|
||||
|
||||
Target data is scraped from upstream sources (buildbot nightly listings, board configs,
|
||||
scriptmodules) and stored in `platforms/targets/<platform>.yml`.
|
||||
|
||||
### Usage
|
||||
|
||||
Filter packs or verification to only include systems reachable by cores available
|
||||
on the target hardware:
|
||||
|
||||
```bash
|
||||
python scripts/generate_pack.py --platform retroarch --target switch
|
||||
python scripts/generate_pack.py --all --target x86_64
|
||||
python scripts/verify.py --platform batocera --target rpi4
|
||||
```
|
||||
|
||||
When combined with `--all`, platforms that define the target are filtered. Platforms
|
||||
without a target file for that name are left unfiltered (no information to exclude anything).
|
||||
Platforms that have target data but not the requested target are skipped with an INFO message.
|
||||
|
||||
### How it works
|
||||
|
||||
The filtering pipeline has three stages:
|
||||
|
||||
1. **`load_target_config()`** reads `platforms/targets/<platform>.yml` and returns
|
||||
the set of cores available on the target. Aliases from `_overrides.yml` are resolved
|
||||
(e.g., `--target rpi4` may match `bcm2711` in the target file).
|
||||
|
||||
2. **`resolve_platform_cores()`** determines which emulator profiles are relevant
|
||||
for the platform, then intersects the result with the target's core set. The
|
||||
intersection uses a reverse index built from each profile's `cores:` field, so
|
||||
that upstream names (e.g., `mednafen_psx` on the buildbot) map to profile keys
|
||||
(e.g., `beetle_psx`).
|
||||
|
||||
3. **`filter_systems_by_target()`** removes platform systems where every core that
|
||||
emulates them is absent from the target. Systems with no core information are kept
|
||||
(benefit of the doubt). System ID normalization strips manufacturer prefixes and
|
||||
separators so that `xbox` matches `microsoft-xbox`.
|
||||
|
||||
### List available targets
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --platform retroarch --list-targets
|
||||
```
|
||||
|
||||
### Overrides
|
||||
|
||||
`platforms/targets/_overrides.yml` provides two mechanisms:
|
||||
|
||||
- **Aliases**: map user-facing names to internal target IDs
|
||||
(e.g., `rpi4` -> `bcm2711`).
|
||||
- **add/remove cores**: patch the scraped core list for a specific target
|
||||
without overwriting the entire file. Useful when a core is known to work
|
||||
but is not listed on the buildbot, or vice versa.
|
||||
|
||||
### Single-target platforms
|
||||
|
||||
Platforms with only one target (e.g., RetroBat with `windows`, RomM with `browser`)
|
||||
treat `--target <their-only-target>` as a no-op: the output is identical to running
|
||||
without `--target`.
|
||||
|
||||
|
||||
## Truth Generation and Diffing
|
||||
|
||||
### What truth is
|
||||
|
||||
Truth data is ground truth generated from emulator profiles. It represents what each
|
||||
core actually needs based on source code analysis, independent of what platform
|
||||
scrapers declare. The purpose is gap analysis: finding files that platforms miss
|
||||
or declare incorrectly.
|
||||
|
||||
### Generate truth
|
||||
|
||||
Build truth YAMLs from emulator profiles for a platform or all platforms:
|
||||
|
||||
```bash
|
||||
python scripts/generate_truth.py --platform retroarch
|
||||
python scripts/generate_truth.py --all --output-dir dist/truth/
|
||||
```
|
||||
|
||||
Each truth YAML lists every system with its files, hashes, and the emulator profiles
|
||||
that reference them. The output mirrors the platform YAML structure so the two can
|
||||
be diffed directly.
|
||||
|
||||
### Diff truth vs scraped
|
||||
|
||||
Find divergences between generated truth and scraped platform data:
|
||||
|
||||
```bash
|
||||
python scripts/diff_truth.py --platform retroarch
|
||||
python scripts/diff_truth.py --all
|
||||
```
|
||||
|
||||
The diff reports:
|
||||
|
||||
- Files present in truth but absent from the platform YAML (undeclared).
|
||||
- Files present in the platform YAML but absent from truth (orphaned or from cores
|
||||
not profiled yet).
|
||||
- Hash mismatches between truth and platform data.
|
||||
|
||||
### Export to native formats
|
||||
|
||||
Convert truth data to the native format each platform consumes:
|
||||
|
||||
```bash
|
||||
python scripts/export_native.py --platform batocera # Python dict (batocera-systems)
|
||||
python scripts/export_native.py --platform recalbox # XML (es_bios.xml)
|
||||
python scripts/export_native.py --all --output-dir dist/upstream/
|
||||
```
|
||||
|
||||
This allows submitting corrections upstream in the format maintainers expect.
|
||||
|
||||
|
||||
## Emulator-Level Verification
|
||||
|
||||
### Per-emulator checks
|
||||
|
||||
Verify files against a single emulator's ground truth (size, hashes, crypto):
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --emulator handy
|
||||
python scripts/verify.py --emulator handy --verbose
|
||||
```
|
||||
|
||||
Default output shows aggregate results per file: the core name and which checks apply.
|
||||
With `--verbose`, each file expands to one line per core with the exact validation
|
||||
parameters and source code reference:
|
||||
|
||||
```
|
||||
lynxboot.img
|
||||
handy validates size=512 crc32=0x0d973c9d [src/handy/system.h:45]
|
||||
```
|
||||
|
||||
### Per-system checks
|
||||
|
||||
Aggregate verification across all cores that emulate a system:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --system atari-lynx
|
||||
```
|
||||
|
||||
### Standalone mode
|
||||
|
||||
Some cores have both libretro and standalone modes with different file requirements.
|
||||
Filter to standalone-only:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --emulator dolphin --standalone
|
||||
```
|
||||
|
||||
### Ground truth in verbose output
|
||||
|
||||
The verbose report includes a coverage footer:
|
||||
|
||||
```
|
||||
Ground truth: 142/160 files have emulator validation (88%)
|
||||
```
|
||||
|
||||
This indicates how many files in the platform can be cross-checked against source-verified
|
||||
emulator profiles. Files without ground truth rely solely on platform-level verification.
|
||||
JSON output (`--json`) always includes the full per-emulator detail regardless of verbosity.
|
||||
|
||||
|
||||
## Offline Workflow
|
||||
|
||||
### Full offline pipeline
|
||||
|
||||
Run the entire pipeline without network access:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline
|
||||
```
|
||||
|
||||
This skips data directory refresh, MAME/FBNeo hash fetch, and buildbot staleness checks.
|
||||
All other steps (database generation, verification, pack building, consistency check,
|
||||
README, site generation) run normally using cached data.
|
||||
|
||||
### Partial runs
|
||||
|
||||
Skip pack generation when you only need verification results:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline --skip-packs
|
||||
```
|
||||
|
||||
Skip documentation generation:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline --skip-docs
|
||||
```
|
||||
|
||||
### Truth pipeline
|
||||
|
||||
Include truth generation and diffing in the pipeline:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline --with-truth
|
||||
```
|
||||
|
||||
Include truth + native format export:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline --with-export
|
||||
```
|
||||
|
||||
### Combining flags
|
||||
|
||||
Flags compose freely:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline --skip-docs --with-truth --target switch
|
||||
```
|
||||
|
||||
This runs: database generation, verification (filtered to Switch cores), truth generation
|
||||
and diff, consistency check. Packs and docs are skipped, no network access.
|
||||
@@ -6,16 +6,22 @@
|
||||
bios/ BIOS and firmware files, organized by Manufacturer/Console/
|
||||
Manufacturer/Console/ canonical files (one per unique content)
|
||||
.variants/ alternate versions (different hash, same purpose)
|
||||
emulators/ one YAML profile per core (285 profiles)
|
||||
emulators/ one YAML profile per core/engine
|
||||
platforms/ one YAML config per platform (scraped from upstream)
|
||||
_shared.yml shared file groups across platforms
|
||||
_registry.yml platform metadata (logos, scrapers, status)
|
||||
_registry.yml platform metadata (logos, scrapers, status, install config)
|
||||
_data_dirs.yml data directory definitions (Dolphin Sys, PPSSPP...)
|
||||
targets/ hardware target configs + _overrides.yml
|
||||
scripts/ all tooling (Python, pyyaml only dependency)
|
||||
scraper/ upstream scrapers (libretro, batocera, recalbox...)
|
||||
scraper/targets/ hardware target scrapers (retroarch, batocera, emudeck, retropie)
|
||||
exporter/ native format exporters (batocera, recalbox, emudeck...)
|
||||
install/ JSON install manifests per platform
|
||||
targets/ JSON target manifests per platform (cores per architecture)
|
||||
data/ cached data directories (not BIOS, fetched at build)
|
||||
schemas/ JSON schemas for validation
|
||||
tests/ E2E test suite with synthetic fixtures
|
||||
_mame_clones.json MAME parent/clone set mappings
|
||||
dist/ generated packs (gitignored)
|
||||
.cache/ hash cache and large file downloads (gitignored)
|
||||
```
|
||||
@@ -28,11 +34,38 @@ Upstream sources Scrapers parse generate_db.py scans
|
||||
batocera-systems builds database.json
|
||||
es_bios.xml (recalbox) (SHA1 primary key,
|
||||
core-info .info files indexes: by_md5, by_name,
|
||||
by_crc32, by_path_suffix)
|
||||
FirmwareDatabase.cs by_crc32, by_path_suffix)
|
||||
MAME/FBNeo source
|
||||
|
||||
emulators/*.yml verify.py checks generate_pack.py resolves
|
||||
source-verified platform-native files by hash, builds ZIP
|
||||
from code verification packs per platform
|
||||
|
||||
truth.py generates diff_truth.py export_native.py
|
||||
ground truth from compares truth vs exports to native formats
|
||||
emulator profiles scraped platform (DAT, XML, JSON, Bash)
|
||||
```
|
||||
|
||||
Pipeline runs all steps in sequence: DB, data dirs, MAME/FBNeo hashes,
|
||||
verify, packs, install manifests, target manifests, consistency check,
|
||||
README, site. See [tools](tools.md) for the full pipeline reference.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[generate_db] --> B[refresh_data_dirs]
|
||||
B --> C[MAME/FBNeo hashes]
|
||||
C --> D[verify --all]
|
||||
D --> E[generate_pack --all]
|
||||
E --> F[install manifests]
|
||||
F --> G[target manifests]
|
||||
G --> H[consistency check]
|
||||
H --> I[generate_readme]
|
||||
I --> J[generate_site]
|
||||
|
||||
style A fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style D fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style E fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style J fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
```
|
||||
|
||||
## Three layers of data
|
||||
@@ -46,12 +79,39 @@ emulators/*.yml verify.py checks generate_pack.py resolves
|
||||
The pack combines platform baseline (layer 1) with core requirements (layer 3).
|
||||
Neither too much (no files from unused cores) nor too few (no missing files for active cores).
|
||||
|
||||
The emulator's source code serves as ground truth for what files are needed,
|
||||
what names they use, and what validation the emulator performs. Platform YAML
|
||||
configs are scraped from upstream and are generally accurate, though they can
|
||||
occasionally have gaps or stale entries. The emulator profiles complement the
|
||||
platform data by documenting what the code actually loads. When the two disagree,
|
||||
the profile takes precedence for pack generation: files the code needs are included
|
||||
even if the platform does not declare them. Files the platform declares but no
|
||||
profile references are kept as well (flagged during cross-reference), since the
|
||||
upstream may cover cases not yet profiled.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
PY[Platform YAML<br/>scraped from upstream] --> PG[Pack generation]
|
||||
EP[Emulator profiles<br/>source-verified] --> PG
|
||||
SH[_shared.yml<br/>curated shared files] --> PY
|
||||
SH --> EP
|
||||
PG --> ZIP[ZIP pack per platform]
|
||||
|
||||
style PY fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style EP fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style SH fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style PG fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style ZIP fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
```
|
||||
|
||||
## Pack grouping
|
||||
|
||||
Platforms that produce identical packs are grouped automatically.
|
||||
RetroArch and Lakka share the same files and `base_destination` (`system/`),
|
||||
so they produce one combined pack (`RetroArch_Lakka_BIOS_Pack.zip`).
|
||||
RetroPie uses `BIOS/` as base path, so it gets a separate pack.
|
||||
With `--target`, the fingerprint includes target cores so platforms
|
||||
with different hardware filters get separate packs.
|
||||
|
||||
## Storage tiers
|
||||
|
||||
@@ -99,6 +159,46 @@ If none exists, the platform version is kept.
|
||||
| RPG Maker/ScummVM | excluded from dedup (NODEDUP) to preserve directory structure |
|
||||
| `strip_components` in data dirs | flattens cache prefix to match expected path |
|
||||
| case-insensitive dedup | prevents `font.rom` + `FONT.ROM` conflicts on Windows/macOS |
|
||||
| frozen snapshot cores | `.info` may reflect current version while code is pinned to an old one. Only the frozen source at the pinned tag is reliable (e.g. desmume2015, mame2003) |
|
||||
|
||||
### File resolution chain
|
||||
|
||||
`resolve_local_file` in `common.py` tries each strategy in order, returning the
|
||||
first match. Used by both `verify.py` and `generate_pack.py`.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
START([resolve_local_file]) --> S0{path_suffix<br/>exact match?}
|
||||
S0 -- yes --> EXACT([exact])
|
||||
S0 -- no --> S1{SHA1<br/>exact match?}
|
||||
S1 -- yes --> EXACT
|
||||
S1 -- no --> S2{MD5 direct<br/>or truncated?}
|
||||
S2 -- yes --> MD5([md5_exact])
|
||||
S2 -- no --> S3{name + aliases<br/>no MD5?}
|
||||
S3 -- yes --> EXACT
|
||||
S3 -- no --> S4{name + aliases<br/>md5_composite /<br/>direct MD5?}
|
||||
S4 -- match --> EXACT
|
||||
S4 -- name only --> HM([hash_mismatch])
|
||||
S4 -- no --> S5{zippedFile<br/>inner ROM MD5?}
|
||||
S5 -- yes --> ZE([zip_exact])
|
||||
S5 -- no --> S6{MAME clone<br/>map lookup?}
|
||||
S6 -- yes --> MC([mame_clone])
|
||||
S6 -- no --> S7{data_dir<br/>cache scan?}
|
||||
S7 -- yes --> DD([data_dir])
|
||||
S7 -- no --> S8{agnostic<br/>fallback?}
|
||||
S8 -- yes --> AG([agnostic_fallback])
|
||||
S8 -- no --> NF([not_found])
|
||||
|
||||
style START fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style EXACT fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style MD5 fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style HM fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style ZE fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style MC fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style DD fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style AG fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
style NF fill:#2d333b,stroke:#adbac7,color:#adbac7
|
||||
```
|
||||
|
||||
## Platform inheritance
|
||||
|
||||
@@ -112,17 +212,36 @@ Core resolution (`resolve_platform_cores`) uses three strategies:
|
||||
- `cores: [list]` - include only named profiles
|
||||
- `cores:` absent - fallback to system ID intersection between platform and profiles
|
||||
|
||||
## Hardware target filtering
|
||||
|
||||
`--target TARGET` filters packs and verification by hardware (e.g. `switch`, `rpi4`, `x86_64`).
|
||||
Target configs are in `platforms/targets/`. Overrides in `_overrides.yml` add aliases and
|
||||
adjust core lists per target. `filter_systems_by_target` excludes systems whose cores are
|
||||
not available on the target. Without `--target`, all systems are included.
|
||||
|
||||
## MAME clone map
|
||||
|
||||
`_mame_clones.json` at repo root maps MAME clone ROM names to their canonical parent.
|
||||
When a clone ZIP was deduplicated, `resolve_local_file` uses this map to find the canonical file.
|
||||
|
||||
## Install manifests
|
||||
|
||||
`generate_pack.py --manifest` produces JSON manifests in `install/` for each platform.
|
||||
These contain file lists with SHA1 hashes, platform detection config, and standalone copy
|
||||
instructions. `install/targets/` contains per-architecture core availability.
|
||||
The cross-platform installer (`install.py`) uses these manifests to auto-detect the
|
||||
user's platform, filter files by hardware target, and download with SHA1 verification.
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_e2e.py` contains 75 end-to-end tests with synthetic fixtures.
|
||||
Covers: file resolution, verification, severity, cross-reference, aliases,
|
||||
inheritance, shared groups, data dirs, storage tiers, HLE, launchers,
|
||||
platform grouping, core resolution (3 strategies + alias exclusion).
|
||||
4 test files with synthetic fixtures:
|
||||
|
||||
| File | Coverage |
|
||||
|------|----------|
|
||||
| `test_e2e.py` | file resolution, verification, severity, cross-reference, aliases, inheritance, shared groups, data dirs, storage tiers, HLE, launchers, platform grouping, core resolution, target filtering, truth/diff, exporters |
|
||||
| `test_mame_parser.py` | BIOS root set detection, ROM block parsing, macro expansion |
|
||||
| `test_fbneo_parser.py` | BIOS set detection, ROM info parsing |
|
||||
| `test_hash_merge.py` | MAME/FBNeo YAML merge, diff detection |
|
||||
|
||||
```bash
|
||||
python -m unittest tests.test_e2e -v
|
||||
@@ -132,7 +251,8 @@ python -m unittest tests.test_e2e -v
|
||||
|
||||
| Workflow | File | Trigger | Role |
|
||||
|----------|------|---------|------|
|
||||
| Build & Release | `build.yml` | `workflow_dispatch` (manual) | restore large files, build packs, deploy site, create GitHub release |
|
||||
| Build & Release | `build.yml` | `workflow_dispatch` (manual) | restore large files, build packs, create GitHub release |
|
||||
| Deploy Site | `deploy-site.yml` | push to main (platforms, emulators, wiki, scripts) + manual | generate site, build with MkDocs, deploy to GitHub Pages |
|
||||
| PR Validation | `validate.yml` | pull request on `bios/`/`platforms/` | validate BIOS hashes, schema check, run tests, auto-label PR |
|
||||
| Weekly Sync | `watch.yml` | cron (Monday 6 AM UTC) + manual | scrape upstream sources, detect changes, create update PR |
|
||||
|
||||
|
||||
112
wiki/faq.md
Normal file
112
wiki/faq.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# FAQ - RetroBIOS
|
||||
|
||||
## My game shows a black screen
|
||||
|
||||
Most likely a missing or incorrect BIOS file. Run verification for your platform:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --platform retroarch
|
||||
```
|
||||
|
||||
Look for MISSING or HASH MISMATCH entries. If a file shows HASH MISMATCH, you have a BIOS file but it's the wrong version or a bad dump. Replace it with one that matches the expected hash.
|
||||
|
||||
Some cores also support HLE (see below), so a missing BIOS may not always be the cause. Check the emulator's logs for error messages.
|
||||
|
||||
## What's the difference between required and optional?
|
||||
|
||||
**Required** means the emulator will not start games for that system without the file. **Optional** means the emulator works without it, but with reduced accuracy or missing features (e.g., boot screen animation, wrong font rendering, or degraded audio).
|
||||
|
||||
In verification output, missing required files appear as CRITICAL or WARNING depending on the platform. Missing optional files appear as WARNING or INFO.
|
||||
|
||||
## What's HLE?
|
||||
|
||||
HLE (High-Level Emulation) is a software reimplementation of what the original BIOS does. Some cores can boot games without a real BIOS file by using their built-in HLE fallback. The trade-off is lower accuracy: some games may have glitches or fail to boot entirely.
|
||||
|
||||
When a core has HLE support, the verification tool lowers the severity of a missing BIOS to INFO. The file is still included in packs because the real BIOS gives better results.
|
||||
|
||||
## Why are there multiple hashes for the same file?
|
||||
|
||||
Two main reasons:
|
||||
|
||||
1. **Regional variants.** The same filename (e.g., `IPL.bin` for GameCube) exists in different versions for USA, Europe, and Japan. Each region has a different hash.
|
||||
2. **Revision differences.** Console manufacturers released updated BIOS versions over time. A PlayStation SCPH-5501 BIOS differs from a SCPH-7001.
|
||||
|
||||
Platforms that verify by MD5 accept specific hashes. If yours doesn't match any known hash, it may be a bad dump or an uncommon revision.
|
||||
|
||||
## How do I know which BIOS I need?
|
||||
|
||||
Two approaches:
|
||||
|
||||
1. **Run verify.py** for your platform. It lists every expected file with its hash and status.
|
||||
2. **Check the project site.** Each platform page lists all required and optional BIOS files per system.
|
||||
|
||||
For a specific emulator core:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --emulator beetle_psx --verbose
|
||||
```
|
||||
|
||||
The `--verbose` flag shows source references and expected values from the emulator's source code.
|
||||
|
||||
## Is this legal?
|
||||
|
||||
Yes. Distribution of BIOS files, firmware, and encryption keys for emulation and preservation is supported by established case law and statutory exemptions across multiple jurisdictions.
|
||||
|
||||
### Emulation and BIOS redistribution
|
||||
|
||||
- **Emulation is legal.** *Sony v. Connectix* (2000) and *Sega v. Accolade* (1992) established that creating emulators and reverse-engineering console firmware for interoperability is lawful. BIOS files are functional prerequisites for this legal activity.
|
||||
- **Fair use (US, 17 USC 107).** Non-commercial redistribution of firmware for personal emulation and archival is transformative use. The files serve a different purpose (interoperability) than the original (running proprietary hardware). No commercial market exists for standalone BIOS files.
|
||||
- **Fair dealing (EU, UK, Canada, Australia).** Equivalent doctrines protect research, private study, and interoperability. The EU Software Directive (2009/24/EC, Art. 5-6) explicitly permits decompilation and use for interoperability.
|
||||
- **Abandonware.** The vast majority of firmware here is for discontinued hardware no longer sold, supported, or distributed by the original manufacturer. No active commercial market is harmed.
|
||||
|
||||
### Encryption keys (Switch prod.keys, 3DS AES keys, Wii U keys)
|
||||
|
||||
This is the most contested area. The legal position:
|
||||
|
||||
- **Keys are not copyrightable.** Encryption keys are mathematical values, not creative expression. Copyright protects original works of authorship; a 256-bit number does not meet the threshold of originality. *Bernstein v. DOJ* (1996) established that code and algorithms are protected speech, and the mere publication of numeric values cannot be restricted under copyright.
|
||||
- **DMCA 1201(f) interoperability exemption.** The DMCA prohibits circumvention of technological protection measures, but Section 1201(f) explicitly permits circumvention for the purpose of achieving interoperability between programs. Emulators require these keys to decrypt and run legally purchased game software. The keys enable interoperability, not piracy.
|
||||
- **Library of Congress DMCA exemptions.** The triennial rulemaking process has repeatedly expanded exemptions for video game preservation. The 2024 exemption (37 CFR 201.40) covers circumvention for preservation of software and video games, including when the original hardware is no longer available.
|
||||
- **Keys derived from consumer hardware.** These keys are extracted from retail hardware owned by consumers. Once a product is sold, the manufacturer cannot indefinitely control how the purchaser uses or examines their own property. *Chamberlain v. Skylink* (2004) held that using a product in a way the manufacturer dislikes is not automatically a DMCA violation.
|
||||
- **No trade secret protection.** For keys to qualify as trade secrets, the holder must take reasonable steps to maintain secrecy. Keys embedded in millions of consumer devices and widely published online do not meet this standard.
|
||||
|
||||
### Recent firmware (Switch 19.0.0, PS3UPDAT, PSVUPDAT)
|
||||
|
||||
- **Firmware updates are freely distributed.** Nintendo, Sony, and other manufacturers distribute firmware updates via CDN without authentication or purchase requirements. Redistributing freely available data does not create new legal liability.
|
||||
- **Functional necessity.** Emulators require system firmware to function. Providing firmware is equivalent to providing the operating environment the software was designed to run in.
|
||||
- **Yuzu context.** The Yuzu settlement (2024) concerned the emulator itself and its facilitation of piracy, not the legality of firmware or key distribution. Yuzu settled without admitting liability and the case created no binding precedent against BIOS or key redistribution.
|
||||
|
||||
### Summary
|
||||
|
||||
This project distributes BIOS files, firmware, and encryption keys for personal use, archival, and interoperability with emulation software. The legal basis rests on fair use, statutory interoperability exemptions, preservation precedent, and the non-copyrightable nature of encryption keys.
|
||||
|
||||
## What's a hash/checksum?
|
||||
|
||||
A hash is a fixed-length fingerprint computed from a file's contents. If even one byte differs, the hash changes completely. The project uses three types:
|
||||
|
||||
| Type | Length | Example |
|
||||
|------|--------|---------|
|
||||
| MD5 | 32 hex chars | `924e392ed05558ffdb115408c263dccf` |
|
||||
| SHA1 | 40 hex chars | `10155d8d6e6e832d8ea1571511e40dfb15fede05` |
|
||||
| CRC32 | 8 hex chars | `2F468B96` |
|
||||
|
||||
Different platforms use different hash types for verification. Batocera uses MD5, RetroArch checks existence only, and RomM accepts any of the three.
|
||||
|
||||
## Why does my verification report say UNTESTED?
|
||||
|
||||
UNTESTED means the file exists on disk but its hash was not confirmed against a known value. This happens on existence-mode platforms (RetroArch, Lakka, RetroPie) where the platform only checks that the file is present, without verifying its contents.
|
||||
|
||||
The file may still be correct. Running `verify.py --emulator <core> --verbose` shows the emulator-level ground truth, which can confirm whether the file's hash matches what the source code expects.
|
||||
|
||||
## Can I use BIOS from one platform on another?
|
||||
|
||||
Yes. BIOS files are console-specific, not platform-specific. A PlayStation BIOS works in RetroArch, Batocera, Recalbox, and any other platform that emulates PlayStation. The only differences between platforms are:
|
||||
|
||||
- **Where the file goes** (each platform has its own BIOS directory)
|
||||
- **What filename is expected** (usually the same, occasionally different)
|
||||
- **How verification works** (MD5 check vs. existence check)
|
||||
|
||||
The packs differ per platform because each platform declares its own set of supported systems and expected files.
|
||||
|
||||
## How often are packs updated?
|
||||
|
||||
A weekly automated sync checks upstream sources (libretro System.dat, batocera-systems, etc.) for changes. If differences are found, a pull request is created automatically. Manual releases happen as needed when new BIOS files are added or profiles are updated.
|
||||
156
wiki/getting-started.md
Normal file
156
wiki/getting-started.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Getting started - RetroBIOS
|
||||
|
||||
## What are BIOS files?
|
||||
|
||||
BIOS files are firmware dumps from original console hardware. Emulators need them to boot games for systems that relied on built-in software (PlayStation, Saturn, Dreamcast, etc.). Without the correct BIOS, the emulator either refuses to start the game or falls back to less accurate software emulation.
|
||||
|
||||
## Installation
|
||||
|
||||
Three ways to get BIOS files in place, from easiest to most manual.
|
||||
|
||||
### Option 1: install.py (recommended)
|
||||
|
||||
Self-contained Python script, no dependencies beyond Python 3.10+. Auto-detects your platform and BIOS directory.
|
||||
|
||||
```bash
|
||||
python install.py
|
||||
```
|
||||
|
||||
Override detection if needed:
|
||||
|
||||
```bash
|
||||
python install.py --platform retroarch --dest ~/custom/bios
|
||||
python install.py --check # verify existing files without downloading
|
||||
python install.py --list-platforms # show supported platforms
|
||||
```
|
||||
|
||||
The installer downloads files from GitHub releases, verifies SHA1 checksums, and places them in the correct directory.
|
||||
|
||||
### Option 2: download.sh (Linux/macOS)
|
||||
|
||||
One-liner for systems with `curl` or `wget`:
|
||||
|
||||
```bash
|
||||
bash scripts/download.sh retroarch ~/RetroArch/system/
|
||||
bash scripts/download.sh --list # show available packs
|
||||
```
|
||||
|
||||
### Option 3: manual download
|
||||
|
||||
1. Go to the [releases page](https://github.com/Abdess/retrobios/releases)
|
||||
2. Download the ZIP pack for your platform
|
||||
3. Extract to the BIOS directory listed below
|
||||
|
||||
## BIOS directory by platform
|
||||
|
||||
### RetroArch
|
||||
|
||||
RetroArch uses the `system_directory` setting in `retroarch.cfg`. Default locations:
|
||||
|
||||
| OS | Default path |
|
||||
|----|-------------|
|
||||
| Windows | `%APPDATA%\RetroArch\system\` |
|
||||
| Linux | `~/.config/retroarch/system/` |
|
||||
| Linux (Flatpak) | `~/.var/app/org.libretro.RetroArch/config/retroarch/system/` |
|
||||
| macOS | `~/Library/Application Support/RetroArch/system/` |
|
||||
| Steam Deck | `~/.var/app/org.libretro.RetroArch/config/retroarch/system/` |
|
||||
| Android | `/storage/emulated/0/RetroArch/system/` |
|
||||
|
||||
To check your actual path: open RetroArch, go to **Settings > Directory > System/BIOS**, or look for `system_directory` in `retroarch.cfg`.
|
||||
|
||||
### Batocera
|
||||
|
||||
```
|
||||
/userdata/bios/
|
||||
```
|
||||
|
||||
Accessible via network share at `\\BATOCERA\share\bios\` (Windows) or `smb://batocera/share/bios/` (macOS/Linux).
|
||||
|
||||
### Recalbox
|
||||
|
||||
```
|
||||
/recalbox/share/bios/
|
||||
```
|
||||
|
||||
Accessible via network share at `\\RECALBOX\share\bios\`.
|
||||
|
||||
### RetroBat
|
||||
|
||||
```
|
||||
bios/
|
||||
```
|
||||
|
||||
Relative to the RetroBat installation directory (e.g., `C:\RetroBat\bios\`).
|
||||
|
||||
### RetroDECK
|
||||
|
||||
```
|
||||
~/.var/app/net.retrodeck.retrodeck/retrodeck/bios/
|
||||
```
|
||||
|
||||
### EmuDeck
|
||||
|
||||
```
|
||||
Emulation/bios/
|
||||
```
|
||||
|
||||
Located inside your Emulation folder. On Steam Deck, typically `~/Emulation/bios/`.
|
||||
|
||||
### Lakka
|
||||
|
||||
```
|
||||
/storage/system/
|
||||
```
|
||||
|
||||
Accessible via SSH or Samba.
|
||||
|
||||
### RetroPie
|
||||
|
||||
```
|
||||
~/RetroPie/BIOS/
|
||||
```
|
||||
|
||||
### BizHawk
|
||||
|
||||
```
|
||||
Firmware/
|
||||
```
|
||||
|
||||
Relative to the BizHawk installation directory.
|
||||
|
||||
### RomM
|
||||
|
||||
BIOS files are managed through the RomM web interface. Check the
|
||||
[RomM documentation](https://github.com/rommapp/romm) for setup details.
|
||||
|
||||
## Verifying your setup
|
||||
|
||||
After placing BIOS files, verify that everything is correct:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --platform retroarch
|
||||
python scripts/verify.py --platform batocera
|
||||
python scripts/verify.py --platform recalbox
|
||||
```
|
||||
|
||||
The output shows each expected file with its status: OK, MISSING, or HASH MISMATCH. Platforms that verify by MD5 (Batocera, Recalbox, EmuDeck) will catch wrong versions. RetroArch only checks that files exist.
|
||||
|
||||
For a single system:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --system sony-playstation
|
||||
```
|
||||
|
||||
For a single emulator core:
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --emulator beetle_psx
|
||||
```
|
||||
|
||||
See [Tools](tools.md) for the full CLI reference.
|
||||
|
||||
## Next steps
|
||||
|
||||
- [FAQ](faq.md) - common questions and troubleshooting
|
||||
- [Tools](tools.md) - all available scripts and options
|
||||
- [Architecture](architecture.md) - how the project works internally
|
||||
@@ -2,18 +2,50 @@
|
||||
|
||||
Technical documentation for the RetroBIOS toolchain.
|
||||
|
||||
## Pages
|
||||
|
||||
- **[Architecture](architecture.md)** - directory structure, data flow, platform inheritance, pack grouping, security, edge cases, CI workflows
|
||||
- **[Tools](tools.md)** - CLI reference for every script, pipeline usage, scrapers
|
||||
- **[Profiling guide](profiling.md)** - how to create an emulator profile from source code, step by step, with YAML field reference
|
||||
- **[Data model](data-model.md)** - database.json structure, indexes, file resolution order, YAML formats
|
||||
|
||||
## For users
|
||||
|
||||
- **[Getting started](getting-started.md)** - installation, BIOS directory paths per platform, verification
|
||||
- **[FAQ](faq.md)** - common questions, troubleshooting, hash explanations
|
||||
|
||||
If you just want to download BIOS packs, see the [home page](../index.md).
|
||||
|
||||
## Technical reference
|
||||
|
||||
- **[Architecture](architecture.md)** - directory structure, data flow, platform inheritance, pack grouping, security, edge cases, CI workflows
|
||||
- **[Tools](tools.md)** - CLI reference for every script, pipeline usage, scrapers
|
||||
- **[Advanced usage](advanced-usage.md)** - custom packs, target filtering, truth generation, emulator verification, offline workflow
|
||||
- **[Verification modes](verification-modes.md)** - how each platform verifies BIOS files, severity matrix, resolution chain
|
||||
- **[Data model](data-model.md)** - database.json structure, indexes, file resolution order, YAML formats
|
||||
- **[Troubleshooting](troubleshooting.md)** - diagnosis by symptom: missing BIOS, hash mismatch, pack issues, verify errors
|
||||
|
||||
## For contributors
|
||||
|
||||
Start with the [profiling guide](profiling.md) to understand how emulator profiles are built,
|
||||
then see [contributing](../contributing.md) for submission guidelines.
|
||||
- **[Profiling guide](profiling.md)** - create an emulator profile from source code, YAML field reference
|
||||
- **[Adding a platform](adding-a-platform.md)** - scraper, registry, YAML config, exporter, target scraper, install detection
|
||||
- **[Adding a scraper](adding-a-scraper.md)** - plugin architecture, BaseScraper, parsers, target scrapers
|
||||
- **[Testing guide](testing-guide.md)** - run tests, fixture pattern, how to add tests, CI integration
|
||||
- **[Release process](release-process.md)** - CI workflows, large files, manual release
|
||||
|
||||
See [contributing](../contributing.md) for submission guidelines.
|
||||
|
||||
## Glossary
|
||||
|
||||
- **BIOS** - firmware burned into console hardware, needed by emulators that rely on original boot code
|
||||
- **firmware** - system software loaded by a console at boot; used interchangeably with BIOS in this project
|
||||
- **HLE** - High-Level Emulation; software reimplementation of BIOS functions, avoids needing the original file
|
||||
- **hash** - fixed-length fingerprint of a file's contents; this project uses MD5, SHA1, SHA256, and CRC32
|
||||
- **platform** - a distribution that packages emulators (RetroArch, Batocera, Recalbox, EmuDeck, etc.)
|
||||
- **core** - an emulator packaged as a libretro plugin, loaded by RetroArch or compatible frontends
|
||||
- **profile** - a YAML file in `emulators/` documenting one core's BIOS requirements, verified against source code
|
||||
- **system** - a game console or computer being emulated (e.g. sony-playstation, nintendo-gameboy-advance)
|
||||
- **pack** - a ZIP archive containing all BIOS files needed by a specific platform
|
||||
- **ground truth** - the emulator's source code, treated as the authoritative reference for BIOS requirements
|
||||
- **cross-reference** - comparison of emulator profiles against platform configs to find undeclared files
|
||||
- **scraper** - a script that fetches BIOS requirement data from an upstream source (System.dat, es_bios.xml, etc.)
|
||||
- **exporter** - a script that converts ground truth data back into a platform's native format
|
||||
- **target** - a hardware architecture that a platform runs on (e.g. switch, rpi4, x86_64, steamos)
|
||||
- **variant** - an alternative version of a BIOS file (different revision, region, or dump), stored in `.variants/`
|
||||
- **required** - a file the core needs to function; determined by source code behavior
|
||||
- **optional** - a file the core functions without, possibly with reduced accuracy or missing features
|
||||
- **hle_fallback** - flag on a file indicating the core has an HLE path; absence is downgraded to INFO severity
|
||||
- **severity** - the urgency of a verification result: OK (verified), INFO (negligible), WARNING (degraded), CRITICAL (broken)
|
||||
|
||||
@@ -9,6 +9,34 @@ The source code is the reference because it reflects actual behavior.
|
||||
Documentation, .info files, and wikis are useful starting points
|
||||
but are verified against the code.
|
||||
|
||||
### Source hierarchy
|
||||
|
||||
Documentation and metadata are valuable starting points, but they can
|
||||
fall out of sync with the actual code over time. The desmume2015 .info
|
||||
file is a good illustration: it declares `firmware_count=3`, but the
|
||||
source code at the pinned version opens zero firmware files. Cross-checking
|
||||
against the source helps catch that kind of gap early.
|
||||
|
||||
When sources conflict, priority follows the chain of actual execution:
|
||||
|
||||
1. **Original emulator source** (ground truth, what the code actually does)
|
||||
2. **Libretro port** (may adapt paths, add compatibility shims, or drop features)
|
||||
3. **.info metadata** (declarative, may be outdated or copied from another core)
|
||||
|
||||
For standalone emulators like BizHawk or amiberry, there is only one
|
||||
level. The emulator's own codebase is the single source of truth. No
|
||||
.info, no wrapper, no divergence to track.
|
||||
|
||||
A note on libretro port differences: the most common change is path
|
||||
resolution. The upstream emulator loads files from the current working
|
||||
directory; the libretro wrapper redirects to `retro_system_directory`.
|
||||
This is normal adaptation, not a divergence worth documenting. Similarly,
|
||||
filename changes like `naomi2_eeprom.bin` becoming `n2_eeprom.bin` are
|
||||
often deliberate. RetroArch uses a single shared system directory for
|
||||
all cores, so the port renames files to prevent collisions between cores
|
||||
that emulate different systems but happen to use the same generic
|
||||
filenames. The upstream name goes in `aliases:`.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Find the source code
|
||||
@@ -21,9 +49,27 @@ Check these locations in order:
|
||||
|
||||
Always clone both upstream and libretro port to compare.
|
||||
|
||||
For libretro cores, cloning both repositories and diffing them reveals
|
||||
what the port changed. Path changes (fopen of a relative path becoming
|
||||
a system_dir lookup) are expected. What matters are file additions the
|
||||
port introduces, files the port dropped, or hash values that differ
|
||||
between the two codebases.
|
||||
|
||||
If the source is hosted outside GitHub, it's worth exploring further. Emulator
|
||||
source on GitLab, Codeberg, SourceForge, Bitbucket, archive.org
|
||||
snapshots, and community mirror tarballs. Inspecting copyright headers
|
||||
or license strings in the libretro fork often points to the original
|
||||
author's site. The upstream code exists somewhere; it's worth continuing the search before concluding the source is unavailable.
|
||||
|
||||
One thing worth noting: even when the same repository was analyzed for
|
||||
a related profile (e.g., fbneo for arcade systems), it helps to do a
|
||||
fresh pass for each new profile. When fbneo_neogeo was profiled, the
|
||||
NeoGeo subset referenced BIOS files that the main arcade analysis
|
||||
hadn't encountered. A fresh look avoids carrying over blind spots.
|
||||
|
||||
### 2. Trace file loading
|
||||
|
||||
Read the code flow. Don't grep keywords by assumption.
|
||||
Read the code flow, tracing from the entry point.
|
||||
Each emulator has its own way of loading files.
|
||||
|
||||
Look for:
|
||||
@@ -34,6 +80,19 @@ Look for:
|
||||
- Hash validation (MD5, CRC32, SHA1 comparisons in code)
|
||||
- Size validation (`fseek`/`ftell`, `stat`, fixed buffer sizes)
|
||||
|
||||
Grepping for "bios" or "firmware" across the source tree can be a
|
||||
useful first pass, but it may miss emulators that use different terms
|
||||
(bootrom, system ROM, IPL, program.rom) and can surface false matches
|
||||
from test fixtures or comments.
|
||||
|
||||
A more reliable approach is starting from the entry point
|
||||
(`retro_load_game` for libretro, `main()` for standalone) and tracing
|
||||
the actual file-open calls forward. Each emulator has its own loading
|
||||
flow. Dolphin loads region-specific IPL files through a boot sequence
|
||||
object. BlastEm reads a list of ROM paths from a configuration
|
||||
structure. same_cdi opens CD-i BIOS files through a machine
|
||||
initialization routine. The loading flow varies widely between emulators.
|
||||
|
||||
### 3. Determine required vs optional
|
||||
|
||||
This is decided by code behavior, not by judgment:
|
||||
@@ -42,6 +101,18 @@ This is decided by code behavior, not by judgment:
|
||||
- **optional**: the core works with degraded functionality without it
|
||||
- **hle_fallback: true**: the core has a high-level emulation path when the file is missing
|
||||
|
||||
The decision is based on the code's behavior. If the core crashes or
|
||||
refuses to boot without the file, it is required. If it continues with
|
||||
degraded functionality (missing boot animation, different fonts, reduced
|
||||
audio in menus), it is optional. This keeps the classification objective
|
||||
and consistent across all profiles.
|
||||
|
||||
When a core has HLE (high-level emulation), the real BIOS typically
|
||||
gives better accuracy, but the core functions without it. These files
|
||||
are marked with `hle_fallback: true` and `required: false`. The file
|
||||
still ships in packs (better experience for the user), but its absence
|
||||
does not raise alarms during verification.
|
||||
|
||||
### 4. Document divergences
|
||||
|
||||
When the libretro port differs from the upstream:
|
||||
@@ -54,6 +125,18 @@ Path differences (current dir vs system_dir) are normal adaptation,
|
||||
not a divergence. Name changes (e.g. `naomi2_` to `n2_`) may be intentional
|
||||
to avoid conflicts in the shared system directory.
|
||||
|
||||
RetroArch's system directory is shared by every installed core. When
|
||||
the libretro port renames a file, it is usually solving a real problem:
|
||||
two cores that both expect `bios.rom` would overwrite each other. The
|
||||
upstream name goes in `aliases:` and `mode: libretro` on the port-specific
|
||||
name, so both names are indexed.
|
||||
|
||||
True divergences worth documenting are: files the port adds that the
|
||||
upstream never loads, files the upstream loads that the port dropped
|
||||
(a gap in the port), and hash differences in embedded ROM data between
|
||||
the two codebases. These get noted in the profile because they affect
|
||||
what the user actually needs to provide.
|
||||
|
||||
### 5. Write the YAML profile
|
||||
|
||||
```yaml
|
||||
@@ -80,6 +163,46 @@ files:
|
||||
source_ref: Source/Core/Core/Boot/Boot_BS2Emu.cpp:42
|
||||
```
|
||||
|
||||
### Writing style
|
||||
|
||||
Notes in a profile describe what the core does, kept focused on:
|
||||
what files get loaded, how, and from where. Comparisons with other
|
||||
cores, disclaimers, and feature coverage beyond file requirements
|
||||
belong in external documentation. The profile is a technical spec.
|
||||
|
||||
Profiles are standalone documentation. Someone should be able to take
|
||||
a single YAML file and integrate it into their own project without
|
||||
knowing anything about this repository's database, directory layout,
|
||||
or naming conventions. The YAML documents what the emulator expects.
|
||||
The tooling resolves the YAML against the local file collection
|
||||
separately.
|
||||
|
||||
A few field conventions that protect the toolchain:
|
||||
|
||||
- `type:` is operational. `resolve_platform_cores()` uses it to filter
|
||||
which profiles apply to a platform. Valid values are `libretro`,
|
||||
`standalone + libretro`, `standalone`, `alias`, `launcher`, `game`,
|
||||
`utility`, `test`. Putting a classification concept here (like
|
||||
"bizhawk-native") breaks the filtering. A BizHawk core is
|
||||
`type: standalone`.
|
||||
|
||||
- `core_classification:` is descriptive. It documents the relationship
|
||||
between the core and the original emulator (pure_libretro,
|
||||
official_port, community_fork, frozen_snapshot, etc.). It has no
|
||||
effect on tooling behavior.
|
||||
|
||||
- Alternative filenames go in `aliases:` on the file entry (rather than
|
||||
as separate entries in platform YAMLs or `_shared.yml`). When the same
|
||||
physical ROM is known by three names across different platforms, one
|
||||
name is `name:` and the rest are `aliases:`.
|
||||
|
||||
- Hashes come from source code. If the source has a hardcoded hex
|
||||
string (like emuscv's `635a978...` in memory.cpp), that goes in. If
|
||||
the source embeds ROM data as byte arrays (like ep128emu's roms.hpp),
|
||||
the bytes can be extracted and hashed. If the source performs no hash
|
||||
check at all, the hash is omitted from the profile. The .info or docs
|
||||
may list an MD5, but source confirmation makes it more reliable.
|
||||
|
||||
### 6. Validate
|
||||
|
||||
```bash
|
||||
@@ -87,6 +210,38 @@ python scripts/cross_reference.py --emulator dolphin --json
|
||||
python scripts/verify.py --emulator dolphin
|
||||
```
|
||||
|
||||
### Lessons learned
|
||||
|
||||
These are patterns that have come up while building profiles. Sharing
|
||||
them here in case they save time.
|
||||
|
||||
**.info metadata can lag behind the code.** The desmume2015 .info
|
||||
declares `firmware_count=3`, but the core source at the pinned version
|
||||
never opens any firmware file. The .info is useful as a starting point
|
||||
but benefits from a cross-check against the actual code.
|
||||
|
||||
**Fresh analysis per profile helps.** When fbneo was profiled for
|
||||
arcade systems, NeoGeo-specific BIOS files were outside the analysis
|
||||
scope. Profiling fbneo_neogeo later surfaced files the first pass
|
||||
hadn't covered. Doing a fresh pass for each profile, even on a
|
||||
familiar codebase, avoids carrying over blind spots.
|
||||
|
||||
**Path adaptation vs real divergence.** The libretro wrapper changing
|
||||
`fopen("./rom.bin")` to load from `system_dir` is the standard
|
||||
porting pattern. The file is the same; only the directory resolution
|
||||
changed. True divergences (added/removed files, different embedded
|
||||
data) are the ones worth documenting.
|
||||
|
||||
**Each core has its own loading logic.** snes9x and bsnes both
|
||||
emulate the Super Nintendo, but they handle the Super Game Boy BIOS
|
||||
and DSP firmware through different code paths. Checking the actual
|
||||
code for each core avoids assumptions based on a related profile.
|
||||
|
||||
**Code over docs.** Wiki pages and README files sometimes reference
|
||||
files from older versions or a different fork. If the source code
|
||||
does not load a particular file, it can be left out of the profile
|
||||
even if documentation mentions it.
|
||||
|
||||
## YAML field reference
|
||||
|
||||
### Profile fields
|
||||
@@ -94,18 +249,22 @@ python scripts/verify.py --emulator dolphin
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `emulator` | yes | display name |
|
||||
| `type` | yes | `libretro`, `standalone`, `standalone + libretro`, `alias`, `launcher` |
|
||||
| `type` | yes | `libretro`, `standalone`, `standalone + libretro`, `alias`, `launcher`, `game`, `utility`, `test` |
|
||||
| `core_classification` | no | `pure_libretro`, `official_port`, `community_fork`, `frozen_snapshot`, `enhanced_fork`, `game_engine`, `embedded_hle`, `alias`, `launcher` |
|
||||
| `source` | yes | libretro core repository URL |
|
||||
| `upstream` | no | original emulator repository URL |
|
||||
| `profiled_date` | yes | date of source analysis |
|
||||
| `core_version` | yes | version analyzed |
|
||||
| `display_name` | no | full display name (e.g. "Sega - Mega Drive (BlastEm)") |
|
||||
| `systems` | yes | list of system IDs this core handles |
|
||||
| `cores` | no | list of core names (default: profile filename) |
|
||||
| `cores` | no | list of upstream core names for buildbot/target matching |
|
||||
| `mode` | no | default mode: `standalone`, `libretro`, or `both` |
|
||||
| `verification` | no | how the core verifies BIOS: `existence` or `md5` |
|
||||
| `files` | yes | list of file entries |
|
||||
| `notes` | no | free-form technical notes |
|
||||
| `exclusion_note` | no | why the profile has no files |
|
||||
| `data_directories` | no | references to data dirs in `_data_dirs.yml` |
|
||||
| `exclusion_note` | no | why the profile has no files despite .info declaring firmware |
|
||||
| `analysis` | no | structured per-subsystem analysis (capabilities, supported modes) |
|
||||
| `platform_details` | no | per-system platform-specific details (paths, romsets, forced systems) |
|
||||
|
||||
### File entry fields
|
||||
|
||||
@@ -113,20 +272,20 @@ python scripts/verify.py --emulator dolphin
|
||||
|-------|-------------|
|
||||
| `name` | filename as the core expects it |
|
||||
| `required` | true if the core needs this file to function |
|
||||
| `system` | system ID this file belongs to |
|
||||
| `system` | system ID this file belongs to (for multi-system profiles) |
|
||||
| `size` | expected size in bytes |
|
||||
| `min_size`, `max_size` | size range when the code accepts a range |
|
||||
| `md5`, `sha1`, `crc32`, `sha256` | expected hashes from source code |
|
||||
| `validation` | list of checks the code performs: `size`, `crc32`, `md5`, `sha1` |
|
||||
| `validation` | checks the code performs: `size`, `crc32`, `md5`, `sha1`, `adler32`, `signature`, `crypto`. Can be a list or dict `{core: [...], upstream: [...]}` for divergent checks |
|
||||
| `aliases` | alternate filenames for the same file |
|
||||
| `mode` | `libretro`, `standalone`, or `both` |
|
||||
| `hle_fallback` | true if a high-level emulation path exists |
|
||||
| `category` | `bios` (default), `game_data`, `bios_zip` |
|
||||
| `region` | geographic region (e.g. `north-america`, `japan`) |
|
||||
| `source_ref` | source file and line number |
|
||||
| `path` | path relative to system directory |
|
||||
| `source_ref` | source file and line number (e.g. `boot.cpp:42`) |
|
||||
| `path` | destination path relative to system directory |
|
||||
| `description` | what this file is |
|
||||
| `note` | additional context |
|
||||
| `archive` | parent ZIP if this file is inside an archive |
|
||||
| `contents` | structure of files inside a BIOS ZIP |
|
||||
| `storage` | `embedded` (default), `external`, `user_provided` |
|
||||
| `contents` | structure of files inside a BIOS ZIP (`name`, `description`, `size`, `crc32`) |
|
||||
| `storage` | `large_file` for files > 50 MB stored as release assets |
|
||||
|
||||
|
||||
158
wiki/release-process.md
Normal file
158
wiki/release-process.md
Normal file
@@ -0,0 +1,158 @@
|
||||
# Release Process
|
||||
|
||||
This page documents the CI/CD pipeline: what each workflow does, how releases
|
||||
are built, and how to run the process manually.
|
||||
|
||||
## CI workflows overview
|
||||
|
||||
The project uses 4 GitHub Actions workflows. All use only official GitHub
|
||||
actions (`actions/checkout`, `actions/setup-python`, `actions/upload-pages-artifact`,
|
||||
`actions/deploy-pages`). No third-party actions.
|
||||
|
||||
Budget target: ~175 minutes/month on the GitHub free tier.
|
||||
|
||||
| Workflow | File | Trigger |
|
||||
|----------|------|---------|
|
||||
| Build & Release | `build.yml` | Push to `bios/**` or `platforms/**`, manual dispatch |
|
||||
| Deploy Site | `deploy-site.yml` | Push to main (platforms, emulators, wiki, scripts, database.json, mkdocs.yml), manual |
|
||||
| PR Validation | `validate.yml` | PR touching `bios/**` or `platforms/**` |
|
||||
| Weekly Sync | `watch.yml` | Cron Monday 06:00 UTC, manual dispatch |
|
||||
|
||||
## build.yml - Build & Release
|
||||
|
||||
Currently disabled (`if: false` on the release job) until pack generation is
|
||||
validated in production.
|
||||
|
||||
**Trigger.** Push to `main` on `bios/**` or `platforms/**` paths, or manual
|
||||
`workflow_dispatch` with optional `force_release` flag to bypass rate limiting.
|
||||
|
||||
**Concurrency.** Group `build`, cancel in-progress.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Checkout, Python 3.12, install `pyyaml`
|
||||
2. Run `test_e2e`
|
||||
3. Rate limit check: skip if last release was less than 7 days ago (unless
|
||||
`force_release` is set)
|
||||
4. Restore large files from the `large-files` release into `.cache/large/`
|
||||
5. Refresh data directories (`refresh_data_dirs.py`)
|
||||
6. Build packs (`generate_pack.py --all --output-dir dist/`)
|
||||
7. Create GitHub release with tag `v{YYYY.MM.DD}` (appends `.N` suffix if
|
||||
a same-day release already exists)
|
||||
8. Clean up old releases, keeping the 3 most recent plus `large-files`
|
||||
|
||||
**Release notes** include file count, total size, per-pack sizes, and the last
|
||||
15 non-merge commits touching `bios/` or `platforms/`.
|
||||
|
||||
## deploy-site.yml - Deploy Documentation Site
|
||||
|
||||
**Trigger.** Push to `main` when any of these paths change: `platforms/`,
|
||||
`emulators/`, `wiki/`, `scripts/generate_site.py`, `scripts/generate_readme.py`,
|
||||
`scripts/verify.py`, `scripts/common.py`, `database.json`, `mkdocs.yml`.
|
||||
Also manual dispatch.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Checkout, Python 3.12
|
||||
2. Install `pyyaml`, `mkdocs-material`, `pymdown-extensions`
|
||||
3. Run `generate_site.py` (converts YAML data into MkDocs pages)
|
||||
4. Run `generate_readme.py` (rebuilds README.md and CONTRIBUTING.md)
|
||||
5. `mkdocs build` to produce the static site
|
||||
6. Upload artifact, deploy to GitHub Pages
|
||||
|
||||
The site is deployed via the `github-pages` environment using the official
|
||||
`actions/deploy-pages` action.
|
||||
|
||||
## validate.yml - PR Validation
|
||||
|
||||
**Trigger.** Pull requests that modify `bios/**` or `platforms/**`.
|
||||
|
||||
**Concurrency.** Per-PR group, cancel in-progress.
|
||||
|
||||
Four parallel jobs:
|
||||
|
||||
**validate-bios.** Diffs the PR to find changed BIOS files, runs
|
||||
`validate_pr.py --markdown` on each, and posts the validation report as a PR
|
||||
comment (hash verification, database match status).
|
||||
|
||||
**validate-configs.** Validates all platform YAML files against
|
||||
`schemas/platform.schema.json` using `jsonschema`. Fails if any config does
|
||||
not match the schema.
|
||||
|
||||
**run-tests.** Runs `python -m unittest tests.test_e2e -v`. Must pass before
|
||||
merge.
|
||||
|
||||
**label-pr.** Auto-labels the PR based on changed paths:
|
||||
|
||||
| Path pattern | Label |
|
||||
|-------------|-------|
|
||||
| `bios/` | `bios` |
|
||||
| `bios/{Manufacturer}/` | `system:{manufacturer}` |
|
||||
| `platforms/` | `platform-config` |
|
||||
| `scripts/` | `automation` |
|
||||
|
||||
## watch.yml - Weekly Platform Sync
|
||||
|
||||
**Trigger.** Cron schedule every Monday at 06:00 UTC, or manual dispatch.
|
||||
|
||||
**Flow:**
|
||||
|
||||
1. Scrape live upstream sources (System.dat, batocera-systems, es_bios.xml,
|
||||
etc.) and regenerate platform YAML configs
|
||||
2. Auto-fetch missing BIOS files
|
||||
3. Refresh data directories
|
||||
4. Run dedup
|
||||
5. Regenerate `database.json`
|
||||
6. Create or update a PR with labels `automated` and `platform-update`
|
||||
|
||||
The PR contains all changes from the scrape cycle. A maintainer reviews and
|
||||
merges.
|
||||
|
||||
## Large files management
|
||||
|
||||
Files larger than 50 MB are stored as assets on a permanent GitHub release
|
||||
named `large-files` (to keep the git repository lightweight).
|
||||
|
||||
Known large files: PS3UPDAT.PUP, PSVUPDAT.PUP, PSP2UPDAT.PUP, dsi_nand.bin,
|
||||
maclc3.zip, Firmware.19.0.0.zip (Switch).
|
||||
|
||||
**Storage.** Listed in `.gitignore` so they stay out of git history. The
|
||||
`large-files` release is excluded from cleanup (the build workflow only
|
||||
deletes version-tagged releases).
|
||||
|
||||
**Build-time restore.** The build workflow downloads all assets from
|
||||
`large-files` into `.cache/large/` and copies them to their expected paths
|
||||
before pack generation.
|
||||
|
||||
**Upload.** To add or update a large file:
|
||||
|
||||
```bash
|
||||
gh release upload large-files "bios/Sony/PS3/PS3UPDAT.PUP#PS3UPDAT.PUP"
|
||||
```
|
||||
|
||||
**Local cache.** `generate_pack.py` calls `fetch_large_file()` which downloads
|
||||
from the release and caches in `.cache/large/` for subsequent runs.
|
||||
|
||||
## Manual release process
|
||||
|
||||
When `build.yml` is disabled, build and release manually:
|
||||
|
||||
```bash
|
||||
# Run the full pipeline (DB + verify + packs + consistency check)
|
||||
python scripts/pipeline.py --offline
|
||||
|
||||
# Or step by step:
|
||||
python scripts/generate_db.py --force --bios-dir bios --output database.json
|
||||
python scripts/verify.py --all
|
||||
python scripts/generate_pack.py --all --output-dir dist/
|
||||
|
||||
# Create the release
|
||||
DATE=$(date +%Y.%m.%d)
|
||||
gh release create "v${DATE}" dist/*.zip \
|
||||
--title "BIOS Pack v${DATE}" \
|
||||
--notes "Release notes here" \
|
||||
--latest
|
||||
```
|
||||
|
||||
To re-enable automated releases, remove the `if: false` guard from the
|
||||
`release` job in `build.yml`.
|
||||
148
wiki/testing-guide.md
Normal file
148
wiki/testing-guide.md
Normal file
@@ -0,0 +1,148 @@
|
||||
# Testing Guide
|
||||
|
||||
This page covers how to run, understand, and extend the test suite.
|
||||
|
||||
All tests use synthetic fixtures. No real BIOS files, platform configs, or
|
||||
network access required.
|
||||
|
||||
## Running tests
|
||||
|
||||
Run a single test module:
|
||||
|
||||
```bash
|
||||
python -m unittest tests.test_e2e -v
|
||||
python -m unittest tests.test_mame_parser -v
|
||||
python -m unittest tests.test_fbneo_parser -v
|
||||
python -m unittest tests.test_hash_merge -v
|
||||
```
|
||||
|
||||
Run the full suite:
|
||||
|
||||
```bash
|
||||
python -m unittest discover tests -v
|
||||
```
|
||||
|
||||
The only dependency is `pyyaml`. No test framework beyond the standard
|
||||
library `unittest` module.
|
||||
|
||||
## Test architecture
|
||||
|
||||
### test_e2e.py
|
||||
|
||||
The main regression suite. A single `TestE2E` class exercises every code path
|
||||
through the resolution, verification, pack generation, and cross-reference
|
||||
logic.
|
||||
|
||||
**Fixture pattern.** `setUp` creates a temporary directory tree with:
|
||||
|
||||
- Fake BIOS files (deterministic content for hash computation)
|
||||
- Platform YAML configs (existence mode, MD5 mode, inheritance, shared groups)
|
||||
- Emulator profile YAMLs (required/optional files, aliases, HLE, standalone)
|
||||
- A synthetic `database.json` keyed by SHA1
|
||||
|
||||
`tearDown` removes the temporary tree.
|
||||
|
||||
**Test numbering.** Tests are grouped by category:
|
||||
|
||||
| Range | Category |
|
||||
|-------|----------|
|
||||
| `test_01`--`test_14` | File resolution (SHA1, MD5, name, alias, truncated MD5, composite, zip contents, variants, hash mismatch) |
|
||||
| `test_20`--`test_31` | Verification (existence mode, MD5 mode, required/optional severity, zipped file, multi-hash) |
|
||||
| `test_40`--`test_47` | Cross-reference (undeclared files, standalone skip, alias profiles, data dir suppression, exclusion notes) |
|
||||
| `test_50`+ | Platform config (inheritance, shared groups, data directories, grouping, core resolution, target filtering, ground truth) |
|
||||
|
||||
Each test calls the same functions that `verify.py` and `generate_pack.py` use
|
||||
in production, against the synthetic fixtures.
|
||||
|
||||
### Parser tests
|
||||
|
||||
**test_mame_parser.** Tests the MAME C source parser that extracts BIOS root
|
||||
sets from driver files. Fixtures are inline C source snippets containing
|
||||
`ROM_START`, `ROM_LOAD`, `GAME()`/`COMP()` macros with
|
||||
`MACHINE_IS_BIOS_ROOT`. Tests cover:
|
||||
|
||||
- Standard `GAME` macro detection
|
||||
- `COMP` macro detection
|
||||
- `ROM_LOAD` / `ROMX_LOAD` parsing (name, size, CRC32, SHA1)
|
||||
- `ROM_SYSTEM_BIOS` variant extraction
|
||||
- Multi-region ROM blocks
|
||||
- Macro expansion and edge cases
|
||||
|
||||
**test_fbneo_parser.** Tests the FBNeo C source parser that identifies
|
||||
`BDF_BOARDROM` sets. Same inline fixture approach.
|
||||
|
||||
**test_hash_merge.** Tests the text-based YAML patching module used to merge
|
||||
upstream BIOS hashes into emulator profiles. Covers:
|
||||
|
||||
- Merge operations (add new hashes, update existing)
|
||||
- Diff computation (detect what changed)
|
||||
- Formatting preservation (comments, ordering, flow style)
|
||||
|
||||
Fixtures are programmatically generated YAML/JSON files written to a temp
|
||||
directory.
|
||||
|
||||
## How to add a test
|
||||
|
||||
1. **Pick the right category.** Find the number range that matches the
|
||||
subsystem you are testing. If none fits, start a new range after the last
|
||||
existing one.
|
||||
|
||||
2. **Create synthetic fixtures.** Write the minimum YAML configs and fake
|
||||
files needed to isolate the behavior. Use `tempfile.mkdtemp` for a clean
|
||||
workspace. Avoid depending on the repo's real `bios/` or `platforms/`
|
||||
directories.
|
||||
|
||||
3. **Call production functions.** Import from `common`, `verify`, `validation`,
|
||||
or `truth` and call the same entry points that the CLI scripts use. Do not
|
||||
re-implement logic in tests.
|
||||
|
||||
4. **Assert specific outcomes.** Check `Status`, `Severity`, resolution
|
||||
method, file counts, or pack contents. Avoid brittle assertions on log
|
||||
output or formatting.
|
||||
|
||||
5. **Run the full suite.** After adding your test, run `python -m unittest
|
||||
discover tests -v` to verify nothing else broke.
|
||||
|
||||
Example skeleton:
|
||||
|
||||
```python
|
||||
def test_42_my_new_behavior(self):
|
||||
# Write minimal fixtures to self.root
|
||||
profile = {"emulator": "test_core", "files": [...]}
|
||||
with open(os.path.join(self.emulators_dir, "test_core.yml"), "w") as f:
|
||||
yaml.dump(profile, f)
|
||||
|
||||
# Call production code
|
||||
result = verify_platform(self.config, self.db, ...)
|
||||
|
||||
# Assert specific outcomes
|
||||
self.assertEqual(result[0]["status"], Status.OK)
|
||||
```
|
||||
|
||||
## Verification discipline
|
||||
|
||||
The test suite is one layer of verification. The full quality gate is:
|
||||
|
||||
1. All unit tests pass (`python -m unittest discover tests`)
|
||||
2. The full pipeline completes without error (`python scripts/pipeline.py --offline`)
|
||||
3. No unexpected CRITICAL entries in the verify output
|
||||
4. Pack file counts match verification file counts (consistency check)
|
||||
|
||||
If a change passes tests but breaks the pipeline, it's worth investigating before merging. Similarly, new CRITICAL entries in the verify output after a change usually indicate something to look into. The pipeline is designed so that all steps agree: if verify reports N files for a platform, the pack should contain exactly N files.
|
||||
|
||||
Ideally, tests, code, and documentation ship together. When profiles and platform configs are involved, updating them in the same change helps keep everything in sync.
|
||||
|
||||
## CI integration
|
||||
|
||||
The `validate.yml` workflow runs `test_e2e` on every pull request that touches
|
||||
`bios/` or `platforms/` files. The test job (`run-tests`) runs in parallel
|
||||
with BIOS validation, schema validation, and auto-labeling.
|
||||
|
||||
Tests must pass before merge. If a test fails in CI, reproduce locally with:
|
||||
|
||||
```bash
|
||||
python -m unittest tests.test_e2e -v 2>&1 | head -50
|
||||
```
|
||||
|
||||
The `build.yml` workflow also runs the test suite before building release
|
||||
packs.
|
||||
156
wiki/tools.md
156
wiki/tools.md
@@ -7,11 +7,35 @@ All tools are Python scripts in `scripts/`. Single dependency: `pyyaml`.
|
||||
Run everything in sequence:
|
||||
|
||||
```bash
|
||||
python scripts/pipeline.py --offline # DB + verify + packs + readme + site
|
||||
python scripts/pipeline.py --offline # DB + verify + packs + manifests + readme + site
|
||||
python scripts/pipeline.py --offline --skip-packs # DB + verify only
|
||||
python scripts/pipeline.py --skip-docs # skip readme + site generation
|
||||
python scripts/pipeline.py --offline --skip-docs # skip readme + site generation
|
||||
python scripts/pipeline.py --offline --target switch # filter by hardware target
|
||||
python scripts/pipeline.py --offline --with-truth # include truth generation + diff
|
||||
python scripts/pipeline.py --offline --with-export # include native format export
|
||||
python scripts/pipeline.py --check-buildbot # check buildbot data freshness
|
||||
```
|
||||
|
||||
Pipeline steps:
|
||||
|
||||
| Step | Description | Skipped by |
|
||||
|------|-------------|------------|
|
||||
| 1/9 | Generate database | - |
|
||||
| 2/9 | Refresh data directories | `--offline` |
|
||||
| 2a | Refresh MAME BIOS hashes | `--offline` |
|
||||
| 2a2 | Refresh FBNeo BIOS hashes | `--offline` |
|
||||
| 2b | Check buildbot staleness | only with `--check-buildbot` |
|
||||
| 2c | Generate truth YAMLs | only with `--with-truth` / `--with-export` |
|
||||
| 2d | Diff truth vs scraped | only with `--with-truth` / `--with-export` |
|
||||
| 2e | Export native formats | only with `--with-export` |
|
||||
| 3/9 | Verify all platforms | - |
|
||||
| 4/9 | Generate packs | `--skip-packs` |
|
||||
| 4b | Generate install manifests | `--skip-packs` |
|
||||
| 4c | Generate target manifests | `--skip-packs` |
|
||||
| 5/9 | Consistency check | if verify or pack skipped |
|
||||
| 8/9 | Generate README | `--skip-docs` |
|
||||
| 9/9 | Generate site | `--skip-docs` |
|
||||
|
||||
## Individual tools
|
||||
|
||||
### generate_db.py
|
||||
@@ -29,10 +53,16 @@ python scripts/generate_db.py --force --bios-dir bios --output database.json
|
||||
Check BIOS coverage for each platform using its native verification mode.
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --all # all platforms
|
||||
python scripts/verify.py --platform batocera # single platform
|
||||
python scripts/verify.py --emulator dolphin # single emulator
|
||||
python scripts/verify.py --system atari-lynx # single system
|
||||
python scripts/verify.py --all # all platforms
|
||||
python scripts/verify.py --platform batocera # single platform
|
||||
python scripts/verify.py --platform retroarch --verbose # with ground truth details
|
||||
python scripts/verify.py --emulator dolphin # single emulator
|
||||
python scripts/verify.py --emulator dolphin --standalone # standalone mode only
|
||||
python scripts/verify.py --system atari-lynx # single system
|
||||
python scripts/verify.py --platform retroarch --target switch # filter by hardware
|
||||
python scripts/verify.py --list-emulators # list all emulators
|
||||
python scripts/verify.py --list-systems # list all systems
|
||||
python scripts/verify.py --platform retroarch --list-targets # list available targets
|
||||
```
|
||||
|
||||
Verification modes per platform:
|
||||
@@ -45,6 +75,7 @@ Verification modes per platform:
|
||||
| EmuDeck | md5 | MD5 whitelist per system |
|
||||
| RetroDECK | md5 | MD5 per file via component manifests |
|
||||
| RomM | md5 | size + any hash (MD5/SHA1/CRC32) |
|
||||
| BizHawk | sha1 | SHA1 per firmware from FirmwareDatabase.cs |
|
||||
|
||||
### generate_pack.py
|
||||
|
||||
@@ -67,6 +98,14 @@ python scripts/generate_pack.py --platform retroarch --split --group-by manufact
|
||||
python scripts/generate_pack.py --from-md5 d8f1206299c48946e6ec5ef96d014eaa
|
||||
python scripts/generate_pack.py --platform batocera --from-md5-file missing.txt
|
||||
python scripts/generate_pack.py --platform retroarch --list-systems
|
||||
|
||||
# Hardware target filtering
|
||||
python scripts/generate_pack.py --all --target x86_64
|
||||
python scripts/generate_pack.py --platform retroarch --target switch
|
||||
|
||||
# Install manifests (consumed by install.py)
|
||||
python scripts/generate_pack.py --all --manifest --output-dir install/
|
||||
python scripts/generate_pack.py --manifest-targets --output-dir install/targets/
|
||||
```
|
||||
|
||||
Packs include platform baseline files plus files required by the platform's cores.
|
||||
@@ -82,17 +121,45 @@ If none exists, the platform version is kept and the discrepancy is reported.
|
||||
- `--split --group-by manufacturer`: group split packs by manufacturer (Sony, Nintendo, Sega...)
|
||||
- `--from-md5`: look up a hash in the database, or build a custom pack with `--platform`/`--emulator`
|
||||
- `--from-md5-file`: same, reading hashes from a file (one per line, comments with #)
|
||||
- `--target`: filter by hardware target (e.g. `switch`, `rpi4`, `x86_64`)
|
||||
|
||||
### cross_reference.py
|
||||
|
||||
Compare emulator profiles against platform configs.
|
||||
Reports files that cores need but platforms don't declare.
|
||||
Reports files that cores need beyond what platforms declare.
|
||||
|
||||
```bash
|
||||
python scripts/cross_reference.py # all
|
||||
python scripts/cross_reference.py --emulator dolphin # single
|
||||
python scripts/cross_reference.py --emulator dolphin # single
|
||||
python scripts/cross_reference.py --emulator dolphin --json # JSON output
|
||||
```
|
||||
|
||||
### truth.py, generate_truth.py, diff_truth.py
|
||||
|
||||
Generate ground truth from emulator profiles, diff against scraped platform data.
|
||||
|
||||
```bash
|
||||
python scripts/generate_truth.py --platform retroarch # single platform truth
|
||||
python scripts/generate_truth.py --all --output-dir dist/truth/ # all platforms
|
||||
python scripts/diff_truth.py --platform retroarch # diff truth vs scraped
|
||||
python scripts/diff_truth.py --all # diff all platforms
|
||||
```
|
||||
|
||||
### export_native.py
|
||||
|
||||
Export truth data to native platform formats (System.dat, es_bios.xml, checkBIOS.sh, etc.).
|
||||
|
||||
```bash
|
||||
python scripts/export_native.py --platform batocera
|
||||
python scripts/export_native.py --all --output-dir dist/upstream/
|
||||
```
|
||||
|
||||
### validation.py
|
||||
|
||||
Validation index and ground truth formatting. Used by verify.py for emulator-level checks
|
||||
(size, CRC32, MD5, SHA1, crypto). Separates reproducible hash checks from cryptographic
|
||||
validations that require console-specific keys.
|
||||
|
||||
### refresh_data_dirs.py
|
||||
|
||||
Fetch data directories (Dolphin Sys, PPSSPP assets, blueMSX databases)
|
||||
@@ -107,24 +174,45 @@ python scripts/refresh_data_dirs.py --key dolphin-sys --force
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `common.py` | Shared library: hash computation, file resolution, platform config loading, emulator profiles, target filtering |
|
||||
| `dedup.py` | Deduplicate `bios/`, move duplicates to `.variants/`. RPG Maker and ScummVM excluded (NODEDUP) |
|
||||
| `validate_pr.py` | Validate BIOS files in pull requests |
|
||||
| `auto_fetch.py` | Fetch missing BIOS files from known sources |
|
||||
| `validate_pr.py` | Validate BIOS files in pull requests, post markdown report |
|
||||
| `auto_fetch.py` | Fetch missing BIOS files from known sources (4-step pipeline) |
|
||||
| `list_platforms.py` | List active platforms (used by CI) |
|
||||
| `download.py` | Download packs from GitHub releases |
|
||||
| `common.py` | Shared library: hash computation, file resolution, platform config loading, emulator profiles |
|
||||
| `download.py` | Download packs from GitHub releases (Python, multi-threaded) |
|
||||
| `generate_readme.py` | Generate README.md and CONTRIBUTING.md from database |
|
||||
| `generate_site.py` | Generate all MkDocs site pages (this documentation) |
|
||||
| `deterministic_zip.py` | Rebuild MAME BIOS ZIPs deterministically (same ROMs = same hash) |
|
||||
| `crypto_verify.py` | 3DS RSA signature and AES crypto verification |
|
||||
| `sect233r1.py` | Pure Python ECDSA verification on sect233r1 curve (3DS OTP cert) |
|
||||
| `batch_profile.py` | Batch profiling automation for libretro cores |
|
||||
| `check_buildbot_system.py` | Detect stale data directories by comparing with buildbot |
|
||||
| `migrate.py` | Migrate flat bios structure to Manufacturer/Console/ hierarchy |
|
||||
|
||||
## Installation tools
|
||||
|
||||
Cross-platform BIOS installer for end users:
|
||||
|
||||
```bash
|
||||
# Python installer (auto-detects platform)
|
||||
python install.py
|
||||
|
||||
# Shell one-liner (Linux/macOS)
|
||||
bash scripts/download.sh retroarch ~/RetroArch/system/
|
||||
bash scripts/download.sh --list
|
||||
|
||||
# Or via install.sh wrapper (detects curl/wget, runs install.py)
|
||||
bash install.sh
|
||||
```
|
||||
|
||||
`install.py` auto-detects the user's platform by checking config files,
|
||||
downloads the matching BIOS pack from GitHub releases with SHA1 verification,
|
||||
and extracts files to the correct directory. `install.ps1` provides
|
||||
equivalent functionality for Windows/PowerShell.
|
||||
|
||||
## Large files
|
||||
|
||||
Files over 50 MB are stored as assets on the `large-files` GitHub release.
|
||||
They are listed in `.gitignore` so they don't bloat the git repository.
|
||||
They are listed in `.gitignore` to keep the git repository lightweight.
|
||||
`generate_db.py` downloads them from the release when rebuilding the database,
|
||||
using `fetch_large_file()` from `common.py`. The same function is used by
|
||||
`generate_pack.py` when a file has a hash mismatch with the local variant.
|
||||
@@ -141,11 +229,49 @@ Located in `scripts/scraper/`. Each inherits `BaseScraper` and implements `fetch
|
||||
| `retrobat_scraper` | batocera-systems.json | JSON |
|
||||
| `emudeck_scraper` | checkBIOS.sh | Bash + CSV |
|
||||
| `retrodeck_scraper` | component manifests | JSON per component |
|
||||
| `romm_scraper` | known_bios_files.json | JSON |
|
||||
| `coreinfo_scraper` | .info files from libretro-core-info | INI-like |
|
||||
| `bizhawk_scraper` | FirmwareDatabase.cs | C# source |
|
||||
| `mame_hash_scraper` | mamedev/mame source tree | C source (sparse clone) |
|
||||
| `fbneo_hash_scraper` | FBNeo source tree | C source (sparse clone) |
|
||||
|
||||
Internal modules: `base_scraper.py` (abstract base with `_fetch_raw()` caching
|
||||
and shared CLI), `dat_parser.py` (clrmamepro DAT format parser).
|
||||
and shared CLI), `dat_parser.py` (clrmamepro DAT format parser),
|
||||
`mame_parser.py` (MAME C source BIOS root set parser),
|
||||
`fbneo_parser.py` (FBNeo C source BIOS set parser),
|
||||
`_hash_merge.py` (text-based YAML patching that preserves formatting).
|
||||
|
||||
Adding a scraper: inherit `BaseScraper`, implement `fetch_requirements()`,
|
||||
call `scraper_cli(YourScraper)` in `__main__`.
|
||||
|
||||
## Target scrapers
|
||||
|
||||
Located in `scripts/scraper/targets/`. Each inherits `BaseTargetScraper` and implements `fetch_targets()`.
|
||||
|
||||
| Scraper | Source | Targets |
|
||||
|---------|--------|---------|
|
||||
| `retroarch_targets_scraper` | libretro buildbot nightly | 20+ architectures |
|
||||
| `batocera_targets_scraper` | Config.in + es_systems.yml | 35+ boards |
|
||||
| `emudeck_targets_scraper` | EmuScripts GitHub API | steamos, windows |
|
||||
| `retropie_targets_scraper` | scriptmodules + rp_module_flags | 7 platforms |
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.targets.retroarch_targets_scraper --dry-run
|
||||
python -m scripts.scraper.targets.batocera_targets_scraper --dry-run
|
||||
```
|
||||
|
||||
## Exporters
|
||||
|
||||
Located in `scripts/exporter/`. Each inherits `BaseExporter` and implements `export()`.
|
||||
|
||||
| Exporter | Output format |
|
||||
|----------|--------------|
|
||||
| `systemdat_exporter` | clrmamepro DAT (RetroArch System.dat) |
|
||||
| `batocera_exporter` | Python dict (batocera-systems) |
|
||||
| `recalbox_exporter` | XML (es_bios.xml) |
|
||||
| `retrobat_exporter` | JSON (batocera-systems.json) |
|
||||
| `emudeck_exporter` | Bash script (checkBIOS.sh) |
|
||||
| `retrodeck_exporter` | JSON (component_manifest.json) |
|
||||
| `romm_exporter` | JSON (known_bios_files.json) |
|
||||
| `lakka_exporter` | clrmamepro DAT (delegates to systemdat) |
|
||||
| `retropie_exporter` | clrmamepro DAT (delegates to systemdat) |
|
||||
|
||||
243
wiki/troubleshooting.md
Normal file
243
wiki/troubleshooting.md
Normal file
@@ -0,0 +1,243 @@
|
||||
# Troubleshooting - RetroBIOS
|
||||
|
||||
Diagnosis guide organized by symptom. Each section describes what to check and how to fix it.
|
||||
|
||||
## Game won't start / black screen
|
||||
|
||||
Most launch failures are caused by a missing or incorrect BIOS file.
|
||||
|
||||
**Check if the BIOS exists:**
|
||||
|
||||
```bash
|
||||
python scripts/verify.py --platform retroarch --verbose
|
||||
python scripts/verify.py --system sony-playstation
|
||||
```
|
||||
|
||||
Look for `MISSING` entries in the output. A missing required BIOS means the core
|
||||
cannot start games for that system at all.
|
||||
|
||||
**Check if the hash matches:**
|
||||
|
||||
Look for `HASH_MISMATCH` in the verify output. This means the file exists but
|
||||
contains different data than expected. Common causes:
|
||||
|
||||
- Wrong region (a PAL BIOS instead of NTSC, or vice versa)
|
||||
- Wrong hardware revision (e.g. SCPH-5501 vs SCPH-1001 for PlayStation)
|
||||
- Corrupted download
|
||||
|
||||
Each system page on the site lists the expected hashes. Compare your file's
|
||||
MD5 or SHA1 against those values.
|
||||
|
||||
**Wrong region BIOS:**
|
||||
|
||||
Some cores require region-specific BIOS files. A Japanese BIOS won't boot
|
||||
North American games on cores that enforce region matching. Check the emulator
|
||||
profile for your core to see which regions are supported and which files
|
||||
correspond to each.
|
||||
|
||||
## BIOS not found by emulator
|
||||
|
||||
The file exists on disk, but the emulator reports it as missing.
|
||||
|
||||
**Wrong directory:**
|
||||
|
||||
Each platform expects BIOS files in a specific base directory:
|
||||
|
||||
- RetroArch, Lakka: `system/` inside the RetroArch directory
|
||||
- Batocera: `/userdata/bios/`
|
||||
- Recalbox: `/recalbox/share/bios/`
|
||||
- RetroPie: `~/RetroPie/BIOS/`
|
||||
|
||||
Some cores expect files in subdirectories (e.g. `dc/` for Dreamcast, `pcsx2/bios/`
|
||||
for PlayStation 2). Check the `path:` field in the emulator profile for the exact
|
||||
expected location relative to the base directory.
|
||||
|
||||
**Wrong filename:**
|
||||
|
||||
Cores match BIOS files by exact filename. If a core expects `scph5501.bin` and your
|
||||
file is named `SCPH-5501.BIN`, it won't be found on platforms that do exact name matching.
|
||||
|
||||
Check the emulator profile for the expected filename and any aliases listed under
|
||||
`aliases:`. Aliases are alternative names that the core also accepts.
|
||||
|
||||
**Case sensitivity:**
|
||||
|
||||
Linux filesystems are case-sensitive. A file named `Bios.ROM` won't match a lookup
|
||||
for `bios.rom`. Windows and macOS are case-insensitive by default, so the same
|
||||
file works there but fails on Linux.
|
||||
|
||||
Batocera's verification uses `casefold()` for case-insensitive matching, but
|
||||
the actual emulator may still require exact case. When in doubt, use the exact
|
||||
filename from the emulator profile.
|
||||
|
||||
## Hash mismatch / UNTESTED
|
||||
|
||||
`verify.py` reports `HASH_MISMATCH` or `UNTESTED` for a file.
|
||||
|
||||
**HASH_MISMATCH:**
|
||||
|
||||
The file exists and was hashed, but the computed hash doesn't match any expected
|
||||
value. This means you have a different version of the file than what the platform
|
||||
or emulator expects.
|
||||
|
||||
To find the correct version, check the system page on the site. It lists every
|
||||
known BIOS file with its expected MD5 and SHA1.
|
||||
|
||||
**UNTESTED:**
|
||||
|
||||
On existence-only platforms (RetroArch, Lakka, RetroPie), the file is present
|
||||
but its hash was not verified against a known value. The platform itself only
|
||||
checks that the file exists. The `--verbose` flag shows ground truth data from
|
||||
emulator profiles, which can confirm whether the file's hash is actually correct.
|
||||
|
||||
**The .variants/ directory:**
|
||||
|
||||
When multiple versions of the same BIOS exist (different revisions, regions, or
|
||||
dumps), the primary version lives in the main directory and alternatives live in
|
||||
`.variants/`. `verify.py` checks the primary file first, then falls back to
|
||||
variants when resolving by hash.
|
||||
|
||||
If your file matches a variant hash but not the primary, it's a valid BIOS --
|
||||
just not the preferred version. Some cores accept multiple versions.
|
||||
|
||||
## Pack is missing files
|
||||
|
||||
A generated pack doesn't contain all the files you expected.
|
||||
|
||||
**Severity levels:**
|
||||
|
||||
`verify.py` assigns a severity to each issue. Not all missing files are equally
|
||||
important:
|
||||
|
||||
| Severity | Meaning | Action needed |
|
||||
|----------|---------|---------------|
|
||||
| CRITICAL | Required file missing or hash mismatch on MD5 platforms | Must fix. Core won't function. |
|
||||
| WARNING | Optional file missing, or hash mismatch on existence platforms | Core works but with reduced functionality. |
|
||||
| INFO | Optional file missing on existence-only platforms, or HLE fallback available | Core works fine, BIOS improves accuracy. |
|
||||
| OK | File present and verified | No action needed. |
|
||||
|
||||
Focus on CRITICAL issues first. WARNING files improve the experience but aren't
|
||||
strictly necessary. INFO files are nice to have.
|
||||
|
||||
**Large files (over 50 MB):**
|
||||
|
||||
Files like PS3UPDAT.PUP, PSVUPDAT.PUP, and Switch firmware are too large for the
|
||||
git repository. They are stored as GitHub release assets under the `large-files`
|
||||
release and downloaded at build time.
|
||||
|
||||
If a pack build fails to include these, check your network connection. In offline
|
||||
mode (`--offline`), large files are only included if already cached locally in
|
||||
`.cache/large/`.
|
||||
|
||||
**Data directories:**
|
||||
|
||||
Some cores need entire directory trees rather than individual files (e.g. Dolphin's
|
||||
`Sys/` directory, PPSSPP's `assets/`). These are fetched by `refresh_data_dirs.py`
|
||||
from upstream repositories.
|
||||
|
||||
In offline mode, data directories are only included if already cached in `data/`.
|
||||
Run `python scripts/refresh_data_dirs.py` to fetch them.
|
||||
|
||||
## verify.py reports errors
|
||||
|
||||
How to read and interpret `verify.py` output.
|
||||
|
||||
**Status codes:**
|
||||
|
||||
| Status | Meaning |
|
||||
|--------|---------|
|
||||
| `ok` | File present, hash matches (or existence check passed) |
|
||||
| `untested` | File present, hash not confirmed (existence-only platforms) |
|
||||
| `missing` | File not found in the repository |
|
||||
| `hash_mismatch` | File found but hash doesn't match expected value |
|
||||
| `size_mismatch` | File found but size doesn't match what the emulator expects |
|
||||
|
||||
**Reading the output:**
|
||||
|
||||
Each line shows the file path, its status, and severity. In verbose mode, ground
|
||||
truth data from emulator profiles is appended, showing which cores reference the
|
||||
file and what validations they perform.
|
||||
|
||||
```
|
||||
scph5501.bin ok [OK]
|
||||
dc_boot.bin missing [CRITICAL]
|
||||
gba_bios.bin untested [WARNING]
|
||||
```
|
||||
|
||||
**Cross-reference section:**
|
||||
|
||||
After per-file results, `verify.py` prints a cross-reference report. This lists
|
||||
files that emulator cores need but that the platform YAML doesn't declare. These
|
||||
files are still included in packs automatically, but the report helps identify
|
||||
gaps in platform coverage data.
|
||||
|
||||
The cross-reference uses `resolve_platform_cores()` to determine which emulator
|
||||
profiles are relevant for each platform, then checks whether each profile's files
|
||||
appear in the platform config.
|
||||
|
||||
**Filtering output:**
|
||||
|
||||
```bash
|
||||
# By platform
|
||||
python scripts/verify.py --platform batocera
|
||||
|
||||
# By emulator core
|
||||
python scripts/verify.py --emulator beetle_psx
|
||||
|
||||
# By system
|
||||
python scripts/verify.py --system sony-playstation
|
||||
|
||||
# By hardware target
|
||||
python scripts/verify.py --platform retroarch --target switch
|
||||
|
||||
# JSON for scripted processing
|
||||
python scripts/verify.py --platform retroarch --json
|
||||
```
|
||||
|
||||
## Installation script fails
|
||||
|
||||
Problems with `install.py`, `install.sh`, or `download.sh`.
|
||||
|
||||
**Network issues:**
|
||||
|
||||
The installer downloads packs from GitHub releases. If the download fails:
|
||||
|
||||
- Check your internet connection
|
||||
- Verify that `https://github.com` is reachable
|
||||
- If behind a proxy, set `HTTPS_PROXY` in your environment
|
||||
- Try again later if GitHub is experiencing issues
|
||||
|
||||
**Permission denied:**
|
||||
|
||||
The installer needs write access to the target directory.
|
||||
|
||||
- On Linux/macOS: check directory ownership (`ls -la`) and run with appropriate
|
||||
permissions. Avoid running as root unless the target directory requires it.
|
||||
- On Windows: run PowerShell as Administrator if installing to a protected directory.
|
||||
|
||||
**Platform not detected:**
|
||||
|
||||
`install.py` auto-detects your platform by checking for known config files. If
|
||||
detection fails, specify the platform manually:
|
||||
|
||||
```bash
|
||||
python install.py --platform retroarch --dest ~/RetroArch/system/
|
||||
python install.py --platform batocera --dest /userdata/bios/
|
||||
```
|
||||
|
||||
Use `python install.py --help` to see all available platforms and options.
|
||||
|
||||
**Pack not found in release:**
|
||||
|
||||
If the installer reports that no pack exists for your platform, check available
|
||||
releases:
|
||||
|
||||
```bash
|
||||
python scripts/download.py --list
|
||||
# or
|
||||
bash scripts/download.sh --list
|
||||
```
|
||||
|
||||
Some platforms share packs (Lakka uses the RetroArch pack). The installer handles
|
||||
this mapping automatically, but if you're downloading manually, check which pack
|
||||
name corresponds to your platform.
|
||||
248
wiki/verification-modes.md
Normal file
248
wiki/verification-modes.md
Normal file
@@ -0,0 +1,248 @@
|
||||
# Verification Modes
|
||||
|
||||
Each platform verifies BIOS files differently. `verify.py` replicates the native behavior
|
||||
of each platform so that verification results match what the platform itself would report.
|
||||
|
||||
|
||||
## Existence Mode
|
||||
|
||||
**Platforms**: RetroArch, Lakka, RetroPie
|
||||
|
||||
**Source**: RetroArch `core_info.c`, function `path_is_valid()`
|
||||
|
||||
The most straightforward mode. A file is OK if it exists at the expected path. No hash is checked.
|
||||
Any file with the correct name passes, regardless of content.
|
||||
|
||||
| Condition | Status | Severity (required) | Severity (optional) |
|
||||
|-----------|--------|---------------------|---------------------|
|
||||
| File present | OK | OK | OK |
|
||||
| File missing | MISSING | WARNING | INFO |
|
||||
|
||||
RetroArch does not distinguish between a correct and an incorrect BIOS at the verification
|
||||
level. A corrupt or wrong-region file still shows as present. This is by design in the
|
||||
upstream code: `core_info.c` only calls `path_is_valid()` and does not open or hash the file.
|
||||
|
||||
Lakka and RetroPie inherit this behavior through platform config inheritance
|
||||
(`inherits: retroarch` in the platform YAML).
|
||||
|
||||
|
||||
## MD5 Mode
|
||||
|
||||
**Platforms**: Batocera, RetroBat, Recalbox, EmuDeck, RetroDECK, RomM
|
||||
|
||||
All MD5-mode platforms compute a hash of the file and compare it against an expected value.
|
||||
The details vary by platform.
|
||||
|
||||
### Standard MD5 (Batocera, RetroBat)
|
||||
|
||||
`verify.py` replicates Batocera's `md5sum()` function. The file is read in binary mode,
|
||||
hashed with MD5, and compared case-insensitively against the expected value.
|
||||
|
||||
| Condition | Status | Severity (required) | Severity (optional) |
|
||||
|-----------|--------|---------------------|---------------------|
|
||||
| Hash matches | OK | OK | OK |
|
||||
| File present, hash differs | UNTESTED | WARNING | WARNING |
|
||||
| File missing | MISSING | CRITICAL | WARNING |
|
||||
|
||||
If the `resolve_local_file` step already confirmed the MD5 match (status `md5_exact`),
|
||||
`verify.py` skips re-hashing and returns OK directly.
|
||||
|
||||
### Truncated MD5 (Batocera bug)
|
||||
|
||||
Some entries in Batocera's system data contain 29-character MD5 strings instead of
|
||||
the standard 32. This is a known upstream bug. `verify.py` handles it by prefix matching:
|
||||
if the expected hash is shorter than 32 characters, the actual hash is compared against
|
||||
only its first N characters.
|
||||
|
||||
### md5_composite (Recalbox ZIP verification)
|
||||
|
||||
Recalbox computes `Zip::Md5Composite` for ZIP files: the MD5 of the concatenation of all
|
||||
inner file MD5s (sorted by filename). `verify.py` replicates this with `md5_composite()`
|
||||
from `common.py`. When a ZIP file's direct MD5 does not match, the composite is tried
|
||||
before reporting a mismatch.
|
||||
|
||||
### Multi-hash (Recalbox)
|
||||
|
||||
Recalbox allows comma-separated MD5 values for a single file entry, accepting any one
|
||||
of them as valid. `verify.py` splits on commas and tries each hash. A match against any
|
||||
listed hash is OK.
|
||||
|
||||
### Mandatory levels (Recalbox)
|
||||
|
||||
Recalbox uses three severity levels derived from two YAML fields (`mandatory` and
|
||||
`hashMatchMandatory`):
|
||||
|
||||
| mandatory | hashMatchMandatory | Color | verify.py mapping |
|
||||
|-----------|--------------------|--------|-------------------|
|
||||
| true | true | RED | CRITICAL |
|
||||
| true | false | YELLOW | WARNING |
|
||||
| false | (any) | GREEN | INFO |
|
||||
|
||||
### checkInsideZip (Batocera zippedFile)
|
||||
|
||||
When a platform entry has a `zipped_file` field, the expected MD5 is not the hash of the
|
||||
ZIP container but of a specific ROM file inside the ZIP. `verify.py` replicates Batocera's
|
||||
`checkInsideZip()`:
|
||||
|
||||
1. Open the ZIP.
|
||||
2. Find the inner file by name (case-insensitive via `casefold()`).
|
||||
3. Read its contents and compute MD5.
|
||||
4. Compare against the expected hash.
|
||||
|
||||
If the inner file is not found inside the ZIP, the status is UNTESTED with a reason string.
|
||||
|
||||
### RomM verification
|
||||
|
||||
RomM checks both file size and hash. It accepts any hash type (MD5, SHA1, or CRC32).
|
||||
ZIP files are not opened; only the container is checked. `verify.py` replicates this
|
||||
by checking size first, then trying each available hash.
|
||||
|
||||
|
||||
## SHA1 Mode
|
||||
|
||||
**Platforms**: BizHawk
|
||||
|
||||
BizHawk firmware entries use SHA1 as the primary hash. `verify.py` computes SHA1
|
||||
via `compute_hashes()` and compares case-insensitively.
|
||||
|
||||
| Condition | Status | Severity (required) | Severity (optional) |
|
||||
|-----------|--------|---------------------|---------------------|
|
||||
| SHA1 matches | OK | OK | OK |
|
||||
| File present, SHA1 differs | UNTESTED | WARNING | WARNING |
|
||||
| File missing | MISSING | CRITICAL | WARNING |
|
||||
|
||||
|
||||
## Emulator-Level Validation
|
||||
|
||||
Independent of platform verification mode, `verify.py` runs emulator-level validation
|
||||
from `validation.py`. This layer uses data from emulator profiles (YAML files in
|
||||
`emulators/`), which are source-verified against emulator code.
|
||||
|
||||
### Validation index
|
||||
|
||||
`_build_validation_index()` reads all emulator profiles and builds a per-filename
|
||||
index of validation rules. When multiple emulators reference the same file, checks
|
||||
are merged (union of all check types). Conflicting expected values are kept as sets
|
||||
(e.g., multiple accepted CRC32 values for different ROM versions).
|
||||
|
||||
Each entry in the index tracks:
|
||||
|
||||
- `checks`: list of validation types (e.g., `["size", "crc32"]`)
|
||||
- `sizes`: set of accepted exact sizes
|
||||
- `min_size`, `max_size`: bounds when the code accepts a range
|
||||
- `crc32`, `md5`, `sha1`, `sha256`: sets of accepted hash values
|
||||
- `adler32`: set of accepted Adler-32 values
|
||||
- `crypto_only`: non-reproducible checks (see below)
|
||||
- `per_emulator`: per-core detail with source references
|
||||
|
||||
### Check categories
|
||||
|
||||
Validation checks fall into two categories:
|
||||
|
||||
**Reproducible** (`_HASH_CHECKS`): `crc32`, `md5`, `sha1`, `adler32`. These can be
|
||||
computed from the file alone. `verify.py` calculates hashes and compares against
|
||||
accepted values from the index.
|
||||
|
||||
**Non-reproducible** (`_CRYPTO_CHECKS`): `signature`, `crypto`. These require
|
||||
console-specific cryptographic keys (e.g., RSA-2048 for 3DS, AES-128-CBC for certain
|
||||
firmware). `verify.py` reports these as informational but cannot verify them without
|
||||
the keys. Size checks still apply if combined with crypto.
|
||||
|
||||
### Size validation
|
||||
|
||||
Three forms:
|
||||
|
||||
- **Exact size**: `size: 524288` with `validation: [size]`. File must be exactly this many bytes.
|
||||
- **Range**: `min_size: 40`, `max_size: 131076` with `validation: [size]`. File size must fall within bounds.
|
||||
- **Informational**: `size: 524288` without `validation: [size]`. The size is documented but the emulator does not check it at runtime.
|
||||
|
||||
### Complement to platform checks
|
||||
|
||||
Emulator validation runs after platform verification. When a file passes platform checks
|
||||
(e.g., existence-mode OK) but fails emulator validation (e.g., wrong CRC32), the result
|
||||
includes a `discrepancy` field:
|
||||
|
||||
```
|
||||
file present (OK) but handy says size mismatch: got 256, accepted [512]
|
||||
```
|
||||
|
||||
This catches cases where a file has the right name but wrong content, which existence-mode
|
||||
platforms cannot detect.
|
||||
|
||||
|
||||
## Severity Matrix
|
||||
|
||||
`compute_severity()` maps the combination of status, required flag, verification mode,
|
||||
and HLE fallback to a severity level.
|
||||
|
||||
| Mode | Status | required | hle_fallback | Severity |
|
||||
|------|--------|----------|--------------|----------|
|
||||
| any | OK | any | any | OK |
|
||||
| any | MISSING | any | true | INFO |
|
||||
| existence | MISSING | true | false | WARNING |
|
||||
| existence | MISSING | false | false | INFO |
|
||||
| md5/sha1 | MISSING | true | false | CRITICAL |
|
||||
| md5/sha1 | MISSING | false | false | WARNING |
|
||||
| md5/sha1 | UNTESTED | any | false | WARNING |
|
||||
|
||||
**HLE fallback**: when an emulator profile marks a file with `hle_fallback: true`, the
|
||||
core has a built-in high-level emulation path and functions without the file. Missing
|
||||
files are downgraded to INFO regardless of platform mode or required status. The file
|
||||
is still included in packs (better accuracy with the real BIOS), but its absence is not
|
||||
actionable.
|
||||
|
||||
|
||||
## File Resolution Chain
|
||||
|
||||
Before verification, each file entry is resolved to a local path by `resolve_local_file()`.
|
||||
The function tries these steps in order, returning the first match:
|
||||
|
||||
| Step | Method | Returns | When it applies |
|
||||
|------|--------|---------|-----------------|
|
||||
| 0 | Path suffix exact | `exact` | `dest_hint` matches `by_path_suffix` index (regional variants with same filename, e.g., `GC/USA/IPL.bin` vs `GC/EUR/IPL.bin`) |
|
||||
| 1 | SHA1 exact | `exact` | SHA1 present in the file entry and found in database |
|
||||
| 2 | MD5 direct lookup | `md5_exact` | MD5 present, not a `zipped_file` entry, name matches (prevents cross-contamination from unrelated files sharing an MD5) |
|
||||
| 3 | Name/alias existence | `exact` | No MD5 in entry; any file with matching name or alias exists. Prefers primary over `.variants/` |
|
||||
| 4 | Name + md5_composite/MD5 | `exact` or `hash_mismatch` | Name matches, checks md5_composite for ZIPs and direct MD5 per candidate. Falls back to hash_mismatch if name matches but no hash does |
|
||||
| 5 | ZIP contents index | `zip_exact` | `zipped_file` with MD5; searches inner ROM MD5 across all ZIPs when name-based resolution failed |
|
||||
| 6 | MAME clone fallback | `mame_clone` | File was deduped; resolves via canonical set name (up to 3 levels deep) |
|
||||
| 7 | Data directory scan | `data_dir` | Searches `data/` caches by exact path then case-insensitive basename walk |
|
||||
| 8 | Agnostic fallback | `agnostic_fallback` | File entry marked `agnostic: true`; matches any file under the system path prefix within the size constraints |
|
||||
|
||||
If no step matches, the result is `(None, "not_found")`.
|
||||
|
||||
The `hash_mismatch` status at step 4 means a file with the right name exists but its hash
|
||||
does not match. This still resolves to a local path (the file is present), but verification
|
||||
will report it as UNTESTED with a reason string showing the expected vs actual hash prefix.
|
||||
|
||||
|
||||
## Discrepancy Detection
|
||||
|
||||
When platform verification passes but emulator validation fails, the file has a discrepancy.
|
||||
This happens most often in existence-mode platforms where any file with the right name is
|
||||
accepted.
|
||||
|
||||
### Variant search
|
||||
|
||||
`_find_best_variant()` searches for an alternative file in the repository that satisfies
|
||||
both the platform MD5 requirement and emulator validation:
|
||||
|
||||
1. Look up all files with the same name in the `by_name` index.
|
||||
2. Skip the current file (already known to fail validation).
|
||||
3. For each candidate, check that its MD5 matches the platform expectation.
|
||||
4. Run `check_file_validation()` against the candidate.
|
||||
5. Return the first candidate that passes both checks.
|
||||
|
||||
The search covers files in `.variants/` (alternate hashes stored during deduplication).
|
||||
If a better variant is found, the pack uses it instead of the primary file. If no variant
|
||||
satisfies both constraints, the platform version is kept and the discrepancy is reported
|
||||
in the verification output.
|
||||
|
||||
### Practical example
|
||||
|
||||
A `scph5501.bin` file passes Batocera MD5 verification (hash matches upstream declaration)
|
||||
but fails the emulator profile's size check because the profile was verified against a
|
||||
different revision. `_find_best_variant` scans `.variants/scph5501.bin.*` for a file
|
||||
that matches both the Batocera MD5 and the emulator's size expectation. If found, the
|
||||
variant is used in the pack. If not, the Batocera-verified file is kept and the discrepancy
|
||||
is logged.
|
||||
Reference in New Issue
Block a user