mirror of
https://github.com/Abdess/retroarch_system.git
synced 2026-10-22 19:59:07 -05:00
feat: generate MAME device sets from -listxml
A machine that instantiates a device loads the device's ROMs from <shortname>.zip (src/emu/device.cpp:140-152), and a missing required ROM stops that machine (romload.cpp:529-570, 667-671). The BIOS root scan never sees these sets. mame_hash_scraper --devices --listxml reads them from the -listxml of the release mame.yml pins, refuses another release, and cites each set at the ROM_START block holding its ROMs. The BIOS root merge also wrote refs without the upstream prefix and would have rewritten all 171 of them in mame.yml.
This commit is contained in:
1 parent
7a65a2cf83
commit
8de45b6f45
7 files changed
+974
-13
No files matched your search
+13
-1
@@ -45,7 +45,7 @@ Every module in `tests/`, by the area it guards.
|
||||
| Hardware targets and systems | synthetic + target files | `test_extract_targets`, `test_lakka_targets`, `test_libretro_slugs`, `test_system_ids`, `test_system_selection`, `test_target_names`, `test_target_scrapers`, `test_target_systems`, `test_target_verify` |
|
||||
| Installer and downloads | loopback server | `test_check_release`, `test_download`, `test_install`, `test_install_cores`, `test_install_experience`, `test_install_selection` |
|
||||
| Truth and native export | synthetic + cached originals | `test_export_counts`, `test_export_native_layer`, `test_export_report`, `test_exporters`, `test_truth_diff`, `test_truth_layout` |
|
||||
| Scrapers, upstream and provenance | inline sources, synthetic | `test_check_freshness`, `test_dedup`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_mame_parser`, `test_mame_ref_audit`, `test_profile_sync`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_romset_recipes`, `test_scraper_contract`, `test_upstream`, `test_upstream_index` |
|
||||
| Scrapers, upstream and provenance | inline sources, synthetic | `test_check_freshness`, `test_dedup`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_mame_devices`, `test_mame_parser`, `test_mame_ref_audit`, `test_profile_sync`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_romset_recipes`, `test_scraper_contract`, `test_upstream`, `test_upstream_index` |
|
||||
| Site, docs and repository contracts | rendered pages, sources | `test_contributing_text`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_optional_dependencies`, `test_prose_style`, `test_site_availability`, `test_site_cores`, `test_site_exports`, `test_site_platform_hashes`, `test_site_system_links`, `test_site_validation`, `test_site_write`, `test_validate_pr`, `test_validate_pr_hashes`, `test_workflow_paths` |
|
||||
|
||||
## Test architecture
|
||||
@@ -99,6 +99,18 @@ sets from driver files. Fixtures are inline C source snippets containing
|
||||
- Multi-region ROM blocks
|
||||
- Macro expansion and edge cases
|
||||
|
||||
**test_mame_devices.** Tests the device ROM sets read from a MAME `-listxml`
|
||||
and merged into a profile, on an inline listxml and driver source. Covers:
|
||||
|
||||
- Only `isdevice="yes"` sets with a dumped ROM kept, from a plain or zipped listxml
|
||||
- The `ROM_START` block cited for a set: found through a ROM macro, a block
|
||||
inside a comment ignored, one dump under two names told apart, the label
|
||||
`device_rom_region` returns, not found and ambiguous
|
||||
- Merge: a set added under the generated section in name order, a written
|
||||
entry's contents refreshed with its manual fields kept, BIOS root sets left
|
||||
as written, a second run changing nothing, a new BIOS root placed before the
|
||||
section
|
||||
|
||||
**test_fbneo_parser.** Tests the FBNeo C source parser that identifies
|
||||
`BDF_BOARDROM` sets. Same inline fixture approach.
|
||||
|
||||
|
||||
+21
-2
@@ -622,7 +622,7 @@ Further options, by script:
|
||||
| `validate_pr.py` | `--changed` takes the BIOS files the branch changes, `--markdown`, `--json` |
|
||||
| `validate_site.py` | `--site-dir` (rendered site), `--config` (mkdocs.yml), `--max-errors N` |
|
||||
| `scraper/coreinfo_scraper.py` | `--compare-db database.json` lists the `.info` firmware the database lacks |
|
||||
| `scraper/mame_hash_scraper.py` | `--release mameNNNN --profile FILE --source-commit SHA --core NAME --built-by PLATFORM` writes the profile of a MAME build a platform compiles |
|
||||
| `scraper/mame_hash_scraper.py` | `--release mameNNNN --profile FILE --source-commit SHA --core NAME --built-by PLATFORM` writes the profile of a MAME build a platform compiles. `--devices --listxml FILE` documents in `mame.yml` every device set with dumped ROMs that the release's `-listxml` names (`isdevice="yes"`), with `--dry-run` to preview |
|
||||
| `scraper/romset_dat_importer.py` | `--cache-dir` (downloaded listxml and DATs) |
|
||||
|
||||
## Installation tools
|
||||
@@ -743,9 +743,28 @@ stand alone.
|
||||
| `misterfpga_scraper` | BiosDB_MiSTer bios_db.json.zip | JSON in ZIP |
|
||||
| `coreinfo_scraper` | .info files from libretro-core-info | INI-like |
|
||||
| `bizhawk_scraper` | FirmwareDatabase.cs | C# source |
|
||||
| `mame_hash_scraper` | mamedev/mame source tree. `--release mameNNNN` writes the profile of one release a platform builds | C source (sparse clone) |
|
||||
| `mame_hash_scraper` | mamedev/mame source tree. `--release mameNNNN` writes the profile of one release a platform builds. `--devices --listxml FILE` adds the device ROM sets of the release's `-listxml` | C source (sparse clone), listxml |
|
||||
| `fbneo_hash_scraper` | FBNeo source tree | C source (sparse clone) |
|
||||
|
||||
A machine that instantiates a device loads the device's ROMs from
|
||||
`<device shortname>.zip`, and a missing ROM that is neither optional nor
|
||||
undumped stops that machine. The BIOS root scan never sees these sets, so
|
||||
`--devices` reads them from the `-listxml` of the release `mame.yml` is pinned
|
||||
on and refuses another release. Each set is cited at the `ROM_START` to
|
||||
`ROM_END` block holding its ROMs, in the file the listxml names, read at the
|
||||
release tag through the upstream cache. When several blocks qualify, the one
|
||||
matching every ROM by name and CRC wins, then the one named after the set, then
|
||||
the one the device class returns from `device_rom_region`. A set still
|
||||
undecided is written without a `source_ref` and listed in the output. New sets
|
||||
go under the `Device ROM sets (generated from MAME -listxml)` comment at the end
|
||||
of `files:` with `required: false`. A set the profile already lists keeps every
|
||||
field a reader wrote and only gets its contents refreshed.
|
||||
|
||||
```bash
|
||||
python -m scripts.scraper.mame_hash_scraper --devices --listxml .cache/dats/mame0289lx.zip --dry-run
|
||||
python -m scripts.scraper.mame_hash_scraper --devices --listxml .cache/dats/mame0289lx.zip
|
||||
```
|
||||
|
||||
Internal modules: `base_scraper.py` (abstract base with `_fetch_raw()` caching
|
||||
and shared CLI), `dat_parser.py` (clrmamepro DAT format parser),
|
||||
`logiqx_parser.py` (Logiqx XML DAT parser, used by the dump catalogs),
|
||||
|
||||
Reference in new issue
Block a user