chore: merge the core probe and issue 85 runs

wiki/testing-guide.md

@@ -56,6 +56,7 @@ Every module in `tests/`, by the area it guards.
 | Hardware targets and systems | synthetic + target files | `test_extract_targets`, `test_lakka_targets`, `test_libretro_slugs`, `test_system_ids`, `test_system_selection`, `test_target_names`, `test_target_scrapers`, `test_target_systems`, `test_target_verify` |
 | Installer and downloads | loopback server | `test_check_release`, `test_download`, `test_install`, `test_install_cores`, `test_install_experience`, `test_install_app`, `test_install_from`, `test_install_offline`, `test_install_selection`, `test_install_tui` |
 | Handheld launchers | a pty as the console, fake installer and firmware tools, stubbed PortMaster dialogs | `test_handheld_launchers`, `test_port` |
+| Running cores | parsers on fixed inputs, a libretro core built from C in the test when a compiler exists | `test_core_probe` |
 | Truth and native export | synthetic + cached originals | `test_export_counts`, `test_export_native_layer`, `test_export_report`, `test_exporters`, `test_truth_data_dir_readers`, `test_truth_diff`, `test_truth_layout` |
 | Scrapers, upstream and provenance | inline sources, synthetic | `test_batocera_md5_list`, `test_check_freshness`, `test_contributing_text`, `test_coverage_cache`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_dedup`, `test_doc_pages`, `test_doc_surfaces`, `test_elec_scrapers`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_launchers`, `test_launchers_emudeck`, `test_launchers_recalbox`, `test_launchers_retrobat`, `test_launchers_retrodeck`, `test_launchers_retropie`, `test_launchers_rocknix`, `test_mame_devices`, `test_mame_parser`, `test_mame_ref_audit`, `test_optional_dependencies`, `test_profile_sync`, `test_prose_style`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_romset_recipes`, `test_schema_semantic_gate`, `test_scraper_contract`, `test_site_availability`, `test_site_content_data`, `test_site_copy`, `test_site_cores`, `test_site_emulator_page`, `test_site_exports`, `test_site_funnel`, `test_site_images`, `test_site_picker`, `test_site_platform_hashes`, `test_site_r2_components`, `test_site_r2_css`, `test_site_r2_emulators`, `test_site_r2_entry`, `test_site_r2_gate`, `test_site_r2_nav`, `test_site_r2_p0`, `test_site_r2_reference`, `test_site_r2_systems`, `test_site_r2_wiki`, `test_site_r3_collection`, `test_site_r3_components`, `test_site_r3_css`, `test_site_r3_emulators`, `test_site_r3_entry`, `test_site_r3_nav`, `test_site_r3_p0`, `test_site_r3_reference`, `test_site_r3_systems`, `test_site_r3_wiki`, `test_site_r4_components`, `test_site_r4_css`, `test_site_r4_emulators`, `test_site_r4_entry`, `test_site_r4_merge`, `test_site_r4_p0`, `test_site_r4_reference`, `test_site_r4_search`, `test_site_r4_systems`, `test_site_r4_wiki`, `test_site_r5_components`, `test_site_r5_css`, `test_site_r5_emulators`, `test_site_r5_entry`, `test_site_r5_reference`, `test_site_r5_search`, `test_site_r5_systems`, `test_site_r5_wiki`, `test_site_render`, `test_site_system_links`, `test_site_validation`, `test_site_weight`, `test_site_write`, `test_source_ref_targets`, `test_upstream`, `test_upstream_index`, `test_validate_pr`, `test_validate_pr_flow`, `test_validate_pr_hashes`, `test_workflow_paths` |
 

wiki/tools.md

@@ -704,6 +704,63 @@ python scripts/refresh_stale.py
 python scripts/refresh_stale.py --only platforms,targets --jobs 6
 ```
 
+### core_probe.py
+
+Runs a libretro core with no window, no sound and no input device, and
+reports what it did. It is the test in real conditions behind a claim about
+a core: which files it opens, what it shows, how fast it runs.
+
+```bash
+# one run, printed as JSON
+python scripts/core_probe.py run --core mednafen_saturn_libretro.so \
+  --content game.cue --system-file sega_101.bin=bios/Sega/Saturn/sega_101.bin \
+  --option beetle_saturn_region=Japan --frames 600
+
+# timed repetitions after warm-up, written as a record
+python scripts/core_probe.py bench --core core.so --content game.cue \
+  --warmup 300 --frames 3000 --repeat 5 --record measurements/
+
+# one system file swapped between runs, compared on one observable
+python scripts/core_probe.py check --core core.so --content disc.cue \
+  --vary mpeg_card.bin=a.bin --vary mpeg_card.bin=b.bin --vary mpeg_card.bin= \
+  --observable video_sha1 --claim "..." --method "..." --record measurements/
+```
+
+Each repetition runs in a fresh process under `strace -f -e trace=openat,open`
+when strace is installed. The files the core opened under the system
+directory come from the kernel, each hashed again after the run. Without
+strace the record says that tracing was unavailable.
+
+| Option | Effect |
+|--------|--------|
+| `--system-file DEST=SRC` | copy SRC into a staged system directory as DEST |
+| `--option KEY=VALUE` | core option. Unset options take the first value the core declares |
+| `--frames`, `--warmup` | frames timed, and frames run before timing starts |
+| `--press FRAME:BUTTON[:LEN]` | hold a libretro joypad button on port 0 (`a`, `b`, `start`, `up`...). A menu may need 15 frames or more to see it |
+| `--joypad-ports N` | ports given a joypad after loading, as RetroArch does (default 2) |
+| `--env KEY=VALUE` | environment of the core process. A core that reads the host clock repeats only with the clock fixed, for example `LD_PRELOAD` of libfaketime with `FAKETIME`. A value naming a file is recorded by file name and sha256 |
+| `--frame-hash-every K`, `--png-dir` | sha1 of every K-th frame, and those frames as PNG |
+| `--state-var SECTION.VAR` | a variable of a Mednafen save state, as an observable |
+| `--state-dir` | keep the save state taken at the end of the first repetition |
+| `--core-repo`, `--core-commit`, `--build-command` | where the core build comes from, copied into the record |
+| `--record DIR` | write `DIR/<core>/<date>-<sha12>.json` (schema `core-probe-record.schema.json`) |
+| `--max-load` | refuse to record when the 1-minute load average is above it (default: a quarter of the CPU count) |
+| `--no-trace` | run without strace |
+| `@FILE` | read arguments from FILE, one per line |
+
+Observables are `frame_sha1` (last frame), `video_sha1` (every frame, chained),
+`audio_sha1`, `state_sha1` (save state at the end), plus `state:SECTION.VAR`
+and `frame_hashes` when asked for. A repetition that disagrees with the first
+one is flagged. A `check` reports `inconclusive` instead of a comparison when,
+in any variant, the compared observable differs between repetitions, a run did
+not load, or a file changed during the run. A disagreement on another
+observable stays flagged in the record. In core log lines, the staged system,
+save and content directories read `<system>`, `<save>` and `<content>`. A record carries the core build (sha256 of the
+`.so`, the compilers its `.comment` section names), the frontend commit, the
+host (CPU, kernel, Python, load before and after) and each input by hash.
+Core log lines arrive through the libretro log interface. Floating-point
+arguments of a log line cannot be read from Python and print as `<double>`.
+
 ### Other tools
 
 | Script | Purpose |