Commit Graph
100 Commits
Author SHA1 Message Date
Abdessamad Derraz d2ba7e18ba fix: search past a folder this user cannot open
One closed folder under HOME (~/.config/EmuDeck at 000) made the whole
detection return nothing: a RetroArch whose retroarch.cfg was readable
was lost and the run fell back to a folder it does not read. Every
detection probe now goes through _exists, _is_dir or _is_file, which
count a refused place as absent, name the closed folder once, and let
the other probes go on. The decorator stays as the last net.

The refusal is an unsearched event before detect. The interface keeps
the line above the list of setups. At HOME or a drive root A goes into
a visible subfolder, never a card or drive row nor .cache, and with
none it stays on Use this folder and says where it is. A default folder
taken because the platform was not found is no longer called detected.
2026-10-24 02:23:57 +02:00
Abdessamad Derraz 801b8bbda0 fix: one pack answer and fuller reference data
The API reads release_pack from the registry entry, not the inherited config, and the picker gets release-advertised.json. The lookup counts emulators that read a file by name, every record names what was declared, the Data page lists all schemas, the target lists and nested manifest fields, and systems.json items carry their api address.
2026-10-24 00:51:13 +02:00
Abdessamad Derraz 329b9e1bda docs: describe every install manifest field
Nested fields and each omission reason code, read from the code that writes them.
2026-10-24 00:51:13 +02:00
Abdessamad Derraz 3db8ddae2b feat: name the save folder and the pack on screen
The plan and the result now say how many files go to the save folder and
where, from a new saves event. With --from-pack they say the files come
from the pack, not a download. A stop asked for (Start+Select, SIGHUP)
ends on the plain outcome, without the log's last lines.
2026-10-23 22:34:24 +02:00
Abdessamad Derraz 6f958382b7 chore: merge the engine options round 2026-10-23 19:25:34 +02:00
Abdessamad Derraz 8f50015a0c feat: round 4 reference and data 2026-10-23 19:23:58 +02:00
Abdessamad Derraz b34a6058f7 chore: merge main into the engine round 2026-10-23 19:07:44 +02:00
Abdessamad Derraz 35e7300465 feat: add --declared-only and the events schema
--declared-only keeps the entries the platform's own list declares
(cores: null), each slot settled to the platform's copy with no core
alternative left behind. Install, --check, --dry-run, --events,
--list-systems, --system, --region, --emulator, --target,
--installed-cores, --manifest and --from-pack apply it. --core,
--list-cores, --list-emulators, --list-platforms, --list-targets,
--standalone-copies, --restore and --tui refuse it, and the interface
does not open by itself with it. On Batocera it gives 353 files of
14531, the list batocera-systems checks.

Event decision: no schema bump. A `declared` key on `file` would add a
key to an existing kind, which schema 1 forbids. A new kind, `declared`,
follows `check` with the same counts over the platform's own files and
their dests in file-event order. Readers already skip a kind they do
not know (rbevents.py and the interface reader were checked), so
EVENT_SCHEMA stays 1.

An unknown --system is answered with the systems the list files its
key file under: --platform retroarch --system snk-neogeo now suggests
arcade, which holds neogeo.zip, instead of snk-neogeo-cd. neogeo, mvs
and aes follow the same route. Never taken, only suggested. A system
title the manifest carries (SNK Neo Geo CD) is taken for its id when
no other system shares it and no platform name reads the same.

schemas/install-events.schema.json is written from EVENT_KEYS by
scripts/generate_event_schema.py. validate_schemas.py reports a stale
file, and tests compare it with EVENT_KEYS and validate a real run.
Bootstrap pins recomputed.
2026-10-23 19:07:18 +02:00
Abdessamad Derraz 6d938f3d54 feat: reference pages answer every hash and field
A platform or emulator record whose value designates a collection
file carries its sha1 and is filed under the file's other hashes, so
a SHA1 paste finds the setups listing it by MD5. The data page shows
every manifest field, writes URL templates as code, states the EmuDeck
rule, the browser lookup and the empty lists. systems.json gives every
id holding files a page, Files to source splits at 100 rows with an
upload link per file, and MiSTer and BizHawk get no empty core table.
2026-10-23 18:52:13 +02:00
Abdessamad Derraz 00cd203b23 feat: add schema_version to release.json 2026-10-23 18:52:04 +02:00
Abdessamad Derraz 50b42823ca feat: add a per-os data_root to profiles
Apps such as AetherSX2 named their user data folder only in prose, so
the site could neither print it nor show only the OS tabs that apply.
data_root keys android, ios, windows, macos and linux, each with the
path the user finds and the source_ref that builds it.
2026-10-23 18:44:25 +02:00
Abdessamad Derraz 82bf236a5a feat: hold an upstream pin a port vendors
upstream_frozen says why upstream_commit stays: the port vendors that
upstream tree, synced to that release, or merged that commit. The view
is then read at its own pin, the way pin_frozen holds source_commit, and
a bump leaves it. Outside a GitHub fork network nothing else could tell
aps3e's vendored RPCS3 1f080485 from an upstream read on a date.
2026-10-23 17:30:20 +02:00
Abdessamad Derraz eeef4fc6fb feat: packs for the most downloaded setups 2026-10-23 17:18:00 +02:00
Abdessamad Derraz 32c648b67d feat: release packs for marked platforms only
release_pack: true in the registry is the list a release builds: RetroArch,
EmuDeck, RetroBat, Batocera, RomM and MiSTer, 83% of the downloads of
v2026.10.10. generate_pack --all, --verify-packs, the pipeline consistency
step, release_record and check_release read it. Manifests are still written
for every platform.
2026-10-23 17:06:46 +02:00
Abdessamad Derraz 4a1d875f43 feat: name systems in the install manifests
Each manifest carries system_titles, the name the site prints for every system its files list (Sony PlayStation, not sony-playstation or psx), so the installer can show it. Older installers ignore the field.
2026-10-23 15:55:46 +02:00
Abdessamad Derraz 94d18bd7e5 feat: name the system in each lookup record 2026-10-23 10:54:34 +02:00
Abdessamad Derraz e3a54d2423 feat: serve each schema at its own $id 2026-10-23 10:46:05 +02:00
Abdessamad Derraz 0b4504992b feat: read batocera md5 written as a list
batocera.linux 6b08b873 lets a biosFiles item declare its md5 as a
list of strings, and checkBios accepts a file matching any of them.
The scrapers join the values with commas, as for Recalbox, and record
md5_list so the export writes the list back as a list, byte for byte.
2026-10-23 07:39:25 +02:00
Abdessamad Derraz 8b26a8fcac feat: cite sources and files on reference pages
Platform pages link each reader's source line at its pinned revision, name the collection file that serves each entry, show launcher records with their lines, name systems with the id beside them and the list they were read from in words. The largest system lists move to pages of their own so a platform page stays under 500 KB.

Gaps: plain numbers with a bar, a badge only for missing required firmware, Files to source as a heading. Cross-reference: counts per platform, one page per platform, one row per upstream project. Exports: gaps listed once, status, category and the serving file in the CSVs and SQLite, the commit in each API envelope, the provenance Wanted list as CSV. Catalog dates in words on the provenance page.
2026-10-23 02:58:51 +02:00
Abdessamad Derraz 2d26385efe feat: place save-dir files in each system folder
Batocera, KNULLI, Recalbox, RetroBat and RetroDECK hand a libretro core <saves>/<system> at each launch. Each copy lands below every system that runs the core. EmuDeck keeps one unsorted saves/retroarch/saves. The unsupported platforms record why.
2026-10-22 21:16:59 +02:00
Abdessamad Derraz 9a1bf4baa6 feat: record the systems each libretro core runs 2026-10-22 21:16:47 +02:00
Abdessamad Derraz 1cd6c9ee4d feat: place save-dir files in packs and lists
load_from: save_dir entries of a libretro build were left out of every
pack. They now go below the platform's save folder (registry
save_directory: RetroArch saves/, Lakka savefiles/, by core), the
manifest marks them root: save, and verify replays the same placement.
Platforms keeping saves per system or per game list them as omitted.
2026-10-22 19:58:26 +02:00
Abdessamad Derraz 33041b6bd1 feat: find a file by any hash, out of the search
Hashes, link lists, install commands, member tables and long reference
tables leave the site search: the index drops from 21.0 MB (4.88 MB
gzip) to 6.8 MB (1.33 MB gzip). Names and prose stay searchable.

files/?hash= takes a SHA1, MD5, CRC32 or SHA256 and reads one shard of
api/v1/lookup/ (about 22 KB gzip, was 675 KB) listing the collection
card, the platforms that declare the value and the profiles that check
it. files/?sha1= still opens the card, and a sha1- anchor missing from
a page is looked up there, so a card can move between pages.

Excluded blocks are section elements: the search plugin compares
elements by tag, so a div ended at the first nested div.
2026-10-22 12:46:40 +02:00
Abdessamad Derraz 4bc90cf06e chore: merge the elec core names 2026-10-22 06:29:22 +02:00
Abdessamad Derraz 9648b9fc3b feat: read elec ports and per-system cores
EmuELEC's ports system has one emulator, `ports`, the dispatch to a
script ports.yaml generates. The scraper now reads each port's engine
(the libretro core handed to emuelecRunEmu.sh with -C, else port_exe),
so devilutionx, sonic2013, soniccd, sonic-mania, reminiscence and
rigelengine are examined. Standalone emulators keep the spelling the
launch code dispatches on instead of invented lowercase forms, and
media players and runtimes go to the overrides' remove_cores.

Each system of the AmberELEC, EmuELEC and KNULLI lists now carries the
cores its ES list offers it, in menu order, under `cores:`.
2026-10-22 06:28:24 +02:00
Abdessamad Derraz 8b952968d6 feat: add a --no-game-data pack narrowing
The install manifest marks each game data file "category": "game_data":
a platform entry only game data profile entries read, or a core file its
profile declares game data. Firmware carries no category. Every manifest
says "categorized": true so the installer can refuse an older list.

generate_pack.py --no-game-data cuts those files from the platform list
and the core profiles before the build, names each one with the cores that
read it as game data, and tags the pack _NoGameData through _narrowings.
verify.py and the pack check take the same cut, so --verify-packs and the
report describe the pack the builder wrote. Modes without a platform
(--emulator, --system alone, --from-md5) and --all-variants refuse it.
2026-10-22 06:19:05 +02:00
Abdessamad Derraz 841017c887 chore: merge the minui family platforms 2026-10-22 04:09:17 +02:00
Abdessamad Derraz d0802c5360 feat: place core files per handed system dir
A platform can hand each libretro core a system directory of its own
(core_system_dirs). platform_layouts then gives one view per directory,
so the gap report, the builder, the manifest, slots and cross_reference
place a core's files below each folder it is handed, and a core handed
none gets no file. Resolution strips the folder again and searches the
collection by the core's own path.
2026-10-22 04:04:31 +02:00
Abdessamad Derraz 2bdac466a9 feat: add knulli, amberelec and emuelec platforms
Each OS publishes Batocera's systems dict (knulli-systems, and
batocera-systems at AmberELEC's newest prerelease commit and EmuELEC v4.8),
read with the shared parser. Cores come from each OS's own ES list.
AmberELEC and EmuELEC declare neogeo/ and ports/ beside bios/ from
/storage/roms, so their packs extract there and core_destination puts
core files under bios/.
2026-10-22 03:42:20 +02:00
Abdessamad Derraz d5a25190a3 fix: place standalone files in the linked folder
standalone_path is relative to the directories a standalone build
resolves it against, now declared per profile (standalone_directories).
A launcher record that links or passes a whole folder of the BIOS tree
under one of them gives that folder, and platform_layouts rebases the
standalone view once for the builder, the manifest, verify and slots
(launchers.standalone_root, common.standalone_destination).

Flycast on RetroPie and ROCKNIX reads bios/dc, on EmuDeck
bios/flycast/bios, on Batocera (FLYCAST_BIOS_PATH) and RetroBat
(Dreamcast.BiosPath) bios/dc. The ROCKNIX, Batocera and RetroBat
scrapers now record the patched data directory, the environment
variable and the setting that say so.
2026-10-21 12:39:03 +02:00
Abdessamad Derraz d6a0639330 chore: merge audit fixes (packrules)
# Conflicts:
#	scripts/packextras.py
2026-10-21 11:52:28 +02:00
Abdessamad Derraz cd589b2742 chore: merge audit fixes (mame) 2026-10-21 11:48:30 +02:00
Abdessamad Derraz 8643d4a4ea chore: merge audit fixes (site)
# Conflicts:
#	emulators/flycast.yml
#	emulators/primehack.yml
2026-10-21 11:48:30 +02:00
Abdessamad Derraz 56e8cbe046 feat: file mame device sets by their machines
The -listxml names, for every machine, its parent (romof), the devices of its default configuration (device_ref) and the cards its slots offer (slotoption). A machine takes the system of the first set on its romof chain a written entry lists, and passes it to each device it can instantiate. A device set reached from one system gets system:, from several the new per-entry systems:, from none neither, and keeps serving every system of the profile. The gap report, the core extras and slots read the entry's own systems, so generate_pack --system and the installer pick by them.
2026-10-21 11:33:39 +02:00
Abdessamad Derraz ef69687fe1 fix: replay held omissions in install regions
The builder keeps a held regional fallback when the requested file is
missing (since 2026-10-17), and the comment beside the manifest's
omissions still said the opposite. A file the builder holds but the
manifest cannot download sat in omitted_files counted unheld, so the
installer kept a fallback the pack drops. Such an omission now carries
held: true and narrow() counts it held, as the builder does.
2026-10-21 11:18:53 +02:00
Abdessamad Derraz c8c4f3f486 chore: merge round 2 fixes (installer) 2026-10-21 10:17:44 +02:00
Abdessamad Derraz 69965ebb25 feat: link each platform and system to its data
An emulator page linked its YAML and api/v1 JSON, platform and system
pages nothing. Each platform page now links its YAML, its own
api/v1/platforms/<id>.json and its install manifest, and each system
page its api/v1/systems/<slug>.json, both validated by the envelope
schema.
2026-10-21 10:15:11 +02:00
Abdessamad Derraz 5e4d9199c2 feat: accept platform system names in --system
The manifest now carries system_names, each system's native_system names from the platform's own file, and --system, its suggestions and --list-systems read them: --platform batocera --system psx installs sony-playstation.
2026-10-21 09:59:25 +02:00
Abdessamad Derraz 43337c6719 fix: match release packs by exact name
release_record rebuilt nothing from the name builder: a substring match
took RetroArch_v1.22.2_AndroidArm64V8A_BIOS_Pack.zip for the RetroArch
pack. The record now refuses any pack that is not a registered platform's
full pack and stores the platforms each pack serves, which the readers
use instead of the pack name.
2026-10-21 09:51:55 +02:00
Abdessamad Derraz 5fc2423ca5 fix: apply --required-only to data directories
The Required packs and manifests carried every data tree, the FreeIntv
overlays included. A data_directories entry now accepts required: false,
and the platform, emulator and manifest walks leave such a tree out under
--required-only. FreeIntv reads its overlays only with the multi-screen
option and draws a placeholder keypad without one (src/libretro.c:294-296,
350-369).
2026-10-21 05:05:49 +02:00
Abdessamad Derraz 15ffeddfe3 feat: read RetroPie launchers from scriptmodules
The scraper walks each package's functions the way RetroPie runs them in one shell (assignments, arrays, for loops, brace expansion, $md_id, calls into other packages) and records every link, copy, move and setting that names $biosdir, the addEmulator flags (sdltrs -romfile3), the BIOS read outside $biosdir (basilisk --rom, gngeopi -B) and the launch scripts that exit without their file (fs-uae kick13.rom, reicast dc_boot.bin).

Each package also gets a package record with its rp_module_repo and the systems it registers, which maps a package to the build it compiles. The schema gains package, repo, systems and the moves and package kinds. platforms/retropie.yml is regenerated at 46128cd2: only the launchers section is new.
2026-10-20 21:13:09 +02:00
Abdessamad Derraz 57af8e4921 feat: read ROCKNIX launchers from its packages
The scraper reads the start scripts and the configs each package installs at the tag the BIOS list is pinned to: configs that name a BIOS path, the yabasanshiro search, directories handed to DuckStation and the PCSX2 builds, and the ln -s commands that put bios/ where an emulator reads. A package that ships one config per device gets a device field only where the values differ, as melonDS does on AMD64.
2026-10-20 20:45:19 +02:00
Abdessamad Derraz 2556a03ed3 feat: describe platform launchers in the schema
A launcher record says what a platform's own launch code looks for and hands to an emulator, read from that code at the pinned revision.
2026-10-20 20:14:30 +02:00
Abdessamad Derraz f111e1c505 docs: built_by names who ships a build
The resolver reads built_by as the platforms on which a core name means this build. Its definition said platforms that compile a libretro build, which excludes a standalone build a platform installs from the project's release. The field now says what it decides: platforms that ship this build under its core name, compiled or installed from the release.
2026-10-20 16:57:24 +02:00
Abdessamad Derraz e5174a07bf feat: mark files a core finds by content
DuckStation identifies its BIOS by MD5 in a folder it scans
(bios.cpp:208), so its entries' names are labels. The export wrote them
into Batocera's and ROCKNIX's lists as files to hold: a user with only
scph5501.bin read 104 missing BIOS where DuckStation boots. Labels also
lent their md5 to a name another core opens, and paired by name they
replaced System.dat's scph5000.bin hashes.

found_by: name | content, on a profile or a file and keyed by build like
load_from. The truth records which cores open each name (_named_by). A
label pairs with a platform entry by hash only, is never added, merges
with an opened name only on a shared hash, and is counted in the export
report rather than listed as missing.
2026-10-20 16:32:22 +02:00
Abdessamad Derraz 40bc07aa3a feat: record files a launcher never opens
unread_on names the platforms whose launcher points the emulator at other files. RetroBat writes absolute BIOS paths into standalone BlastEm's config, so the default names were reported as required and missing there. Such an entry leaves that platform's gap report and pack, and its emulator page says where it is never opened.
2026-10-20 10:27:15 +02:00
Abdessamad Derraz 1ff2bfc3c0 fix: the API schema holds each endpoint's fields
The data page promises that v1 fields are only ever added, and the envelope schema left items unconstrained, so a renamed field passed validation. Each kind now lists the fields its items carry and their types.
2026-10-20 06:18:25 +02:00
Abdessamad Derraz f7d57dd91d feat: a platform package can differ from upstream
bundled describes the emulator's own release, and a platform package can differ both ways: Batocera's hcl package installs the binary alone and sends users to fetch the data, while its blueMSX core package copies the machines and databases beside the core. not_bundled_on and installed_on name those platforms, and the builder reads them through one rule.
2026-10-20 06:11:10 +02:00
Abdessamad Derraz ddc7432825 feat: packs leave out what the emulator installs
A file or tree installed with the emulator is already on the user's machine, so packs, install manifests and verify's pack count leave it out. bundled is scoped by mode like any entry: Dolphin's Sys and PPSSPP's assets ship with the standalone releases, not with the libretro cores, so they now follow the libretro build only and still reach RetroArch packs. PrimeHack, standalone only, ships its own Sys.
2026-10-20 06:04:10 +02:00
Abdessamad Derraz 77389236e6 feat: record a release with no genuine copy
Paid Android emulators announce versions that circulate only rebuilt or re-signed. The profile stays on the newest unmodified build, and version_source.no_genuine_copy names the release searched for, so check_freshness stops flagging it until the page announces a later one.
2026-10-20 04:49:24 +02:00