dedup.py removes a copy only when another file has the same SHA-1 and
the same name, keeping the higher-priority path, so resolution by hash
still finds the content. Verified rather than assumed: all 17268
manifest entries ship the same bytes as before, no destination gained
or lost, only the source paths consolidate. Every deleted path's content
is still held elsewhere.
12.1 MB and 48 empty directories go with them. The index and the
published artefacts are regenerated on top.
A destination can be declared by more than one system, bare in one and
hash-constrained in another. generate_pack resolves that with
_preferred_entries so the constrained sibling claims the destination;
generate_manifest never did, and named whatever answered to the name.
RetroDECK's bios/d2fdc.zip is the case: declared with an md5 in the
arcade system and bare under apple-ii. The pack carried the right
archive, 262 bytes reached through the MAME clone map, while the
manifest sent install.py to a 256-byte Apple II ROM. Downloading the
ZIP and running the installer gave different files.
A test now hashes every manifest entry a platform pins and compares it
against what the platform declares: 3295 entries, on the platforms whose
frontend reads the bytes. It accepts any of several declarations for one
destination, the member-composite MD5 Recalbox pins for arcade archives,
and Batocera's 29-character prefixes. Reinstating the old manifest entry
fails it.
Rebuilt from the complete state: 8923 files indexed, 436 emulators, and
coverage up on every platform that gained a profile. The gap count falls
from 9 to 7, the three that closed being the C128 kernal, the Cave Story
build and Supermodel's Games.xml.
Whole pipeline green on this revision: database, provenance, recipes,
data directories, hash scrapers, truth, verify, packs, manifests,
consistency and pack integrity across the twelve platforms.
The published manifests still described what the builder produced before
today's fixes. RetroArch's listed 55 files from another emulator's tree -
an Android package's banner, icons and Noto fonts - to be installed into
a BIOS directory, and omitted the six regional DSi NAND dumps a
filename-agnostic core accepts.
Across the twelve platforms: 199 files no longer listed, 36 newly
listed. Generated from the committed profile set and the committed
database so the revision stays consistent with itself.
The pack builder accepts the aliases declared in the target overrides,
so --target switch works there, but the installer's target manifests
carried only canonical names. The documented word was the one that
failed, and the installer then carried on with every file: 1911 files
and 4.1 GB where the user had asked for the 863 that target needs.
Aliases are emitted beside their canonical target, and an unknown
target now stops the run and lists what is available. A filter is
applied or refused, never ignored.
The database drops the duplicate PS3UPDAT.PUP entry and regains
adler32 on every file. RetroPie gains an install manifest, so the
download table reports its size instead of a dash.
Every other manifest field was treated as hostile input, but
standalone_copies targets were only length-checked before being
expanded and written to: a traversal component or a symlink already
sitting at the destination sent the copy outside the directory the
user opted into. Targets are now validated like the other paths and a
symlinked destination is never followed.
RETROBIOS_BASE_URL serves the manifest and the files it declares, so
it now has to be HTTPS the way both bootstraps already require of the
installer URL; loopback stays open for the end-to-end tests.
install.ps1 left TLS at the Windows PowerShell 5.1 default, which
GitHub refuses, so the download failed before any hash was checked.
check_local read every file once per declared digest, single threaded.
One read now feeds both, across the same pool the downloads use.
RetroPie had no manifest, so the one-line installer answered 'unknown
platform' for a frontend whose packs do ship.
YAML 1.1 reads an unquoted 01 as the integer 1 and 81 as 81, so the
manager set stopped naming the file its driver opens and the API
published "cores": [2048] as a number where consumers match strings.
Ground truth from MAME 0.289 at the revision the profile cites:
src/mame/vtech/crvision.cpp:957 loads "01" and "23".
The root cause was the scraper: _hash_merge wrote these names through an
f-string, so quoting the profiles alone would be undone on the next
refresh. _yaml_scalar now quotes every name, archive and description it
writes.
Add MSX2J.rom (sha1: 0081ea0d25bc5cd8d70b60ad8cfdc7307812c0fd, size: 32768) to multiple install manifests and the RetroDECK bios list. Update generated timestamps and adjust total_files/total_size counts in batocera, lakka, recalbox, retroarch, retrobat, and retrodeck manifests. Also bump README verified file count and regenerate the auto-generated timestamp to reflect the new entry.
find_undeclared_files was enriching declared_names with DB aliases,
filtering core extras that were never packed by Phase 1 under that
name. Pass strict YAML names to _collect_emulator_extras so alias-
only files (dc_bios.bin, amiga-os-310-a1200.rom, scph102.bin, etc.)
get packed at the emulator's expected path. Also fix truth mode
output message and --all-variants --verify-packs quick-exit bypass.
RetroDECK: core extras with subdirectory paths (e.g. vice/C64/,
fbneo/, dc/) were placed outside bios/ because the prefix was only
inferred for bare filenames. Add _detect_extras_prefix() to infer
the dominant BIOS prefix from YAML destinations.
RomM: core extras landed flat at bios/{file} instead of the required
bios/{platform_slug}/{file}. Add _detect_slug_structure() to detect
per-system slug layouts and _map_emulator_to_slug() to route each
extra to the correct slug subfolder.
Also skip manifest writes when only the generated timestamp changed,
preventing unnecessary diffs in install/*.json.
Closes#43
FBNeo and Kronos expect BIOS archives in core-specific subdirectories
(system/fbneo/, system/kronos/). RetroArch firmware check uses .info
paths which include these prefixes, so files at root show as Missing.
Add archive_prefix field to emulator profiles. The pack code now places
archive copies in the prefixed subdirectory while keeping root copies
for cores that expect them there (e.g. Geolith for neogeo.zip).
Add support for Coleco Adam, Entex Adventure Vision and APF M-1000 BIOS/ROM sets in MAME and MESS metadata (multiple Adam device MCU ROMs and optional FDC/SPI variants, Advision and APF BIOS entries). Update generated metadata across the repo: README coverage numbers and per-platform coverage rows, database generated timestamp and totals (total_files 7245), and various install manifests (notably batocera.json) with new timestamps, adjusted file counts/sizes, SHA1s, repo_path fixes and an added adam_fdc_320kb.zip entry. Also update notes to reflect the new system ROM sets in the emulators entries.
Abdessamad Derraz