dedup.py removes a copy only when another file has the same SHA-1 and
the same name, keeping the higher-priority path, so resolution by hash
still finds the content. Verified rather than assumed: all 17268
manifest entries ship the same bytes as before, no destination gained
or lost, only the source paths consolidate. Every deleted path's content
is still held elsewhere.
12.1 MB and 48 empty directories go with them. The index and the
published artefacts are regenerated on top.
A destination can be declared by more than one system, bare in one and
hash-constrained in another. generate_pack resolves that with
_preferred_entries so the constrained sibling claims the destination;
generate_manifest never did, and named whatever answered to the name.
RetroDECK's bios/d2fdc.zip is the case: declared with an md5 in the
arcade system and bare under apple-ii. The pack carried the right
archive, 262 bytes reached through the MAME clone map, while the
manifest sent install.py to a 256-byte Apple II ROM. Downloading the
ZIP and running the installer gave different files.
A test now hashes every manifest entry a platform pins and compares it
against what the platform declares: 3295 entries, on the platforms whose
frontend reads the bytes. It accepts any of several declarations for one
destination, the member-composite MD5 Recalbox pins for arcade archives,
and Batocera's 29-character prefixes. Reinstating the old manifest entry
fails it.
Rebuilt from the complete state: 8923 files indexed, 436 emulators, and
coverage up on every platform that gained a profile. The gap count falls
from 9 to 7, the three that closed being the C128 kernal, the Cave Story
build and Supermodel's Games.xml.
Whole pipeline green on this revision: database, provenance, recipes,
data directories, hash scrapers, truth, verify, packs, manifests,
consistency and pack integrity across the twelve platforms.
The published manifests still described what the builder produced before
today's fixes. RetroArch's listed 55 files from another emulator's tree -
an Android package's banner, icons and Noto fonts - to be installed into
a BIOS directory, and omitted the six regional DSi NAND dumps a
filename-agnostic core accepts.
Across the twelve platforms: 199 files no longer listed, 36 newly
listed. Generated from the committed profile set and the committed
database so the revision stays consistent with itself.
The database drops the duplicate PS3UPDAT.PUP entry and regains
adler32 on every file. RetroPie gains an install manifest, so the
download table reports its size instead of a dash.
Every other manifest field was treated as hostile input, but
standalone_copies targets were only length-checked before being
expanded and written to: a traversal component or a symlink already
sitting at the destination sent the copy outside the directory the
user opted into. Targets are now validated like the other paths and a
symlinked destination is never followed.
RETROBIOS_BASE_URL serves the manifest and the files it declares, so
it now has to be HTTPS the way both bootstraps already require of the
installer URL; loopback stays open for the end-to-end tests.
install.ps1 left TLS at the Windows PowerShell 5.1 default, which
GitHub refuses, so the download failed before any hash was checked.
check_local read every file once per declared digest, single threaded.
One read now feeds both, across the same pool the downloads use.
RetroPie had no manifest, so the one-line installer answered 'unknown
platform' for a frontend whose packs do ship.
Abdessamad Derraz
