Commit Graph
118 Commits
Author SHA1 Message Date
Abdessamad Derraz cd7d993cac feat: settle entries by their system's cores
A system that lists the cores it runs is settled by those cores alone, so an unprofiled core of another system no longer leaves its entries open. Readers whose core runs the entry's system are named first, and cores that only share the file name are listed as such.
2026-10-22 06:30:56 +02:00
Abdessamad Derraz a75ea05b86 feat: document what settles each category
Each missing platform entry now carries its evidence: the profile entries that read it with their source reference, the cores of its system examined without finding a reader, or why the question stays open. verify --verbose prints it, --json carries it and the platform pages show it.
2026-10-22 05:43:06 +02:00
Abdessamad Derraz f66adfdcbc feat: categorize missing platform entries
A platform list can name a game's data files or a file no core it runs opens. Verify gives each missing entry a category read from the profiles of the platform's cores: firmware, game data or unread. Only firmware gaps carry a severity and fail --strict; the others are counted on their own line, and the pack check no longer counts them as pack errors. Missing firmware stays a conformance error.
2026-10-22 05:38:57 +02:00
Abdessamad Derraz 2bdac466a9 feat: add knulli, amberelec and emuelec platforms
Each OS publishes Batocera's systems dict (knulli-systems, and
batocera-systems at AmberELEC's newest prerelease commit and EmuELEC v4.8),
read with the shared parser. Cores come from each OS's own ES list.
AmberELEC and EmuELEC declare neogeo/ and ports/ beside bios/ from
/storage/roms, so their packs extract there and core_destination puts
core files under bios/.
2026-10-22 03:42:20 +02:00
Abdessamad Derraz fd1d1b7b96 chore: merge round 2 fixes (pins)
# Conflicts:
#	install.ps1
#	install.sh
2026-10-21 10:31:09 +02:00
Abdessamad Derraz ccf61b96d7 fix: report names a RetroPie sentence gains
flycast now reads naomi2.zip, which the RetroArch list RetroPie inherits
already declares. The render added it to the flycast help sentences but
landed_changes counted additions only for entries the platform layer
lacks, so the report missed a name the diff carries. names_anew lets an
exporter whose unit is not the entry say so.
2026-10-21 10:26:04 +02:00
Abdessamad Derraz 83a4b21d30 fix: keep platform hashes cores disagree on
When two cores declare different values for one file, the truth keeps the first and marks the entry hash_conflict. Its hashes pair an entry and never replace the platform's: armsx2's GameIndex.yaml no longer overwrites System.dat's line for LRPS2.
2026-10-21 09:57:20 +02:00
Abdessamad Derraz f7b758a41d fix: patch batocera entries past commented items
_patched_entry read '#{ ... }' lines as live items, found no match in the model and gave up, so atarist was written again whole (52 lines changed for 3 additions). A commented item is now left as written.
2026-10-21 09:52:34 +02:00
Abdessamad Derraz 5ec1208a83 fix: write romm keys with a bare file name
RomM lists the platform firmware folder without recursing and looks each file up as slug:file_name (models/firmware.py, base_handler.list_files at 5.4.0). An addition named with a directory (fuse/48.rom) is not written.
2026-10-21 09:51:32 +02:00
Abdessamad Derraz 4205bc2233 fix: count one retrobat addition per path
The render writes one biosFiles item per path, as checkBios reads one md5 per item. A second addition at a held path was dropped from the file and still counted added (78 announced, 75 written).
2026-10-21 09:50:46 +02:00
Abdessamad Derraz 28435230a7 fix: read each named core's own entry in recalbox
An element's mandatory flag came from the one truth entry its md5 paired
with. saturn/saturn_bios.bin names yabause, yabasanshiro and kronos,
which require it, and was paired with the beetle_saturn entry, which
does not. Each named core's entries for the file, found by name, now
decide, except a namesake elsewhere in the same tree (bluemsx KANJI.rom
under Shared Roms is not the one under MSX2+ - European).
2026-10-21 05:39:17 +02:00
Abdessamad Derraz d9548e3522 fix: count a recalbox element kept as written
An element whose md5 the truth contradicts without naming another is
written back whole. It was counted as an entry the format cannot keep.
It now counts as kept, and the md5 it keeps is named as withheld.
2026-10-21 05:14:44 +02:00
Abdessamad Derraz 4a6b654bdc fix: keep a romm entry whole without a size
RomM compares file_size_bytes with the entry's size before any hash. A
new hash the truth gives no size for went out beside the old size and
was reported as a size removal. The entry now stays as written and the
correction is withheld and named.
2026-10-21 05:13:46 +02:00
Abdessamad Derraz 4670e0442e fix: withhold a mister hash only beside a size
An entry that states no size has none to contradict: the hash is written
as before.
2026-10-21 05:13:46 +02:00
Abdessamad Derraz d48e81861c fix: keep a mister entry whole without a size
Downloader rejects an entry without an integer size (db_entity.py:187)
and compares it after each fetch (fetch_file_worker.py:127). A new hash
the truth gives no size for is withheld and named, not written beside
the old size nor reported as a size removal.
2026-10-21 05:12:02 +02:00
Abdessamad Derraz 2e29b3be03 fix: read recalbox mandatory per named core
An element lists its cores. Its mandatory flag now follows whether those
cores need the file, and stays as written where none of them reads it.
cdi/cdibios.zip keeps mandatory=false (same_cdi does not read it), the
two amiberry kickstarts keep theirs, saturn_bios.bin becomes optional
for the three HLE cores it names.
2026-10-21 05:10:54 +02:00
Abdessamad Derraz fbf7cb191b fix: keep recalbox composite md5 on archives
Bios.cpp:64-68 hashes a .zip with Zip::Md5Composite, the md5 of its
members in name order. A profile's md5 on an archive is the container's,
so it neither pairs with, corrects nor fills that field, and an archive
addition with only a container md5 is left out.
2026-10-21 05:08:45 +02:00
Abdessamad Derraz f796155e1a fix: count only the names retropie help gains
The report listed every file a profiled core reads as added (1360),
while the export appends one name to one scriptmodule.
2026-10-21 05:05:00 +02:00
Abdessamad Derraz 6d38e597db fix: keep a RetroPie list's comma before its and
quasi88 writes its list "N88KNJ2.ROM, and N88SUB.ROM". Extended with
N88EXT0.ROM, which QUASI88kai reads, it came back as "N88SUB.ROM and
N88EXT0.ROM": every list was rewritten in one idiom. A list that puts a
comma before its final and keeps it.
2026-10-20 22:38:32 +02:00
Abdessamad Derraz 540374b40d fix: count a launcher's names as read
The removal candidates looked for a reading in the profiles only, so a
name the platform's own launch code opens, links or copies was listed
as read by no core: Batocera's DuckStation generator walks 22 names
(duckstationGenerator.py:501-515). Every name a launcher record names
is now a reading on that platform. Recalbox's unexplained entries drop
from 20 to 13, RetroBat's from 24 to 20, EmuDeck's from 25 to 22.
2026-10-20 22:20:42 +02:00
Abdessamad Derraz a9c175d322 chore: regenerate the updated platform files 2026-10-20 20:44:42 +02:00
Abdessamad Derraz 479ada29fe docs: word removal candidates by what was searched
"No core opens it" claimed more than the search shows: what was
checked is that no profile of a core the platform runs names the
entry by name, path or hash. The reason and the page sentence now
say that.
2026-10-20 20:00:54 +02:00
Abdessamad Derraz 545e3f3ba1 feat: record each export change before and after
The page named the entries a change touched and never the values, so
a maintainer could not check a correction. The report now carries
changes, one object per change with kind, entry, field, old and new,
read from the platform entry and from the value the render writes
(written() per format: record value for System.dat and RomM, one md5
where a single one is checked). The existing label lists are derived
from it. The page writes "md5 X becomes Y" and names each required
flag with its new value. A test parses the original and updated
System.dat and es_bios.xml and checks the list equals the diff.
2026-10-20 20:00:54 +02:00
Abdessamad Derraz 5cba005d67 fix: keep the platform's hash order when unchanged
Where the truth names no value the platform lacks, hashes() still put
the truth's values first and rewrote the list in another order:
Recalbox's msx/CARTS.SHA md5 attribute came out with its two values
swapped, a diff line that changes nothing the frontend checks. The
platform's own list now stands when it already holds every value.
2026-10-20 20:00:54 +02:00
Abdessamad Derraz 0b4f98ce79 feat: report entries no examined core opens
The export keeps every entry a platform declares. An entry the truth
does not pair was still read in 1189 of 2776 cases (alias, hash, a
type: directory tree, a core outside the resolved set), so absence of
a pairing proves nothing. Each report now lists, under
removal_candidates, the unpaired entries that no profile of the cores
the platform runs answers by name, path tail, hash, directory or data
directory, with the cores examined and a reason. A core the platform
names without a profile, or a ROM set under a core that opens sets by
name, leaves the verdict undetermined. The exports page states the
count and lists the entries per platform.
2026-10-20 20:00:54 +02:00
Abdessamad Derraz 7e0b3a08b9 fix: keep System.dat lines whose values agree
The rom line renderer upper-cased every crc, so Dinothawr.zip's
683ed4ad, the one lower-case crc libretro writes, changed with no
correction behind it. A line whose values match ours now keeps its
bytes, a corrected crc follows the case of its own line, and an added
line keeps the file's upper case. An empty truth hands System.dat back
byte for byte for RetroArch and Lakka.
2026-10-20 19:39:36 +02:00
Abdessamad Derraz eba5f1dd62 fix: patch Recalbox's attributes, not its elements
A changed element was rendered again, losing its alignment and the
place of its attributes, and an element whose md5 list was only
reordered (neogeo.zip, the same 24 digests) was rewritten too: Recalbox
accepts any digest of the list (Bios.cpp:109-130), so the order is not
content. A changed attribute is now replaced where it stands, one
dropped goes with its space, one added goes before />, and each system's
opening tag keeps its own indentation. An empty truth gives back the
cached es_bios.xml byte for byte.
2026-10-20 19:36:09 +02:00
Abdessamad Derraz aaffcdcb63 fix: patch Batocera's values, not its entries
An entry with one md5 to correct was written again whole: atarist lost
its eight blank lines, every '"x"},' became '"x" },', and most of the
819 lines Batocera's export changed were layout. Each item now keeps
its own text and only the md5 inside it changes; an addition follows
the last item in its indentation and spacing. The patched entry must
read as the one that would have been written, or that one is written.
Batocera's diff to the original drops to 273 lines, ROCKNIX's to 3.
2026-10-20 19:22:58 +02:00
Abdessamad Derraz 659e9b591f fix: keep RetroBat's blank line between systems
RetroBat separates every system after the first with a blank line.
json.dumps dropped all 100 of them: 100 hunks of noise in a diff meant
to show corrections. The separator is written again where the original
has it, and an added system takes it too where the original separates
them all. An empty truth gives back the cached file byte for byte.
2026-10-20 19:07:46 +02:00
Abdessamad Derraz 804534397f fix: write the MiSTer database on one line
The BiosDB is published as a single line of JSON without a final
newline. Written with indent=2 it came out as 1499 changed lines for a
database whose content had not changed, which would bury any real
correction. It is now written in MiSTer's own form: an empty truth
gives back the cached database byte for byte.
2026-10-20 18:54:28 +02:00
Abdessamad Derraz 3ed5f68b82 fix: hand RetroDECK manifests back as written
With nothing to correct, nine of ten manifests still changed: a final
newline was added where the original ends without one, "md5": null was
written 39 times where no md5 key was, and PPSSPP's single bios object
became a list rebuilt from our data. A manifest now ends as the
original does, keeps no key it did not have, and a single object stays
one while nothing is added. An empty truth gives back every cached
manifest byte for byte.
2026-10-20 18:41:26 +02:00
Abdessamad Derraz 85c2433c98 fix: write each export as its bytes and mode
write_text translated line ends on Windows, so FirmwareDatabase.cs's
CRLF would have come out CR CR LF; it replaced the published file in
place, so a run stopped mid-write left it truncated; and it dropped the
mode, while rocknix-systems is 100755 in ROCKNIX's tree. Files are
written atomically as UTF-8 bytes, and an exporter states the mode of
any file the platform does not publish as 0644.
2026-10-20 18:27:22 +02:00
Abdessamad Derraz cbe769e74c fix: read all RetroPie helps, propose only its own
Eight scriptmodules write "bios" in lower case (xm7, np2pi, quasi88,
minivmac, px68k...) and none was examined. A module reported "left as
is" for a name no profile documents was written all the same: the skip
had no continue. The cap of six names came from that case-sensitive
reading, where xm7 writes thirteen: a sentence now grows no longer than
the longest RetroPie writes. lr- was stripped whatever the build, so the
standalone minivmac package matched the libretro profile, which reads
another file. A file of a system RetroPie lacks is proposed to no
package: lr-geargrafx adds pcengine only (lr-geargrafx.sh:41-42) and
was given the LaserActive's pac-n1.bin and pac-n10.bin.
2026-10-20 18:12:56 +02:00
Abdessamad Derraz a5d7a6572d fix: ask for no file the emulator installs
The pack leaves out what an emulator installs itself
(validation.deliverable); the truth did not. RetroPie's gngeopi help
asked users to copy gngeo_data.zip, which make install puts in the
package's data directory, and amiberry's whdboot tree, scummvm's extra
data and dosbox-x's FREECG98.BMP went into platform lists as files to
bring.

The truth applies deliverable per platform and per build: an entry
nobody has to bring is kept and marked bundled, never added, still
corrected where the platform declares it. A rebuilt archive is
installed only when every member is. The runner written beside an
addition is the build that needs the file (_brought_by), and the
RetroPie sentence names only what its package's core must be given.
2026-10-20 17:39:21 +02:00
Abdessamad Derraz f957b5f893 fix: open no System.dat section libretro lacks
libretro titles each section with the system as it spells it,
"Commodore - Amiga". An addition for a system it has no section for
opened one under our slug: 21 sections came out as commodore-c64,
quake, laseractive or xrick, which no libretro DAT carries. An addition
now lands only in a section the original holds and is otherwise counted
as left out, so the export keeps the original's 66 sections. A platform
entry is always written, under its own section.
2026-10-20 17:21:16 +02:00
Abdessamad Derraz e5174a07bf feat: mark files a core finds by content
DuckStation identifies its BIOS by MD5 in a folder it scans
(bios.cpp:208), so its entries' names are labels. The export wrote them
into Batocera's and ROCKNIX's lists as files to hold: a user with only
scph5501.bin read 104 missing BIOS where DuckStation boots. Labels also
lent their md5 to a name another core opens, and paired by name they
replaced System.dat's scph5000.bin hashes.

found_by: name | content, on a profile or a file and keyed by build like
load_from. The truth records which cores open each name (_named_by). A
label pairs with a platform entry by hash only, is never added, merges
with an opened name only on a shared hash, and is counted in the export
report rather than listed as missing.
2026-10-20 16:32:22 +02:00
Abdessamad Derraz 7a65a2cf83 fix: refuse an export without its original
System.dat, RomM, Recalbox and RetroBat were rebuilt from our data when
their file was not cached: libretro's quoted names, Recalbox's comments
and one of its two zxspectrum blocks were lost, and the run said OK.
needs_original now follows native_sources, so every format we patch
fails without the file it patches and writes nothing.
2026-10-20 15:09:16 +02:00
Abdessamad Derraz e81c63804a fix: correct no hash the code does not check
validation: names what the code verifies. dosbox checks
MT32_CONTROL.ROM against nine sha1 and size; the truth's one md5 and
crc32 describe one of those revisions, and the export wrote them beside
System.dat's own sha1, a rom line no file satisfies, and replaced
Batocera's md5. pcsx_rearmed and DuckStation check scph1001.bin by size,
and the v2.0 md5 platforms name was replaced by v4.1's.

checked_hashes keeps a hash the code does not name only where a checked
hash pins one content. Otherwise it still pairs a renamed file but never
corrects or fills a value the platform wrote. Additions keep every hash.
diff_truth reads the same rule.
2026-10-20 14:40:10 +02:00
Abdessamad Derraz 181ad0c630 fix: pair truth entries hash-consistent first
Each truth entry claimed its platform line in turn, so the first one to
arrive won. dosbox-x's system.rom (alias bios.rom, no path) took
Batocera's np2kai/BIOS.ROM by name before np2kai's own np2kai/bios.rom
was tried, and the export wrote an IBM PCjr md5 on it. A path also
outranked a contradicting hash: the truth's MSX.rom joins bluemsx's path
and fmsx's md5, which replaced the blueMSX dump on Recalbox's and
System.dat's Machines/Shared Roms/MSX.rom while fmsx's own line held it.

pair_entries runs each test over every entry before the next, tests that
contradict no hash first. export_native and diff_truth both read it.
2026-10-20 14:24:00 +02:00
Abdessamad Derraz 048334cc82 docs: offer the platform files as peers would
The page called the original "their file" and listed hashes "corrected" and additions "left out", a verdict on work the page only proposes to extend. It now says what each updated file changes in neutral terms built from the report's numbers, gives the reason a file is not added, credits the effort a list takes to keep current, and invites an issue when a change does not fit a platform. A test keeps the judging words off the page.
2026-10-20 13:20:34 +02:00
Abdessamad Derraz b6a1304893 feat: commit the corrected file of each platform
exports/ now holds the twelve platforms' own files, corrected, with a
report of every change, read by the site's corrected files page. Two
summaries are reworded for the page: an addition the format cannot
carry is counted as left out, and RetroPie names each skipped package
by sentence instead of joining reasons with semicolons. Its parser no
longer reads "e.g" as a file name.
2026-10-20 09:26:12 +02:00
Abdessamad Derraz 19a7128ec6 fix: compare md5 when keeping Recalbox text
The attribute pattern took letters only, so md5 never entered the comparison and an element whose digest the truth corrects kept Recalbox's text. A test written before the change caught it. The RetroDECK fixture now names the core that reads its addition, as the placement rule requires.
2026-10-20 01:25:25 +02:00
Abdessamad Derraz c5fd079b78 fix: keep the text of unchanged Recalbox elements
Every <bios> element the model held was rendered again, so attribute order, alignment, a line break inside md5 and the case of a digest changed in 222 elements that stated the same thing. The diff deleted 451 lines for 136 corrections. An element whose attributes say what Recalbox wrote now keeps Recalbox's own text: 175 lines, the corrections.
2026-10-20 01:23:29 +02:00
Abdessamad Derraz cef6d943e2 fix: a Batocera note stays with its own item
Notes were keyed by file path, so a comment before one of the items reading an archive came back before every item reading it: bk's '# MAME' twelve times. A comment after the list's opening bracket matched neither rule and was dropped (enterprise's ep128emu-core). Notes are now keyed by file and archive member, written once, and a head-line comment goes with the first item.
2026-10-20 01:22:27 +02:00
Abdessamad Derraz e74d182d32 fix: a RetroDECK addition needs its component core
Additions took the component their system's entries sit in, whatever core reads them. 104 armsx2 and yaps2 files went into PCSX2's manifest, beetle_psx and rustation dumps into DuckStation's, a directxbox image into xemu's, all counted as added. An addition now lands only where a core reading it is that component's emulator: 132 additions instead of 581.
2026-10-20 01:21:38 +02:00
Abdessamad Derraz 132c69a4b2 fix: a ROCKNIX runner follows its build
ROCKNIX registers DuckStation twice, as the libretro core under retroarch and as duckstation-sa under its own emulator. The exporter looked a standalone runner up as a core name, so standalone DuckStation's 98 additions were written for users of the libretro core, PS2 images included, and standalone ARMSX2 was refused. A libretro runner now answers with the retroarch pair, a standalone with its emulator's own.
2026-10-20 01:20:45 +02:00
Abdessamad Derraz 4231902a6c docs: write prose without dashes or semicolons
Profiles, the wiki, comments, docstrings, command output and schema descriptions carried em-dashes, clause-joining semicolons, unicode arrows, box-drawing banners and check-mark glyphs. Each sentence was rewritten rather than having its character swapped. Hashes, paths, source_ref values and every cited file:line are unchanged, which a comparison of each parsed profile against the previous revision confirms.

validate_pr comments now say PASS and FAIL in words, and source_ref values in a profile's structured blocks render as code. test_prose_style fails if any of these comes back, in tracked files or in the generated site.
2026-10-19 06:47:31 +02:00
Abdessamad Derraz fc47ac9f1f refactor: pair truth and platform entries one way
diff_truth matched by name with a rank, export_native by path, guarded name and hash: on current data each named files the other did not. Both now read one matcher, which also accepts aliases and sha256.
2026-10-18 20:48:29 +02:00
Abdessamad Derraz 66caa0e363 fix: read batocera entries as literals
A comment after an entry swallowed the closing brace, so gba was rewritten with its emulator gate lost. The entry is now read with literal_eval, which also stops executing the platform's text.
2026-10-18 20:37:38 +02:00
Abdessamad Derraz eac2eaec83 fix: count only the hash changes an export writes
RomM kept hashes the truth contradicts beside the corrected one, RetroDECK kept a cleared md5, Recalbox counted changes on an element it writes back whole, and two record formats counted fills no record could hold.
2026-10-18 20:36:47 +02:00