fix: read redump bios dats from /bios/<code>

redump.info moved its BIOS DAT links from /static/bios/<name>.dat to one
route per system (/bios/PSX, /bios/PS2, /bios/GC, /bios/XBOX), served as
an attachment. The scraper matched only the old shape and found nothing.
Both shapes are read now.

Refreshed snapshot: the PS2 DAT moved from 2026-06-16 to 2026-10-10,
191 entries become 225 (34 mechacon, DVD and font ROMs added, the
ps2-dvd-302c.bin sha1 corrected, five descriptions completed). Only
the provenance fields of database.json change.
This commit is contained in:
Abdessamad Derraz committed 2026-11-03 14:37:09 +01:00
1 parent bb634661d5
commit aa0158c65a
4 files changed
+352 -23

No files matched your search

+7 -4
View File
@@ -1,9 +1,11 @@
"""Fetch Redump BIOS DAT files and write a provenance snapshot.
Redump serves BIOS DATs as static clrmamepro files linked from
https://redump.info/downloads under /static/bios/. The downloads page
is scanned for those links, so new BIOS DATs are picked up without
code changes.
Redump serves BIOS DATs as Logiqx files linked from
https://redump.info/downloads. The links were static files under
/static/bios/<name>.dat and are now one route per system,
/bios/<CODE> (PSX, PS2, GC, XBOX), served as an attachment. The
downloads page is scanned for either shape, so new BIOS DATs are
picked up without code changes.
Run manually, then commit the snapshot:
python -m scripts.scraper.redump_dat_scraper --dry-run
@@ -40,6 +42,7 @@ def _fetch(url: str) -> str:
def discover_bios_datfiles(downloads_html: str) -> list[str]:
"""Extract BIOS DAT paths from the downloads page."""
paths = set(re.findall(r'href="(/static/bios/[^"]+\.dat)"', downloads_html))
paths.update(re.findall(r'href="(/bios/[A-Za-z0-9_-]+)"', downloads_html))
return sorted(paths)