fix: sha1-based large file restore, fix broken data dir urls

Replace grep-based restore with SHA1 matching via database.json.
The old grep heuristic failed for assets with renamed basenames
(dsi_nand_batocera42.bin) or special characters (MAME dots vs
spaces), and only restored to the first .gitignore match when
multiple paths shared a basename.

Fix 3 broken data directory sources:
- opentyrian: buildbot URL 404, use release asset
- syobonaction: invalid git_subtree URL, use GitHub archive
- stonesoup: same fix, adds 532 game data files

.github/workflows/build.yml

@@ -58,16 +58,32 @@ jobs:
         run: |
           mkdir -p .cache/large
           gh release download large-files -D .cache/large/ 2>/dev/null || true
-          for f in .cache/large/*; do
+          python3 -c "
-            [ -f "$f" ] || continue
+          import hashlib, json, os, shutil
-            name=$(basename "$f")
+          db = json.load(open('database.json'))
-            target=$(grep "$name" .gitignore | head -1)
+          with open('.gitignore') as f:
-            if [ -n "$target" ] && [ ! -f "$target" ]; then
+              ignored = {l.strip() for l in f if l.strip().startswith('bios/')}
-              mkdir -p "$(dirname "$target")"
+          cache = '.cache/large'
-              cp "$f" "$target"
+          if not os.path.isdir(cache):
-              echo "Restored: $target"
+              exit(0)
-            fi
+          idx = {}
-          done
+          for fn in os.listdir(cache):
+              fp = os.path.join(cache, fn)
+              if os.path.isfile(fp):
+                  h = hashlib.sha1(open(fp, 'rb').read()).hexdigest()
+                  idx[h] = fp
+          restored = 0
+          for sha1, entry in db['files'].items():
+              path = entry['path']
+              if path in ignored and not os.path.exists(path):
+                  src = idx.get(sha1)
+                  if src:
+                      os.makedirs(os.path.dirname(path), exist_ok=True)
+                      shutil.copy2(src, path)
+                      print(f'Restored: {path}')
+                      restored += 1
+          print(f'Total: {restored} files restored')
+          "
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 

.github/workflows/deploy-site.yml

@@ -36,6 +36,43 @@ jobs:
 
       - run: pip install pyyaml mkdocs-material pymdown-extensions
 
+      - name: Restore large files from release
+        run: |
+          mkdir -p .cache/large
+          gh release download large-files -D .cache/large/ 2>/dev/null || true
+          python3 -c "
+          import hashlib, json, os, shutil
+          db = json.load(open('database.json'))
+          with open('.gitignore') as f:
+              ignored = {l.strip() for l in f if l.strip().startswith('bios/')}
+          cache = '.cache/large'
+          if not os.path.isdir(cache):
+              exit(0)
+          idx = {}
+          for fn in os.listdir(cache):
+              fp = os.path.join(cache, fn)
+              if os.path.isfile(fp):
+                  h = hashlib.sha1(open(fp, 'rb').read()).hexdigest()
+                  idx[h] = fp
+          restored = 0
+          for sha1, entry in db['files'].items():
+              path = entry['path']
+              if path in ignored and not os.path.exists(path):
+                  src = idx.get(sha1)
+                  if src:
+                      os.makedirs(os.path.dirname(path), exist_ok=True)
+                      shutil.copy2(src, path)
+                      print(f'Restored: {path}')
+                      restored += 1
+          print(f'Total: {restored} files restored')
+          "
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Refresh data directories
+        run: python scripts/refresh_data_dirs.py
+        continue-on-error: true
+
       - name: Generate site
         run: |
           python scripts/generate_site.py

.gitignore

@@ -29,13 +29,14 @@ data/
 # Large files stored as GitHub Release assets (additional)
 bios/Arcade/MAME/artwork/snspell.zip
 bios/Arcade/MAME/MAME 0.174 Arcade XML.dat
-bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP
+bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP.3ae832c9
 bios/Nintendo/DS/DSi_Nand_JPN.bin
 bios/Nintendo/DS/DSi_Nand_EUR.bin
 bios/Nintendo/DS/DSi_Nand_USA.bin
 bios/Nintendo/DS/DSi_Nand_AUS.bin
 bios/Nintendo/DS/DSi_Nand_CHN.bin
 bios/Nintendo/DS/DSi_Nand_KOR.bin
+bios/Nintendo/DS/dsi_nand.bin
 
 # QEMU EDK2 firmware (64MB each)
 bios/QEMU/edk2-aarch64-code.fd
@@ -51,8 +52,22 @@ bios/Arcade/Arcade/ps2.zip
 bios/Arcade/Arcade/kf2k3pcb.zip
 bios/Arcade/Arcade/ms5pcb.zip
 
+# Dolphin/Ishiiruka SD card images (128MB each)
+bios/Nintendo/Wii/WiiSD.raw
+bios/Nintendo/Wii/sd.raw
+
 # EasyRPG soundfont (142MB)
 bios/RPG Maker/easyrpg/easyrpg.soundfont
 
 # Temporary files
 tmp/
+
+# FBA2012 Toaplan samples (>100MB)
+bios/Arcade/fba2012/samples/fireshrk.zip
+bios/Arcade/fba2012/samples/ghox.zip
+bios/Arcade/fba2012/samples/samesame.zip
+bios/Arcade/fba2012/samples/tekipaki.zip
+bios/Arcade/fba2012/samples/vimana.zip
+bios/Arcade/fba2012/samples/fireshrk.zip
+bios/Arcade/fba2012/samples/samesame.zip
+bios/Arcade/fba2012/samples/vimana.zip

CONTRIBUTING.md

@@ -7,6 +7,16 @@
 3. Variants (alternate hashes): `bios/Manufacturer/Console/.variants/`
 4. Create a Pull Request - checksums are verified automatically
 
+## Add a new platform
+
+1. Write a scraper in `scripts/scraper/`
+2. Create the platform YAML in `platforms/`
+3. Register in `platforms/_registry.yml`
+4. Submit a Pull Request
+
+Contributors who add platform support are credited in the README,
+on the documentation site, and in the BIOS packs.
+
 ## File conventions
 
 - Files >50 MB go in GitHub release assets (`large-files` release)

README.md

@@ -2,7 +2,7 @@
 
 Complete BIOS and firmware packs for Batocera, BizHawk, EmuDeck, Lakka, Recalbox, RetroArch, RetroBat, RetroDECK, RetroPie, and RomM.
 
-**7,562** verified files across **352** systems, ready to extract into your emulator's BIOS directory.
+**7,302** verified files across **396** systems, ready to extract into your emulator's BIOS directory.
 
 ## Quick Install
 
@@ -27,14 +27,14 @@ Pick your platform, download the ZIP, extract to the BIOS path.
 
 | Platform | BIOS files | Extract to | Download |
 |----------|-----------|-----------|----------|
-| Batocera | 359 | `/userdata/bios/` | [Download](../../releases/latest) |
+| Batocera | 361 | `/userdata/bios/` | [Download](../../releases/latest) |
 | BizHawk | 118 | `Firmware/` | [Download](../../releases/latest) |
 | EmuDeck | 161 | `Emulation/bios/` | [Download](../../releases/latest) |
 | Lakka | 448 | `system/` | [Download](../../releases/latest) |
 | Recalbox | 346 | `/recalbox/share/bios/` | [Download](../../releases/latest) |
 | RetroArch | 448 | `system/` | [Download](../../releases/latest) |
-| RetroBat | 331 | `bios/` | [Download](../../releases/latest) |
+| RetroBat | 339 | `bios/` | [Download](../../releases/latest) |
-| RetroDECK | 2007 | `~/retrodeck/bios/` | [Download](../../releases/latest) |
+| RetroDECK | 2006 | `~/retrodeck/bios/` | [Download](../../releases/latest) |
 | RetroPie | 448 | `BIOS/` | [Download](../../releases/latest) |
 | RomM | 374 | `bios/{platform_slug}/` | [Download](../../releases/latest) |
 
@@ -44,14 +44,14 @@ BIOS, firmware, and system files for consoles from Atari to PlayStation 3.
 Each file is checked against the emulator's source code to match what the code actually loads at runtime.
 
 - **10 platforms** supported with platform-specific verification
-- **328 emulators** profiled from source (RetroArch cores + standalone)
+- **329 emulators** profiled from source (RetroArch cores + standalone)
-- **352 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
+- **396 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
-- **7,562 files** verified with MD5, SHA1, CRC32 checksums
+- **7,302 files** verified with MD5, SHA1, CRC32 checksums
-- **8183 MB** total collection size
+- **8765 MB** total collection size
 
 ## Supported systems
 
-NES, SNES, Nintendo 64, GameCube, Wii, Game Boy, Game Boy Advance, Nintendo DS, Nintendo 3DS, Switch, PlayStation, PlayStation 2, PlayStation 3, PSP, PS Vita, Mega Drive, Saturn, Dreamcast, Game Gear, Master System, Neo Geo, Atari 2600, Atari 7800, Atari Lynx, Atari ST, MSX, PC Engine, TurboGrafx-16, ColecoVision, Intellivision, Commodore 64, Amiga, ZX Spectrum, Arcade (MAME), and 318+ more.
+NES, SNES, Nintendo 64, GameCube, Wii, Game Boy, Game Boy Advance, Nintendo DS, Nintendo 3DS, Switch, PlayStation, PlayStation 2, PlayStation 3, PSP, PS Vita, Mega Drive, Saturn, Dreamcast, Game Gear, Master System, Neo Geo, Atari 2600, Atari 7800, Atari Lynx, Atari ST, MSX, PC Engine, TurboGrafx-16, ColecoVision, Intellivision, Commodore 64, Amiga, ZX Spectrum, Arcade (MAME), and 362+ more.
 
 Full list with per-file details: **[https://abdess.github.io/retrobios/](https://abdess.github.io/retrobios/)**
 
@@ -59,16 +59,16 @@ Full list with per-file details: **[https://abdess.github.io/retrobios/](https:/
 
 | Platform | Coverage | Verified | Untested | Missing |
 |----------|----------|----------|----------|---------|
-| Batocera | 359/359 (100.0%) | 359 | 0 | 0 |
+| Batocera | 361/361 (100.0%) | 361 | 0 | 0 |
 | BizHawk | 118/118 (100.0%) | 118 | 0 | 0 |
 | EmuDeck | 161/161 (100.0%) | 161 | 0 | 0 |
 | Lakka | 448/448 (100.0%) | 448 | 0 | 0 |
 | Recalbox | 346/346 (100.0%) | 346 | 0 | 0 |
 | RetroArch | 448/448 (100.0%) | 448 | 0 | 0 |
-| RetroBat | 331/331 (100.0%) | 330 | 1 | 0 |
+| RetroBat | 339/339 (100.0%) | 339 | 0 | 0 |
-| RetroDECK | 2007/2007 (100.0%) | 1998 | 9 | 0 |
+| RetroDECK | 2006/2006 (100.0%) | 2006 | 0 | 0 |
 | RetroPie | 448/448 (100.0%) | 448 | 0 | 0 |
-| RomM | 374/374 (100.0%) | 371 | 3 | 0 |
+| RomM | 374/374 (100.0%) | 374 | 0 | 0 |
 
 ## Build your own pack
 
@@ -104,7 +104,7 @@ The [documentation site](https://abdess.github.io/retrobios/) provides:
 - **Per-emulator profiles** with source code references for every file
 - **Per-system pages** showing which emulators and platforms cover each console
 - **Gap analysis** identifying missing files and undeclared core requirements
-- **Cross-reference** mapping files across 10 platforms and 328 emulators
+- **Cross-reference** mapping files across 10 platforms and 329 emulators
 
 ## How it works
 
@@ -130,4 +130,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
 
 This repository provides BIOS files for personal backup and archival purposes.
 
-*Auto-generated on 2026-03-28T17:22:19Z*
+*Auto-generated on 2026-04-02T13:38:26Z*

bios/Apple/Apple II/DiskII/state-machine-13.rom

@@ -0,0 +1,65 @@
+ИИ�Ш
+
+
+
+�Щ�Щ�Ы�Ы�ШШ
+
+
+
+�Щ�Щ�Ы�ЫИ=ИИ
+
+
+
+�й�й�л�л�н�и
+
+
+
+�й�й�л�лИИИИ
+
+
+
+ЈшЈшЈшЈшЈшЈш
+
+
+
+ЈшЈшЈшЈшЙ§Иј
+
+
+
+ИјИјИјИјЙ§Pј
+
+
+
+ИјИјИјИјИИHИ
+
+
+
+H(H(H(H(H(H(
+
+
+
+H(H(H(H(ИЙИИ
+
+
+
+X8X8X8X8	ЩX8
+
+
+
+X8X8X8X8ИИИИ
+
+
+
+hhhhhh
+
+
+
+hhhh�Нxp
+
+
+
+xxxx
+-xp
+
+
+

bios/Apple/Apple II/DiskII/state-machine-16.rom

@@ -0,0 +1,65 @@
+Ш
+
+
+
+99;;8(
+
+
+
+99;;-Ш8H
+
+
+
+(H(H(H(H-H8H
+
+
+
+(H(H(H(HШШШШ
+
+
+
+XxXxXxXxXxXx
+
+
+
+XxXxXxXxШШШШ
+
+
+
+hh€hh€h€h€
+
+
+
+hh€hh€ШНШШ
+
+
+
+�№�№�»�»�Ѕ�ё
+
+
+
+�№�№�»�»ШЩШШ
+
+
+
+ЁИЁИЁИЁИ)YЁИ
+
+
+
+ЁИЁИЁИЁИЩэШш
+
+
+
+ШшШшШшШшЩэ ш
+
+
+
+ШшШшШшШшШЭиа
+
+
+
+и€ии€иMиа
+
+
+
+и€ии€и

bios/Commodore/Amiga/rom.key

@@ -0,0 +1,36 @@
+
+ Cloanto ® Amiga Forever ROM key file. Copyright © Cloanto 1997.
+
+ If you find this file in a location other than the Amiga Forever
+ collection, please report this to Amiga International, Inc., and/or to
+ Cloanto (E-mail: support@cloanto.com).
+
+ *************************************************************************
+ *                                                                       *
+ *                              COPYRIGHTS                               *
+ *                                                                       *
+ *                     THE AMIGA ROM AND OS FILES ARE                    *
+ *                                                                       *
+ *  Copyright © 1985-1997 Amiga International, Inc. All Rights Reserved  *
+ *                                                                       *
+ *************************************************************************
+
+
+ ************************************************************************
+ *                                                                      *
+ *                            DISCLAIMER                                *
+ *                                                                      *
+ *   ALL SOFTWARE/INFORMATION IN AMIGA FOREVER IS PROVIDED "AS IS".     *
+ *   NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE      *
+ *   ACCURACY, RELIABILITY, PERFORMANCE, CURRENTNESS, OR OPERATION      *
+ *   OF THIS SOFTWARE/INFORMATION, AND ALL USE IS AT YOUR OWN RISK.     *
+ *   NEITHER AMIGA INTERNATIONAL, INC. NOR CLOANTO NOR THE AUTHORS      *
+ *   ASSUME ANY RESPONSIBILITY OR LIABILITY WHATSOEVER WITH RESPECT     *
+ *   TO YOUR USE OF THIS  SOFTWARE/INFORMATION.                         *
+ *                                                                      *
+ ************************************************************************
+
+ Cloanto is a registered trademark of Cloanto. Amiga is a registered
+ trademark of Amiga International, Inc.
+
+ Amiga ® Amiga ® Amiga ® Amiga ® Amiga ® Amiga ® Amiga ® Amiga ® Amiga ®