refactor: deduplicate hash comparison loop in check_file_validation

Add support for Coleco Adam, Entex Adventure Vision and APF M-1000 BIOS/ROM sets in MAME and MESS metadata (multiple Adam device MCU ROMs and optional FDC/SPI variants, Advision and APF BIOS entries). Update generated metadata across the repo: README coverage numbers and per-platform coverage rows, database generated timestamp and totals (total_files 7245), and various install manifests (notably batocera.json) with new timestamps, adjusted file counts/sizes, SHA1s, repo_path fixes and an added adam_fdc_320kb.zip entry. Also update notes to reflect the new system ROM sets in the emulators entries.

.github/workflows/build.yml

@@ -18,13 +18,11 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  regenerate:
+  release:
+    if: false  # disabled until pack generation is validated in production
     runs-on: ubuntu-latest
     permissions:
       contents: write
-      pages: write
-    outputs:
-      changed: ${{ steps.commit.outputs.changed }}
     steps:
       - uses: actions/checkout@v6
 
@@ -34,57 +32,9 @@ jobs:
 
       - run: pip install pyyaml
 
-      - name: Restore large files from release
-        run: |
-          mkdir -p .cache/large
-          gh release download large-files -D .cache/large/ 2>/dev/null || true
-          # Copy large files to their bios/ paths so generate_db sees them
-          for f in .cache/large/*; do
-            [ -f "$f" ] || continue
-            name=$(basename "$f")
-            # Match against .gitignore entries to find target path
-            target=$(grep "$name" .gitignore | head -1)
-            if [ -n "$target" ] && [ ! -f "$target" ]; then
-              mkdir -p "$(dirname "$target")"
-              cp "$f" "$target"
-              echo "Restored: $target"
-            fi
-          done
-        env:
-          GH_TOKEN: ${{ github.token }}
-
-      - name: Regenerate database, readme, and site
-        run: |
-          pip install mkdocs-material
-          python scripts/generate_db.py --force --bios-dir bios --output database.json
-          python scripts/generate_readme.py --db database.json --platforms-dir platforms
-          python scripts/generate_site.py
-
       - name: Run tests
         run: python -m unittest tests.test_e2e -v
 
-      - name: Deploy site to GitHub Pages
-        if: github.ref == 'refs/heads/main'
-        run: mkdocs gh-deploy --force --clean
-
-      - name: Commit if changed
-        id: commit
-        run: |
-          git diff --quiet database.json README.md CONTRIBUTING.md && echo "changed=false" >> "$GITHUB_OUTPUT" && exit 0
-          git config user.name "github-actions[bot]"
-          git config user.email "github-actions[bot]@users.noreply.github.com"
-          git add database.json README.md CONTRIBUTING.md
-          git commit -m "regenerate database and docs"
-          git push
-          echo "changed=true" >> "$GITHUB_OUTPUT"
-
-  release:
-    needs: regenerate
-    if: (needs.regenerate.outputs.changed == 'true' || github.event.inputs.force_release == 'true')
-    runs-on: ubuntu-latest
-    permissions:
-      contents: write
-    steps:
       - name: Rate limit
         if: github.event.inputs.force_release != 'true'
         id: rate
@@ -103,19 +53,23 @@ jobs:
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
-      - uses: actions/checkout@v6
+      - name: Restore large files from release
         if: steps.rate.outputs.skip != 'true'
-        with:
-          ref: main
-
-      - uses: actions/setup-python@v6
-        if: steps.rate.outputs.skip != 'true'
-        with:
-          python-version: "3.12"
-
-      - name: Install dependencies
-        if: steps.rate.outputs.skip != 'true'
-        run: pip install pyyaml
+        run: |
+          mkdir -p .cache/large
+          gh release download large-files -D .cache/large/ 2>/dev/null || true
+          for f in .cache/large/*; do
+            [ -f "$f" ] || continue
+            name=$(basename "$f")
+            target=$(grep "$name" .gitignore | head -1)
+            if [ -n "$target" ] && [ ! -f "$target" ]; then
+              mkdir -p "$(dirname "$target")"
+              cp "$f" "$target"
+              echo "Restored: $target"
+            fi
+          done
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Refresh data directories
         if: steps.rate.outputs.skip != 'true'
@@ -124,9 +78,7 @@ jobs:
 
       - name: Build packs
         if: steps.rate.outputs.skip != 'true'
-        run: |
-          python scripts/generate_db.py --bios-dir bios --output database.json
-          python scripts/generate_pack.py --all --output-dir dist/
+        run: python scripts/generate_pack.py --all --output-dir dist/
 
       - name: Release
         if: steps.rate.outputs.skip != 'true'
@@ -150,14 +102,17 @@ jobs:
           ${PACKS}
 
           ### Install
-          Download, extract to your emulator's BIOS directory.
+          Download the pack matching your frontend, extract to the BIOS directory.
 
-          | Platform | Path |
-          |----------|------|
-          | RetroArch / Lakka | system/ |
-          | Batocera | /userdata/bios/ |
-          | Recalbox | /recalbox/share/bios/ |
-          | RetroBat | bios/ |
+          | Platform | Pack | Path |
+          |----------|------|------|
+          | RetroArch / Lakka | RetroArch_Lakka_BIOS_Pack.zip | system/ |
+          | Batocera | Batocera_BIOS_Pack.zip | /userdata/bios/ |
+          | Recalbox | Recalbox_BIOS_Pack.zip | /recalbox/share/bios/ |
+          | RetroBat | RetroBat_BIOS_Pack.zip | bios/ |
+          | RetroDECK | RetroDECK_BIOS_Pack.zip | ~/retrodeck/bios/ |
+          | EmuDeck | EmuDeck_BIOS_Pack.zip | Emulation/bios/ |
+          | RomM | RomM_BIOS_Pack.zip | bios/{platform_slug}/ |
 
           ### Changes
           ${CHANGES}

.github/workflows/deploy-site.yml

@@ -0,0 +1,59 @@
+name: Deploy Site
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "platforms/**"
+      - "emulators/**"
+      - "wiki/**"
+      - "scripts/generate_site.py"
+      - "scripts/generate_readme.py"
+      - "scripts/verify.py"
+      - "scripts/common.py"
+      - "database.json"
+      - "mkdocs.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: deploy-site
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+
+      - run: pip install pyyaml mkdocs-material pymdown-extensions
+
+      - name: Generate site
+        run: |
+          python scripts/generate_site.py
+          python scripts/generate_readme.py --db database.json --platforms-dir platforms
+          mkdocs build
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v4
+        with:
+          path: site/
+
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -11,13 +11,8 @@ site/
 *.log
 node_modules/
 
-# Generated site pages (built in CI)
-docs/index.md
-docs/platforms/
-docs/systems/
-docs/emulators/
-docs/contributing.md
-docs/gaps.md
+# Generated site pages (built by generate_site.py, deployed in CI)
+docs/
 
 # Large files stored as GitHub Release assets (> 50MB)
 bios/Arcade/Arcade/Firmware.19.0.0.zip
@@ -35,3 +30,43 @@ data/
 bios/Arcade/MAME/artwork/snspell.zip
 bios/Arcade/MAME/MAME 0.174 Arcade XML.dat
 bios/Sony/PlayStation Vita/.variants/PSP2UPDAT.PUP
+bios/Nintendo/DS/DSi_Nand_JPN.bin
+bios/Nintendo/DS/DSi_Nand_EUR.bin
+bios/Nintendo/DS/DSi_Nand_USA.bin
+bios/Nintendo/DS/DSi_Nand_AUS.bin
+bios/Nintendo/DS/DSi_Nand_CHN.bin
+bios/Nintendo/DS/DSi_Nand_KOR.bin
+
+# QEMU EDK2 firmware (64MB each)
+bios/QEMU/edk2-aarch64-code.fd
+bios/QEMU/edk2-arm-code.fd
+bios/QEMU/edk2-arm-vars.fd
+
+# ScummVM data bundle (79MB, fetched from large-files release)
+bios/ScummVM/scummvm.zip
+
+# Arcade BIOS ZIPs > 100MB
+bios/Arcade/Arcade/svcpcb.zip
+bios/Arcade/Arcade/ps2.zip
+bios/Arcade/Arcade/kf2k3pcb.zip
+bios/Arcade/Arcade/ms5pcb.zip
+
+# Dolphin/Ishiiruka SD card images (128MB each)
+bios/Nintendo/Wii/WiiSD.raw
+bios/Nintendo/Wii/sd.raw
+
+# EasyRPG soundfont (142MB)
+bios/RPG Maker/easyrpg/easyrpg.soundfont
+
+# Temporary files
+tmp/
+
+# FBA2012 Toaplan samples (>100MB)
+bios/Arcade/fba2012/samples/fireshrk.zip
+bios/Arcade/fba2012/samples/ghox.zip
+bios/Arcade/fba2012/samples/samesame.zip
+bios/Arcade/fba2012/samples/tekipaki.zip
+bios/Arcade/fba2012/samples/vimana.zip
+bios/Arcade/fba2012/samples/fireshrk.zip
+bios/Arcade/fba2012/samples/samesame.zip
+bios/Arcade/fba2012/samples/vimana.zip

README.md

@@ -1,41 +1,124 @@
-# Retrogaming BIOS & Firmware Collection
+# RetroBIOS
 
-Complete, verified collection of BIOS, firmware, and system files for retrogaming emulators.
+Complete BIOS and firmware packs for Batocera, BizHawk, EmuDeck, Lakka, Recalbox, RetroArch, RetroBat, RetroDECK, RetroPie, and RomM.
 
-> **6724** files | **5013.3 MB** | **8** platforms | **275** emulator profiles
+**7,245** verified files across **355** systems, ready to extract into your emulator's BIOS directory.
 
-## Download
+## Quick Install
 
-| Platform | Files | Verification | Pack |
-|----------|-------|-------------|------|
-| Batocera | 359 | md5 | [Download](../../releases/latest) |
-| EmuDeck | 161 | md5 | [Download](../../releases/latest) |
-| Lakka | 448 | existence | [Download](../../releases/latest) |
-| Recalbox | 346 | md5 | [Download](../../releases/latest) |
-| RetroArch | 448 | existence | [Download](../../releases/latest) |
-| RetroBat | 331 | md5 | [Download](../../releases/latest) |
-| RetroDECK | 2007 | md5 | [Download](../../releases/latest) |
-| RetroPie | 448 | existence | [Download](../../releases/latest) |
+Copy one command into your terminal:
+
+```bash
+# Linux / macOS / Steam Deck
+curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh
+
+# Windows (PowerShell)
+irm https://raw.githubusercontent.com/Abdess/retrobios/main/install.ps1 | iex
+
+# Handheld (SD card mounted on PC)
+curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --platform retroarch --dest /path/to/sdcard
+```
+
+The script auto-detects your platform, downloads only missing files, and verifies checksums.
+
+## Download BIOS packs
+
+Pick your platform, download the ZIP, extract to the BIOS path.
+
+| Platform | BIOS files | Extract to | Download |
+|----------|-----------|-----------|----------|
+| Batocera | 359 | `/userdata/bios/` | [Download](../../releases/latest) |
+| BizHawk | 118 | `Firmware/` | [Download](../../releases/latest) |
+| EmuDeck | 161 | `Emulation/bios/` | [Download](../../releases/latest) |
+| Lakka | 448 | `system/` | [Download](../../releases/latest) |
+| Recalbox | 346 | `/recalbox/share/bios/` | [Download](../../releases/latest) |
+| RetroArch | 448 | `system/` | [Download](../../releases/latest) |
+| RetroBat | 331 | `bios/` | [Download](../../releases/latest) |
+| RetroDECK | 2007 | `~/retrodeck/bios/` | [Download](../../releases/latest) |
+| RetroPie | 448 | `BIOS/` | [Download](../../releases/latest) |
+| RomM | 374 | `bios/{platform_slug}/` | [Download](../../releases/latest) |
+
+## What's included
+
+BIOS, firmware, and system files for consoles from Atari to PlayStation 3.
+Each file is checked against the emulator's source code to match what the code actually loads at runtime.
+
+- **10 platforms** supported with platform-specific verification
+- **328 emulators** profiled from source (RetroArch cores + standalone)
+- **355 systems** covered (NES, SNES, PlayStation, Saturn, Dreamcast, ...)
+- **7,245 files** verified with MD5, SHA1, CRC32 checksums
+- **9266 MB** total collection size
+
+## Supported systems
+
+NES, SNES, Nintendo 64, GameCube, Wii, Game Boy, Game Boy Advance, Nintendo DS, Nintendo 3DS, Switch, PlayStation, PlayStation 2, PlayStation 3, PSP, PS Vita, Mega Drive, Saturn, Dreamcast, Game Gear, Master System, Neo Geo, Atari 2600, Atari 7800, Atari Lynx, Atari ST, MSX, PC Engine, TurboGrafx-16, ColecoVision, Intellivision, Commodore 64, Amiga, ZX Spectrum, Arcade (MAME), and 321+ more.
+
+Full list with per-file details: **[https://abdess.github.io/retrobios/](https://abdess.github.io/retrobios/)**
 
 ## Coverage
 
 | Platform | Coverage | Verified | Untested | Missing |
 |----------|----------|----------|----------|---------|
-| Batocera | 359/359 (100.0%) | 358 | 1 | 0 |
+| Batocera | 359/359 (100.0%) | 354 | 5 | 0 |
+| BizHawk | 118/118 (100.0%) | 118 | 0 | 0 |
 | EmuDeck | 161/161 (100.0%) | 161 | 0 | 0 |
-| Lakka | 448/448 (100.0%) | 448 | 0 | 0 |
-| Recalbox | 346/346 (100.0%) | 346 | 0 | 0 |
-| RetroArch | 448/448 (100.0%) | 448 | 0 | 0 |
-| RetroBat | 331/331 (100.0%) | 331 | 0 | 0 |
-| RetroDECK | 2007/2007 (100.0%) | 1698 | 309 | 0 |
-| RetroPie | 448/448 (100.0%) | 448 | 0 | 0 |
+| Lakka | 443/448 (98.9%) | 443 | 0 | 5 |
+| Recalbox | 276/346 (79.8%) | 273 | 3 | 70 |
+| RetroArch | 443/448 (98.9%) | 443 | 0 | 5 |
+| RetroBat | 330/331 (99.7%) | 326 | 4 | 1 |
+| RetroDECK | 1958/2007 (97.6%) | 1876 | 82 | 49 |
+| RetroPie | 443/448 (98.9%) | 443 | 0 | 5 |
+| RomM | 372/374 (99.5%) | 372 | 0 | 2 |
 
-## Documentation
+## Build your own pack
 
-Full file listings, platform coverage, emulator profiles, and gap analysis: **[https://abdess.github.io/retrobios/](https://abdess.github.io/retrobios/)**
+Clone the repo and generate packs for any platform, emulator, or system:
+
+```bash
+# Full platform pack
+python scripts/generate_pack.py --platform retroarch --output-dir dist/
+python scripts/generate_pack.py --platform batocera --output-dir dist/
+
+# Single emulator or system
+python scripts/generate_pack.py --emulator dolphin
+python scripts/generate_pack.py --system sony-playstation-2
+
+# List available emulators and systems
+python scripts/generate_pack.py --list-emulators
+python scripts/generate_pack.py --list-systems
+
+# Verify your BIOS collection
+python scripts/verify.py --all
+python scripts/verify.py --platform batocera
+python scripts/verify.py --emulator flycast
+python scripts/verify.py --platform retroarch --verbose  # emulator ground truth
+```
+
+Only dependency: Python 3 + `pyyaml`.
+
+## Documentation site
+
+The [documentation site](https://abdess.github.io/retrobios/) provides:
+
+- **Per-platform pages** with file-by-file verification status and hashes
+- **Per-emulator profiles** with source code references for every file
+- **Per-system pages** showing which emulators and platforms cover each console
+- **Gap analysis** identifying missing files and undeclared core requirements
+- **Cross-reference** mapping files across 10 platforms and 328 emulators
+
+## How it works
+
+Documentation and metadata can drift from what emulators actually load.
+To keep packs accurate, each file is checked against the emulator's source code.
+
+1. **Read emulator source code** - trace every file the code loads, its expected hash and size
+2. **Cross-reference with platforms** - match against what each platform declares
+3. **Build packs** - include baseline files plus what each platform's cores need
+4. **Verify** - run platform-native checks and emulator-level validation
 
 ## Contributors
 
+<a href="https://github.com/PixNyb"><img src="https://avatars.githubusercontent.com/u/40770831?v=4" width="50" title="PixNyb"></a>
 <a href="https://github.com/monster-penguin"><img src="https://avatars.githubusercontent.com/u/266009589?v=4" width="50" title="monster-penguin"></a>
 
 
@@ -47,4 +130,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
 
 This repository provides BIOS files for personal backup and archival purposes.
 
-*Auto-generated on 2026-03-20T19:10:25Z*
+*Auto-generated on 2026-03-29T14:04:16Z*