From 6353ee53dfcd26e8343e74a67dd89bf17226b4ad Mon Sep 17 00:00:00 2001 From: Abdessamad Derraz <3028866+Abdess@users.noreply.github.com> Date: Thu, 22 Oct 2026 11:50:53 +0200 Subject: [PATCH] feat: lay the site out as a reading funnel Five tabs from Start here to Contribute, a level on every page, a device picker on Get your files that lists every registered platform, and platform pages that open on where the files go, the command, the pack and how to check. FAQ and troubleshooting keep the questions asked before installing, the technical ones move to their own page, and getting started points to the pages that took its content. --- README.md | 4 +- mkdocs.yml | 1478 +++++++++++++++++------------------ scripts/generate_readme.py | 2 +- scripts/generate_site.py | 1287 ++++++++++++++++++------------ scripts/sitewrite.py | 6 + tests/test_doc_pages.py | 21 +- tests/test_doc_surfaces.py | 28 +- tests/test_site_exports.py | 7 +- tests/test_site_funnel.py | 227 ++++++ wiki/adding-a-scraper.md | 4 + wiki/advanced-usage.md | 21 +- wiki/architecture.md | 4 + wiki/community-tools.md | 6 +- wiki/faq.md | 217 +++-- wiki/first-install.md | 41 +- wiki/getting-started.md | 314 +------- wiki/glossary.md | 153 ++++ wiki/index.md | 96 +-- wiki/installer.md | 10 +- wiki/profiling.md | 4 + wiki/release-process.md | 4 + wiki/technical-faq.md | 112 +++ wiki/testing-guide.md | 6 +- wiki/tools.md | 6 +- wiki/troubleshooting.md | 213 ++--- wiki/verification-modes.md | 149 +++- wiki/what-are-bios-files.md | 76 ++ 27 files changed, 2574 insertions(+), 1922 deletions(-) create mode 100644 tests/test_site_funnel.py create mode 100644 wiki/glossary.md create mode 100644 wiki/technical-faq.md create mode 100644 wiki/what-are-bios-files.md diff --git a/README.md b/README.md index d56ba441..ed908602 100644 --- a/README.md +++ b/README.md @@ -177,7 +177,7 @@ The [documentation site](https://abdess.github.io/retrobios/) provides: A platform's BIOS list and an emulator's code are maintained separately. Each list is compared with the emulator source code, file by file where a profile exists, and where the two differ the code decides. -Hashes document what emulator code loads and accepts, not dump provenance. That boundary, and how it relates to preservation catalogs such as No-Intro, is drawn in the [FAQ](https://abdess.github.io/retrobios/wiki/faq/#are-these-files-verified-against-original-hardware-dumps). +Hashes document what emulator code loads and accepts, not dump provenance. That boundary, and how it relates to preservation catalogs such as No-Intro, is drawn in the [technical questions](https://abdess.github.io/retrobios/wiki/technical-faq/#are-these-files-verified-against-original-hardware-dumps). 1. **Read emulator source code**: trace every file the code loads, its expected hash and size 2. **Cross-reference with platforms**: match against what each platform declares @@ -208,4 +208,4 @@ The scripts and tooling are released under the [MIT License](LICENSE). The BIOS and firmware files are not covered by that license: they are third-party system software, preserved and provided for personal backup, archival, and interoperability with emulation software. [NOTICE](NOTICE) sets out their status and how to ask for a file to be removed. The reasoning, and where it is weakest, is in the [FAQ](https://abdess.github.io/retrobios/wiki/faq/#is-this-legal). -*Auto-generated on 2026-10-22T09:06:29Z* +*Auto-generated on 2026-10-22T09:44:26Z* diff --git a/mkdocs.yml b/mkdocs.yml index 9fb3b6bc..357d4d6e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -53,6 +53,8 @@ theme: - navigation.top - navigation.tracking - navigation.indexes + # Breadcrumbs above the title: on a deep page they say which tab it is in. + - navigation.path # 400+ pages: pruning keeps the navigation out of every page's HTML. - navigation.prune - navigation.footer @@ -107,762 +109,728 @@ validation: unrecognized_links: warn anchors: warn nav: -- Home: index.md -- Download: which-pack.md -- Platforms: - - Overview: platforms/index.md - - AmberELEC: platforms/amberelec.md - - Batocera: platforms/batocera.md - - BizHawk: platforms/bizhawk.md - - EmuDeck: platforms/emudeck.md - - EmuELEC: platforms/emuelec.md - - KNULLI: platforms/knulli.md - - Lakka: platforms/lakka.md - - MiSTer FPGA: platforms/misterfpga.md - - MinUI: platforms/minui.md - - MyMinUI: platforms/myminui.md - - NextUI: platforms/nextui.md - - ROCKNIX: platforms/rocknix.md - - Recalbox: platforms/recalbox.md - - RetroArch: platforms/retroarch.md - - RetroBat: platforms/retrobat.md - - RetroDECK: platforms/retrodeck.md - - RetroPie: platforms/retropie.md - - RomM: platforms/romm.md -- Systems: - - Overview: systems/index.md - - 3D Realms: systems/3d-realms.md - - 3DO Company: systems/3do-company.md - - APF: systems/apf.md - - Acorn: systems/acorn.md - - Amstrad: systems/amstrad.md - - Apogee: systems/apogee.md - - Apple: systems/apple.md - - Aquaplus: systems/aquaplus.md - - Arcade: systems/arcade.md - - Atari: systems/atari.md - - BBK: systems/bbk.md - - Bally: systems/bally.md - - Bandai: systems/bandai.md - - Bit Corporation: systems/bit-corporation.md - - Blizzard: systems/blizzard.md - - Bungie: systems/bungie.md - - Camputers: systems/camputers.md - - Casio: systems/casio.md - - Coleco: systems/coleco.md - - Commodore: systems/commodore.md - - DOS: systems/dos.md - - Dinothawr: systems/dinothawr.md - - Dragon: systems/dragon.md - - EACA: systems/eaca.md - - Electronic Arts: systems/electronic-arts.md - - Elektronika: systems/elektronika.md - - Enterprise: systems/enterprise.md - - Entex: systems/entex.md - - Epic MegaGames: systems/epic-megagames.md - - Epoch: systems/epoch.md - - Fairchild: systems/fairchild.md - - Fujitsu: systems/fujitsu.md - - Funtech: systems/funtech.md - - GCE: systems/gce.md - - Galaksija: systems/galaksija.md - - Game Engines: systems/game-engines.md - - GamePark: systems/gamepark.md - - Gemei: systems/gemei.md - - Grundy: systems/grundy.md - - Hartung: systems/hartung.md - - IBM: systems/ibm.md - - Id Software: systems/id-software.md - - Infocom: systems/infocom.md - - Interact: systems/interact.md - - Interplay: systems/interplay.md - - Java: systems/java.md - - Lexaloffle: systems/lexaloffle.md - - Magnavox: systems/magnavox.md - - Mattel: systems/mattel.md - - Microsoft: systems/microsoft.md - - Monolith: systems/monolith.md - - NEC: systems/nec.md - - Nintendo: systems/nintendo.md - - Nokia: systems/nokia.md - - Oric: systems/oric.md - - Other: systems/other.md - - PC: systems/pc.md - - Palm: systems/palm.md - - Philips: systems/philips.md - - Pioneer: systems/pioneer.md - - QEMU: systems/qemu.md - - RCA: systems/rca.md - - RPG Maker: systems/rpg-maker.md - - Real3D: systems/real3d.md - - Roland: systems/roland.md - - SDL: systems/sdl.md - - SNK: systems/snk.md - - ScummVM: systems/scummvm.md - - Sega: systems/sega.md - - Sharp: systems/sharp.md - - Sinclair: systems/sinclair.md - - Softdisk: systems/softdisk.md - - Sony: systems/sony.md - - Sord: systems/sord.md - - Synertek: systems/synertek.md - - Tandy: systems/tandy.md - - Tangerine: systems/tangerine.md - - Texas Instruments: systems/texas-instruments.md - - Thomson: systems/thomson.md - - Tiger: systems/tiger.md - - TimeTop: systems/timetop.md - - Tomy: systems/tomy.md - - Toys for Bob: systems/toys-for-bob.md - - VTech: systems/vtech.md - - Valve: systems/valve.md - - VideoBrain: systems/videobrain.md - - Videoton: systems/videoton.md - - Vircon: systems/vircon.md - - Wenquxing: systems/wenquxing.md - - ZC: systems/zc.md - - sdlpal: systems/sdlpal.md -- Emulators: - - Overview: emulators/index.md - - Official ports (78): - - amiarcadia: emulators/amiarcadia.md - - Amiberry: emulators/amiberry.md - - Ardens: emulators/ardens.md - - Atari800: emulators/atari800.md - - BlastEm: emulators/blastem.md - - Boytacean: emulators/boytacean.md - - bsnes: emulators/bsnes.md - - ClownMDEmu: emulators/clownmdemu.md - - CoolbarEmu: emulators/coolbaremu.md - - CrocoDS: emulators/crocods.md - - DirkSimple: emulators/dirksimple.md - - DOSBox Pure Unleashed: emulators/dosbox-pure-unleashed.md - - DuckStation: emulators/duckstation.md - - emux (CHIP-8): emulators/emux_chip8.md - - emux (Game Boy): emulators/emux_gb.md - - emux (NES): emulators/emux_nes.md - - emux (SMS): emulators/emux_sms.md - - FinalBurn Neo: emulators/fbneo.md - - FinalBurn Neo (CPS-1/CPS-2): emulators/fbneo_cps12.md - - FinalBurn Neo (Neo Geo): emulators/fbneo_neogeo.md - - fixGB: emulators/fixgb.md - - fixNES: emulators/fixnes.md - - Flycast: emulators/flycast.md - - FreeJ2ME: emulators/freej2me.md - - galaksija: emulators/galaksija.md - - Gearboy: emulators/gearboy.md - - Gearcoleco: emulators/gearcoleco.md - - Geargrafx: emulators/geargrafx.md - - Gearlynx: emulators/gearlynx.md - - Gearsystem: emulators/gearsystem.md - - Geartowns: emulators/geartowns.md - - Geolith: emulators/geolith.md - - Hatari: emulators/hatari.md - - Holani: emulators/holani.md - - IroGB: emulators/irogb.md - - JAXE: emulators/jaxe.md - - JollyCV: emulators/jollycv.md - - Kronos: emulators/kronos.md - - LowRes NX: emulators/lowresnx.md - - M2000: emulators/m2000.md - - MAME: emulators/mame.md - - MAME Arcade: emulators/mamearcade.md - - MAME MESS: emulators/mamemess.md - - Mesen: emulators/mesen.md - - mesen-s: emulators/mesen-s.md - - mGBA: emulators/mgba.md - - Mr.Boom: emulators/mrboom.md - - Nuance: emulators/nuance.md - - Panda3DS: emulators/panda3ds.md - - PCSX2: emulators/pcsx2.md - - PicoDrive: emulators/picodrive.md - - piece-emu: emulators/piece.md - - play: emulators/play.md - - pokketstation: emulators/pokketstation.md - - PPSSPP: emulators/ppsspp.md - - RB5000: emulators/rb5000.md - - Picocomputer 6502: emulators/rp6502.md - - RPCS3: emulators/rpcs3.md - - Rust-DOS: emulators/rust_dos.md - - Rustation: emulators/rustation.md - - RustyNES: emulators/rustynes.md - - RVVM: emulators/rvvm.md - - SimCoupe (standalone): emulators/samcoupe.md - - SameBoy: emulators/sameboy.md - - sameduck: emulators/sameduck.md - - SDLPAL: emulators/sdlpal.md - - skyemu: emulators/skyemu.md - - snes9x: emulators/snes9x.md - - SquirrelJME: emulators/squirreljme.md - - Stella: emulators/stella.md - - VaporSpec: emulators/vaporspec.md - - VBA-M: emulators/vbam.md - - VeMUlator: emulators/vemulator.md - - Vircon32: emulators/vircon32.md - - vitaQuakeII: emulators/vitaquake2.md - - WQXEmu: emulators/wqxemu.md - - yabasanshiro: emulators/yabasanshiro.md - - Yuzu: emulators/yuzu.md - - Community forks (190): - - 2600.emu: emulators/2600-emu.md - - EightyOne: emulators/81.md - - a5200: emulators/a5200.md - - ACE-DL: emulators/ace-dl.md - - AetherSX2: emulators/aethersx2.md - - Anarch: emulators/anarch.md - - AppleWin: emulators/applewin.md - - aPS3e: emulators/aps3e.md - - ArcadeDuck: emulators/arcadeduck.md - - ARMSX1: emulators/armsx1.md - - ARMSX2: emulators/armsx2.md - - aX360e: emulators/ax360e.md - - Azahar: emulators/azahar.md - - AzaharPlus: emulators/azaharplus.md - - b2: emulators/b2.md - - Beetle Lynx (Mednafen Lynx): emulators/beetle_lynx.md - - Beetle NGP (Mednafen Neo Geo Pocket): emulators/beetle_ngp.md - - Beetle PCE (Mednafen PCE): emulators/beetle_pce.md - - Beetle PC-FX (Mednafen): emulators/beetle_pcfx.md - - Beetle PSX (Mednafen PSX): emulators/beetle_psx.md - - beetle_saturn: emulators/beetle_saturn.md - - Beetle VB (Mednafen Virtual Boy): emulators/beetle_vb.md - - BennuGD: emulators/bennugd.md - - bk-emulator: emulators/bk.md - - blueMSX: emulators/bluemsx.md - - bsnes-jg: emulators/bsnes-jg.md - - C64.emu: emulators/c64-emu.md - - Caprice32: emulators/cap32.md - - CapriceRPI: emulators/capricerpi.md - - Cemu: emulators/cemu.md - - Chihiro-DS: emulators/chihiro-ds.md - - ChimeraSNES: emulators/chimerasnes.md - - Citra: emulators/citra.md - - Citra MMJ: emulators/citra-mmj.md - - Citra Canary: emulators/citra_canary.md - - Citron: emulators/citron.md - - Daphne: emulators/daphne.md - - DeSmuME: emulators/desmume.md - - dhewm3: emulators/dhewm3.md - - DICE: emulators/dice.md - - Dolphin: emulators/dolphin.md - - Dolphin MMJR: emulators/dolphin-mmjr.md - - Dolphin MMJR2: emulators/dolphin-mmjr2.md - - DOSBox: emulators/dosbox.md - - DOSBox-core: emulators/dosbox_core.md - - DOSBox-SVN: emulators/dosbox_svn.md - - DOSBox-SVN CE: emulators/dosbox_svn_ce.md - - Eden: emulators/eden.md - - EightyOne (Recalbox): emulators/eightyone.md - - eka2l1: emulators/eka2l1.md - - EmuCoreV: emulators/emucorev.md - - EmuCoreX: emulators/emucorex.md - - EmuSCV: emulators/emuscv.md - - ep128emu_core: emulators/ep128emu_core.md - - Eternal Computer: emulators/eternal.md - - FCEUmm: emulators/fceumm.md - - FCEUmm Mod: emulators/fceumm_mod.md - - fMSX: emulators/fmsx.md - - Frodo: emulators/frodo.md - - FS-UAE: emulators/fsuae.md - - Fuse: emulators/fuse.md - - Gambatte: emulators/gambatte.md - - GameTank: emulators/gametank.md - - GBA.emu: emulators/gba-emu.md - - GBC.emu: emulators/gbc-emu.md - - Genesis Plus GX: emulators/genesis_plus_gx.md - - GnGeo-Pi: emulators/gngeopi.md - - gpSP: emulators/gpsp.md - - GSplus: emulators/gsplus.md - - GSplus (legacy): emulators/gsplus-legacy.md - - hakuX: emulators/hakux.md - - Handy: emulators/handy.md - - hatariB: emulators/hatarib.md - - Infinity: emulators/infinity.md - - ioRTCW: emulators/iortcw.md - - J2ME Loader: emulators/j2me-loader.md - - JL-Mod: emulators/jl-mod.md - - KEmulator nnmod: emulators/kemulator.md - - Kenji-NX: emulators/kenji-nx.md - - Lime3DS: emulators/lime3ds.md - - LinApple: emulators/linapple.md - - LRPS2: emulators/lrps2.md - - Lynx.emu: emulators/lynx-emu.md - - M64Plus FZ: emulators/m64plus-fz.md - - MAME4droid 2024: emulators/mame4droid-2024.md - - Mandarine: emulators/mandarine.md - - Mariani: emulators/mariani.md - - MD.emu: emulators/md-emu.md - - mednafen_pce: emulators/mednafen_pce.md - - mednafen_pce_fast: emulators/mednafen_pce_fast.md - - Mednafen STV: emulators/mednafen_stv.md - - mednafen_supafaust: emulators/mednafen_supafaust.md - - mednafen_supergrafx: emulators/mednafen_supergrafx.md - - melonDS: emulators/melonds.md - - melonDS Nightly: emulators/melonds-nightly.md - - melonDS DS: emulators/melonds_ds.md - - Meteor GBA: emulators/meteor.md - - mGBA Splitscreen: emulators/mgba_splitscreen.md - - Mini vMac: emulators/minivmac.md - - Mini vMac 37: emulators/minivmac37.md - - Moonlight: emulators/moonlight.md - - mpv: emulators/mpv.md - - MSX.emu: emulators/msx-emu.md - - Mupen64Plus AE: emulators/mupen64plus-ae.md - - nekop2: emulators/nekop2.md - - NEO.emu: emulators/neo-emu.md - - nes: emulators/nes.md - - NES.emu: emulators/nes-emu.md - - Nesoid: emulators/nesoid.md - - Nestopia UE: emulators/nestopia.md - - NetherSX2: emulators/nethersx2.md - - NetherSX2-Turnip: emulators/nethersx2-turnip.md - - NetherSX2-Turnip Classic: emulators/nethersx2-turnip-classic.md - - NGP.emu: emulators/ngp-emu.md - - NooDS: emulators/noods.md - - np2pi: emulators/np2pi.md - - Numero: emulators/numero.md - - O2EM: emulators/o2em.md - - Oberon: emulators/oberon.md - - ONScripter: emulators/onscripter.md - - Opera (4DO): emulators/opera.md - - Osmose (RetroPie): emulators/osmose.md - - ParaLLEl N64: emulators/parallel_n64.md - - pce-emu: emulators/pce-emu.md - - PCEE2: emulators/pcee2.md - - PCem: emulators/pcem.md - - PCSX1: emulators/pcsx1.md - - PD777: emulators/pd777.md - - PokeMini: emulators/pokemini.md - - Potator: emulators/potator.md - - ProSystem: emulators/prosystem.md - - PUAE (P-UAE): emulators/puae.md - - px68k: emulators/px68k.md - - px68k (SDL): emulators/px68k-sdl.md - - QEMU: emulators/qemu.md - - QUASI88: emulators/quasi88.md - - QuickNES: emulators/quicknes.md - - RACE (Neo Geo Pocket): emulators/race.md - - Reicast: emulators/reicast.md - - REminiscence: emulators/reminiscence.md - - RetroDream: emulators/retrodream.md - - Rosalie's Mupen GUI: emulators/rosalies-mupen-gui.md - - SAME CDi: emulators/same_cdi.md - - Saturn.emu: emulators/saturn-emu.md - - SDL2TRS: emulators/sdl2trs.md - - SimCoupe: emulators/simcp.md - - simple64: emulators/simple64.md - - Snes9x EX+: emulators/snes9x-explus.md - - Sudachi: emulators/sudachi.md - - Sugarbox2: emulators/sugarbox2.md - - SUPER3: emulators/super3.md - - Supermodel: emulators/supermodel.md - - Suyu: emulators/suyu.md - - Swan.emu: emulators/swan-emu.md - - swanstation: emulators/swanstation.md - - Syobon Action: emulators/syobonaction.md - - TamaLIBretro: emulators/tamalibretro.md - - TempGBA: emulators/tempgba.md - - TGB Dual: emulators/tgbdual.md - - trident: emulators/trident.md - - UAE4All2: emulators/uae4all.md - - UAE4ALL (libretro, Chips-fr): emulators/uae4all-rpi.md - - uae4arm: emulators/uae4arm.md - - UXN: emulators/uxn.md - - vecx: emulators/vecx.md - - VICE x128: emulators/vice_x128.md - - VICE x64: emulators/vice_x64.md - - VICE x64dtv: emulators/vice_x64dtv.md - - VICE x64sc: emulators/vice_x64sc.md - - VICE xcbm2: emulators/vice_xcbm2.md - - VICE xcbm5x0: emulators/vice_xcbm5x0.md - - VICE xpet: emulators/vice_xpet.md - - VICE xplus4: emulators/vice_xplus4.md - - VICE xscpu64: emulators/vice_xscpu64.md - - VICE xvic: emulators/vice_xvic.md - - Virtual Jaguar: emulators/virtualjaguar.md - - vitaQuakeIII: emulators/vitaquake3.md - - vitavoyager: emulators/vitavoyager.md - - X Millennium: emulators/x1.md - - X1 BOX: emulators/x1-box.md - - x64sdl: emulators/x64sdl.md - - xemu: emulators/xemu.md - - xemu Chihiro: emulators/xemu-chihiro.md - - XenDroid: emulators/xendroid.md - - Xenia Canary: emulators/xenia.md - - Xenia Edge: emulators/xenia-edge.md - - XeniOS: emulators/xenios.md - - yabause: emulators/yabause.md - - yaps2: emulators/yaps2.md - - ymir: emulators/ymir.md - - Pure libretro (52): - - 00_example: emulators/00_example.md - - '2048': emulators/2048.md - - 3DEngine: emulators/3dengine.md - - advanced_tests: emulators/advanced_tests.md - - Arduous: emulators/arduous.md - - BBKEmu: emulators/bbkemu.md - - Beetle WonderSwan (Mednafen WonderSwan): emulators/beetle_wswan.md - - Craft: emulators/craft.md - - Cruzes: emulators/cruzes.md - - DingooEmu: emulators/dingooemu.md - - DirectXBox: emulators/directxbox.md - - EmuCV: emulators/emucv.md - - Exel-Hence: emulators/exel-hence.md - - FFmpeg: emulators/ffmpeg.md - - FreeChaF: emulators/freechaf.md - - FreeIntv: emulators/freeintv.md - - GAM4980: emulators/gam4980.md - - Game Music Emu: emulators/gme.md - - Gong: emulators/gong.md - - Game & Watch: emulators/gw.md - - Image Viewer: emulators/imageviewer.md - - MCSoftserve: emulators/mcsoftserve.md - - MojoZork: emulators/mojozork.md - - Mu: emulators/mu.md - - Native32Emu: emulators/native32emu.md - - NeoCD: emulators/neocd.md - - Pascal Pong: emulators/pascal_pong.md - - PlaydiaEmu: emulators/playdiaemu.md - - PocketCDG: emulators/pocketcdg.md - - radio: emulators/radio.md - - Redbook: emulators/redbook.md - - RemoteJoy: emulators/remotejoy.md - - ROM Cleaner: emulators/romcleaner.md - - SPMP8000Emu: emulators/spmp8000emu.md - - test_netplay: emulators/test_netplay.md - - testaudio_callback: emulators/testaudio_callback.md - - testaudio_no_callback: emulators/testaudio_no_callback.md - - testaudio_playback_wav: emulators/testaudio_playback_wav.md - - testgl_compute_shaders: emulators/testgl_compute_shaders.md - - testgl_ff: emulators/testgl_ff.md - - testinput_buttontest: emulators/testinput_buttontest.md - - testretroluxury: emulators/testretroluxury.md - - testsw: emulators/testsw.md - - testsw_vram: emulators/testsw_vram.md - - testvulkan_async_compute: emulators/testvulkan_async_compute.md - - Theodore: emulators/theodore.md - - The Powder Toy: emulators/thepowdertoy.md - - Tia: emulators/tia.md - - MicroW8: emulators/uw8.md - - uzem: emulators/uzem.md - - video_processor: emulators/video_processor.md - - VirtualXT: emulators/virtualxt.md - - Game engines (108): - - 2 Ship 2 Harkinian: emulators/2ship.md - - Abuse: emulators/abuse.md - - Adventure Game Studio: emulators/ags.md - - Aleph One: emulators/alephone.md - - boom3: emulators/boom3.md - - BStone: emulators/bstone.md - - Bermuda Syndrome: emulators/bsyndrome.md - - Cannonball: emulators/cannonball.md - - CatacombGL: emulators/catacombgl.md - - C-Dogs SDL: emulators/cdogs.md - - Commander Genius: emulators/cgenius.md - - ChaiLove: emulators/chailove.md - - corsixth: emulators/corsixth.md - - DarkPlaces: emulators/darkplaces.md - - devilutionx: emulators/devilutionx.md - - Dinothawr: emulators/dinothawr.md - - doukutsu-rs: emulators/doukutsu_rs.md - - dsda-doom: emulators/dsda-doom.md - - Dusklight: emulators/dusklight.md - - dxx-rebirth: emulators/dxx-rebirth.md - - EasyRPG Player: emulators/easyrpg.md - - ECWolf: emulators/ecwolf.md - - EDuke32: emulators/eduke32.md - - ET Legacy: emulators/etlegacy.md - - FAKE-08: emulators/fake08.md - - fallout1-ce: emulators/fallout1-ce.md - - fallout2-ce: emulators/fallout2-ce.md - - Frotz: emulators/frotz.md - - Ion Fury: emulators/fury.md - - future-pinball: emulators/future-pinball.md - - Gargoyle: emulators/gargoyle.md - - GemRB: emulators/gemrb.md - - Ghostship: emulators/ghostship.md - - GZDoom: emulators/gzdoom.md - - Hydra Castle Labyrinth: emulators/hcl.md - - Hurrican: emulators/hurrican.md - - idTech4A++: emulators/idtech4a++.md - - Ikemen GO: emulators/ikemen.md - - ioquake3: emulators/ioquake3.md - - Jazz² Resurrection: emulators/jazz2-native.md - - Julius: emulators/julius.md - - Jump 'n Bump: emulators/jumpnbump.md - - lightspark: emulators/lightspark.md - - LOVE: emulators/love.md - - LOVE 0.10.2: emulators/love-0.10.2.md - - Lutro: emulators/lutro.md - - LZDoom: emulators/lzdoom.md - - mkxp-z: emulators/mkxp_z.md - - NXEngine: emulators/nxengine.md - - Omicron: emulators/omicron.md - - ONScripter Yuri: emulators/onsyuri.md - - openbor: emulators/openbor.md - - OpenGOAL: emulators/opengoal.md - - OpenJazz: emulators/openjazz.md - - openjk: emulators/openjk.md - - OpenJKDF2: emulators/openjkdf2.md - - OpenLara: emulators/openlara.md - - OpenMoHAA: emulators/openmohaa.md - - OpenTTD: emulators/openttd.md - - OpenTyrian: emulators/opentyrian.md - - Perfect Dark (PC port): emulators/pdark.md - - PrBoom: emulators/prboom.md - - PrBoom+: emulators/prboom-plus.md - - PuzzleScript: emulators/puzzlescript.md - - Pyxel: emulators/pyxel.md - - Quake 3 (Raspberry Pi): emulators/quake3.md - - Raze: emulators/raze.md - - Retro8: emulators/retro8.md - - RigelEngine: emulators/rigelengine.md - - ruffle: emulators/ruffle.md - - ScummVM: emulators/scummvm.md - - ScummVM (SDL1): emulators/scummvm-sdl1.md - - sdlpop: emulators/sdlpop.md - - Singe 2: emulators/singe2.md - - Ship of Harkinian: emulators/soh.md - - solarus: emulators/solarus.md - - Solarus 2.0.0: emulators/solarus-2.0.0.md - - Sonic Mania Decompilation: emulators/sonic-mania.md - - Sonic 1 & 2 (2013) Decompilation: emulators/sonic2013.md - - Sonic 3 A.I.R.: emulators/sonic3-air.md - - Sonic CD Decompilation: emulators/soniccd.md - - SplitWolf: emulators/splitwolf.md - - Sonic Robo Blast 2: emulators/srb2.md - - Starship: emulators/starship.md - - Stratagus: emulators/stratagus.md - - Super Bros War: emulators/superbroswar.md - - Taradino: emulators/taradino.md - - The Force Engine: emulators/theforceengine.md - - TheXTech: emulators/thextech.md - - TIC-80: emulators/tic80.md - - TR1X: emulators/tr1x.md - - TR2X: emulators/tr2x.md - - TyrQuake: emulators/tyrquake.md - - uqm: emulators/uqm.md - - UZDoom: emulators/uzdoom.md - - visual-pinball: emulators/visual-pinball.md - - vkQuake: emulators/vkquake.md - - vkQuake2: emulators/vkquake2.md - - vkQuake3: emulators/vkquake3.md - - VVVVVV: emulators/vvvvvv.md - - WASM-4: emulators/wasm4.md - - Wolf4SDL: emulators/wolf4sdl.md - - Xash3D FWGS: emulators/xash3d_fwgs.md - - XRick: emulators/xrick.md - - Yamagi Quake II: emulators/yquake2.md - - Zelda Classic v2.10: emulators/zc210.md - - Zelda Classic 2.55 Prototype: emulators/zc255.md - - ZQuest Classic: emulators/zquestclassic.md - - Enhanced forks (34): - - A7800: emulators/a7800.md - - AdvanceMAME: emulators/advancemame.md - - bsnes-hd beta: emulators/bsnes_hd_beta.md - - bsnes-mercury: emulators/bsnes_mercury.md - - DOSBox Staging: emulators/dosbox-staging.md - - DOSBox-X: emulators/dosbox-x.md - - DOSBox Pure: emulators/dosbox_pure.md - - DoubleCherryGB: emulators/doublecherrygb.md - - Flycast Dojo: emulators/flycast-dojo.md - - FreeJ2ME-Plus: emulators/freej2me_plus.md - - future-pinball-fploader: emulators/future-pinball-fploader.md - - Genesis Plus GX Expanded: emulators/genesis_plus_gx_expanded.md - - Genesis Plus GX Wide: emulators/genesis_plus_gx_wide.md - - GroovyMAME: emulators/groovymame.md - - HBMAME (Homebrew MAME): emulators/hbmame.md - - nSide (SFC Balanced): emulators/higan_sfc_balanced.md - - Hypseus Singe: emulators/hypseus-singe.md - - Ishiiruka: emulators/ishiiruka.md - - MAME 2003-Plus: emulators/mame2003_plus.md - - MesenCE: emulators/mesence.md - - Mupen64Plus-Next: emulators/mupen64plus_next.md - - NP2kai: emulators/np2kai.md - - PCSX2x6: emulators/pcsx2x6.md - - PrimeHack: emulators/primehack.md - - pSxMAME: emulators/psxmame.md - - QUASI88kai: emulators/quasi88kai.md - - RealRTCW: emulators/realrtcw.md - - SMS Plus GX: emulators/smsplus.md - - Supermodel Dojo: emulators/supermodel-dojo.md - - Supermodel (Sinden fork): emulators/supermodel3.md - - SuperSnes9x: emulators/supersnes9x.md - - Triforce: emulators/triforce.md - - xm6pro68k: emulators/xm6pro68k.md - - xm6typeg: emulators/xm6typeg.md - - Frozen snapshots (50): - - AdvanceMAME 0.94: emulators/advmame-0.94.md - - AdvanceMAME 1.4: emulators/advmame-1.4.md - - bnes: emulators/bnes.md - - bsnes 2014: emulators/bsnes2014.md - - bsnes C++98: emulators/bsnes_cplusplus98.md - - CDi 2015: emulators/cdi2015.md - - Citra 2018: emulators/citra2018.md - - Cxbx-Reloaded (31ff15b): emulators/cxbx-reloaded-2023.md - - DeSmuME 2015: emulators/desmume2015.md - - piFBA (fba2x): emulators/fba2x.md - - FB Alpha 2012: emulators/fbalpha2012.md - - FB Alpha 2012 CPS-1: emulators/fbalpha2012_cps1.md - - FB Alpha 2012 CPS-2: emulators/fbalpha2012_cps2.md - - FB Alpha 2012 CPS-3: emulators/fbalpha2012_cps3.md - - FB Alpha 2012 Neo Geo: emulators/fbalpha2012_neogeo.md - - FB Alpha 2019: emulators/fbalpha2019.md - - Flycast 2021: emulators/flycast2021.md - - Hatari 2014: emulators/hatari2014.md - - higan (SFC Accuracy): emulators/higan_sfc.md - - MAME 0.258: emulators/mame0258.md - - MAME 0.278: emulators/mame0278.md - - MAME 2000: emulators/mame2000.md - - MAME 2003: emulators/mame2003.md - - MAME 2003 Midway: emulators/mame2003_midway.md - - MAME 2009: emulators/mame2009.md - - MAME 2010: emulators/mame2010.md - - MAME 2014: emulators/mame2014.md - - MAME 2015: emulators/mame2015.md - - MAME 2016: emulators/mame2016.md - - MAME4ALL-Pi: emulators/mame4all.md - - MAME4droid (0.139u1): emulators/mame4droid.md - - mednafen_gba: emulators/mednafen_gba.md - - mednafen_snes: emulators/mednafen_snes.md - - MESS 2015: emulators/mess2015.md - - NicaiEmu: emulators/nicaiemu.md - - PCSX2 Legacy: emulators/pcsx2-legacy.md - - PiSNES: emulators/pisnes.md - - PPSSPP 1.5.4: emulators/ppsspp-1.5.4.md - - PUAE 2021: emulators/puae2021.md - - Snes9x 2002: emulators/snes9x2002.md - - Snes9x 2005: emulators/snes9x2005.md - - Snes9x 2005 Plus: emulators/snes9x2005_plus.md - - Snes9x 2010: emulators/snes9x2010.md - - Stella 2014: emulators/stella2014.md - - Stella 2023: emulators/stella2023.md - - Stone Soup: emulators/stonesoup.md - - Supermodel Legacy: emulators/supermodel-legacy.md - - ti99sim-sdl1: emulators/ti99sim-sdl1.md - - UME 2015: emulators/ume2015.md - - VBA-Next: emulators/vba_next.md - - Embedded HLE (19): - - 3dSen: emulators/3dsen.md - - Altirra: emulators/altirra.md - - Cxbx-Reloaded: emulators/cxbx-reloaded.md - - DaedalusX64: emulators/daedalusx64.md - - DroidArcadia: emulators/droidarcadia.md - - izapple2: emulators/izapple2.md - - Lindbergh Loader: emulators/lindbergh-loader.md - - Linkboy: emulators/linkboy.md - - Linux Loader: emulators/linuxloader.md - - MFME: emulators/mfme.md - - My Boy!: emulators/my-boy.md - - My OldBoy!: emulators/my-oldboy.md - - no$gba: emulators/nosgba.md - - PCSX-ReARMed: emulators/pcsx_rearmed.md - - PCSX-ReARMed (standalone): emulators/pcsx_rearmed_standalone.md - - Pizza Boy GBA: emulators/pizza-boy-gba.md - - Pizza Boy GBC: emulators/pizza-boy-gbc.md - - sixtyforce: emulators/sixtyforce.md - - touchHLE: emulators/touchhle.md - - Launchers (5): - - ArcadeFlashWeb: emulators/arcadeflashweb.md - - Dolphin Launcher: emulators/dolphin_launcher.md - - Parallel Launcher: emulators/parallel-launcher.md - - Pygame: emulators/pygame.md - - SWF Player: emulators/swf-player.md - - Other (73): - - ares: emulators/ares.md - - Basilisk II: emulators/basiliskii.md - - beebem: emulators/beebem.md - - Beetle GBA (Mednafen): emulators/beetle_gba.md - - BigPEmu: emulators/bigpemu.md - - BizHawk: emulators/bizhawk.md - - CaPriCe Forever: emulators/capriceforever.md - - Clock Signal (CLK): emulators/clk.md - - ColEm: emulators/colem.md - - CoolCV: emulators/coolcv.md - - CPCemu: emulators/cpcemu.md - - CSpect: emulators/cspect.md - - Demul: emulators/demul.md - - DGen/SDL: emulators/dgen.md - - DOSBox (SDL2 fork): emulators/dosbox-sdl2.md - - DraStic: emulators/drastic.md - - DREAMM: emulators/dreamm.md - - D.Smile: emulators/dsmile.md - - DSperate: emulators/dsperate.md - - ep128emu-core: emulators/ep128emu.md - - ePSXe: emulators/epsxe.md - - FBZX: emulators/fbzx.md - - FPse: emulators/fpse.md - - FPseNG: emulators/fpse-ng.md - - FS-UAE 3.1: emulators/fs-uae.md - - Gopher2600: emulators/gopher2600.md - - gopher64: emulators/gopher64.md - - iNES: emulators/ines.md - - IrataJaguar: emulators/iratajaguar.md - - jgenesis: emulators/jgenesis.md - - Jynx: emulators/jynx.md - - jzIntv: emulators/jzintv.md - - Kega Fusion: emulators/kega-fusion.md - - KEGS: emulators/kegs.md - - Lexaloffle: emulators/lexaloffle.md - - MagicEngine: emulators/magicengine.md - - MasterGear: emulators/mastergear.md - - Mednafen: emulators/mednafen.md - - Model 2 Emulator: emulators/model2.md - - NanoBoyAdvance: emulators/nanoboyadvance.md - - openMSX: emulators/openmsx.md - - Oricutron: emulators/oricutron.md - - Phoenix: emulators/phoenix.md - - Pizza Boy SC: emulators/pizza-boy-sc.md - - Plastic: emulators/plastic.md - - POM2: emulators/pom2.md - - project64: emulators/project64.md - - puNES: emulators/punes.md - - RAINE: emulators/raine.md - - Real3DOPlayer: emulators/real3doplayer.md - - Redream: emulators/redream.md - - rpix86: emulators/rpix86.md - - Ryujinx: emulators/ryujinx.md - - shadps4: emulators/shadps4.md - - SheepShaver: emulators/sheepshaver.md - - Skyline: emulators/skyline.md - - Speccy: emulators/speccy.md - - SSF: emulators/ssf.md - - Starboard: emulators/starboard.md - - SUPER ZSNES: emulators/superzsnes.md - - TeknoParrot: emulators/teknoparrot.md - - ti99sim: emulators/ti99sim.md - - tsugaru: emulators/tsugaru.md - - veesem: emulators/veesem.md - - VICE: emulators/vice.md - - Virtual Virtual Boy: emulators/virtual-virtual-boy.md - - Vita3K: emulators/vita3k.md - - WinUAE: emulators/winuae.md - - x16emu: emulators/x16emu.md - - XM7: emulators/xm7.md - - XRoar: emulators/xroar.md - - ZEsarUX: emulators/zesarux.md - - ZiNc: emulators/zinc.md -- Cross-reference: cross-reference.md -- Gap analysis: gaps.md -- Dump provenance: provenance.md -- Updated platform files: exports.md -- Data & API: data.md -- Wiki: - - Overview: wiki/index.md - - First install: wiki/first-install.md - - Getting started: wiki/getting-started.md - - Installer: wiki/installer.md - - FAQ: wiki/faq.md - - Troubleshooting: wiki/troubleshooting.md - - Architecture: wiki/architecture.md - - Tools: wiki/tools.md - - Advanced usage: wiki/advanced-usage.md - - Verification modes: wiki/verification-modes.md +- Start here: + - index.md + - What BIOS files are: wiki/what-are-bios-files.md + - Get your files: which-pack.md + - First install, step by step: wiki/first-install.md + - When a game does not start: wiki/troubleshooting.md + - Common questions: wiki/faq.md + - Glossary: wiki/glossary.md +- Your setup: + - platforms/index.md + - Computers and Steam Deck: + - BizHawk: platforms/bizhawk.md + - EmuDeck: platforms/emudeck.md + - RetroArch: platforms/retroarch.md + - RetroBat: platforms/retrobat.md + - RetroDECK: platforms/retrodeck.md + - Game systems for PCs and small boards: + - Batocera: platforms/batocera.md + - Lakka: platforms/lakka.md + - Recalbox: platforms/recalbox.md + - RetroPie: platforms/retropie.md + - Handhelds and TV boxes: + - AmberELEC: platforms/amberelec.md + - EmuELEC: platforms/emuelec.md + - KNULLI: platforms/knulli.md + - MinUI: platforms/minui.md + - MyMinUI: platforms/myminui.md + - NextUI: platforms/nextui.md + - ROCKNIX: platforms/rocknix.md + - MiSTer FPGA: + - MiSTer FPGA: platforms/misterfpga.md + - Self-hosted game library: + - RomM: platforms/romm.md +- Explore: + - Systems: + - systems/index.md + - Consoles and computers: + - 3DO Company: systems/3do-company.md + - Acorn: systems/acorn.md + - Amstrad: systems/amstrad.md + - APF: systems/apf.md + - Apple: systems/apple.md + - Aquaplus: systems/aquaplus.md + - Arcade: systems/arcade.md + - Atari: systems/atari.md + - Bally: systems/bally.md + - Bandai: systems/bandai.md + - BBK: systems/bbk.md + - Bit Corporation: systems/bit-corporation.md + - Bungie: systems/bungie.md + - Camputers: systems/camputers.md + - Casio: systems/casio.md + - Coleco: systems/coleco.md + - Commodore: systems/commodore.md + - DOS: systems/dos.md + - Dragon: systems/dragon.md + - EACA: systems/eaca.md + - Elektronika: systems/elektronika.md + - Enterprise: systems/enterprise.md + - Entex: systems/entex.md + - Epoch: systems/epoch.md + - Fairchild: systems/fairchild.md + - Fujitsu: systems/fujitsu.md + - Funtech: systems/funtech.md + - Galaksija: systems/galaksija.md + - GamePark: systems/gamepark.md + - GCE: systems/gce.md + - Gemei: systems/gemei.md + - Grundy: systems/grundy.md + - Hartung: systems/hartung.md + - IBM: systems/ibm.md + - Infocom: systems/infocom.md + - Interact: systems/interact.md + - Java: systems/java.md + - Lexaloffle: systems/lexaloffle.md + - Magnavox: systems/magnavox.md + - Mattel: systems/mattel.md + - Microsoft: systems/microsoft.md + - NEC: systems/nec.md + - Nintendo: systems/nintendo.md + - Nokia: systems/nokia.md + - Oric: systems/oric.md + - Other: systems/other.md + - Palm: systems/palm.md + - PC: systems/pc.md + - Philips: systems/philips.md + - Pioneer: systems/pioneer.md + - QEMU: systems/qemu.md + - RCA: systems/rca.md + - Real3D: systems/real3d.md + - Roland: systems/roland.md + - SDL: systems/sdl.md + - Sega: systems/sega.md + - Sharp: systems/sharp.md + - Sinclair: systems/sinclair.md + - SNK: systems/snk.md + - Sony: systems/sony.md + - Sord: systems/sord.md + - Synertek: systems/synertek.md + - Tandy: systems/tandy.md + - Tangerine: systems/tangerine.md + - Texas Instruments: systems/texas-instruments.md + - Thomson: systems/thomson.md + - Tiger: systems/tiger.md + - TimeTop: systems/timetop.md + - Tomy: systems/tomy.md + - VideoBrain: systems/videobrain.md + - Videoton: systems/videoton.md + - Vircon: systems/vircon.md + - VTech: systems/vtech.md + - Wenquxing: systems/wenquxing.md + - ZC: systems/zc.md + - Game engines and PC games: + - 3D Realms: systems/3d-realms.md + - Apogee: systems/apogee.md + - Blizzard: systems/blizzard.md + - Dinothawr: systems/dinothawr.md + - Electronic Arts: systems/electronic-arts.md + - Epic MegaGames: systems/epic-megagames.md + - Game Engines: systems/game-engines.md + - Id Software: systems/id-software.md + - Interplay: systems/interplay.md + - Monolith: systems/monolith.md + - RPG Maker: systems/rpg-maker.md + - ScummVM: systems/scummvm.md + - sdlpal: systems/sdlpal.md + - Softdisk: systems/softdisk.md + - Toys for Bob: systems/toys-for-bob.md + - Valve: systems/valve.md + - Emulators: + - emulators/index.md + - 0 to 9: + - 2 Ship 2 Harkinian: emulators/2ship.md + - 2600.emu: emulators/2600-emu.md + - 3dSen: emulators/3dsen.md + - A to C: + - a5200: emulators/a5200.md + - A7800: emulators/a7800.md + - Abuse: emulators/abuse.md + - ACE-DL: emulators/ace-dl.md + - AdvanceMAME: emulators/advancemame.md + - AdvanceMAME 0.94: emulators/advmame-0.94.md + - AdvanceMAME 1.4: emulators/advmame-1.4.md + - Adventure Game Studio: emulators/ags.md + - AetherSX2: emulators/aethersx2.md + - Aleph One: emulators/alephone.md + - Altirra: emulators/altirra.md + - amiarcadia: emulators/amiarcadia.md + - Amiberry: emulators/amiberry.md + - AppleWin: emulators/applewin.md + - aPS3e: emulators/aps3e.md + - ArcadeDuck: emulators/arcadeduck.md + - ArcadeFlashWeb: emulators/arcadeflashweb.md + - Ardens: emulators/ardens.md + - Arduous: emulators/arduous.md + - ares: emulators/ares.md + - ARMSX1: emulators/armsx1.md + - ARMSX2: emulators/armsx2.md + - Atari800: emulators/atari800.md + - aX360e: emulators/ax360e.md + - Azahar: emulators/azahar.md + - AzaharPlus: emulators/azaharplus.md + - b2: emulators/b2.md + - Basilisk II: emulators/basiliskii.md + - BBKEmu: emulators/bbkemu.md + - beebem: emulators/beebem.md + - Beetle GBA (Mednafen): emulators/beetle_gba.md + - Beetle Lynx (Mednafen Lynx): emulators/beetle_lynx.md + - Beetle NGP (Mednafen Neo Geo Pocket): emulators/beetle_ngp.md + - Beetle PC-FX (Mednafen): emulators/beetle_pcfx.md + - Beetle PCE (Mednafen PCE): emulators/beetle_pce.md + - Beetle PSX (Mednafen PSX): emulators/beetle_psx.md + - Beetle VB (Mednafen Virtual Boy): emulators/beetle_vb.md + - Beetle WonderSwan (Mednafen WonderSwan): emulators/beetle_wswan.md + - beetle_saturn: emulators/beetle_saturn.md + - BennuGD: emulators/bennugd.md + - Bermuda Syndrome: emulators/bsyndrome.md + - BigPEmu: emulators/bigpemu.md + - BizHawk: emulators/bizhawk.md + - bk-emulator: emulators/bk.md + - BlastEm: emulators/blastem.md + - blueMSX: emulators/bluemsx.md + - bnes: emulators/bnes.md + - boom3: emulators/boom3.md + - Boytacean: emulators/boytacean.md + - bsnes: emulators/bsnes.md + - bsnes 2014: emulators/bsnes2014.md + - bsnes C++98: emulators/bsnes_cplusplus98.md + - bsnes-hd beta: emulators/bsnes_hd_beta.md + - bsnes-jg: emulators/bsnes-jg.md + - bsnes-mercury: emulators/bsnes_mercury.md + - BStone: emulators/bstone.md + - C-Dogs SDL: emulators/cdogs.md + - C64.emu: emulators/c64-emu.md + - Cannonball: emulators/cannonball.md + - CaPriCe Forever: emulators/capriceforever.md + - Caprice32: emulators/cap32.md + - CapriceRPI: emulators/capricerpi.md + - CatacombGL: emulators/catacombgl.md + - CDi 2015: emulators/cdi2015.md + - Cemu: emulators/cemu.md + - Chihiro-DS: emulators/chihiro-ds.md + - ChimeraSNES: emulators/chimerasnes.md + - Citra: emulators/citra.md + - Citra 2018: emulators/citra2018.md + - Citra Canary: emulators/citra_canary.md + - Citra MMJ: emulators/citra-mmj.md + - Citron: emulators/citron.md + - Clock Signal (CLK): emulators/clk.md + - ClownMDEmu: emulators/clownmdemu.md + - ColEm: emulators/colem.md + - Commander Genius: emulators/cgenius.md + - CoolbarEmu: emulators/coolbaremu.md + - CoolCV: emulators/coolcv.md + - corsixth: emulators/corsixth.md + - CPCemu: emulators/cpcemu.md + - CrocoDS: emulators/crocods.md + - Cruzes: emulators/cruzes.md + - CSpect: emulators/cspect.md + - Cxbx-Reloaded: emulators/cxbx-reloaded.md + - Cxbx-Reloaded (31ff15b): emulators/cxbx-reloaded-2023.md + - D to F: + - D.Smile: emulators/dsmile.md + - DaedalusX64: emulators/daedalusx64.md + - Daphne: emulators/daphne.md + - DarkPlaces: emulators/darkplaces.md + - Demul: emulators/demul.md + - DeSmuME: emulators/desmume.md + - DeSmuME 2015: emulators/desmume2015.md + - devilutionx: emulators/devilutionx.md + - DGen/SDL: emulators/dgen.md + - dhewm3: emulators/dhewm3.md + - DICE: emulators/dice.md + - DingooEmu: emulators/dingooemu.md + - Dinothawr: emulators/dinothawr.md + - DirectXBox: emulators/directxbox.md + - DirkSimple: emulators/dirksimple.md + - Dolphin: emulators/dolphin.md + - Dolphin Launcher: emulators/dolphin_launcher.md + - Dolphin MMJR: emulators/dolphin-mmjr.md + - Dolphin MMJR2: emulators/dolphin-mmjr2.md + - DOSBox: emulators/dosbox.md + - DOSBox (SDL2 fork): emulators/dosbox-sdl2.md + - DOSBox Pure: emulators/dosbox_pure.md + - DOSBox Pure Unleashed: emulators/dosbox-pure-unleashed.md + - DOSBox Staging: emulators/dosbox-staging.md + - DOSBox-core: emulators/dosbox_core.md + - DOSBox-SVN: emulators/dosbox_svn.md + - DOSBox-SVN CE: emulators/dosbox_svn_ce.md + - DOSBox-X: emulators/dosbox-x.md + - DoubleCherryGB: emulators/doublecherrygb.md + - doukutsu-rs: emulators/doukutsu_rs.md + - DraStic: emulators/drastic.md + - DREAMM: emulators/dreamm.md + - DroidArcadia: emulators/droidarcadia.md + - dsda-doom: emulators/dsda-doom.md + - DSperate: emulators/dsperate.md + - DuckStation: emulators/duckstation.md + - Dusklight: emulators/dusklight.md + - dxx-rebirth: emulators/dxx-rebirth.md + - EasyRPG Player: emulators/easyrpg.md + - ECWolf: emulators/ecwolf.md + - Eden: emulators/eden.md + - EDuke32: emulators/eduke32.md + - EightyOne: emulators/81.md + - EightyOne (Recalbox): emulators/eightyone.md + - eka2l1: emulators/eka2l1.md + - EmuCoreV: emulators/emucorev.md + - EmuCoreX: emulators/emucorex.md + - EmuCV: emulators/emucv.md + - EmuSCV: emulators/emuscv.md + - emux (CHIP-8): emulators/emux_chip8.md + - emux (Game Boy): emulators/emux_gb.md + - emux (NES): emulators/emux_nes.md + - emux (SMS): emulators/emux_sms.md + - ep128emu-core: emulators/ep128emu.md + - ep128emu_core: emulators/ep128emu_core.md + - ePSXe: emulators/epsxe.md + - ET Legacy: emulators/etlegacy.md + - Eternal Computer: emulators/eternal.md + - Exel-Hence: emulators/exel-hence.md + - FAKE-08: emulators/fake08.md + - fallout1-ce: emulators/fallout1-ce.md + - fallout2-ce: emulators/fallout2-ce.md + - FB Alpha 2012: emulators/fbalpha2012.md + - FB Alpha 2012 CPS-1: emulators/fbalpha2012_cps1.md + - FB Alpha 2012 CPS-2: emulators/fbalpha2012_cps2.md + - FB Alpha 2012 CPS-3: emulators/fbalpha2012_cps3.md + - FB Alpha 2012 Neo Geo: emulators/fbalpha2012_neogeo.md + - FB Alpha 2019: emulators/fbalpha2019.md + - FBZX: emulators/fbzx.md + - FCEUmm: emulators/fceumm.md + - FCEUmm Mod: emulators/fceumm_mod.md + - FinalBurn Neo: emulators/fbneo.md + - FinalBurn Neo (CPS-1/CPS-2): emulators/fbneo_cps12.md + - FinalBurn Neo (Neo Geo): emulators/fbneo_neogeo.md + - fixGB: emulators/fixgb.md + - fixNES: emulators/fixnes.md + - Flycast: emulators/flycast.md + - Flycast 2021: emulators/flycast2021.md + - Flycast Dojo: emulators/flycast-dojo.md + - fMSX: emulators/fmsx.md + - FPse: emulators/fpse.md + - FPseNG: emulators/fpse-ng.md + - FreeChaF: emulators/freechaf.md + - FreeIntv: emulators/freeintv.md + - FreeJ2ME: emulators/freej2me.md + - FreeJ2ME-Plus: emulators/freej2me_plus.md + - Frodo: emulators/frodo.md + - Frotz: emulators/frotz.md + - FS-UAE: emulators/fsuae.md + - FS-UAE 3.1: emulators/fs-uae.md + - Fuse: emulators/fuse.md + - future-pinball: emulators/future-pinball.md + - future-pinball-fploader: emulators/future-pinball-fploader.md + - G to L: + - galaksija: emulators/galaksija.md + - GAM4980: emulators/gam4980.md + - Gambatte: emulators/gambatte.md + - Game & Watch: emulators/gw.md + - GameTank: emulators/gametank.md + - Gargoyle: emulators/gargoyle.md + - GBA.emu: emulators/gba-emu.md + - GBC.emu: emulators/gbc-emu.md + - Gearboy: emulators/gearboy.md + - Gearcoleco: emulators/gearcoleco.md + - Geargrafx: emulators/geargrafx.md + - Gearlynx: emulators/gearlynx.md + - Gearsystem: emulators/gearsystem.md + - Geartowns: emulators/geartowns.md + - GemRB: emulators/gemrb.md + - Genesis Plus GX: emulators/genesis_plus_gx.md + - Genesis Plus GX Expanded: emulators/genesis_plus_gx_expanded.md + - Genesis Plus GX Wide: emulators/genesis_plus_gx_wide.md + - Geolith: emulators/geolith.md + - Ghostship: emulators/ghostship.md + - GnGeo-Pi: emulators/gngeopi.md + - Gopher2600: emulators/gopher2600.md + - gopher64: emulators/gopher64.md + - gpSP: emulators/gpsp.md + - GroovyMAME: emulators/groovymame.md + - GSplus: emulators/gsplus.md + - GSplus (legacy): emulators/gsplus-legacy.md + - GZDoom: emulators/gzdoom.md + - hakuX: emulators/hakux.md + - Handy: emulators/handy.md + - Hatari: emulators/hatari.md + - Hatari 2014: emulators/hatari2014.md + - hatariB: emulators/hatarib.md + - HBMAME (Homebrew MAME): emulators/hbmame.md + - higan (SFC Accuracy): emulators/higan_sfc.md + - Holani: emulators/holani.md + - Hurrican: emulators/hurrican.md + - Hydra Castle Labyrinth: emulators/hcl.md + - Hypseus Singe: emulators/hypseus-singe.md + - idTech4A++: emulators/idtech4a++.md + - Ikemen GO: emulators/ikemen.md + - iNES: emulators/ines.md + - Infinity: emulators/infinity.md + - Ion Fury: emulators/fury.md + - ioquake3: emulators/ioquake3.md + - ioRTCW: emulators/iortcw.md + - IrataJaguar: emulators/iratajaguar.md + - IroGB: emulators/irogb.md + - Ishiiruka: emulators/ishiiruka.md + - izapple2: emulators/izapple2.md + - J2ME Loader: emulators/j2me-loader.md + - JAXE: emulators/jaxe.md + - Jazz² Resurrection: emulators/jazz2-native.md + - jgenesis: emulators/jgenesis.md + - JL-Mod: emulators/jl-mod.md + - JollyCV: emulators/jollycv.md + - Julius: emulators/julius.md + - Jynx: emulators/jynx.md + - jzIntv: emulators/jzintv.md + - Kega Fusion: emulators/kega-fusion.md + - KEGS: emulators/kegs.md + - KEmulator nnmod: emulators/kemulator.md + - Kenji-NX: emulators/kenji-nx.md + - Kronos: emulators/kronos.md + - Lexaloffle: emulators/lexaloffle.md + - lightspark: emulators/lightspark.md + - Lime3DS: emulators/lime3ds.md + - LinApple: emulators/linapple.md + - Lindbergh Loader: emulators/lindbergh-loader.md + - Linkboy: emulators/linkboy.md + - Linux Loader: emulators/linuxloader.md + - LowRes NX: emulators/lowresnx.md + - LRPS2: emulators/lrps2.md + - Lynx.emu: emulators/lynx-emu.md + - LZDoom: emulators/lzdoom.md + - M to O: + - M2000: emulators/m2000.md + - M64Plus FZ: emulators/m64plus-fz.md + - MagicEngine: emulators/magicengine.md + - MAME: emulators/mame.md + - MAME 0.258: emulators/mame0258.md + - MAME 0.278: emulators/mame0278.md + - MAME 2000: emulators/mame2000.md + - MAME 2003: emulators/mame2003.md + - MAME 2003 Midway: emulators/mame2003_midway.md + - MAME 2003-Plus: emulators/mame2003_plus.md + - MAME 2009: emulators/mame2009.md + - MAME 2010: emulators/mame2010.md + - MAME 2014: emulators/mame2014.md + - MAME 2015: emulators/mame2015.md + - MAME 2016: emulators/mame2016.md + - MAME Arcade: emulators/mamearcade.md + - MAME MESS: emulators/mamemess.md + - MAME4ALL-Pi: emulators/mame4all.md + - MAME4droid (0.139u1): emulators/mame4droid.md + - MAME4droid 2024: emulators/mame4droid-2024.md + - Mandarine: emulators/mandarine.md + - Mariani: emulators/mariani.md + - MasterGear: emulators/mastergear.md + - MD.emu: emulators/md-emu.md + - Mednafen: emulators/mednafen.md + - Mednafen STV: emulators/mednafen_stv.md + - mednafen_gba: emulators/mednafen_gba.md + - mednafen_pce: emulators/mednafen_pce.md + - mednafen_pce_fast: emulators/mednafen_pce_fast.md + - mednafen_snes: emulators/mednafen_snes.md + - mednafen_supafaust: emulators/mednafen_supafaust.md + - mednafen_supergrafx: emulators/mednafen_supergrafx.md + - melonDS: emulators/melonds.md + - melonDS DS: emulators/melonds_ds.md + - melonDS Nightly: emulators/melonds-nightly.md + - Mesen: emulators/mesen.md + - mesen-s: emulators/mesen-s.md + - MesenCE: emulators/mesence.md + - MESS 2015: emulators/mess2015.md + - Meteor GBA: emulators/meteor.md + - MFME: emulators/mfme.md + - mGBA: emulators/mgba.md + - mGBA Splitscreen: emulators/mgba_splitscreen.md + - Mini vMac: emulators/minivmac.md + - Mini vMac 37: emulators/minivmac37.md + - mkxp-z: emulators/mkxp_z.md + - Model 2 Emulator: emulators/model2.md + - MojoZork: emulators/mojozork.md + - MSX.emu: emulators/msx-emu.md + - Mu: emulators/mu.md + - Mupen64Plus AE: emulators/mupen64plus-ae.md + - Mupen64Plus-Next: emulators/mupen64plus_next.md + - My Boy!: emulators/my-boy.md + - My OldBoy!: emulators/my-oldboy.md + - NanoBoyAdvance: emulators/nanoboyadvance.md + - Native32Emu: emulators/native32emu.md + - nekop2: emulators/nekop2.md + - NEO.emu: emulators/neo-emu.md + - NeoCD: emulators/neocd.md + - nes: emulators/nes.md + - NES.emu: emulators/nes-emu.md + - Nesoid: emulators/nesoid.md + - Nestopia UE: emulators/nestopia.md + - NetherSX2: emulators/nethersx2.md + - NetherSX2-Turnip: emulators/nethersx2-turnip.md + - NetherSX2-Turnip Classic: emulators/nethersx2-turnip-classic.md + - NGP.emu: emulators/ngp-emu.md + - NicaiEmu: emulators/nicaiemu.md + - no$gba: emulators/nosgba.md + - NooDS: emulators/noods.md + - NP2kai: emulators/np2kai.md + - np2pi: emulators/np2pi.md + - nSide (SFC Balanced): emulators/higan_sfc_balanced.md + - Nuance: emulators/nuance.md + - Numero: emulators/numero.md + - NXEngine: emulators/nxengine.md + - O2EM: emulators/o2em.md + - Oberon: emulators/oberon.md + - Omicron: emulators/omicron.md + - ONScripter: emulators/onscripter.md + - ONScripter Yuri: emulators/onsyuri.md + - openbor: emulators/openbor.md + - OpenGOAL: emulators/opengoal.md + - OpenJazz: emulators/openjazz.md + - openjk: emulators/openjk.md + - OpenJKDF2: emulators/openjkdf2.md + - OpenLara: emulators/openlara.md + - OpenMoHAA: emulators/openmohaa.md + - openMSX: emulators/openmsx.md + - OpenTTD: emulators/openttd.md + - OpenTyrian: emulators/opentyrian.md + - Opera (4DO): emulators/opera.md + - Oricutron: emulators/oricutron.md + - Osmose (RetroPie): emulators/osmose.md + - P to R: + - Panda3DS: emulators/panda3ds.md + - Parallel Launcher: emulators/parallel-launcher.md + - ParaLLEl N64: emulators/parallel_n64.md + - pce-emu: emulators/pce-emu.md + - PCEE2: emulators/pcee2.md + - PCem: emulators/pcem.md + - PCSX-ReARMed: emulators/pcsx_rearmed.md + - PCSX-ReARMed (standalone): emulators/pcsx_rearmed_standalone.md + - PCSX1: emulators/pcsx1.md + - PCSX2: emulators/pcsx2.md + - PCSX2 Legacy: emulators/pcsx2-legacy.md + - PCSX2x6: emulators/pcsx2x6.md + - PD777: emulators/pd777.md + - Perfect Dark (PC port): emulators/pdark.md + - Phoenix: emulators/phoenix.md + - Picocomputer 6502: emulators/rp6502.md + - PicoDrive: emulators/picodrive.md + - piece-emu: emulators/piece.md + - piFBA (fba2x): emulators/fba2x.md + - PiSNES: emulators/pisnes.md + - Pizza Boy GBA: emulators/pizza-boy-gba.md + - Pizza Boy GBC: emulators/pizza-boy-gbc.md + - Pizza Boy SC: emulators/pizza-boy-sc.md + - Plastic: emulators/plastic.md + - play: emulators/play.md + - PlaydiaEmu: emulators/playdiaemu.md + - PokeMini: emulators/pokemini.md + - pokketstation: emulators/pokketstation.md + - POM2: emulators/pom2.md + - Potator: emulators/potator.md + - PPSSPP: emulators/ppsspp.md + - PPSSPP 1.5.4: emulators/ppsspp-1.5.4.md + - PrBoom: emulators/prboom.md + - PrBoom+: emulators/prboom-plus.md + - PrimeHack: emulators/primehack.md + - project64: emulators/project64.md + - ProSystem: emulators/prosystem.md + - pSxMAME: emulators/psxmame.md + - PUAE (P-UAE): emulators/puae.md + - PUAE 2021: emulators/puae2021.md + - puNES: emulators/punes.md + - px68k: emulators/px68k.md + - px68k (SDL): emulators/px68k-sdl.md + - QEMU: emulators/qemu.md + - Quake 3 (Raspberry Pi): emulators/quake3.md + - QUASI88: emulators/quasi88.md + - QUASI88kai: emulators/quasi88kai.md + - QuickNES: emulators/quicknes.md + - RACE (Neo Geo Pocket): emulators/race.md + - radio: emulators/radio.md + - RAINE: emulators/raine.md + - Raze: emulators/raze.md + - RB5000: emulators/rb5000.md + - Real3DOPlayer: emulators/real3doplayer.md + - RealRTCW: emulators/realrtcw.md + - Redream: emulators/redream.md + - Reicast: emulators/reicast.md + - REminiscence: emulators/reminiscence.md + - Retro8: emulators/retro8.md + - RetroDream: emulators/retrodream.md + - RigelEngine: emulators/rigelengine.md + - Rosalie's Mupen GUI: emulators/rosalies-mupen-gui.md + - RPCS3: emulators/rpcs3.md + - rpix86: emulators/rpix86.md + - ruffle: emulators/ruffle.md + - Rust-DOS: emulators/rust_dos.md + - Rustation: emulators/rustation.md + - RustyNES: emulators/rustynes.md + - RVVM: emulators/rvvm.md + - Ryujinx: emulators/ryujinx.md + - S to U: + - SAME CDi: emulators/same_cdi.md + - SameBoy: emulators/sameboy.md + - sameduck: emulators/sameduck.md + - Saturn.emu: emulators/saturn-emu.md + - ScummVM: emulators/scummvm.md + - ScummVM (SDL1): emulators/scummvm-sdl1.md + - SDL2TRS: emulators/sdl2trs.md + - SDLPAL: emulators/sdlpal.md + - sdlpop: emulators/sdlpop.md + - shadps4: emulators/shadps4.md + - SheepShaver: emulators/sheepshaver.md + - Ship of Harkinian: emulators/soh.md + - SimCoupe: emulators/simcp.md + - SimCoupe (standalone): emulators/samcoupe.md + - simple64: emulators/simple64.md + - Singe 2: emulators/singe2.md + - sixtyforce: emulators/sixtyforce.md + - skyemu: emulators/skyemu.md + - Skyline: emulators/skyline.md + - SMS Plus GX: emulators/smsplus.md + - snes9x: emulators/snes9x.md + - Snes9x 2002: emulators/snes9x2002.md + - Snes9x 2005: emulators/snes9x2005.md + - Snes9x 2005 Plus: emulators/snes9x2005_plus.md + - Snes9x 2010: emulators/snes9x2010.md + - Snes9x EX+: emulators/snes9x-explus.md + - solarus: emulators/solarus.md + - Solarus 2.0.0: emulators/solarus-2.0.0.md + - Sonic 1 & 2 (2013) Decompilation: emulators/sonic2013.md + - Sonic 3 A.I.R.: emulators/sonic3-air.md + - Sonic CD Decompilation: emulators/soniccd.md + - Sonic Mania Decompilation: emulators/sonic-mania.md + - Sonic Robo Blast 2: emulators/srb2.md + - Speccy: emulators/speccy.md + - SplitWolf: emulators/splitwolf.md + - SPMP8000Emu: emulators/spmp8000emu.md + - SquirrelJME: emulators/squirreljme.md + - SSF: emulators/ssf.md + - Starboard: emulators/starboard.md + - Starship: emulators/starship.md + - Stella: emulators/stella.md + - Stella 2014: emulators/stella2014.md + - Stella 2023: emulators/stella2023.md + - Sudachi: emulators/sudachi.md + - Sugarbox2: emulators/sugarbox2.md + - SUPER ZSNES: emulators/superzsnes.md + - SUPER3: emulators/super3.md + - Supermodel: emulators/supermodel.md + - Supermodel (Sinden fork): emulators/supermodel3.md + - Supermodel Dojo: emulators/supermodel-dojo.md + - Supermodel Legacy: emulators/supermodel-legacy.md + - SuperSnes9x: emulators/supersnes9x.md + - Suyu: emulators/suyu.md + - Swan.emu: emulators/swan-emu.md + - swanstation: emulators/swanstation.md + - SWF Player: emulators/swf-player.md + - Syobon Action: emulators/syobonaction.md + - TamaLIBretro: emulators/tamalibretro.md + - Taradino: emulators/taradino.md + - TeknoParrot: emulators/teknoparrot.md + - TempGBA: emulators/tempgba.md + - TGB Dual: emulators/tgbdual.md + - The Force Engine: emulators/theforceengine.md + - Theodore: emulators/theodore.md + - TheXTech: emulators/thextech.md + - ti99sim: emulators/ti99sim.md + - ti99sim-sdl1: emulators/ti99sim-sdl1.md + - Tia: emulators/tia.md + - TIC-80: emulators/tic80.md + - touchHLE: emulators/touchhle.md + - TR1X: emulators/tr1x.md + - TR2X: emulators/tr2x.md + - trident: emulators/trident.md + - Triforce: emulators/triforce.md + - tsugaru: emulators/tsugaru.md + - TyrQuake: emulators/tyrquake.md + - UAE4ALL (libretro, Chips-fr): emulators/uae4all-rpi.md + - UAE4All2: emulators/uae4all.md + - uae4arm: emulators/uae4arm.md + - UME 2015: emulators/ume2015.md + - uqm: emulators/uqm.md + - UZDoom: emulators/uzdoom.md + - uzem: emulators/uzem.md + - V to Z: + - VaporSpec: emulators/vaporspec.md + - VBA-M: emulators/vbam.md + - VBA-Next: emulators/vba_next.md + - vecx: emulators/vecx.md + - veesem: emulators/veesem.md + - VeMUlator: emulators/vemulator.md + - VICE: emulators/vice.md + - VICE x128: emulators/vice_x128.md + - VICE x64: emulators/vice_x64.md + - VICE x64dtv: emulators/vice_x64dtv.md + - VICE x64sc: emulators/vice_x64sc.md + - VICE xcbm2: emulators/vice_xcbm2.md + - VICE xcbm5x0: emulators/vice_xcbm5x0.md + - VICE xpet: emulators/vice_xpet.md + - VICE xplus4: emulators/vice_xplus4.md + - VICE xscpu64: emulators/vice_xscpu64.md + - VICE xvic: emulators/vice_xvic.md + - Vircon32: emulators/vircon32.md + - Virtual Jaguar: emulators/virtualjaguar.md + - Virtual Virtual Boy: emulators/virtual-virtual-boy.md + - VirtualXT: emulators/virtualxt.md + - visual-pinball: emulators/visual-pinball.md + - Vita3K: emulators/vita3k.md + - vitaQuakeII: emulators/vitaquake2.md + - vitaQuakeIII: emulators/vitaquake3.md + - vitavoyager: emulators/vitavoyager.md + - vkQuake: emulators/vkquake.md + - vkQuake2: emulators/vkquake2.md + - vkQuake3: emulators/vkquake3.md + - VVVVVV: emulators/vvvvvv.md + - WinUAE: emulators/winuae.md + - Wolf4SDL: emulators/wolf4sdl.md + - WQXEmu: emulators/wqxemu.md + - X Millennium: emulators/x1.md + - X1 BOX: emulators/x1-box.md + - x16emu: emulators/x16emu.md + - x64sdl: emulators/x64sdl.md + - Xash3D FWGS: emulators/xash3d_fwgs.md + - xemu: emulators/xemu.md + - xemu Chihiro: emulators/xemu-chihiro.md + - XenDroid: emulators/xendroid.md + - Xenia Canary: emulators/xenia.md + - Xenia Edge: emulators/xenia-edge.md + - XeniOS: emulators/xenios.md + - xm6pro68k: emulators/xm6pro68k.md + - xm6typeg: emulators/xm6typeg.md + - XM7: emulators/xm7.md + - XRick: emulators/xrick.md + - XRoar: emulators/xroar.md + - yabasanshiro: emulators/yabasanshiro.md + - yabause: emulators/yabause.md + - Yamagi Quake II: emulators/yquake2.md + - yaps2: emulators/yaps2.md + - ymir: emulators/ymir.md + - Yuzu: emulators/yuzu.md + - Zelda Classic 2.55 Prototype: emulators/zc255.md + - Zelda Classic v2.10: emulators/zc210.md + - ZEsarUX: emulators/zesarux.md + - ZiNc: emulators/zinc.md + - ZQuest Classic: emulators/zquestclassic.md + - All installer options: wiki/installer.md +- Reference: + - wiki/index.md + - How each platform checks files: wiki/verification-modes.md + - What the packs do not hold: gaps.md + - Cross-reference: cross-reference.md + - Dump provenance: provenance.md + - Updated platform files: exports.md + - Data and API: data.md + - Find a file by its SHA1: files/index.md - Data model: wiki/data-model.md - - Profiling guide: wiki/profiling.md + - Architecture: wiki/architecture.md + - Build your own packs: wiki/advanced-usage.md + - Command-line tools: wiki/tools.md + - Technical questions: wiki/technical-faq.md + - Projects using this data: wiki/community-tools.md +- Contribute: + - contributing.md + - Profiling an emulator: wiki/profiling.md - Adding a platform: wiki/adding-a-platform.md - Adding a scraper: wiki/adding-a-scraper.md - - Testing guide: wiki/testing-guide.md + - Testing: wiki/testing-guide.md - Release process: wiki/release-process.md - - Community tools: wiki/community-tools.md -- Contributing: contributing.md diff --git a/scripts/generate_readme.py b/scripts/generate_readme.py index 57a5cb4b..1a8ab504 100644 --- a/scripts/generate_readme.py +++ b/scripts/generate_readme.py @@ -659,7 +659,7 @@ def generate_readme(db: dict, platforms_dir: str) -> str: "Hashes document what emulator code loads and accepts, not dump" " provenance. That boundary, and how it relates to preservation" " catalogs such as No-Intro, is drawn in the" - f" [FAQ]({SITE_URL}wiki/faq/" + f" [technical questions]({SITE_URL}wiki/technical-faq/" "#are-these-files-verified-against-original-hardware-dumps).", "", "1. **Read emulator source code**: trace every file the code loads, its expected hash and size", diff --git a/scripts/generate_site.py b/scripts/generate_site.py index 55573f1e..2335f69a 100644 --- a/scripts/generate_site.py +++ b/scripts/generate_site.py @@ -36,9 +36,11 @@ sys.path.insert(0, os.path.dirname(__file__)) from common import ( upstream_profile_index, build_zip_contents_index, + GAME_DATA_TOPS, compute_composition, count_catalog_matched, list_registered_platforms, + load_platform_registry, load_database, load_emulator_profiles, load_platform_config, @@ -53,7 +55,7 @@ from common import ( ) from artifacts import ArtifactLockBusy, artifact_lock from cross_reference import FileConsumers, held_copy -from nativemode import reads_file_contents +from nativemode import digest_algorithm, reads_file_contents @@ -71,6 +73,17 @@ from generate_readme import ( ) from provenance_report import build_report, distinct_missing from siteexports import generate_exports_page, load_export_reports +from siteguides import ( + ALSO_RUNS, + DEVICE_GROUPS, + GROUP_ANCHORS, + GUIDES, + INSTALL_BASE, + install_command, + platform_args, + terminal_tabs, + trust_box, +) import release_record DOCS_DIR = "docs" @@ -366,14 +379,27 @@ def _timestamp() -> str: # Home page -def _extract_table(platforms_dir: str) -> list[str]: - """The home page's extraction table, indented for its admonition.""" - rows = [ - f" | {display} | `{folder}` |" - for display, folder in extract_targets(platforms_dir) +def _stat(value: str, label: str) -> list[str]: + return [ + '
', + f'{value}', + f'{label}', + "
", + "", ] - notes = [f" {note}" for note in extract_notes(platforms_dir)] - return [" | Platform | Extract to |", " |----------|-----------|", *rows, "", *notes] + + +def _group_members(coverages: dict, registry: dict, group: str) -> list[str]: + """Registered platforms of a device group, by display name.""" + return sorted( + (name for name in coverages if registry.get(name, {}).get("device_group") == group), + key=lambda name: coverages[name]["platform"].casefold(), + ) + + +def _next_block(links: list[str]) -> list[str]: + """Where to go from here, at the end of a page.""" + return ['!!! tip "Next"', "", *(f" - {link}" for link in links), ""] def generate_home( @@ -383,166 +409,156 @@ def generate_home( registry: dict | None = None, platforms_dir: str = "platforms", ) -> str: + """The page someone lands on knowing nothing: what a BIOS file is, + whether they need one, and the three steps to get the right ones. + Figures about the collection come last.""" + registry = registry or {} total_files = db.get("total_files", 0) total_size = db.get("total_size", 0) - ts = _timestamp() - - unique = unique_emulator_profiles(profiles) - emulator_count = len(unique) - - classifications: dict[str, int] = {} - for p in unique.values(): - cls = p.get("core_classification", "other") - if cls not in CLS_LABELS or cls == "unclassified": - cls = "other" - classifications[cls] = classifications.get(cls, 0) + 1 + emulator_count = len(unique_emulator_profiles(profiles)) + names = sorted((cov["platform"] for cov in coverages.values()), key=str.casefold) lines = [ - f"# {SITE_NAME}", + "# BIOS files for your emulators, in the right folder", "", - "BIOS and firmware metadata checked the way each platform checks it, " - "cross-referenced against revision-pinned emulator source code.", + "Many consoles start their games with the help of a small program built " + "into the machine, called a BIOS or firmware. An emulator needs a copy of " + "that file to start the games of those consoles. RetroBIOS gives you the " + "right files for the setup you already have, in the folder where your " + "emulators look for them.", + "", + "[Get your files](which-pack.md){ .md-button .md-button--primary } " + "[What is a BIOS file?](wiki/what-are-bios-files.md){ .md-button }", + "", + f"Free, with no account. It works with {', '.join(names[:-1])} and {names[-1]}.", + "", + "", + "", + "## Do I need this?", + "", + "- **Yes**, if a game or an emulator says a BIOS file is missing.", + "- **Usually yes** for PlayStation, PlayStation 2, Saturn, Dreamcast, Neo " + "Geo and many home computers. Details: [the files each system " + "needs](systems/index.md).", + "- **Usually not** for NES, Super Nintendo and Mega Drive cartridges: those " + "consoles start games without a BIOS. Their add-ons do need one, such as " + "the Famicom Disk System and the Mega-CD.", + "", + "## Three steps", + "", + '
', + "", + "- **1. Find your setup**", + "", + " Pick your device, or the system you play on. Its page shows the folder " + "where the files go.", + "", + " [Pick your setup](which-pack.md#pick-your-setup)", + "", + "- **2. Get the files**", + "", + " One command downloads only what is missing and checks each file. You " + "can also download a ZIP pack and extract it yourself.", + "", + " [First install, step by step](wiki/first-install.md)", + "", + "- **3. Start a game**", + "", + " Your emulator finds the files by itself. If a game still does not " + "start, the help page goes through the usual causes.", + "", + " [When a game does not start](wiki/troubleshooting.md)", + "", + "
", + "", + "## Pick your device", + "", + '
', + "", + ] + for group, (label, description) in DEVICE_GROUPS.items(): + members = _group_members(coverages, registry, group) + if not members: + continue + lines += [ + f"- **{label}**", + "", + f" {description}", + "", + " " + ", ".join( + f"[{coverages[name]['platform']}](platforms/{name}.md)" for name in members + ), + "", + f" [See where the files go](which-pack.md#{GROUP_ANCHORS[group]})", + "", + ] + lines += [ + "
", + "", + "## Is this safe?", + "", + "The installer writes only inside your BIOS folder, checks every file " + "before saving it, and never touches games or saves. " + "[How the command is checked](wiki/faq.md#is-piping-the-installer-into-a-shell-safe).", + "", + "Laws on copying console firmware differ by country. " + "[The project's position](wiki/faq.md#is-this-legal) says where it stands " + "and how to ask for a file to be removed.", + "", + "## Go further", + "", + "These pages go into detail, for when you want to know more than where the " + "files go.", + "", + '
', + "", + "- **Systems**", + "", + " What each console and computer needs, file by file.", + "", + " Details: [Browse the systems](systems/index.md)", + "", + "- **Emulators**", + "", + " What each emulator reads, with the source code that reads it.", + "", + " Details: [Browse the emulators](emulators/index.md)", + "", + "- **How files are checked**", + "", + " Each setup checks its files its own way: by name, by MD5 or by SHA1.", + "", + " Reference: [How each platform checks files](wiki/verification-modes.md)", + "", + "- **Data and API**", + "", + " Every file and hash as JSON, CSV and SQLite, for your own tools.", + "", + " Reference: [Data and API](data.md)", + "", + "
", "", '
', "", - '
', - f'{total_files:,}', - f'{COLLECTION_FILES_LABEL}', - "
", - "", - '
', - f'{len(coverages)}', - 'Platforms', - "
", - "", - '
', - f'{emulator_count}', - 'Emulators profiled', - "
", - "", - '
', - f'{_fmt_size(total_size)}', - 'Total size', - "
", - "", + *_stat(f"{total_files:,}", COLLECTION_FILES_LABEL), + *_stat(str(len(coverages)), "Setups supported"), + *_stat(str(emulator_count), "Emulators documented"), + *_stat(_fmt_size(total_size), "Size of the collection"), "
", "", composition_sentence(db), "", + "## Help the project", + "", + "Files, emulator profiles and corrections are welcome. " + "For contributors: [How to contribute](contributing.md).", + "", + *_next_block([ + "[Get your files](which-pack.md)", + "[What BIOS files are](wiki/what-are-bios-files.md)", + ]), + f'
Generated on {_timestamp()}.
', ] - - # Platforms FIRST (main action) - lines.extend( - [ - "## Platforms", - "", - "| Icon | Platform | Files in the pack | Checked by | Download |", - "|---|----------|------:|-----------|----------|", - ] - ) - - mode_icons = {"md5": "MD5", "sha1": "SHA1", "existence": "exists"} - - for name, cov in sorted(coverages.items(), key=lambda x: x[1]["platform"]): - display = cov["platform"] - logo_url = (registry or {}).get(name, {}).get("logo", "") - logo_md = ( - f"![{display}]({logo_url}){{ width=20 loading=lazy }}" if logo_url else "" - ) - mode_label = mode_icons.get(cov["mode"], cov["mode"]) - - lines.append( - f"| {logo_md} | [{display}](platforms/{name}.md) | " - f"{_released_files(name)} | {mode_label} | " - f"{_pack_buttons(name)} |" - ) - - lines.extend( - [ - "", - "Checked by is the test each platform runs on its own, replicated " - "from its source code. RetroArch only looks for the filename, while " - "Batocera compares MD5 checksums, the fingerprint of a file's " - "contents. " - "[How each mode works](wiki/verification-modes.md).", - ] - ) - - catalog_matched = count_catalog_matched(db) - if catalog_matched: - system_files = compute_composition(db)["systems"]["files"] - lines.extend( - [ - "", - f"**{catalog_matched:,}** of {system_files:,} system files are " - "byte-identical to a dump catalogued by No-Intro, Redump, or " - "TOSEC, and say so on their system page. " - "[What that means](provenance.md).", - ] - ) - - # Quick start, collapsed: secondary information - lines.extend( - [ - "", - '??? info "Where to extract"', - "", - *_extract_table(platforms_dir), - " Every other pack extracts straight into the BIOS folder." - " [Full instructions per setup](which-pack.md).", - "", - ] - ) - - lines.extend( - [ - "## Emulator profiles", - "", - "| Classification | Count |", - "|---------------|-------|", - ] - ) - for cls, count in sorted(classifications.items(), key=lambda x: -x[1]): - label = CLS_LABELS.get(cls, cls) - lines.append(f"| [{label}](emulators/index.md#{cls}) | {count} |") - - # Methodology (collapsible) - lines.extend( - [ - "", - '??? abstract "Methodology"', - "", - " Platform lists are checked against emulator source code, file " - "by file where a profile exists ([how far that reaches](gaps.md)). " - "Documentation and metadata can differ from actual runtime behavior, " - "so the source is the primary reference.", - "", - " 1. **Upstream emulator source**: what the original project " - "loads (Dolphin, PCSX2, Mednafen...)", - " 2. **Libretro core source**: the RetroArch port, which may " - "adapt paths or add files", - " 3. **`.info` declarations**: metadata that platforms rely on, " - "checked for accuracy", - "", - ] - ) - - lines.extend( - [ - "---", - "", - "[Systems](systems/index.md){ .md-button } " - "[Emulators](emulators/index.md){ .md-button } " - "[Cross-reference](cross-reference.md){ .md-button } " - "[Gap analysis](gaps.md){ .md-button } " - "[Dump provenance](provenance.md){ .md-button } " - "[Data & API](data.md){ .md-button } " - "[Contributing](contributing.md){ .md-button .md-button--primary }", - "", - f'
Generated on {ts}.
', - ] - ) - return "\n".join(lines) + "\n" @@ -1394,10 +1410,10 @@ def _browser_title(relative: Path, title: str) -> str: # Material appends " - RetroBIOS": the home title says what the site # is, instead of "RetroBIOS - RetroBIOS". "index.md": "BIOS and firmware packs for RetroArch, Batocera and more", - "platforms/index.md": "Platforms", + "platforms/index.md": "Your setup", "systems/index.md": "Systems", "emulators/index.md": "Emulators", - "wiki/index.md": "Guide and methodology", + "wiki/index.md": "Reference", } if key in index_titles: return index_titles[key] @@ -1435,18 +1451,19 @@ def _shorten(plain: str, limit: int = 157) -> str: def _page_description(markdown: str, relative: Path, title: str) -> str: key = relative.as_posix() specific = { - "index.md": "Source-traced BIOS and firmware metadata, platform verification, emulator profiles, gaps and reproducible retrogaming data exports.", + "index.md": "The BIOS files your emulators need, for RetroArch, Batocera, EmuDeck and more: the folder they go in, one command that installs and checks them, and packs to download.", "data.md": "Versioned RetroBIOS JSON API, CSV exports, SQLite snapshot, schemas, checksums and data interpretation guidance.", "cross-reference.md": "Cross-reference from retrogaming platforms to emulator cores, systems, upstream projects and profiled firmware files.", "gaps.md": "Current RetroBIOS verification gaps, platform-to-emulator divergences, missing files and documented source limitations.", "exports.md": "Each platform's own BIOS list (System.dat, batocera-systems, es_bios.xml and the rest) with the hashes and files the emulator source code shows, offered to its maintainers.", "provenance.md": "Hash-based comparison of RetroBIOS metadata with No-Intro, Redump and TOSEC catalog snapshots.", - "which-pack.md": "One-line automatic RetroBIOS installation and platform-specific pack destinations, with verification and release caveats.", + "which-pack.md": "Pick your device or setup and get its BIOS files: the folder they go in, one command that installs and checks them, or a ZIP pack to extract.", } if key in specific: return specific[key] if key.startswith("platforms/") and relative.stem != "index": - return f"{title}: declared BIOS contract, verification mode, coverage, destinations and emulator complement." + return (f"{title}: where the BIOS files go, the command that installs them, the pack, " + "and every file by system with its hashes.") if key.startswith("emulators/") and relative.stem != "index": # Pinned only when the profile names the revision it was read at: # 48 pages without one called themselves source-pinned. @@ -1804,6 +1821,55 @@ def emulator_structured_data(profile: dict) -> dict: LOCAL_NOTE_DIRS = ("plans", "specs") +# The funnel: who each page is written for, from someone who knows nothing +# (start) to someone adding to the project (contribute). The label sits above +# the title, and the search ranks the first two levels higher. +LEVELS = { + "start": "Start here", + "setup": "Setup guide", + "explore": "Details", + "reference": "Reference", + "contribute": "For contributors", +} +SEARCH_BOOST = {"start": 5, "setup": 3} +_REFERENCE_PAGES = { + "gaps.md", "cross-reference.md", "provenance.md", "exports.md", "data.md", + "files/index.md", "wiki/data-model.md", +} + + +def page_level(relative: Path) -> str: + """The level of a generated page. A wiki page states its own.""" + path = relative.as_posix() + if path in ("index.md", "which-pack.md"): + return "start" + if relative.parts[0] == "platforms": + return "setup" + if relative.parts[0] in ("systems", "emulators"): + return "explore" + if path in _REFERENCE_PAGES: + return "reference" + if path == "contributing.md": + return "contribute" + return "" + + +def level_label(level: str) -> str: + """The visible label of a level, plain text for the theme to style.""" + return f'{LEVELS[level]}' + + +def _front_matter(markdown: str) -> dict: + """The YAML front matter of a page, empty when it has none.""" + if not markdown.startswith("---\n"): + return {} + end = markdown.find("\n---\n", 4) + if end == -1: + return {} + data = yaml.safe_load(markdown[4:end]) + return data if isinstance(data, dict) else {} + + def decorate_markdown_pages(docs: Path) -> None: """Add per-page descriptions and machine-readable structured data.""" for path in sorted(docs.rglob("*.md")): @@ -1811,11 +1877,14 @@ def decorate_markdown_pages(docs: Path) -> None: if any(part in LOCAL_NOTE_DIRS for part in relative.parts[:-1]): continue markdown = path.read_text(encoding="utf-8") + authored = _front_matter(markdown) stripped = _undecorated(markdown) if stripped is not markdown: markdown = stripped - elif markdown.startswith("---\n"): - continue + elif authored: + # A wiki page states its own level: keep it, wrap the body. + markdown = markdown[markdown.find("\n---\n", 4) + 5:].lstrip("\n") + level = authored.get("level") or page_level(relative) title = _page_title(markdown, relative) browser_title = _browser_title(relative, title) @@ -1851,12 +1920,15 @@ def decorate_markdown_pages(docs: Path) -> None: "generated_by: retrobios-site\n" f"title: {json.dumps(browser_title, ensure_ascii=False)}\n" f"description: {json.dumps(description, ensure_ascii=False)}\n" + + (f"level: {level}\n" if level else "") + + (f"search:\n boost: {SEARCH_BOOST[level]}\n" if level in SEARCH_BOOST else "") + (f"date_modified: {json.dumps(str(structured['dateModified']))}\n" if structured.get("dateModified") else "") + "---\n\n" '\n\n" + + (f"{level_label(level)}\n\n" if level else "") ) write_decorated(path, front_matter + markdown) @@ -1865,13 +1937,39 @@ def decorate_markdown_pages(docs: Path) -> None: def generate_platform_index(coverages: dict, registry: dict | None = None) -> str: + """The Your setup tab: every platform by the device it runs on, then the + figures of each one under a reference heading.""" + registry = registry or {} total_present = sum(c["present"] for c in coverages.values()) lines = [ - "# Platforms", + "# Your setup", "", - f"{len(coverages)} supported platforms with " - f"{total_present:,} verified files.", + "Pick the system you play on. Its page says where the BIOS files go, " + "gives the command that installs them and links its pack.", + "", + ] + for group, (label, description) in DEVICE_GROUPS.items(): + members = _group_members(coverages, registry, group) + if not members: + continue + lines += [f"## {label}", "", description, "", '
', ""] + for name in members: + guide = GUIDES[name] + lines += [ + f"- **[{coverages[name]['platform']}]({name}.md)**", + "", + f" {guide.summary}", + "", + ] + lines += ["
", ""] + + lines += [ + "## Reference: every platform at a glance { #reference }", + "", + level_label("reference"), + "", + f"{len(coverages)} supported platforms with {total_present:,} verified files.", "", "| Platform | On its BIOS list | Files in the pack | Checked by | Status | Download |", "|----------|------:|------:|-----------|--------|----------|", @@ -1891,7 +1989,7 @@ def generate_platform_index(coverages: dict, registry: dict | None = None) -> st cov["mode"], f'{cov["mode"]}', ) - status = (registry or {}).get(name, {}).get("status", "active") + status = registry.get(name, {}).get("status", "active") if status == "archived": archived_any = True status_html = 'archived' @@ -1902,7 +2000,7 @@ def generate_platform_index(coverages: dict, registry: dict | None = None) -> st f"| [{display}]({name}.md) | " f"{cov['present']:,} | {_released_files(name)} | " f"{mode_html} | {status_html} | " - f"{_pack_buttons(name)} |" + f"{_pack_cell(name, release_record.load_record(), '../')} |" ) lines.extend( @@ -1912,8 +2010,9 @@ def generate_platform_index(coverages: dict, registry: dict | None = None) -> st "files its emulators load and the data directories they read, " "which is the count an extraction shows. " "Checked by is the test each platform runs on its own, replicated " - "from its source code. " - "[How each mode works](../wiki/verification-modes.md).", + "from its source code.", + "", + "Reference: [How each platform checks files](../wiki/verification-modes.md).", ] ) @@ -1964,7 +2063,7 @@ def _render_platform_systems( lines.append("") pad = " " else: - lines.append(f"## {sys_link} {{ #{anchor} }}") + lines.append(f"### {sys_link} {{ #{anchor} }}") lines.append("") pad = "" @@ -2127,7 +2226,7 @@ def _core_folders_section(name: str, config: dict) -> list[str]: manifest = _REPO_ROOT / "install" / f"{name}.json" files = json.loads(manifest.read_text())["files"] if manifest.is_file() else [] lines = [ - "## Core folders", + "### Core folders", "", "Each folder below is the one the frontend gives that core. The pack " "puts the files the core reads there.", @@ -2147,18 +2246,137 @@ def _core_folders_section(name: str, config: dict) -> list[str]: return lines +def _check_sentence(display: str, mode: str, config: dict, no_list: bool) -> str: + """What the frontend itself checks, from the mode its code uses.""" + if no_list: + return (f"{display} publishes no BIOS list, so it does not check files " + "itself. Each emulator core looks for its own files when a game starts.") + if config.get("hash_rule") == "size_and_any_hash": + return (f"{display} itself compares the size and a hash (MD5, SHA1 or " + "CRC32) of each file with its list, so a file with other contents " + "does not count there.") + algorithm = digest_algorithm(mode) + if algorithm: + return (f"{display} itself compares the {algorithm.upper()} of each file " + "with its list, so a file with other contents does not count there.") + return (f"{display} itself only checks that each file is there, by its name. " + "The check above also reads the contents, so it finds a file that has " + "the right name and other contents.") + + +def _platform_guide(name: str, display: str, cov: dict, registry: dict, no_list: bool) -> list[str]: + """The guide part of a platform page: folder, command, pack, check, next.""" + guide = GUIDES[name] + entry = registry.get(name, {}) + extract_to = entry.get("extract_to") + lines = [ + "## Where the files go", + "", + "| Where | BIOS folder |", + "|-------|-------------|", + *(f"| {label} | {path} |" for label, path in guide.folders), + ] + if extract_to: + lines.append(f"| Extract the pack into | `{extract_to}` |") + lines.append("") + for note in guide.notes: + lines += [note, ""] + lines += [line for note in extract_notes("platforms") if f"The {display} pack " in note + for line in (note, "")] + if name == "retroarch": + lines += [ + "Handheld systems that start RetroArch for their games keep its files " + "in the folder below. The command finds it by itself.", + "", + "| System | BIOS folder |", + "|--------|-------------|", + *(f"| {system} | `{folder}` |" for system, folder in _retroarch_handhelds(registry)), + "", + ] + lines += [ + *(["", ""] if guide.on_card else []), + "## Install", + "", + f"The command finds the {display} BIOS folder, downloads the files that " + "are missing or differ, checks each one, and resumes where it stopped.", + "", + ] + if guide.on_card: + lines += [ + f"Put the {display} SD card in a computer. On Windows, note its drive " + "letter (for example **E:**). Then run the line with the root of the " + "card after `--dest`.", + "", + ] + lines += [ + *terminal_tabs(guide.run, platform_args(name, guide)), + *trust_box("../"), + "To install fewer systems, add `--system` with the system ids listed in " + "the reference below.", + "", + "## Download", + "", + ] + record = release_record.load_record() + pack = release_record.pack_for(name, record) + if pack and pack.get("assets"): + parts = len(pack["assets"]) + lines += [ + _pack_buttons(name, f"Download {display} Pack"), + "", + f"{_fmt_size(pack['download_size'])} to download" + + (f" in {parts} parts" if parts > 1 else "") + + f", {_fmt_size(pack['extracted_size'])} once extracted. " + + (f"Extract it into `{extract_to}`. " if extract_to else "") + + ("Every part is needed: extract them all into the same folder." + if parts > 1 else ""), + "", + ] + elif record.get("tag"): + lines += [ + f"The {record['tag']} release has no {display} pack yet. The command " + "above installs the same files.", + "", + ] + else: + lines += [_pack_buttons(name, f"Download {display} Pack"), ""] + check_args = {key: f"{args} --check" for key, args in platform_args(name, guide).items()} + lines += [ + "## How to check", + "", + "The same command with `--check` reads the BIOS folder, checks every file " + "and writes nothing. It ends on a line such as " + "`5865/5865 present (5865 verified, 0 wrong hash)`.", + "", + *terminal_tabs(guide.run, check_args, steps=False), + _check_sentence(display, cov["mode"], cov["config"], no_list), + "", + *_next_block([ + "[First install, step by step](../wiki/first-install.md), with what " + "each step prints", + "[When a game does not start](../wiki/troubleshooting.md)", + "[Every file by system](#reference-every-file-by-system), further down " + "this page", + ]), + ] + return lines + + def generate_platform_page( name: str, cov: dict, registry: dict | None = None, emulator_files: dict | None = None, ) -> str: + """A guide first (folder, command, pack, check), then every file the + platform's list and its emulators need, under a reference heading.""" + registry = registry or {} config = cov["config"] display = cov["platform"] mode = cov["mode"] pct = _pct(cov["present"], cov["total"]) - logo_url = (registry or {}).get(name, {}).get("logo", "") + logo_url = registry.get(name, {}).get("logo", "") logo_md = ( f"![{display}]({logo_url}){{ width=48 align=right }}\n\n" if logo_url else "" ) @@ -2178,7 +2396,7 @@ def generate_platform_page( logo_md, ] - if (registry or {}).get(name, {}).get("status") == "archived": + if registry.get(name, {}).get("status") == "archived": lines.extend( [ '!!! warning "Archived platform"', @@ -2190,7 +2408,23 @@ def generate_platform_page( ] ) - no_list = not cov["total"] and (registry or {}).get(name, {}).get("native_list") == "none" + summary = GUIDES[name].summary if name in GUIDES else "" + if summary: + lines += [summary + (f" [{display} website]({homepage})." if homepage else ""), ""] + + no_list = not cov["total"] and registry.get(name, {}).get("native_list") == "none" + if name in GUIDES: + lines += _platform_guide(name, display, cov, registry, no_list) + + lines += [ + "## Reference: every file by system { #reference-every-file-by-system }", + "", + level_label("reference"), + "", + "What follows is for checking the work: the counts, where the list was " + "read from, and each file with its hashes.", + "", + ] if no_list: # MinUI and its forks hand each core a folder and name no file: a # 0/0 coverage band read as "this device is not supported". @@ -2237,13 +2471,8 @@ def generate_platform_page( lines.append(f"| Hash type | {hash_type} |") if version: lines.append(f"| Version | {version} |") - # Where a user puts the pack, from the registry the README and the pack - # guide read too. base_destination is only the root the platform's own - # list writes its paths from: the pack strips it, so it is no folder to - # extract into, and showing it as "BIOS path" sent EmuDeck users to bios/. - extract_to = (registry or {}).get(name, {}).get("extract_to") - if extract_to: - lines.append(f"| Extract the pack into | `{extract_to}` |") + # base_destination is only the root the platform's own list writes its + # paths from: the pack strips it, so it is no folder to extract into. if base_dest: lines.append(f"| Paths in the platform's list start at | `{base_dest}/` |") if homepage: @@ -2260,7 +2489,7 @@ def generate_platform_page( if name in _EXPORTED: lines.append(f"| Updated file | [Download it and see the changes](../exports.md#{name}) |") lines.append(f"| Data | {_platform_data_links(name)} |") - contrib_list = (registry or {}).get(name, {}).get("contributed_by", []) + contrib_list = registry.get(name, {}).get("contributed_by", []) if contrib_list: for cb in contrib_list: username = cb.get("username", "") @@ -2271,16 +2500,12 @@ def generate_platform_page( f"| Contributed by | [@{username}](https://github.com/{username}):" f" {contribution}{pr_link} |" ) - detect = ((registry or {}).get(name, {}).get("install") or {}).get("detect") or [] - on_card = any(rule.get("method") == "card_marker" for rule in detect) - lines.extend(["", *_platform_install_block(name, display, on_card)]) - lines.extend( - [ - "", - _pack_buttons(name, f"Download {display} Pack"), - "", - ] - ) + if name not in GUIDES: + detect = (registry.get(name, {}).get("install") or {}).get("detect") or [] + on_card = any(rule.get("method") == "card_marker" for rule in detect) + lines.extend(["", *_platform_install_block(name, display, on_card)]) + lines.extend(["", _pack_buttons(name, f"Download {display} Pack")]) + lines.append("") # The declaration behind each detail, by system and name in declaration # order: keyed by name alone, the last of MiSTer's 23 boot.rom gave its @@ -2303,7 +2528,7 @@ def generate_platform_page( # System summary table (quick navigation) lines.extend( [] if no_list and not by_system else [ - "## Systems overview", + "### Systems overview", "", "| System | Files | Status | Emulators |", "|--------|-------|--------|-----------|", @@ -2512,15 +2737,31 @@ def generate_systems_index(manufacturers: dict) -> str: f"{total_mfr} manufacturers, {total_consoles} consoles, " f"{total_files:,} files in the repository.", "", - "| Manufacturer | Consoles | Files |", - "|-------------|----------|-------|", ] - for mfr in sorted(manufacturers.keys()): - consoles = manufacturers[mfr] - file_count = sum(len(files) for files in consoles.values()) - slug = mfr.lower().replace(" ", "-") - lines.append(f"| [{mfr}]({slug}.md) | {len(consoles)} | {file_count} |") + machines = [m for m in sorted(manufacturers, key=str.casefold) if m not in GAME_DATA_TOPS] + games = [m for m in sorted(manufacturers, key=str.casefold) if m in GAME_DATA_TOPS] + for heading, names in (("Consoles and computers", machines), + ("Game engines and PC games", games)): + if not names: + continue + lines += [ + f"## {heading}", + "", + "| Manufacturer | Consoles | Files |", + "|-------------|----------|-------|", + ] + for mfr in names: + consoles = manufacturers[mfr] + file_count = sum(len(files) for files in consoles.values()) + lines.append(f"| [{mfr}]({_slug(mfr)}.md) | {len(consoles)} | {file_count} |") + lines.append("") + if games: + lines += [ + "Game engines and PC games hold the files a game engine reads, such " + "as the data of a game, rather than the BIOS of a machine.", + "", + ] return "\n".join(lines) + "\n" @@ -2644,7 +2885,7 @@ def generate_provenance_page( "", "When the two views disagree, this project follows the code, because " "that is what decides whether your emulator boots. The reasoning is in " - "the [FAQ](wiki/faq.md#are-these-files-verified-against-original-" + "the [technical questions](wiki/technical-faq.md#are-these-files-verified-against-original-" "hardware-dumps).", "", "## Coverage", @@ -3039,6 +3280,11 @@ def generate_system_page( lines.append("") + lines.extend(_next_block([ + "[Get your files](../which-pack.md), for the setup you play on", + "[When a game does not start](../wiki/troubleshooting.md)", + "[Every emulator](../emulators/index.md), and the files each one reads", + ])) lines.append(f'
Generated on {_timestamp()}.
') return "\n".join(lines) + "\n" @@ -3633,22 +3879,6 @@ def _render_metadata_rows(profile: dict) -> list[str]: return lines -INSTALL_BASE = "https://raw.githubusercontent.com/Abdess/retrobios/main" - - -def install_command(shell: str, args: str = "") -> str: - """The one-liner for a shell, with installer options when given. - - PowerShell passes options only through a scriptblock: `irm ... | iex` - takes none. - """ - if shell == "sh": - return f"curl -fsSL {INSTALL_BASE}/install.sh | sh" + (f" -s -- {args}" if args else "") - if args: - return f"&([scriptblock]::Create((irm {INSTALL_BASE}/install.ps1))) {args}" - return f"irm {INSTALL_BASE}/install.ps1 | iex" - - def install_commands(args: str = "", indent: str = "") -> list[str]: """The same install for each place a user runs it: one generator for the download page and the platform, system and emulator pages, so none of @@ -4168,6 +4398,11 @@ def generate_emulator_page( lines.append("") lines.append("") + lines.extend(_next_block([ + "[Get your files](../which-pack.md), for the setup you play on", + "[Every system](../systems/index.md), and the files each one needs", + "[When a game does not start](../wiki/troubleshooting.md)", + ])) lines.extend([f"*Generated on {_timestamp()}*"]) return "\n".join(lines) + "\n" @@ -5066,7 +5301,20 @@ def generate_cross_reference( def generate_contributing() -> str: return contributing_text( lambda page: f"{page}.md", "# Contributing", "on this site" - ) + ) + "\n" + "\n".join(_next_block([ + "[Profiling an emulator](wiki/profiling.md), the first task most " + "contributions start from", + ])) + + +# The end of a reference page: back to the index, or on to contributing. +REFERENCE_NEXT = "\n" + "\n".join([ + '!!! tip "Next"', + "", + " - [Reference index](wiki/index.md), every reference page in one list", + " - For contributors: [How to contribute](contributing.md)", + "", +]) # Wiki pages @@ -5281,301 +5529,329 @@ def _mkdocs_config(docs: Path, text: str) -> tuple[Path, str]: # mkdocs.yml nav generator +_OS_NAMES = {"windows": "Windows", "darwin": "macOS", "linux": "Linux"} + + +def _runs_on(entry: dict, guide) -> str: + """Where a setup runs, read from the systems the installer detects it on.""" + if guide.on_card: + return "SD card, filled from a computer" + if "server" in guide.run: + return "A server" + if "device" in guide.run: + return "Its own system" + systems = {rule.get("os") for rule in (entry.get("install") or {}).get("detect") or []} + names = [label for key, label in _OS_NAMES.items() if key in systems] + if "steamdeck" in guide.run: + names.append("Steam Deck") + if "android" in guide.run: + names.append("Android") + return ", ".join(names) + + +def _folder_cell(name: str, guide, prefix: str = "") -> str: + """The BIOS folder in a table cell, or a link where it depends on the system.""" + if len(guide.folders) > 3: + return f"Depends on the system: [listed on its page]({prefix}platforms/{name}.md#where-the-files-go)" + return "
".join(f"{label}: {path}" for label, path in guide.folders) + + +def _pack_cell(name: str, record: dict, prefix: str = "") -> str: + """The pack's parts and size, or the command where the release has none.""" + pack = release_record.pack_for(name, record) + if record.get("tag") and not (pack and pack.get("assets")): + return (f"Not in {record['tag']}: " + f"[use the command]({prefix}platforms/{name}.md#install)") + display = load_platform_config(name, "platforms").get("platform", name) + links = download_links(name, record, REPO_URL, display) + size = pack.get("download_size") if pack else None + return f"{links} ({_fmt_size(size)})" if size else links + + +def _retroarch_handhelds(registry: dict) -> list[tuple[str, str]]: + """Handheld systems that run RetroArch, with the BIOS folder the installer + writes on each, read from the detection rules of the registry.""" + seen: dict[str, str] = {} + for rule in (registry.get("retroarch", {}).get("install") or {}).get("detect") or []: + name, folder = rule.get("name"), rule.get("bios_path") + if name and folder and rule.get("installs", True) and name not in seen: + seen[name] = folder + return sorted(seen.items(), key=lambda item: item[0].casefold()) + + +def _group_keys(registry: dict, group: str, display: dict[str, str]) -> list[str]: + return sorted( + (k for k in registry if registry[k].get("device_group") == group), + key=lambda k: display[k].casefold(), + ) + + def generate_which_pack() -> str: - """Generate the 'Which pack?' decision page.""" - rel = RELEASE_URL - quick = "\n".join(install_commands()) - check_sh, check_ps = install_command("sh", "--check"), install_command("powershell", "--check") - sd_card = install_command( - "sh", "--platform retroarch --dest /path/to/sdcard/retroarch/cores/system") - return _link_pack_cells(f"""\ -# Download - -Some retro consoles need firmware files (commonly called BIOS) to run games. -Without them, the emulator either refuses to start the game or runs it with -reduced accuracy. RetroBIOS maps those requirements to source-traced emulator -profiles and verifies local content against the evidence each platform exposes. - -BIOS directory paths per platform, verification and the CLI are in -[Getting started](wiki/getting-started.md). - -## Quick install - -One line detects the platform and BIOS directory, downloads only missing or -incorrect files, verifies their hashes, and installs them atomically. The small -bootstrap checks the Python installer against an embedded SHA-256 before running -it, and the installer reads its file list from that same revision. - -{quick} -That is the complete default flow. Extra copies into detected standalone -emulator directories are deliberately opt-in with `--standalone-copies`, so -automatic setup never writes outside the selected platform tree unexpectedly. -Adding `--check` reads the BIOS folder, hashes what is there and writes -nothing: - -```sh -{check_sh} -``` - -```powershell -{check_ps} -``` - -An SD card mounted on a PC is installed with `--platform` and `--dest`, the -BIOS folder on the card (RetroArch on Switch: `retroarch/cores/system`, on -Android: `RetroArch/system`): - -```sh -{sd_card} -``` - -Options, environment overrides, how each platform is detected and what the -installer is allowed to write: [Installer](wiki/installer.md). - ---- - -## Manual download - -Pick the pack that matches the setup from the [latest release]({rel}), -download it, and extract the files into the BIOS folder listed below. -The metadata and website can be newer than that release: pack publication is -manual and only happens after the release gates pass. - -A pack over 2 GB comes in several parts (`PackName.part1of2.zip`, -`PackName.part2of2.zip`), and every part is needed. Each part is an ordinary -ZIP: extract them all into the same folder. - -### Steam Deck - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| [EmuDeck](https://www.emudeck.com/) | Installs and configures multiple emulators, adds each game to the Steam library | [EmuDeck]({rel}) | `~/Emulation/bios/` | -| [RetroDECK](https://retrodeck.net/) | Single Flatpak app with its emulators bundled, installed from Flathub or Discover | [RetroDECK]({rel}) | `~/retrodeck/` (the pack carries its own `bios/`) | -| RetroArch standalone | Installed from Discover, Steam, or Flatpak | [RetroArch]({rel}) | Open RetroArch > Settings > Directory > System, that is the folder | - -### Windows - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| [RetroArch](https://www.retroarch.com/) | Multi-system emulator, loads different cores for each console | [RetroArch]({rel}) | The `system` folder next to `retroarch.exe` | -| [RetroBat](https://www.retrobat.org/) | Windows frontend with EmulationStation, includes RetroArch and standalone emulators | [RetroBat]({rel}) | The `bios` folder inside the RetroBat installation | -| [BizHawk](https://tasvideos.org/BizHawk) | Multi-system emulator with rerecording and debugging tools, hosted by TASVideos | [BizHawk]({rel}) | The `Firmware` folder inside the BizHawk installation | -| [LaunchBox](https://www.launchbox-app.com/) | Game library manager and launcher, uses RetroArch or standalone emulators behind the scenes | [RetroArch]({rel}) | Open RetroArch (via LaunchBox) > Settings > Directory > System | -| [EmuDeck](https://www.emudeck.com/) | Installs and configures multiple emulators with ES-DE | [EmuDeck]({rel}) | `bios` in the Emulation folder chosen in EmuDeck, `C:\\Emulation\\bios\\` by default | - -### Linux - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| RetroArch (native) | Installed via package manager or AppImage | [RetroArch]({rel}) | `~/.config/retroarch/system/` | -| RetroArch (Flatpak) | Installed from Flathub | [RetroArch]({rel}) | `~/.var/app/org.libretro.RetroArch/config/retroarch/system/` | -| [Batocera](https://batocera.org/) | Bootable OS dedicated to gaming, runs from USB or full install, supports PC and SBC | [Batocera]({rel}) | `/userdata/bios/` | -| [Recalbox](https://www.recalbox.com/) | Bootable OS for retro gaming that configures its emulators itself | [Recalbox]({rel}) | `/recalbox/share/bios/` | - -### macOS - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| [RetroArch](https://www.retroarch.com/) | Multi-system emulator | [RetroArch]({rel}) | `~/Library/Application Support/RetroArch/system/` | -| [EmuDeck](https://www.emudeck.com/) | Installs and configures multiple emulators with ES-DE | [EmuDeck]({rel}) | `~/Emulation/bios/` | - -### Raspberry Pi and single-board computers - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| [RetroPie](https://retropie.org.uk/) | Scripts that install RetroArch and standalone emulators on Raspberry Pi OS | [RetroPie]({rel}) | `~/RetroPie/BIOS/` | -| [Lakka](https://www.lakka.tv/) | Operating system built around RetroArch, boots into its menu | [Lakka]({rel}) | `/storage/system/` | -| [Batocera](https://batocera.org/) | Bootable OS, with images for Raspberry Pi 3, 4 and 5 and other boards (Odroid XU4 among them) | [Batocera]({rel}) | `/userdata/bios/` | -| [Recalbox](https://www.recalbox.com/) | Bootable OS for retro gaming that configures its emulators itself | [Recalbox]({rel}) | `/recalbox/share/bios/` | - -### Handhelds - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| Android (Retroid Pocket, Odin, etc.) | RetroArch for Android | [RetroArch]({rel}) | `RetroArch/system/` on internal storage or SD card | -| [ROCKNIX](https://rocknix.org/) | Linux OS for ARM and x86 handhelds, among them the RG35XX (H700) and the RG552 (RK3399) | [ROCKNIX]({rel}) | `/storage/roms/bios/`, over SSH or the network share | -| [Batocera](https://batocera.org/) | Also has images for handheld chips, among them H700 and RK3326 | [Batocera]({rel}) | `/userdata/bios/` | -| [KNULLI](https://knulli.org/) | Batocera-based OS for handhelds, among them the RG35XX family (H700) and the TrimUI Smart Pro | [KNULLI]({rel}) | `/userdata/bios/` | -| [AmberELEC](https://amberelec.org/) | Linux OS for Anbernic handhelds: RG351P, RG351V, RG351MP, RG353, RG503 and RG552 | [AmberELEC]({rel}) | `/storage/roms/`, which the pack fills with `bios/` and `neogeo/` | -| [EmuELEC](https://emuelec.org/) | Linux OS for Amlogic TV boxes, the Odroid Go Advance, GameForce and RK356x devices | [EmuELEC]({rel}) | `/storage/roms/`, which the pack fills with `bios/`, `neogeo/` and `ports/` | - -On a handheld that runs PortMaster, the RetroBIOS port checks and fills the -BIOS folder from the device, with gamepad screens: -[on a handheld](wiki/installer.md#on-a-handheld-the-retrobios-port). - -### FPGA - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| [MiSTer FPGA](https://mister-devel.github.io/MkDocs_MiSTer/) | FPGA cores that reimplement console hardware on a DE10-Nano board | [MiSTer FPGA]({rel}) | `/media/fat/games/`, one subfolder per core | - -### Self-hosted ROM manager - -| Setup | What it is | Pack | Extract to | -|-------|-----------|------|-----------| -| [RomM](https://github.com/rommapp/romm) | Web-based ROM manager, plays games in the browser via EmulatorJS | [RomM]({rel}) | `bios/` in the RomM library, one subfolder per platform. With `firmware: "{{platform}}/bios"` in `config.yml`, each subfolder goes to `/bios/` instead | - ---- - -## One pack per platform - -The [latest release]({rel}) carries one pack for each platform, and that -pack holds everything the platform runs: the BIOS list it publishes, plus -every file its emulator cores load, read from their source code. There is no -lighter variant to choose, because a pack that leaves out a file a core needs -means a game that does not boot, with no message saying why. It is not a -guarantee either: source profiles can document files nobody has dumped or -that only the user can provide, all of them visible in the -[gap analysis](gaps.md). - -Want less than everything? At a terminal the installer asks whether to install -everything or to choose systems, cores and regions, and `--system`, `--core`, -`--region` and `--target` make the same choice without questions (see the -[installer reference](wiki/installer.md#choosing-what-to-install)). From a -clone of the repository, `python scripts/generate_pack.py --platform retroarch ---region us` builds the same narrowing as a pack and `--required-only` the bare -minimum each core needs to start. `--no-game-data`, for the pack and the -installer alike, keeps the firmware and leaves out the files a game reads -([details](wiki/advanced-usage.md#firmware-without-game-data)), and `--help` -lists every way to build your own. - ---- - -## After extraction - -Launch a game and use the emulator's firmware status screen where available. -Most supported frontends find files in the documented directory automatically, -but standalone emulators may still need their BIOS path configured. - -If a game still asks for a missing file, check the -[platforms section](platforms/index.md) for the full file list, or the -[emulators section](emulators/index.md) for what each core expects. -""") - - -def _link_pack_cells(page: str) -> str: - """Point each table cell naming a platform's pack at that pack's files. - - The cell names the platform by its display name, resolved through the - registry, and the links come from the release record. - """ + """The device picker: every registered platform, grouped by the device it + runs on, with its BIOS folder, its pack and its page.""" + registry = load_platform_registry("platforms") record = release_record.load_record() - keys = { - load_platform_config(key, "platforms").get("platform", key): key - for key in list_registered_platforms("platforms", include_archived=True) + display = {key: load_platform_config(key, "platforms").get("platform", key) for key in registry} + check_sh, check_ps = install_command("sh", "--check"), install_command("powershell", "--check") + card = install_command("sh", "--platform retroarch --dest /path/to/sdcard/retroarch/cores/system") + lines = [ + "# Get your files", + "", + "Each setup keeps its BIOS files in a folder of its own. Pick yours below " + "to see where that folder is. Then get the files in one of two ways:", + "", + "- **One command**, the easiest. It finds your setup and its BIOS folder, " + "downloads only the files that are missing, and checks each one before " + "saving it.", + "- **A ZIP pack.** You download the pack of your setup and extract it into " + "the BIOS folder yourself.", + "", + "", + "", + "## Pick your setup", + "", + '
', + "", + ] + for group, (label, description) in DEVICE_GROUPS.items(): + members = _group_keys(registry, group, display) + if not members: + continue + lines += [ + f"- **{label}**", + "", + f" {description}", + "", + f" [{', '.join(display[k] for k in members)}](#{GROUP_ANCHORS[group]})", + "", + ] + lines += [ + "
", + "", + "## One command { #quick-install }", + "", + "Run this line in a terminal. It finds your setup by itself. On a system " + "that starts into a game menu, such as Batocera, the line runs on the " + "device: the page of your setup gives the exact command.", + "", + *terminal_tabs(("windows", "macos", "linux", "steamdeck", "android"), {}), + *trust_box(""), + "At a terminal, the installer shows the setup it found and asks what to " + "install:", + "", + "```", + " Enter) install everything", + " i) only what the installed cores read", + " c) choose systems, cores and regions", + " q) quit", + "```", + "", + "Press **Enter** to install everything. Press **c** to pick only some " + "consoles. The **i** line appears when the installer finds emulator cores " + "installed, and keeps only the files they read.", + "", + "To see what is already in your BIOS folder, add `--check`. It reads the " + "folder, checks every file and writes nothing:", + "", + "```sh", + check_sh, + "```", + "", + "```powershell", + check_ps, + "```", + "", + "A memory card from another device can be filled from a computer: name the " + "setup with `--platform` and the BIOS folder on the card with `--dest`. " + "For RetroArch on a Nintendo Switch:", + "", + "```sh", + card, + "```", + "", + ] + for group, (label, description) in DEVICE_GROUPS.items(): + members = _group_keys(registry, group, display) + if not members: + continue + lines += [ + f"## {label} {{ #{GROUP_ANCHORS[group]} }}", + "", + description, + "", + "| Setup | What it is | Runs on | BIOS folder | Pack |", + "|-------|-----------|---------|-------------|------|", + ] + for key in members: + guide = GUIDES[key] + lines.append( + f"| [{display[key]}](platforms/{key}.md) | {guide.summary} | " + f"{_runs_on(registry[key], guide)} | {_folder_cell(key, guide)} | " + f"{_pack_cell(key, record)} |" + ) + for setup, summary, uses, runs_on, folder in ALSO_RUNS.get(group, ()): + lines.append( + f"| {setup} | {summary} It uses the [{display[uses]}](platforms/{uses}.md) " + f"files. | {runs_on} | {folder} | {_pack_cell(uses, record)} |" + ) + lines.append("") + if group == "handheld": + lines += [ + "Many other handheld systems start RetroArch for their games. On " + "them, the RetroArch files go in the folder below, and the command " + "finds it by itself.", + "", + "| System | Runs | BIOS folder |", + "|--------|------|-------------|", + *(f"| {name} | [RetroArch](platforms/retroarch.md) | `{folder}` |" + for name, folder in _retroarch_handhelds(registry)), + "", + ] + lines += [ + "## Download a pack { #manual-download }", + "", + f"Each pack in the tables above comes from the [latest release]({RELEASE_URL}). " + "Download the pack of your setup and extract it into its BIOS folder. " + "This site can describe files newer than that release: a release is " + "published by hand, once its checks pass.", + "", + "A pack over 2 GB comes in several parts (`PackName.part1of2.zip`, " + "`PackName.part2of2.zip`), and every part is needed. Each part is an " + "ordinary ZIP: extract them all into the same folder.", + "", + *[line for note in extract_notes("platforms") for line in (note, "")], + "## Fewer files", + "", + "Each pack holds everything its setup runs: the BIOS list the setup " + "publishes, and every file its emulators load. The command can install " + "less: press **c** when it asks, or name the consoles with `--system`.", + "", + "Details: [All installer options](wiki/installer.md#choosing-what-to-install). " + "Reference: [Build your own packs](wiki/advanced-usage.md).", + "", + *_next_block([ + "[First install, step by step](wiki/first-install.md)", + "The page of your setup, from the tables above: where the files go and " + "how to check them", + "[When a game does not start](wiki/troubleshooting.md)", + ]), + ] + return "\n".join(lines) + "\n" + + +# Letters each alphabetical group of the emulators navigation covers. +EMULATOR_NAV_GROUPS = ("ABC", "DEF", "GHIJKL", "MNO", "PQR", "STU", "VWXYZ") + + +def nav_emulators(profiles: dict) -> dict[str, dict]: + """The emulators listed in the navigation, by key. + + A profile that emulates no system and reads no file (the libretro sample + and test cores, games with nothing to load) answers no question about + BIOS files. Its page stays, reachable from the overview table and the + search, and the navigation is left to the cores people look up. + """ + return { + key: profile for key, profile in unique_emulator_profiles(profiles).items() + if profile.get("systems") or profile.get("files") } - cell = re.compile(rf"\| \[([^\]]+)\]\({re.escape(RELEASE_URL)}\) \|") - def link(match: re.Match[str]) -> str: - label = match.group(1) - if label not in keys: - raise KeyError(f"no registered platform is displayed as {label!r}") - return f"| {download_links(keys[label], record, REPO_URL, label)} |" - def released(line: str) -> bool: - # A platform registered after the recorded release has no pack in - # it yet: its row would send the reader to a release without one. - found = cell.search(line) - if not found or not record.get("packs"): - return True - label = found.group(1) - return label not in keys or release_record.pack_for(keys[label], record) is not None +def _emulator_nav(profiles: dict) -> list: + """Emulators in alphabetical groups ("A to C"), by the name a user knows.""" + groups: dict[str, list[tuple[str, str]]] = {} + for key, profile in nav_emulators(profiles).items(): + display = str(profile.get("emulator", key)) + initial = display[:1].upper() + letters = next((g for g in EMULATOR_NAV_GROUPS if initial in g), "") + label = f"{letters[0]} to {letters[-1]}" if letters else "0 to 9" + groups.setdefault(label, []).append((display, f"emulators/{key}.md")) + order = ["0 to 9"] + [f"{g[0]} to {g[-1]}" for g in EMULATOR_NAV_GROUPS] + return ["emulators/index.md"] + [ + {label: [{d: path} for d, path in sorted(groups[label], key=lambda e: e[0].casefold())]} + for label in order if label in groups + ] - page = "\n".join(line for line in page.split("\n") if released(line)) - return cell.sub(link, page) + +def _system_nav(manufacturers: dict) -> list: + """Manufacturer pages, the machines apart from the game and engine trees.""" + machines, games = [], [] + for mfr in sorted(manufacturers, key=str.casefold): + entry = {mfr: f"systems/{_slug(mfr)}.md"} + (games if mfr in GAME_DATA_TOPS else machines).append(entry) + nav: list = ["systems/index.md", {"Consoles and computers": machines}] + if games: + nav.append({"Game engines and PC games": games}) + return nav + + +def _platform_nav(coverages: dict, registry: dict) -> list: + """Platforms by the kind of device they run on, as on the device picker.""" + nav: list = ["platforms/index.md"] + for group, (label, _description) in DEVICE_GROUPS.items(): + members = [ + {coverages[name]["platform"]: f"platforms/{name}.md"} + for name in sorted(coverages, key=lambda n: coverages[n]["platform"].casefold()) + if registry.get(name, {}).get("device_group") == group + ] + if members: + nav.append({label: members}) + return nav def generate_mkdocs_nav( coverages: dict, manufacturers: dict, profiles: dict, + registry: dict | None = None, ) -> list: - """Generate the nav section for mkdocs.yml.""" - platform_nav = [{"Overview": "platforms/index.md"}] - for name in sorted(coverages.keys(), key=lambda x: coverages[x]["platform"]): - display = coverages[name]["platform"] - platform_nav.append({display: f"platforms/{name}.md"}) - - system_nav = [{"Overview": "systems/index.md"}] - for mfr in sorted(manufacturers.keys()): - slug = mfr.lower().replace(" ", "-") - system_nav.append({mfr: f"systems/{slug}.md"}) - - unique_profiles = { - k: v for k, v in profiles.items() if v.get("type") not in ("alias", "test") - } - - by_class: dict[str, list[tuple[str, str]]] = {} - for name in sorted(unique_profiles.keys()): - p = unique_profiles[name] - cls = p.get("core_classification", "other") - display = p.get("emulator", name) - by_class.setdefault(cls, []).append((display, f"emulators/{name}.md")) - - cls_labels = { - "pure_libretro": "Pure libretro", - "official_port": "Official ports", - "community_fork": "Community forks", - "frozen_snapshot": "Frozen snapshots", - "enhanced_fork": "Enhanced forks", - "game_engine": "Game engines", - "embedded_hle": "Embedded HLE", - "launcher": "Launchers", - "other": "Other", - } - - emu_nav: list = [{"Overview": "emulators/index.md"}] - for cls in [ - "official_port", - "community_fork", - "pure_libretro", - "game_engine", - "enhanced_fork", - "frozen_snapshot", - "embedded_hle", - "launcher", - "other", - ]: - entries = by_class.get(cls, []) - if not entries: - continue - label = cls_labels.get(cls, cls) - sub = [{display: path} for display, path in entries] - emu_nav.append({f"{label} ({len(entries)})": sub}) - - wiki_nav = [ - {"Overview": "wiki/index.md"}, - {"First install": "wiki/first-install.md"}, - {"Getting started": "wiki/getting-started.md"}, - {"Installer": "wiki/installer.md"}, - {"FAQ": "wiki/faq.md"}, - {"Troubleshooting": "wiki/troubleshooting.md"}, - {"Architecture": "wiki/architecture.md"}, - {"Tools": "wiki/tools.md"}, - {"Advanced usage": "wiki/advanced-usage.md"}, - {"Verification modes": "wiki/verification-modes.md"}, - {"Data model": "wiki/data-model.md"}, - {"Profiling guide": "wiki/profiling.md"}, - {"Adding a platform": "wiki/adding-a-platform.md"}, - {"Adding a scraper": "wiki/adding-a-scraper.md"}, - {"Testing guide": "wiki/testing-guide.md"}, - {"Release process": "wiki/release-process.md"}, - {"Community tools": "wiki/community-tools.md"}, - ] + """The navigation, five tabs in reading order. + Start here is for someone who knows nothing yet, Your setup for someone + who knows their device, Explore for one system or emulator, Reference for + checking the work, Contribute for adding to it. Each tab opens on its + index page. + """ return [ - {"Home": "index.md"}, - {"Download": "which-pack.md"}, - {"Platforms": platform_nav}, - {"Systems": system_nav}, - {"Emulators": emu_nav}, - {"Cross-reference": "cross-reference.md"}, - {"Gap analysis": "gaps.md"}, - {"Dump provenance": "provenance.md"}, - {"Updated platform files": "exports.md"}, - {"Data & API": "data.md"}, - {"Wiki": wiki_nav}, - {"Contributing": "contributing.md"}, + {"Start here": [ + "index.md", + {"What BIOS files are": "wiki/what-are-bios-files.md"}, + {"Get your files": "which-pack.md"}, + {"First install, step by step": "wiki/first-install.md"}, + {"When a game does not start": "wiki/troubleshooting.md"}, + {"Common questions": "wiki/faq.md"}, + {"Glossary": "wiki/glossary.md"}, + ]}, + {"Your setup": _platform_nav(coverages, registry or {})}, + {"Explore": [ + {"Systems": _system_nav(manufacturers)}, + {"Emulators": _emulator_nav(profiles)}, + {"All installer options": "wiki/installer.md"}, + ]}, + {"Reference": [ + "wiki/index.md", + {"How each platform checks files": "wiki/verification-modes.md"}, + {"What the packs do not hold": "gaps.md"}, + {"Cross-reference": "cross-reference.md"}, + {"Dump provenance": "provenance.md"}, + {"Updated platform files": "exports.md"}, + {"Data and API": "data.md"}, + {"Find a file by its SHA1": "files/index.md"}, + {"Data model": "wiki/data-model.md"}, + {"Architecture": "wiki/architecture.md"}, + {"Build your own packs": "wiki/advanced-usage.md"}, + {"Command-line tools": "wiki/tools.md"}, + {"Technical questions": "wiki/technical-faq.md"}, + {"Projects using this data": "wiki/community-tools.md"}, + ]}, + {"Contribute": [ + "contributing.md", + {"Profiling an emulator": "wiki/profiling.md"}, + {"Adding a platform": "wiki/adding-a-platform.md"}, + {"Adding a scraper": "wiki/adding-a-scraper.md"}, + {"Testing": "wiki/testing-guide.md"}, + {"Release process": "wiki/release-process.md"}, + ]}, ] @@ -5713,7 +5989,7 @@ def _generate(args: argparse.Namespace) -> None: exports = generate_data_exports( docs, db, coverages, profiles, stats, gap_report ) - write_if_changed(str(docs / "data.md"), generate_data_page(exports)) + write_if_changed(str(docs / "data.md"), generate_data_page(exports) + REFERENCE_NEXT) PAGE_DATA["data.md"] = dataset_structured_data(exports, db) # Build system_id -> manufacturer page map (needed by all generators) @@ -5808,7 +6084,8 @@ def _generate(args: argparse.Namespace) -> None: print("Generating cross-reference page...") write_if_changed( - str(docs / "cross-reference.md"), generate_cross_reference(coverages, profiles) + str(docs / "cross-reference.md"), + generate_cross_reference(coverages, profiles) + REFERENCE_NEXT, ) print("Generating gap analysis page...") @@ -5816,7 +6093,7 @@ def _generate(args: argparse.Namespace) -> None: str(docs / "gaps.md"), generate_gap_analysis( profiles, coverages, db, suppl_names, registry, gap_report - ), + ) + REFERENCE_NEXT, ) print("Generating updated platform files page...") @@ -5825,14 +6102,14 @@ def _generate(args: argparse.Namespace) -> None: generate_exports_page( export_reports, {name: cov["platform"] for name, cov in coverages.items()}, - ), + ) + REFERENCE_NEXT, ) print("Generating dump provenance page...") provenance_report = build_report(db, load_provenance_snapshots()) write_if_changed( str(docs / "provenance.md"), - generate_provenance_page(db, provenance_report, load_derivable()), + generate_provenance_page(db, provenance_report, load_derivable()) + REFERENCE_NEXT, ) # Wiki pages: copy manually maintained sources + generate dynamic ones @@ -5860,7 +6137,7 @@ def _generate(args: argparse.Namespace) -> None: # Update mkdocs.yml nav section only (avoid yaml.dump round-trip mangling quotes) print("Updating mkdocs.yml nav...") - nav = generate_mkdocs_nav(coverages, manufacturers, profiles) + nav = generate_mkdocs_nav(coverages, manufacturers, profiles, registry) nav_yaml = yaml.dump( {"nav": nav}, default_flow_style=False, sort_keys=False, allow_unicode=True ) @@ -5922,6 +6199,8 @@ theme: - navigation.top - navigation.tracking - navigation.indexes + # Breadcrumbs above the title: on a deep page they say which tab it is in. + - navigation.path # 400+ pages: pruning keeps the navigation out of every page's HTML. - navigation.prune - navigation.footer diff --git a/scripts/sitewrite.py b/scripts/sitewrite.py index 9f0a73bc..fc8589ce 100644 --- a/scripts/sitewrite.py +++ b/scripts/sitewrite.py @@ -20,6 +20,10 @@ _produced: set[str] = set() def _record(path: str | Path) -> None: _produced.add(os.path.realpath(str(path))) +# The reader level label the decoration pass puts above a page's title. +LEVEL_LABEL_START = ' str: script_end = markdown.find("\n\n") if markdown.startswith('\n\n"):] + if markdown.startswith(LEVEL_LABEL_START): + markdown = markdown[markdown.find("\n\n") + 2:] return markdown def write_if_changed(path: str, content: str) -> bool: diff --git a/tests/test_doc_pages.py b/tests/test_doc_pages.py index 499b263e..a25a43ef 100644 --- a/tests/test_doc_pages.py +++ b/tests/test_doc_pages.py @@ -730,7 +730,8 @@ class RetroArchWindowsSystemFolder(unittest.TestCase): """Getting started gave %APPDATA%\\RetroArch\\system for the Windows installer. RetroArch's default is `:\\system`, the folder of retroarch.exe (platform_win32.c). %APPDATA% is where a retroarch.cfg - can live, and install.py reads it there.""" + can live, and install.py reads it there. The RetroArch page of the site + states the folder now, from the guide data.""" def test_page_and_installer_agree(self): import install @@ -738,10 +739,26 @@ class RetroArchWindowsSystemFolder(unittest.TestCase): source = inspect.getsource(install) self.assertIn('ra_dir or exe_dir / "system"', source) self.assertTrue(str(install._os_default_dests("windows")["retroarch"]).replace("\\", "/").endswith("/system")) - section = _section(_page("getting-started.md"), "RetroArch") + from siteguides import GUIDES + + guide = GUIDES["retroarch"] + section = "\n".join([*(path for _label, path in guide.folders), *guide.notes]) self.assertNotIn("%APPDATA%\\RetroArch\\system", section) self.assertIn("`system\\` next to `retroarch.exe`", section) + def test_the_macos_folder_is_the_installers(self): + """The site gave Application Support/RetroArch/system for macOS. A + RetroArch left at its defaults reads ~/Documents/RetroArch/system + (platform_darwin.m), Application Support only for a portable or + Steam build, and install.py looks there in that order.""" + import install + from siteguides import GUIDES + + source = inspect.getsource(install._macos_retroarch_base) + self.assertIn('Path.home() / "Documents" / "RetroArch"', source) + folders = dict(GUIDES["retroarch"].folders) + self.assertEqual(folders["macOS"], "`~/Documents/RetroArch/system/`") + if __name__ == "__main__": unittest.main() diff --git a/tests/test_doc_surfaces.py b/tests/test_doc_surfaces.py index 770c4690..b87abebe 100644 --- a/tests/test_doc_surfaces.py +++ b/tests/test_doc_surfaces.py @@ -75,22 +75,25 @@ def _which_pack() -> str: class TheDownloadTablesFollowTheRegistry(unittest.TestCase): """EmuDeck has a Windows edition the installer detects, and the Windows - table of the download page left it out.""" + table of the download page left it out. The page is grouped by device + now: the row of each platform names every system it is detected on.""" def test_every_detected_system_lists_the_platform(self): import yaml # noqa: PLC0415 + from common import load_platform_config # noqa: PLC0415 + registry = yaml.safe_load(_read("platforms/_registry.yml")) - names = {"retroarch": "RetroArch", "retrobat": "RetroBat", "emudeck": "EmuDeck", - "bizhawk": "BizHawk"} page = _which_pack() - for os_name, heading in (("windows", "### Windows"), ("darwin", "### macOS")): - table = _section(page, heading) + for os_name, label in (("windows", "Windows"), ("darwin", "macOS")): for platform, entry in registry["platforms"].items(): systems = {rule["os"] for rule in (entry.get("install") or {}).get("detect", [])} if os_name in systems: + name = load_platform_config(platform, "platforms")["platform"] + row = next(line for line in page.splitlines() + if line.startswith(f"| [{name}](")) with self.subTest(os=os_name, platform=platform): - self.assertIn(f"[{names[platform]}](", table) + self.assertIn(label, row.split(" | ")[2]) class DownloadLinksNameTheFile(unittest.TestCase): @@ -159,9 +162,10 @@ class DownloadLinksNameTheFile(unittest.TestCase): for asset in pack["assets"]: self.assertIn(f"/releases/download/{RECORD['tag']}/{asset})", page) - def test_a_platform_the_release_lacks_has_no_row(self): + def test_a_platform_the_release_lacks_links_no_release(self): """KNULLI, AmberELEC and EmuELEC were registered after the recorded - release: a row for them linked a release page holding no such pack.""" + release: a row for them linked a release page holding no such pack. + Every platform is on the picker now, so the row offers the command.""" if not RECORD.get("packs"): self.skipTest("release.json is not written") from common import list_registered_platforms, load_platform_config @@ -172,8 +176,10 @@ class DownloadLinksNameTheFile(unittest.TestCase): if pack_for(key, RECORD) is not None: continue label = load_platform_config(key, "platforms").get("platform", key) + row = next(line for line in page.splitlines() if line.startswith(f"| [{label}](")) with self.subTest(platform=key): - self.assertNotIn(f"| [{label}](", page) + self.assertNotIn("/releases/", row) + self.assertIn(f"Not in {RECORD['tag']}: [use the command](platforms/{key}.md#install)", row) class SlicesLeaveWithTheirRelease(unittest.TestCase): @@ -205,7 +211,7 @@ class EndUsersCheckTheirOwnFolder(unittest.TestCase): ("wiki/troubleshooting.md", "## Game won't start / black screen"), ("wiki/faq.md", "## My game shows a black screen"), ("wiki/faq.md", "## How do I know which BIOS I need?"), - ("wiki/getting-started.md", "## Checking what you installed"), + ("wiki/troubleshooting.md", "## Checking what you installed"), ) def test_user_sections_never_send_to_verify(self): @@ -263,7 +269,7 @@ class ShownOutputIsPrintedOutput(unittest.TestCase): self.assertIn(literal, source) def test_samples_match_a_shape(self): - section = _section(_read("wiki/troubleshooting.md"), "### Reading the output") + section = _section(_read("wiki/verification-modes.md"), "### Reading the output") blocks = re.findall(r"```\n(.*?)```", section, re.DOTALL) self.assertTrue(blocks) for block in blocks: diff --git a/tests/test_site_exports.py b/tests/test_site_exports.py index 65948994..07a31239 100644 --- a/tests/test_site_exports.py +++ b/tests/test_site_exports.py @@ -902,8 +902,13 @@ class TheOneLinerIsWhereUsersLand(unittest.TestCase): "verified": 0, "missing": 0, "details": [], "config": {"systems": {}}} page = generate_site.generate_platform_page("batocera", cov) self.assertIn("install.sh | sh -s -- --platform batocera", page) - self.assertIn("install.ps1))) --platform batocera", page) self.assertLess(page.index("--platform batocera"), page.index("md-button")) + # Batocera runs the command on the device itself. A platform detected + # on Windows gets the PowerShell form. + cov = dict(cov, platform="RetroArch", mode="existence") + page = generate_site.generate_platform_page("retroarch", cov) + self.assertIn("install.ps1))) --platform retroarch", page) + self.assertIn("install.sh | sh -s -- --platform retroarch", page) entry = {"name": "a.bin", "sha1": "a" * 40, "md5": "b" * 32, "size": 3, "path": "bios/Recalbox/MSX/a.bin"} diff --git a/tests/test_site_funnel.py b/tests/test_site_funnel.py new file mode 100644 index 00000000..593e1a81 --- /dev/null +++ b/tests/test_site_funnel.py @@ -0,0 +1,227 @@ +"""The site reads as a funnel, from someone who knows nothing to a contributor. + +Five tabs in reading order, every registered platform reachable from the +device picker, a level on every page, and no page for beginners sending its +reader two levels deeper without saying so. +""" + +from __future__ import annotations + +import contextlib +import os +import re +import sys +import tempfile +import unittest +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(REPO_ROOT / "scripts")) + +import generate_site # noqa: E402 +from common import list_registered_platforms, load_platform_config, load_platform_registry # noqa: E402 +from siteguides import DEVICE_GROUPS, GROUP_ANCHORS, GUIDES # noqa: E402 + +DOCS = REPO_ROOT / "docs" +ORDER = list(generate_site.LEVELS) +LINK = re.compile(r"\[[^\]]*\]\(([^)\s#]+\.md)(?:#[^)]*)?\)") +FENCED = re.compile(r"^\s*```.*?^\s*```", re.S | re.M) + + +def _registered() -> list[str]: + return list_registered_platforms(str(REPO_ROOT / "platforms"), include_archived=True) + + +def _which_pack() -> str: + with contextlib.chdir(REPO_ROOT): + return generate_site.generate_which_pack() + + +def _docs_pages() -> list[Path]: + if not DOCS.is_dir() or not (DOCS / "index.md").is_file(): + if os.environ.get("CI"): + raise AssertionError("CI generates docs/ before the suite") + raise unittest.SkipTest("docs/ not generated (python scripts/generate_site.py)") + skipped = set(generate_site.LOCAL_NOTE_DIRS) + return [p for p in sorted(DOCS.rglob("*.md")) + if not set(p.relative_to(DOCS).parts[:-1]) & skipped] + + +def _level(page: Path) -> str: + return str(generate_site._front_matter(page.read_text(encoding="utf-8")).get("level") or "") + + +class FiveTabs(unittest.TestCase): + """Twelve tabs put the walkthrough at position eleven, under "Wiki".""" + + def test_the_tabs_in_reading_order(self): + nav = generate_site.generate_mkdocs_nav({}, {}, {}, {}) + self.assertEqual( + [next(iter(tab)) for tab in nav], + ["Start here", "Your setup", "Explore", "Reference", "Contribute"], + ) + + def test_each_tab_opens_on_an_index_or_a_first_page(self): + nav = generate_site.generate_mkdocs_nav({}, {}, {}, {}) + tabs = {next(iter(tab)): next(iter(tab.values())) for tab in nav} + self.assertEqual(tabs["Start here"][0], "index.md") + self.assertEqual(tabs["Your setup"][0], "platforms/index.md") + self.assertEqual(tabs["Reference"][0], "wiki/index.md") + self.assertEqual(tabs["Contribute"][0], "contributing.md") + + def test_the_breadcrumbs_are_on(self): + source = (REPO_ROOT / "scripts" / "generate_site.py").read_text(encoding="utf-8") + self.assertIn(" - navigation.path\n", source) + + +class EveryPlatformOnThePicker(unittest.TestCase): + """KNULLI, MinUI, AmberELEC, EmuELEC, NextUI and MyMinUI were on no table + of the download page: a hand-written list leaves out what it forgets.""" + + def test_every_platform_has_a_group_and_a_guide(self): + registry = load_platform_registry(str(REPO_ROOT / "platforms")) + for key in _registered(): + with self.subTest(platform=key): + self.assertIn(registry[key].get("device_group"), DEVICE_GROUPS) + self.assertIn(key, GUIDES) + + def test_every_group_has_an_anchor(self): + self.assertEqual(set(GROUP_ANCHORS), set(DEVICE_GROUPS)) + + def test_every_platform_has_a_row(self): + page = _which_pack() + for key in _registered(): + name = load_platform_config(key, str(REPO_ROOT / "platforms"))["platform"] + with self.subTest(platform=key): + self.assertIn(f"| [{name}](platforms/{key}.md) |", page) + + def test_every_platform_is_in_the_navigation_of_its_group(self): + registry = load_platform_registry(str(REPO_ROOT / "platforms")) + coverages = { + key: {"platform": load_platform_config(key, str(REPO_ROOT / "platforms"))["platform"]} + for key in _registered() + } + nav = generate_site.generate_mkdocs_nav(coverages, {}, {}, registry) + setup = next(tab["Your setup"] for tab in nav if "Your setup" in tab) + groups = {next(iter(item)): next(iter(item.values())) + for item in setup if isinstance(item, dict)} + for key in coverages: + label = DEVICE_GROUPS[registry[key]["device_group"]][0] + with self.subTest(platform=key): + self.assertIn({coverages[key]["platform"]: f"platforms/{key}.md"}, groups[label]) + + def test_handheld_systems_that_run_retroarch_are_named(self): + """muOS, ArkOS and the rest are detected as RetroArch by the + installer and were named on no page a user reads.""" + page = _which_pack() + self.assertIn("| muOS | [RetroArch](platforms/retroarch.md) | `/run/muos/storage/bios` |", page) + self.assertNotIn("| TrimUI |", page) + + +class SampleCoresLeaveTheNavigation(unittest.TestCase): + """13 libretro sample cores (testsw, testgl_ff...) sat in the emulators + navigation among the cores people look up.""" + + def test_a_core_with_no_system_and_no_file_is_left_out(self): + profiles = { + "testsw": {"emulator": "testsw", "type": "libretro", "systems": []}, + "duckstation": {"emulator": "DuckStation", "type": "standalone", + "systems": ["sony-playstation"], "files": [{"name": "a.bin"}]}, + "zx": {"emulator": "zx81", "type": "libretro", "systems": ["sinclair-zx81"]}, + } + nav = generate_site._emulator_nav(profiles) + self.assertEqual(nav[0], "emulators/index.md") + groups = {next(iter(group)): next(iter(group.values())) for group in nav[1:]} + self.assertEqual(groups, { + "D to F": [{"DuckStation": "emulators/duckstation.md"}], + "V to Z": [{"zx81": "emulators/zx.md"}], + }) + + +class EveryPageHasALevel(unittest.TestCase): + def test_wiki_sources_declare_one(self): + for page in sorted((REPO_ROOT / "wiki").glob("*.md")): + level = generate_site._front_matter(page.read_text(encoding="utf-8")).get("level") + with self.subTest(page=page.name): + self.assertIn(level, generate_site.LEVELS) + + def test_every_generated_page_has_one_and_its_label(self): + for page in _docs_pages(): + text = page.read_text(encoding="utf-8") + level = _level(page) + with self.subTest(page=str(page.relative_to(DOCS))): + self.assertIn(level, generate_site.LEVELS) + self.assertIn(generate_site.level_label(level), text) + + def test_the_decoration_keeps_an_authored_level(self): + with tempfile.TemporaryDirectory(dir=REPO_ROOT / "tmp") as tmp: + page = Path(tmp) / "wiki" / "faq.md" + page.parent.mkdir() + page.write_text("---\nlevel: start\n---\n\n# FAQ\n\nQuestions people ask first.\n") + generate_site.decorate_markdown_pages(Path(tmp)) + generate_site.decorate_markdown_pages(Path(tmp)) + text = page.read_text() + self.assertEqual(text.count("level: start"), 1) + self.assertEqual(text.count(generate_site.level_label("start")), 1) + self.assertIn(" boost: 5\n", text) + self.assertIn("# FAQ\n", text) + + +class DeepLinksCarryALabel(unittest.TestCase): + """A beginner page sent its reader to Tools and Architecture as the next + step. A link from a start or setup page to a page two levels deeper or + more says what it leads to, with that page's level label before it on + its line, or in the heading of its section ("## Reference: ...").""" + + def test_start_and_setup_pages(self): + pages = _docs_pages() + levels = {page: _level(page) for page in pages} + offenders = [] + for page, level in levels.items(): + if level not in ("start", "setup"): + continue + text = FENCED.sub("", page.read_text(encoding="utf-8")) + heading = "" + for line in text.splitlines(): + if re.match(r"#{1,3} ", line): + heading = line.lstrip("#").strip() + for match in LINK.finditer(line): + target = (page.parent / match.group(1)).resolve() + if target not in levels: + continue + deeper = levels[target] + if ORDER.index(deeper) - ORDER.index(level) < 2: + continue + label = f"{generate_site.LEVELS[deeper]}:" + if label not in line[: match.start()] and not heading.startswith(label): + offenders.append(f"{page.relative_to(DOCS)}: {match.group(0)}") + self.assertEqual(offenders, [], "\n".join(offenders[:30])) + + +class GuidePagesEndOnANextStep(unittest.TestCase): + def test_home_picker_and_platform_page(self): + for page in ("index.md", "which-pack.md", "platforms/batocera.md", + "wiki/first-install.md", "wiki/troubleshooting.md", + "wiki/what-are-bios-files.md", "wiki/faq.md"): + path = DOCS / page if page.startswith(("index", "which", "platforms")) else REPO_ROOT / page + if not path.is_file(): + _docs_pages() + with self.subTest(page=page): + text = path.read_text(encoding="utf-8") + self.assertTrue('!!! tip "Next"' in text or "## What comes next" in text) + + def test_the_platform_page_opens_on_its_guide(self): + """The folder came after four counters and a metadata table, and was + the pack-relative `bios/` where Batocera reads /userdata/bios/.""" + text = (DOCS / "platforms" / "batocera.md").read_text(encoding="utf-8") \ + if (DOCS / "platforms" / "batocera.md").is_file() else _docs_pages() + order = [text.index(h) for h in ( + "## Where the files go", "## Install", "## Download", "## How to check", + "## Reference: every file by system", + )] + self.assertEqual(order, sorted(order)) + self.assertLess(text.index("`/userdata/bios/`"), text.index("rb-stat-value")) + + +if __name__ == "__main__": + unittest.main() diff --git a/wiki/adding-a-scraper.md b/wiki/adding-a-scraper.md index d67ad4ae..29559ce8 100644 --- a/wiki/adding-a-scraper.md +++ b/wiki/adding-a-scraper.md @@ -1,3 +1,7 @@ +--- +level: contribute +--- + # Adding a scraper How to create or modify a scraper for fetching BIOS requirements from upstream diff --git a/wiki/advanced-usage.md b/wiki/advanced-usage.md index 2780d730..7d06867b 100644 --- a/wiki/advanced-usage.md +++ b/wiki/advanced-usage.md @@ -1,7 +1,26 @@ -# Advanced usage +--- +level: reference +--- + +# Build your own packs Fine-grained control over pack generation, hardware filtering, truth analysis, and verification. +The [latest release](https://github.com/Abdess/retrobios/releases/latest) +carries one pack for each platform, and that pack holds everything the +platform runs: the BIOS list it publishes, plus every file its emulator cores +load, read from their source code. There is no lighter variant to choose, +because a pack that leaves out a file a core needs means a game that does not +boot, with no message saying why. It is not a guarantee either: source +profiles can document files nobody has dumped or that only the user can +provide, all of them visible in the [gap analysis](../gaps.md). + +From a clone of the repository, `python scripts/generate_pack.py --platform +retroarch --region us` builds a narrower pack, `--required-only` the bare +minimum each core needs to start, and `--no-game-data` keeps the firmware and +leaves out the files a game reads. `--help` lists every way to build your own, +and the sections below describe each one. + ## Custom packs ### Build from hash diff --git a/wiki/architecture.md b/wiki/architecture.md index dc301edf..04b4e692 100644 --- a/wiki/architecture.md +++ b/wiki/architecture.md @@ -1,3 +1,7 @@ +--- +level: reference +--- + # Architecture ## Principles diff --git a/wiki/community-tools.md b/wiki/community-tools.md index a0f04e2c..94ec875b 100644 --- a/wiki/community-tools.md +++ b/wiki/community-tools.md @@ -1,4 +1,8 @@ -# Community tools +--- +level: reference +--- + +# Projects using this data Projects built on RetroBIOS data. diff --git a/wiki/faq.md b/wiki/faq.md index 3936a690..04878b16 100644 --- a/wiki/faq.md +++ b/wiki/faq.md @@ -1,89 +1,19 @@ +--- +level: start +--- + # FAQ -## My game shows a black screen +The questions people ask before installing anything. Questions about how files +are identified and checked are in the +Reference: [technical questions](technical-faq.md). -Most likely a missing or incorrect BIOS file. Check what is in the BIOS folder: +## Do I need BIOS files? -```bash -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check -``` - -The count it ends on says what is wrong. A number after `wrong hash` means a -file is there whose contents the platform will reject, so it is the wrong -version or a bad dump, and running the install again replaces it. Files that -are absent are counted as needing downloading. - -Some cores also support HLE (see below), so a missing BIOS may not always be the cause. Check the emulator's logs for error messages. - -## What's the difference between required and optional? - -**Required** means the emulator will not start games for that system without the file. **Optional** means the emulator works without it, but with reduced accuracy or missing features (e.g., boot screen animation, wrong font rendering, or degraded audio). - -In verification output, missing required files appear as CRITICAL or WARNING depending on the platform. Missing optional files appear as WARNING or INFO. - -## What's HLE? - -HLE (High-Level Emulation) is a software reimplementation of what the original BIOS does. Some cores can boot games without a real BIOS file by using their built-in HLE fallback. The trade-off is lower accuracy: some games may have glitches or fail to boot entirely. - -When a core has HLE support, the verification tool lowers the severity of a missing BIOS to INFO. The file is still included in packs because the real BIOS gives better results. - -## Why are there multiple hashes for the same file? - -Two main reasons: - -1. **Regional variants.** The same filename (e.g., `IPL.bin` for GameCube) exists in different versions for USA, Europe, and Japan. Each region has a different hash. -2. **Revision differences.** Console manufacturers released updated BIOS versions over time. A PlayStation SCPH-5501 BIOS differs from a SCPH-7001. - -Platforms that verify by MD5 accept specific hashes. If yours doesn't match any known hash, it may be a bad dump or an uncommon revision. - -A filename is never allowed to stand in for a declared hash. When a file entry -carries a hash, resolution matches on content and reports `hash_mismatch` if the -only same-named copy disagrees. What happens next depends on the platform: an -MD5 or SHA1 platform would reject the file, so the pack omits it and says why, -while an existence platform only checks presence, so the file ships and the divergence -is reported instead. - -## Why are there files that aren't BIOS? - -A file earns its place when an emulator loads it from disk and does not bundle it. That covers actual BIOS ROMs, console firmware, arcade BIOS sets, and also game data like `prboom.wad` for the PrBoom core or soundfonts for EasyRPG. - -This follows the platforms themselves: Batocera lists `prboom.wad` in its BIOS checker, and libretro declares these files in the `firmware` field of its core info. The word "BIOS" is their vocabulary for "external file the emulator needs". Files are tagged by category (`bios`, `game_data`, `bios_zip`) in the emulator profiles, so the distinction stays visible. - -## Are these files verified against original hardware dumps? - -No, and that is a deliberate boundary. The hashes here document what -emulator code loads and accepts, read from the source. When several -revisions or regions pass, all of them are kept (`.variants/` in the -repository, per-emulator pages list each one), and the pack ships the -variant satisfying the most checks, logging any conflict it cannot -resolve. - -Whether a file byte-matches the original hardware ROM is a different -question, answered by dump-preservation catalogs such as No-Intro, -Redump, and TOSEC, not by emulator code. The collection is -cross-referenced against their DATs: files matching a catalog entry -carry a verified dump badge on the system pages, and catalog entries -absent from the collection are tracked as acquisition targets. When -the two views differ, this project follows the code, because that is -what decides whether your emulator boots. - -The [dump provenance](../provenance.md) page has the current coverage -per catalog and the full list of catalogued dumps still missing. - -## Which MAME version do the arcade BIOS sets match? - -Arcade BIOS sets are coupled to the romset version, so there is one -profile per MAME core generation: MAME 2000 (0.37b5), MAME 2003 (0.78), -MAME 2003-Plus, MAME 2009 (0.135u4), MAME 2010 (0.139), MAME 2015 -(0.160), MAME 2016 (0.174), and current MAME (0.289), each read from that -version's own source tree, because the BIOS root sets differ between -versions. Derivatives and ports carry their own profiles on top of those -generations: `mamearcade`, `mamemess`, `mame2003_midway`, the three -`mame4droid` drops, `advancemame`, `groovymame` and `hbmame`. Recalbox -compiles two MAME versions of its own, 0.258 and 0.278, profiled as -`mame0258` and `mame0278`. The pack -ships the sets matching the core you run, and a generic unversioned arcade -pack cannot do that. +For many consoles, yes: PlayStation, PlayStation 2, Saturn, Mega-CD, Neo Geo, +Dreamcast and many home computers. For cartridge consoles such as the NES or +the Super Nintendo, usually not. [What BIOS files are](what-are-bios-files.md) +explains why, console by console. ## How do I know which BIOS I need? @@ -117,7 +47,23 @@ curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh -o less install.sh && sh install.sh ``` -See [Installer](installer.md) for the full behaviour. +Details: [All installer options](installer.md), with the full behaviour. + +## Will the installer change my other files? + +No. It writes only inside the BIOS folder, and games and saves are never +touched. Inside that folder, a BIOS file whose contents differ from the +expected one is replaced with a checked copy. Files the installer does not +know are never read, moved or deleted. Copies into the folders of standalone +emulators happen only when you ask for them with `--standalone-copies`. + +## What if Windows or an antivirus blocks the command? + +The PowerShell execution policy governs script files, and the PowerShell line +writes none, so the policy does not block it. Where a security tool still +blocks it, download the pack of your setup from +[Get your files](../which-pack.md) and extract it into the BIOS folder: it holds +the same files. ## Can I run the installer again? @@ -127,6 +73,11 @@ replaced by the verified copy, and files the manifest does not name are never read, moved or deleted. `--check` does the same inspection and exits without writing. +## Can I undo it? + +Yes. Delete the files from your BIOS folder. The installer changed nothing +outside that folder. + ## Can I install onto an SD card or another machine's drive? Yes, `--dest` takes any path and the one-liner forwards arguments: @@ -140,7 +91,17 @@ curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh \ RetroArch on Switch, `RetroArch/system` on Android. On Windows the arguments need a script block, since `iex` would read them as -its own. The form is on the [Installer](installer.md#passing-options) page. +its own. Details: [the Windows form](installer.md#passing-options). + +## Can I use BIOS from one platform on another? + +Yes. BIOS files are console-specific, not platform-specific. A PlayStation BIOS works in RetroArch, Batocera, Recalbox, and any other platform that emulates PlayStation. The only differences between platforms are: + +- **Where the file goes** (each platform has its own BIOS directory) +- **What filename is expected** (usually the same, occasionally different) +- **How verification works** (MD5 check vs. existence check) + +The packs differ per platform because each platform declares its own set of supported systems and expected files. ## Why is a file the emulator reads not in the pack? @@ -153,6 +114,47 @@ a standalone PPSSPP pack does not. When a platform's own package installs data beside a core (Batocera with blueMSX), that platform's pack leaves it out too. The emulator's page on the site marks these files as installed with the emulator. +## How often are packs updated? + +Upstream sources (libretro System.dat, batocera-systems, etc.) are re-scraped by hand and the diff is reviewed before it lands. Releases happen as needed when new BIOS files are added or profiles are updated. + +## What's the difference between required and optional? + +**Required** means the emulator will not start games for that system without +the file. **Optional** means the emulator works without it, but with reduced +accuracy or missing features (e.g., boot screen animation, wrong font +rendering, or degraded audio). + +## What's HLE? + +HLE (High-Level Emulation) is a software reimplementation of what the original +BIOS does. Some cores can boot games without a real BIOS file by using their +built-in HLE fallback. The trade-off is lower accuracy: some games may have +glitches or fail to boot entirely. The file is still included in packs because +the real BIOS gives better results. + +## What's a hash? + +A hash is a fingerprint computed from a file's contents. If even one byte +differs, the hash changes completely. Batocera, Recalbox, EmuDeck and several +other setups compare the MD5 hash of each BIOS file with their list and refuse +a file that does not match. RetroArch only checks that the file is there. The +kinds of hash and where each is used are in the +Reference: [technical questions](technical-faq.md#whats-a-hashchecksum). + +## Are these files verified against original hardware dumps? + +The hashes here come from what emulator code loads and accepts. Whether a file +matches a catalogued dump of the original chip is a separate question, and the +system pages mark the files that do. The full answer is in the +Reference: [technical questions](technical-faq.md#are-these-files-verified-against-original-hardware-dumps). + +## My game shows a black screen + +Most often a BIOS file is missing, or is not the version the emulator expects. +[When a game does not start](troubleshooting.md) goes through it step by step, +starting with a check of what is in your BIOS folder. + ## Is this legal? Redistributing firmware is not settled law, and this page does not pretend otherwise. What follows is the reasoning the project acts on, with the strength of each argument stated plainly so anyone can weigh it. None of it is legal advice, and none of it has been tested in court. @@ -196,47 +198,8 @@ Age is not what decides inclusion here: availability is. Firmware distribution d The project tries to archive files while they are still available rather than after they have become rare. "Retro" is a moving window: the PlayStation 2 was current hardware when its BIOS was first preserved, and today's consoles will be tomorrow's retro systems. A file is included for the same reason at any age: an emulator needs it, and nothing guarantees it stays available. -## What's a hash/checksum? +!!! tip "Next" -A hash is a fixed-length fingerprint computed from a file's contents. If even one byte differs, the hash changes completely. Every file in the database carries these: - -| Type | Length | Example | Used for | -|------|--------|---------|----------| -| SHA1 | 40 hex chars | `10155d8d6e6e832d8ea1571511e40dfb15fede05` | database primary key, BizHawk, installer downloads | -| MD5 | 32 hex chars | `924e392ed05558ffdb115408c263dccf` | most platform verification | -| SHA256 | 64 hex chars | `9a1c...` | emulator profiles whose upstream publishes SHA256 | -| CRC32 | 8 hex chars | `2F468B96` | ROM-set matching, arcade DATs | - -Emulator profiles add Adler-32 where the code checks it, which is how Dolphin -checks its DSP ROMs (`dsp_rom.bin` and `dsp_coef.bin`, hashed byte-swapped). -`IPL.bin` is not one of them: Dolphin loads it without a hash check. - -Verification uses whichever one the platform itself uses: MD5 for Batocera, -RetroBat, Recalbox, EmuDeck, RetroDECK, RomM, ROCKNIX, KNULLI, AmberELEC, EmuELEC -and MiSTer FPGA, SHA1 for -BizHawk, and nothing at all for RetroArch, Lakka and RetroPie, which only check -that the file exists. - -## Why does my verification report say UNTESTED? - -`untested` means the file exists on disk but its hash does not match the expected value. This happens on MD5 and SHA1 platforms (Batocera, Recalbox, BizHawk, ROCKNIX, MiSTer FPGA, and the rest) when the file is present but contains different data than what the platform declares. - -On existence-mode platforms (RetroArch, Lakka, RetroPie, MinUI, NextUI, MyMinUI), files are never `untested` because the platform only checks presence, not content. Those files show as `ok` if present, whatever they contain. - -Running `verify.py --emulator --verbose` shows the emulator-level ground truth, which can confirm whether the file's hash matches what the source code expects. The platform report applies the same check on its own: `verify.py --platform retroarch` prints a `DISCREPANCY` line for every file the platform accepts and a profiled emulator rejects. - -It reaches only as far as the profiles do. Files no profile states a value for are platform-only, and nothing can be said about their content. The footer of the platform report gives that ratio as `Ground truth: N/M files have emulator validation`. - -## Can I use BIOS from one platform on another? - -Yes. BIOS files are console-specific, not platform-specific. A PlayStation BIOS works in RetroArch, Batocera, Recalbox, and any other platform that emulates PlayStation. The only differences between platforms are: - -- **Where the file goes** (each platform has its own BIOS directory) -- **What filename is expected** (usually the same, occasionally different) -- **How verification works** (MD5 check vs. existence check) - -The packs differ per platform because each platform declares its own set of supported systems and expected files. - -## How often are packs updated? - -Upstream sources (libretro System.dat, batocera-systems, etc.) are re-scraped by hand and the diff is reviewed before it lands. Releases happen as needed when new BIOS files are added or profiles are updated. + - [Get your files](../which-pack.md) + - [When a game does not start](troubleshooting.md) + - [Glossary](glossary.md) diff --git a/wiki/first-install.md b/wiki/first-install.md index fc421357..d5dd03f3 100644 --- a/wiki/first-install.md +++ b/wiki/first-install.md @@ -1,3 +1,7 @@ +--- +level: start +--- + # First install One path, from an empty BIOS folder to files the emulator can read. Every step @@ -6,6 +10,8 @@ shows what the screen looks like when it worked. If something on this page does not match what happens, the last section covers the cases that come up. + + ## Before starting Three things, and a few minutes: @@ -25,6 +31,11 @@ touched. ## Step 1. Run the command +The command runs in a terminal. On Windows, open the Start menu, type +PowerShell and open it. On a Mac, open Terminal from Applications, then +Utilities. On a Steam Deck, switch to Desktop Mode and open Konsole. Paste the +line and press Enter. + On Linux, macOS, a Steam Deck, or a handheld running Batocera, Recalbox, KNULLI, ROCKNIX, AmberELEC, EmuELEC or Lakka: @@ -75,6 +86,19 @@ SHA-256 the command checked before running it. That path is the one thing worth reading twice. If it is not where the emulator keeps its BIOS files, stop with Ctrl+C and see the last section. +At a terminal, the installer then asks what to install: + +``` + Enter) install everything + i) only what the installed cores read + c) choose systems, cores and regions + q) quit +``` + +Enter installs everything, which is what the rest of this page shows. **c** +lets you pick consoles, emulator cores and regions. The **i** line appears +only when the installer finds emulator cores installed. + ## Step 3. Let it work The examples on this page follow the Batocera list as published on 2026-10-21. @@ -181,8 +205,8 @@ curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh \ | sh -s -- --platform retroarch --dest /path/to/system ``` -The accepted names come from `--list-platforms`. For where each platform keeps -its BIOS folder, see [Getting started](getting-started.md). +The accepted names come from `--list-platforms`. The page of each setup, +listed on [Get your files](../which-pack.md), gives its BIOS folder. ### Error: Python 3.8 or newer is required. @@ -222,11 +246,16 @@ much is free. `--dest` installs to another drive. A file can be in place and still be the wrong one for that game, or the console may need a file that nobody has dumped yet. -[Troubleshooting](troubleshooting.md) goes through that case. +[When a game does not start](troubleshooting.md) goes through that case. ## What comes next -- [Troubleshooting](troubleshooting.md), when a game does not start +- The page of your setup, from [Get your files](../which-pack.md): where its + folder is and how the setup itself checks the files +- [When a game does not start](troubleshooting.md) - [FAQ](faq.md), for what the words mean and why a pack is this large -- [Installer](installer.md), for every option, what it detects and what it is - allowed to write +- Details: [All installer options](installer.md), for every option, what it + detects and what it is allowed to write + +To remove what the installer wrote, delete the files from the BIOS folder. +Nothing outside it was changed. diff --git a/wiki/getting-started.md b/wiki/getting-started.md index 74730485..1e763bf6 100644 --- a/wiki/getting-started.md +++ b/wiki/getting-started.md @@ -1,312 +1,28 @@ +--- +level: start +--- + # Getting started +What this page held now lives on the pages written for each step. + ## What are BIOS files? -BIOS files are firmware dumps from original console hardware. Emulators need them to boot games for systems that relied on built-in software (PlayStation, Saturn, Dreamcast, etc.). Without the correct BIOS, the emulator either refuses to start the game or falls back to less accurate software emulation. +[What BIOS files are](what-are-bios-files.md) explains what they do, and which +consoles need them. ## Installation -Three ways to get BIOS files in place. The installer is the one to use, -and the two sections after it are for when it cannot run. For a walkthrough of a -first install with what each step prints, see -[First install](first-install.md). - -### Option 1: the installer (recommended) - -One command, nothing to clone. The bootstrap verifies `install.py` against the -SHA-256 embedded in it, then the installer detects the platform and its BIOS -directory, downloads only what is missing or incorrect, checks size and -SHA-256/SHA-1 before writing, and installs each file atomically. - -```bash -# Linux / macOS / Steam Deck -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh - -# Windows (PowerShell) -irm https://raw.githubusercontent.com/Abdess/retrobios/main/install.ps1 | iex - -# Android, from Termux, after pkg install python and termux-setup-storage -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -``` - -Running `install.py` directly works the same way and needs nothing beyond -Python 3.8+, the floor both bootstraps enforce before they hand over: - -```bash -python install.py -``` - -Override detection when needed: - -```bash -python install.py --platform retroarch --dest ~/custom/bios -python install.py --target switch # keep only files for that hardware -python install.py --list-systems # systems a platform holds, with sizes -python install.py --system sony-playstation,nintendo-gba -python install.py --region us,eu,jp # one BIOS per region, best first -python install.py --no-input # ask nothing, even at a terminal -python install.py --check # verify existing files, download nothing -python install.py --list-platforms # supported platforms and what was detected -python install.py --list-targets # hardware targets for a platform -python install.py --jobs 4 # parallel downloads (default 8) -python install.py --verbose -python install.py --standalone-copies # opt in to extra standalone-emulator paths -``` - -The default flow writes inside the detected platform tree only. Copies into -separate standalone-emulator directories are opt-in, so discovery cannot cause -unexpected writes elsewhere on the machine. Entries the collection cannot -satisfy are reported as safely omitted and the run continues. The installer -never substitutes a same-named file for one a hash-verifying platform would -reject. - -Arguments pass through the one-liner too, which is how you target an SD card -mounted on another machine: - -```bash -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh \ - | sh -s -- --platform retroarch --dest /path/to/sdcard/retroarch/cores/system -``` - -`--dest` is the BIOS folder itself, so on a card it names that folder on the -card: `retroarch/cores/system` for RetroArch on Switch, `RetroArch/system` on -Android. - -PowerShell needs another form, the environment overrides and the detection -rules are listed per platform, and the trust boundary is described in full on -the [Installer](installer.md) page. - -### Option 2: download.sh (Linux/macOS, from a clone) - -Downloads a whole pack rather than the missing files. Needs `curl` and `unzip`: - -```bash -bash scripts/download.sh retroarch ~/RetroArch/system/ -bash scripts/download.sh --list # show available packs -``` - -A pack published in several parts is downloaded part by part and checked -against the SHA-256 the release publishes before anything is extracted. -`python scripts/download.py` does the same on Windows. - -### Option 3: manual download - -1. Go to the [releases page](https://github.com/Abdess/retrobios/releases) -2. Download the ZIP pack for your platform -3. Extract to the BIOS directory listed below - -A pack over 2 GB comes in several parts (`Pack.part1of2.zip`, -`Pack.part2of2.zip`), and every part is needed. Each part is an ordinary ZIP: -extract them all into the same folder. See [Download](../which-pack.md) for -the per-setup instructions. +[Get your files](../which-pack.md) gives the command and the packs for every +setup. [First install, step by step](first-install.md) shows what each step of +the command prints. ## BIOS directory by platform -### RetroArch - -RetroArch uses the `system_directory` setting in `retroarch.cfg`. Default locations: - -| OS | Default path | -|----|-------------| -| Windows (installer or portable .7z) | `system\` next to `retroarch.exe` | -| Windows (UWP/MSIX package) | `%LOCALAPPDATA%\Packages\1e4cf179-f3c2-404f-b9f3-cb2070a5aad8_*\LocalState\system\` | -| Linux | `~/.config/retroarch/system/` | -| Linux (Flatpak) | `~/.var/app/org.libretro.RetroArch/config/retroarch/system/` | -| macOS | `~/Library/Application Support/RetroArch/system/` | -| Steam Deck (Flatpak) | `~/.var/app/org.libretro.RetroArch/config/retroarch/system/` | -| Android | `/storage/emulated/0/RetroArch/system/` | - -On Windows, `retroarch.cfg` is read beside `retroarch.exe` first, then from -`%APPDATA%\RetroArch\`. Either way, a `system_directory` left at its default is -the `system` folder beside `retroarch.exe` (`:\system` in RetroArch's -`frontend/drivers/platform_win32.c`, where `:` stands for the folder of the -executable). - -To check your actual path: open RetroArch, go to **Settings > Directory > System/BIOS**, or look for `system_directory` in `retroarch.cfg`. - -### Batocera - -``` -/userdata/bios/ -``` - -Accessible via network share at `\\BATOCERA\share\bios\` (Windows) or `smb://batocera/share/bios/` (macOS/Linux). - -### Recalbox - -``` -/recalbox/share/bios/ -``` - -Accessible via network share at `\\RECALBOX\share\bios\`. - -### RetroBat - -``` -bios/ -``` - -Relative to the RetroBat installation directory (e.g., `C:\RetroBat\bios\`). - -### RetroDECK - -``` -~/retrodeck/bios/ -``` - -On a MicroSD install the root moves, e.g. `/run/media/mmcblk0p1/retrodeck/bios/`. - -The RetroDECK pack is the exception to "extract into the BIOS directory": its -entries already start with `bios/` (and one with `roms/`), so extract it into -`~/retrodeck/` and the files land in the right place. Extracting into -`~/retrodeck/bios/` would create `~/retrodeck/bios/bios/`. - -### EmuDeck - -``` -Emulation/bios/ -``` - -Located inside your Emulation folder. On Steam Deck, typically `~/Emulation/bios/`. - -### Lakka - -``` -/storage/system/ -``` - -Accessible via SSH or Samba. - -### RetroPie - -``` -~/RetroPie/BIOS/ -``` - -RetroPie is archived in this project: its configuration is kept and packs are -still built, but the upstream data is no longer scraped on a schedule. Use the -RetroPie pack, not the RetroArch one: RetroPie inherits RetroArch's systems -but also installs standalone emulators (openMSX, PCSX2, XRoar, Amiberry and -others) whose files only the RetroPie pack carries, laid out for -`~/RetroPie/BIOS/` rather than RetroArch's `system/`. - -### ROCKNIX - -``` -/storage/roms/bios/ -``` - -Accessible via SSH or Samba, like Lakka. - -### KNULLI - -``` -/userdata/bios/ -``` - -### AmberELEC and EmuELEC - -``` -/storage/roms/ -``` - -Their BIOS lists name files under `/storage/roms`: `bios/` for most, and -`neogeo/` (and `ports/` on EmuELEC) for the rest. The pack holds those -folders, so it is extracted into `/storage/roms/` itself. - -### MiSTer FPGA - -``` -/media/fat/games/ -``` - -Files go under the per-core subdirectory the MiSTer BIOS database declares -(e.g. `/media/fat/games/3DO/boot.rom`). Only the entries the BIOS database -lists with a download URL are in scope: the rest ship with the MiSTer -distribution and install themselves. - -### MinUI, NextUI and MyMinUI - -``` -/Bios// -``` - -These frontends give each core a folder of its own, named after the tag in -parentheses of the Roms folder the game sits in: a game in -`Roms/Sony PlayStation (PS)/` runs with `Bios/PS/` as its system directory, one -in `Roms/Game Boy (GB)/` with `Bios/GB/` (MinUI -`workspace/all/minarch/minarch.c:2943`, NextUI -`workspace/all/minarch/ma_core.c:76`, MyMinUI -`workspace/all/minarch/minarch.c:3532`). The pack holds the `Bios/` folder: -extract it at the root of the card. A core started from two tags (Gambatte for -GB and GBC) gets its files in both folders. - -The installer runs on the computer the card is mounted on, with the card's root -as `--dest`: - -```bash -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --platform minui --dest /media/$USER/CARD -``` - -Without `--platform`, it reads the release name the frontend writes in -`.system/version.txt` on the card. A pak that starts a standalone program -instead of a libretro core (DraStic for NDS, native PICO-8) is not covered. - -### BizHawk - -``` -Firmware/ -``` - -Relative to the BizHawk installation directory. - -### RomM - -BIOS files live in the RomM library under `bios/{platform_slug}/`, one -subfolder per system, and are managed through the web interface. Check the -[RomM documentation](https://github.com/rommapp/romm) for setup details. +The page of each setup opens on where its files go. All setups are on +[Get your files](../which-pack.md), grouped by the device they run on. ## Checking what you installed -`--check` reads the BIOS folder, hashes what is there and writes nothing: - -```bash -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check -``` - -It ends on a count: - -``` -Checking existing files... - 5865/5865 present (5865 verified, 0 wrong hash) - - All files up to date. -``` - -`0 wrong hash` is the line that matters. A number there means a file is present -whose contents the platform will reject, which is how a wrong revision or a bad -dump shows up. - -The installer's `--check` hashes every file against the list on every -platform, so it catches a wrong revision even where the frontend would not: -RetroArch, Lakka, RetroPie, MinUI, NextUI and MyMinUI only check that a file -exists, while Batocera, RetroBat, Recalbox, EmuDeck, RetroDECK, RomM, ROCKNIX, -KNULLI, AmberELEC, EmuELEC and MiSTer FPGA compare -MD5 and BizHawk compares SHA1. - -For a single system, or the files a single emulator core reads (`--list-systems` -and `--list-cores` give the names): - -```bash -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check --system sony-playstation -curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check --emulator beetle_psx -``` - -`--verbose` names every file still to fetch, wrong hashes included. See [Installer](installer.md) -for every option, and [Tools](tools.md) for the repository's own scripts. - -## Next steps - -- [FAQ](faq.md) - common questions and troubleshooting -- [Tools](tools.md) - all available scripts and options -- [Architecture](architecture.md) - how the project works internally +[When a game does not start](troubleshooting.md#checking-what-you-installed) +shows how to check the files in your BIOS folder. diff --git a/wiki/glossary.md b/wiki/glossary.md new file mode 100644 index 00000000..8cd3b40a --- /dev/null +++ b/wiki/glossary.md @@ -0,0 +1,153 @@ +--- +level: start +--- + +# Glossary + +The first part holds the words anyone meets when setting up BIOS files. The +second part holds the words this project uses in its reference pages. + +## Words you will meet + +**BIOS, or firmware** +: A small program built into a console. Many emulators need a copy of it to + start games. [What BIOS files are](what-are-bios-files.md) says more. + +**Emulator** +: A program that lets a computer, a phone or a handheld act like an older + console. + +**Core** +: One emulator inside RetroArch. RetroArch loads a different core for each + console. + +**Frontend** +: The menu you pick and start games from, such as EmulationStation, MinUI or + the RetroArch menu. It starts the emulator for you. + +**Setup, or platform** +: The software you play with: RetroArch, Batocera, EmuDeck, KNULLI and the + others. Each one keeps its BIOS files in a folder of its own. + +**BIOS folder** +: The folder where your setup looks for BIOS files. Its place depends on the + setup, and the page of each setup gives it. + +**Pack** +: One ZIP file with every BIOS file a setup uses. A pack over 2 GB comes in + several parts, and every part is needed. + +**Installer** +: A small program started by one line pasted into a terminal. It finds your + BIOS folder and downloads only the files that are missing. + +**Terminal, or command line** +: A window where you type commands. On Windows it is PowerShell, on a Mac it + is Terminal, on a Steam Deck it is Konsole. + +**Hash, or checksum (MD5, SHA1)** +: A fingerprint of a file's contents. Two files with the same hash hold the + same bytes. Some setups use it to refuse a file that is not the expected + version. + +**Region** +: The market a console was sold in, such as Japan, North America or Europe. + Some emulators pick the BIOS of the same region as the game. + +**Required and optional** +: Required: the emulator does not start games without the file. Optional: + games start without it, with small differences such as a missing start-up + screen. + +**HLE** +: High-level emulation: code in the emulator that stands in for the BIOS. + Many games work with it, some glitch or do not start. + +**Game data** +: Files of a game that a game engine reads, such as the data of Doom or + Diablo. They come from your own copy of the game, and their absence does not + affect any console. + +## Project terms + +**Profile** +: A YAML file in `emulators/` that documents what one emulator loads, read + from its source code. + +**Ground truth** +: The emulator's source code, the reference every profile is checked against. + +**System** +: A console or computer being emulated, named by an id such as + `sony-playstation` or `nintendo-gba`. + +**Verification mode** +: How a setup checks a file: by its name only (existence), by MD5 or by SHA1. + Reference: [How each platform checks files](verification-modes.md). + +**Status** +: The outcome of one file check: `ok`, `untested` (present, but not the + expected hash) or `missing`. + +**Severity** +: How much a check result matters: OK, INFO (negligible), WARNING (degraded), + CRITICAL (the emulator cannot start games). + +**Discrepancy** +: A file that passes the setup's own check but fails the emulator's size or + hash check. + +**hle_fallback** +: A flag on a profile entry: the emulator has HLE for this file, so its + absence is reported at INFO. + +**Variant** +: Another version of a BIOS file (a revision, a region or another dump), kept + in `.variants/` in the repository. + +**Cross-reference** +: The comparison of emulator profiles with a setup's own list, which finds the + files the setup's emulators load and its list does not name. + +**Shared group** +: A group of files in `_shared.yml` that several setups include, with the + folder an emulator expects. + +**Data directory** +: A whole folder an emulator needs, such as Dolphin's `Sys/` or the PPSSPP + assets, kept in `data/` and not indexed in the database. + +**Storage tier** +: Where a file comes from: `embedded` (in `bios/`), `external` (downloaded at + build time) or `user_provided`. + +**Target** +: A hardware architecture a setup runs on, such as `switch`, `rpi4` or + `x86_64`. + +**Manifest** +: The JSON list of files per setup in `install/`, which the installer reads. + +**Scraper** +: A script that reads a setup's own BIOS list (System.dat, es_bios.xml and + others) into this project's format. + +**Exporter** +: A script that writes the project's data back in a setup's own format. + +**Truth** +: A setup's file list generated from the emulator profiles, compared with the + list the setup publishes. + +**Dump catalog** +: A preservation project (Redump, No-Intro, TOSEC) that publishes lists of + verified hardware dumps. + +**Provenance** +: The catalogs that list a file, joined to the database by hash. It informs, + it never overrides the emulator code. + +!!! tip "Next" + + - [What BIOS files are](what-are-bios-files.md) + - [Get your files](../which-pack.md) diff --git a/wiki/index.md b/wiki/index.md index fc3e4ded..755d1ee3 100644 --- a/wiki/index.md +++ b/wiki/index.md @@ -1,75 +1,59 @@ -# Wiki +--- +level: reference +--- -Technical documentation for the RetroBIOS toolchain. +# Reference -Pages are grouped by what you came to do. **For users** walks through -installing and checking files. **Technical reference** describes how the -toolchain behaves, one subject per page. **For contributors** is task-oriented: -each page takes one job from start to finish. +These pages describe how the files are checked, what the data holds and how the +toolchain works. To set up BIOS files on a device, start with +[Get your files](../which-pack.md). -## For users +## How the files are checked -- **[First install](first-install.md)** - one path from an empty folder to working files, with what each step prints -- **[Getting started](getting-started.md)** - installation, BIOS directory paths per platform, verification -- **[Installer](installer.md)** - the one-liner in full: bootstrap, options, environment overrides, platform detection, trust boundary -- **[Troubleshooting](troubleshooting.md)** - diagnosis by symptom: missing BIOS, hash mismatch, pack issues -- **[FAQ](faq.md)** - common questions, hash explanations +- **[How each platform checks files](verification-modes.md)** - existence, MD5 and SHA1 modes, severity matrix, resolution chain, reading `verify.py` output +- **[What the packs do not hold](../gaps.md)** - files emulators load that no pack can carry yet, by platform and by emulator +- **[Cross-reference](../cross-reference.md)** - files the emulators of a platform load and its own list does not name +- **[Dump provenance](../provenance.md)** - how the collection lines up against the No-Intro, Redump and TOSEC catalogs +- **[Updated platform files](../exports.md)** - each platform's own file, with the changes the emulator source supports +- **[Technical questions](technical-faq.md)** - multiple hashes, MAME versions, UNTESTED, dump verification -If you just want to download BIOS packs, see the [home page](../index.md). +## Data -## Technical reference - -- **[Architecture](architecture.md)** - directory structure, data flow, platform inheritance, pack grouping, security, edge cases, CI workflows -- **[Tools](tools.md)** - CLI reference for every script, pipeline usage, scrapers -- **[Advanced usage](advanced-usage.md)** - custom packs, region filtering, one file per slot, target filtering, truth generation, emulator verification, offline workflow -- **[Verification modes](verification-modes.md)** - how each platform verifies BIOS files, severity matrix, resolution chain +- **[Data and API](../data.md)** - JSON, CSV and SQLite exports with their checksums +- **[Find a file by its SHA1](../files/index.md)** - the permanent address of a file by its contents - **[Data model](data-model.md)** - database.json structure, indexes, file resolution order, YAML formats -See also [dump provenance](../provenance.md) for how the collection lines up -against the No-Intro, Redump and TOSEC catalogs. +## The toolchain + +- **[Architecture](architecture.md)** - directory structure, data flow, platform inheritance, pack grouping, security, edge cases, CI workflows +- **[Build your own packs](advanced-usage.md)** - custom packs, region filtering, one file per slot, target filtering, truth generation, emulator verification, offline workflow +- **[Command-line tools](tools.md)** - every script, pipeline usage, scrapers +- **[Projects using this data](community-tools.md)** - projects built on RetroBIOS data + +## Methodology + +Platform lists are checked against emulator source code, file by file where a +profile exists. Documentation and metadata can differ from what a program does +at run time, so the source is the primary reference: + +1. **Upstream emulator source**: what the original project loads (Dolphin, + PCSX2, Mednafen and others) +2. **Libretro core source**: the RetroArch port, which may adapt paths or add + files +3. **`.info` declarations**: metadata that platforms rely on, checked against + the two above ## For contributors -- **[Profiling guide](profiling.md)** - create an emulator profile from source code, YAML field reference +- **[Profiling an emulator](profiling.md)** - create an emulator profile from source code, YAML field reference - **[Adding a platform](adding-a-platform.md)** - scraper, registry, YAML config, exporter, target scraper, install detection - **[Adding a scraper](adding-a-scraper.md)** - plugin architecture, BaseScraper, parsers, target scrapers -- **[Testing guide](testing-guide.md)** - run tests, fixture pattern, how to add tests, CI integration +- **[Testing](testing-guide.md)** - run tests, fixture pattern, how to add tests, CI integration - **[Release process](release-process.md)** - CI workflows, large files, manual release See [contributing](../contributing.md) for submission guidelines. -## Community - -- **[Community tools](community-tools.md)** - projects built on RetroBIOS data - ## Glossary -- **BIOS** - firmware burned into console hardware, needed by emulators that rely on original boot code -- **firmware** - system software loaded by a console at boot, used interchangeably with BIOS in this project -- **HLE** - High-Level Emulation: software reimplementation of BIOS functions, avoids needing the original file -- **hash** - fixed-length fingerprint of a file's contents. This project uses MD5, SHA1, SHA256, CRC32, and Adler-32 -- **platform** - a distribution that packages emulators (RetroArch, Batocera, Recalbox, EmuDeck, etc.) -- **core** - an emulator packaged as a libretro plugin, loaded by RetroArch or compatible frontends -- **profile** - a YAML file in `emulators/` documenting one core's BIOS requirements, verified against source code -- **system** - a game console or computer being emulated (e.g. sony-playstation, nintendo-gameboy-advance) -- **pack** - a ZIP archive containing all BIOS files needed by a specific platform -- **ground truth** - the emulator's source code, treated as the authoritative reference for BIOS requirements -- **cross-reference** - comparison of emulator profiles against platform configs to find undeclared files -- **scraper** - a script that fetches BIOS requirement data from an upstream source (System.dat, es_bios.xml, etc.) -- **exporter** - a script that converts ground truth data back into a platform's native format -- **target** - a hardware architecture that a platform runs on (e.g. switch, rpi4, x86_64, steamos) -- **region** - the territory an emulator's code branches on to select a file. `--region` builds a pack from an ordered priority list, `--one-per-slot` keeps one file per system and region -- **variant** - an alternative version of a BIOS file (different revision, region, or dump), stored in `.variants/` -- **required** - a file the core needs to function, determined by source code behavior -- **optional** - a file the core functions without, possibly with reduced accuracy or missing features -- **hle_fallback** - flag on a file indicating the core has an HLE path, and absence is downgraded to INFO severity -- **severity** - the urgency of a verification result: OK (verified), INFO (negligible), WARNING (degraded), CRITICAL (broken) -- **status** - the outcome of a single file check: `ok`, `untested` (present, hash not the expected one), or `missing` -- **discrepancy** - a file that passes the platform check but fails the emulator's own size or hash validation -- **shared group** - a file group in `_shared.yml` that several platforms include, carrying the destination a core expects -- **data directory** - a whole directory tree a core needs (Dolphin `Sys/`, PPSSPP assets), cached in `data/`, not indexed in the database -- **storage tier** - where a file comes from: `embedded` (in `bios/`), `external` (downloaded at build), `user_provided` -- **truth** - platform-shaped data generated from emulator profiles, used to diff against what the platform declares -- **dump catalog** - a preservation project (Redump, No-Intro, TOSEC) publishing DATs of verified hardware dumps -- **provenance** - the catalogs that list a file, joined into the database by hash. An annotation, never an authority -- **manifest** - the JSON file list per platform in `install/`, consumed by `install.py` +The words these pages use, from BIOS to provenance, are in the +[glossary](glossary.md). diff --git a/wiki/installer.md b/wiki/installer.md index 75521a80..d35b1c54 100644 --- a/wiki/installer.md +++ b/wiki/installer.md @@ -1,9 +1,13 @@ -# Installer +--- +level: explore +--- + +# All installer options Reference for `install.sh`, `install.ps1` and `install.py`: what the one-liner runs, what it detects, where the bytes come from and what it is allowed to -write. For BIOS directory paths per platform, see -[Getting started](getting-started.md). +write. For the BIOS folder of each setup, see +[Get your files](../which-pack.md). ## One line diff --git a/wiki/profiling.md b/wiki/profiling.md index 726e8620..f5a7cba4 100644 --- a/wiki/profiling.md +++ b/wiki/profiling.md @@ -1,3 +1,7 @@ +--- +level: contribute +--- + # Profiling guide How to create an emulator profile from source code. diff --git a/wiki/release-process.md b/wiki/release-process.md index d0a8bcd8..68c81fb6 100644 --- a/wiki/release-process.md +++ b/wiki/release-process.md @@ -1,3 +1,7 @@ +--- +level: contribute +--- + # Release process Two GitHub workflows check pull requests and publish the site. Releases are diff --git a/wiki/technical-faq.md b/wiki/technical-faq.md new file mode 100644 index 00000000..10e7dff9 --- /dev/null +++ b/wiki/technical-faq.md @@ -0,0 +1,112 @@ +--- +level: reference +--- + +# Technical questions + +Questions about how the files are identified and checked. The questions asked +before installing are in the [FAQ](faq.md). + +## How is a missing file ranked? + +In verification output, missing required files appear as CRITICAL or WARNING +depending on the platform. Missing optional files appear as WARNING or INFO. + +When a core has HLE support, the verification tool lowers the severity of a +missing BIOS to INFO. The file is still included in packs because the real BIOS +gives better results. The full table is the +[severity matrix](verification-modes.md#severity-matrix). + +## Why are there multiple hashes for the same file? + +Two main reasons: + +1. **Regional variants.** The same filename (e.g., `IPL.bin` for GameCube) exists in different versions for USA, Europe, and Japan. Each region has a different hash. +2. **Revision differences.** Console manufacturers released updated BIOS versions over time. A PlayStation SCPH-5501 BIOS differs from a SCPH-7001. + +Platforms that verify by MD5 accept specific hashes. If yours doesn't match any known hash, it may be a bad dump or an uncommon revision. + +A filename is never allowed to stand in for a declared hash. When a file entry +carries a hash, resolution matches on content and reports `hash_mismatch` if the +only same-named copy disagrees. What happens next depends on the platform: an +MD5 or SHA1 platform would reject the file, so the pack omits it and says why, +while an existence platform only checks presence, so the file ships and the divergence +is reported instead. + +## Why are there files that aren't BIOS? + +A file earns its place when an emulator loads it from disk and does not bundle it. That covers actual BIOS ROMs, console firmware, arcade BIOS sets, and also game data like `prboom.wad` for the PrBoom core or soundfonts for EasyRPG. + +This follows the platforms themselves: Batocera lists `prboom.wad` in its BIOS checker, and libretro declares these files in the `firmware` field of its core info. The word "BIOS" is their vocabulary for "external file the emulator needs". Files are tagged by category (`bios`, `game_data`, `bios_zip`) in the emulator profiles, so the distinction stays visible. + +## Are these files verified against original hardware dumps? + +No, and that is a deliberate boundary. The hashes here document what +emulator code loads and accepts, read from the source. When several +revisions or regions pass, all of them are kept (`.variants/` in the +repository, per-emulator pages list each one), and the pack ships the +variant satisfying the most checks, logging any conflict it cannot +resolve. + +Whether a file byte-matches the original hardware ROM is a different +question, answered by dump-preservation catalogs such as No-Intro, +Redump, and TOSEC, not by emulator code. The collection is +cross-referenced against their DATs: files matching a catalog entry +carry a verified dump badge on the system pages, and catalog entries +absent from the collection are tracked as acquisition targets. When +the two views differ, this project follows the code, because that is +what decides whether your emulator boots. + +The [dump provenance](../provenance.md) page has the current coverage +per catalog and the full list of catalogued dumps still missing. + +## Which MAME version do the arcade BIOS sets match? + +Arcade BIOS sets are coupled to the romset version, so there is one +profile per MAME core generation: MAME 2000 (0.37b5), MAME 2003 (0.78), +MAME 2003-Plus, MAME 2009 (0.135u4), MAME 2010 (0.139), MAME 2015 +(0.160), MAME 2016 (0.174), and current MAME (0.289), each read from that +version's own source tree, because the BIOS root sets differ between +versions. Derivatives and ports carry their own profiles on top of those +generations: `mamearcade`, `mamemess`, `mame2003_midway`, the three +`mame4droid` drops, `advancemame`, `groovymame` and `hbmame`. Recalbox +compiles two MAME versions of its own, 0.258 and 0.278, profiled as +`mame0258` and `mame0278`. The pack +ships the sets matching the core you run, and a generic unversioned arcade +pack cannot do that. + +## What's a hash/checksum? + +A hash is a fixed-length fingerprint computed from a file's contents. If even one byte differs, the hash changes completely. Every file in the database carries these: + +| Type | Length | Example | Used for | +|------|--------|---------|----------| +| SHA1 | 40 hex chars | `10155d8d6e6e832d8ea1571511e40dfb15fede05` | database primary key, BizHawk, installer downloads | +| MD5 | 32 hex chars | `924e392ed05558ffdb115408c263dccf` | most platform verification | +| SHA256 | 64 hex chars | `9a1c...` | emulator profiles whose upstream publishes SHA256 | +| CRC32 | 8 hex chars | `2F468B96` | ROM-set matching, arcade DATs | + +Emulator profiles add Adler-32 where the code checks it, which is how Dolphin +checks its DSP ROMs (`dsp_rom.bin` and `dsp_coef.bin`, hashed byte-swapped). +`IPL.bin` is not one of them: Dolphin loads it without a hash check. + +Verification uses whichever one the platform itself uses: MD5 for Batocera, +RetroBat, Recalbox, EmuDeck, RetroDECK, RomM, ROCKNIX, KNULLI, AmberELEC, EmuELEC +and MiSTer FPGA, SHA1 for +BizHawk, and nothing at all for RetroArch, Lakka and RetroPie, which only check +that the file exists. + +## Why does my verification report say UNTESTED? + +`untested` means the file exists on disk but its hash does not match the expected value. This happens on MD5 and SHA1 platforms (Batocera, Recalbox, BizHawk, ROCKNIX, MiSTer FPGA, and the rest) when the file is present but contains different data than what the platform declares. + +On existence-mode platforms (RetroArch, Lakka, RetroPie, MinUI, NextUI, MyMinUI), files are never `untested` because the platform only checks presence, not content. Those files show as `ok` if present, whatever they contain. + +Running `verify.py --emulator --verbose` shows the emulator-level ground truth, which can confirm whether the file's hash matches what the source code expects. The platform report applies the same check on its own: `verify.py --platform retroarch` prints a `DISCREPANCY` line for every file the platform accepts and a profiled emulator rejects. + +It reaches only as far as the profiles do. Files no profile states a value for are platform-only, and nothing can be said about their content. The footer of the platform report gives that ratio as `Ground truth: N/M files have emulator validation`. + +!!! tip "Next" + + - [Reference index](index.md), every reference page in one list + - For contributors: [How to contribute](../contributing.md) diff --git a/wiki/testing-guide.md b/wiki/testing-guide.md index fcb8793f..8ed72bfd 100644 --- a/wiki/testing-guide.md +++ b/wiki/testing-guide.md @@ -1,3 +1,7 @@ +--- +level: contribute +--- + # Testing guide The suite is discovered dynamically across focused modules, so its exact count @@ -47,7 +51,7 @@ Every module in `tests/`, by the area it guards. | Handheld port | stubbed PortMaster dialogs, fake installer | `test_port` | | Truth and native export | synthetic + cached originals | `test_export_counts`, `test_export_native_layer`, `test_export_report`, `test_exporters`, `test_truth_data_dir_readers`, `test_truth_diff`, `test_truth_layout` | | Scrapers, upstream and provenance | inline sources, synthetic | `test_check_freshness`, `test_dedup`, `test_elec_scrapers`, `test_fbneo_parser`, `test_find_pinned_revision`, `test_hash_merge`, `test_launchers`, `test_launchers_emudeck`, `test_launchers_recalbox`, `test_launchers_retrobat`, `test_launchers_retrodeck`, `test_launchers_retropie`, `test_launchers_rocknix`, `test_mame_devices`, `test_mame_parser`, `test_mame_ref_audit`, `test_profile_sync`, `test_provenance`, `test_refresh_data_dirs`, `test_refresh_failures`, `test_refresh_stale`, `test_retrodeck_scraper_errors`, `test_romset_recipes`, `test_scraper_contract`, `test_upstream`, `test_upstream_index` | -| Site, docs and repository contracts | rendered pages, sources | `test_contributing_text`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_doc_pages`, `test_doc_surfaces`, `test_optional_dependencies`, `test_prose_style`, `test_site_availability`, `test_site_cores`, `test_site_emulator_page`, `test_site_exports`, `test_site_platform_hashes`, `test_site_render`, `test_site_system_links`, `test_site_validation`, `test_site_write`, `test_validate_pr`, `test_validate_pr_flow`, `test_validate_pr_hashes`, `test_workflow_paths` | +| Site, docs and repository contracts | rendered pages, sources | `test_contributing_text`, `test_data_dir_refs`, `test_dataclass_defaults`, `test_doc_pages`, `test_doc_surfaces`, `test_optional_dependencies`, `test_prose_style`, `test_site_availability`, `test_site_cores`, `test_site_emulator_page`, `test_site_exports`, `test_site_funnel`, `test_site_platform_hashes`, `test_site_render`, `test_site_system_links`, `test_site_validation`, `test_site_write`, `test_validate_pr`, `test_validate_pr_flow`, `test_validate_pr_hashes`, `test_workflow_paths` | ## Test architecture diff --git a/wiki/tools.md b/wiki/tools.md index ce21306b..7b9fab1d 100644 --- a/wiki/tools.md +++ b/wiki/tools.md @@ -1,4 +1,8 @@ -# Tools +--- +level: reference +--- + +# Command-line tools All tools are Python scripts in `scripts/`. They need Python 3.9 or newer and `pyyaml`, and only `validate_schemas.py` also needs `jsonschema[format-nongpl]`. `install.py` diff --git a/wiki/troubleshooting.md b/wiki/troubleshooting.md index 62240b22..8b3be4d5 100644 --- a/wiki/troubleshooting.md +++ b/wiki/troubleshooting.md @@ -1,4 +1,11 @@ -# Troubleshooting +--- +level: start +--- + +# When a game does not start + +Start here when a game does not start, or says that a BIOS file is missing. +Each section is one check, in the order that finds the cause soonest. ## Game won't start / black screen @@ -48,6 +55,45 @@ North American games on cores that enforce region matching. Check the emulator profile for your core to see which regions are supported and which files correspond to each. +## Checking what you installed + +`--check` reads the BIOS folder, hashes what is there and writes nothing: + +```bash +curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check +``` + +It ends on a count: + +``` +Checking existing files... + 5865/5865 present (5865 verified, 0 wrong hash) + + All files up to date. +``` + +`0 wrong hash` is the line that matters. A number there means a file is present +whose contents the platform will reject, which is how a wrong revision or a bad +dump shows up. + +The installer's `--check` hashes every file against the list on every +platform, so it catches a wrong revision even where the frontend would not: +RetroArch, Lakka, RetroPie, MinUI, NextUI and MyMinUI only check that a file +exists, while Batocera, RetroBat, Recalbox, EmuDeck, RetroDECK, RomM, ROCKNIX, +KNULLI, AmberELEC, EmuELEC and MiSTer FPGA compare +MD5 and BizHawk compares SHA1. + +For a single system, or the files a single emulator core reads (`--list-systems` +and `--list-cores` give the names): + +```bash +curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check --system sony-playstation +curl -fsSL https://raw.githubusercontent.com/Abdess/retrobios/main/install.sh | sh -s -- --check --emulator beetle_psx +``` + +`--verbose` names every file still to fetch, wrong hashes included. +Details: [All installer options](installer.md). + ## BIOS not found by emulator The file exists on disk, but the emulator reports it as missing. @@ -100,11 +146,8 @@ name per file: `BIOS.col`, never `bios.col` beside it, since extracting both on Windows or macOS would overwrite one with the other. On Linux, Mesen-CE opens `bios.col` and NP2kai opens the lowercase `np2kai/2608_*.wav`, which the pack only holds as `BIOS.col` and `2608_*.WAV`. The installer checks whether the -BIOS directory keeps case and, if it does, also fetches these names. If you -extract the ZIP on Linux by hand, take the Lakka pack instead: it is the -RetroArch pack with both spellings. Releases up to v2026.10.10 have no Lakka -pack of their own (`RetroArch_Lakka_*_BIOS_Pack.zip` serves both and holds one -spelling), so with those use the installer. +BIOS directory keeps case and, if it does, also fetches these names. On Linux, use the installer +rather than extracting the ZIP by hand. The reverse happens on Batocera and Recalbox, whose lists hold both spellings because their systems keep case: NP2kai reads `np2kai/bios.rom` and @@ -113,149 +156,6 @@ because their systems keep case: NP2kai reads `np2kai/bios.rom` and installer checks the folder first, keeps one file per name there and lists the ones it left out. -## Hash mismatch / UNTESTED - -This section and the two after it are about this repository's own tools, run -from a clone. `scripts/verify.py` checks the collection in `bios/` against each -platform, never your BIOS folder: for that, use `--check` above. - -`verify.py` reports `UNTESTED` for a file of the collection that was hashed -and does not match what the platform declares. The line gives the reason: - -``` - UNTESTED (required): nintendo-ds/dsifirmware.bin: declared hash contradicted by the local file -``` - -The system page on the site lists every known version of the file with its -expected MD5 and SHA1. - -### Existence-mode platforms never report UNTESTED - -On RetroArch, Lakka and RetroPie, the platform check only asks whether the file -is there, so the only outcomes are OK and MISSING. `verify.py` adds the emulator -ground truth from the profiles and flags a `DISCREPANCY` when a file passes the -platform check but fails the emulator's own size or hash validation. - -### The .variants/ directory - -When multiple versions of the same BIOS exist (different revisions, regions, or -dumps), the primary version lives in the main directory and alternatives live in -`.variants/`. `verify.py` checks the primary file first, then falls back to -variants when resolving by hash. - -If your file matches a variant hash but not the primary, it's a valid BIOS -- -just not the preferred version. Some cores accept multiple versions. - -## Pack is missing files - -A generated pack doesn't contain all the files you expected. - -### Severity levels - -`verify.py` assigns a severity to each issue, counted under `severity_counts` -in its `--json` output. Not all missing files are equally important: - -| Severity | Meaning | Action needed | -|----------|---------|---------------| -| CRITICAL | Required file missing on an MD5 or SHA1 platform | Must fix. Core won't function. | -| WARNING | Optional file missing on an MD5 or SHA1 platform, hash mismatch (UNTESTED) on any hash platform, or a required file missing on an existence platform | Core works but with reduced functionality, or runs on content nobody verified. | -| INFO | Optional file missing on an existence-only platform, or a missing file the core covers with an HLE fallback | Core works fine, BIOS improves accuracy. | -| OK | File present and verified | No action needed. | - -The full mapping, including how `hle_fallback` overrides the platform mode, is -in [verification modes](verification-modes.md#severity-matrix). - -Focus on CRITICAL issues first. WARNING files improve the experience but aren't -strictly necessary. INFO files are nice to have. - -### Large files (release assets) - -Files like PS3UPDAT.PUP, PSVUPDAT.PUP, and Switch firmware are too large for the -git repository. They are stored as GitHub release assets under the `large-files` -release and downloaded at build time. - -If a pack build fails to include these, check your network connection. In offline -mode (`--offline`), large files are only included if already cached locally in -`.cache/large/`. - -### Data directories - -Some cores need entire directory trees rather than individual files (e.g. Dolphin's -`Sys/` directory, PPSSPP's `assets/`). These are fetched by `refresh_data_dirs.py` -from upstream repositories. - -In offline mode, data directories are only included if already cached in `data/`. -Run `python scripts/refresh_data_dirs.py` to fetch them. - -## verify.py reports errors - -How to read and interpret `verify.py` output. - -### Status codes - -| Status | Meaning | -|--------|---------| -| `OK` | File present, hash matches (or existence check passed) | -| `UNTESTED` | File present, hash not confirmed against the expected value | -| `MISSING` | File not found in the repository | -| `DISCREPANCY` | The platform check passes, an emulator's own check fails | - -### Reading the output - -The first line counts the platform's files and names its check. Under it come -the entries that need attention, one per line as `STATUS (required|optional): -system/file`, followed by a reason where there is one, then the counts of the -files the emulator cores add. From `python scripts/verify.py --platform retrodeck`: - -``` -RetroDECK: 2007/2008 OK, 1 untested [md5] - UNTESTED (required): nintendo-ds/dsifirmware.bin: declared hash contradicted by the local file - DISCREPANCY: nintendo-snes/sgb2.boot.rom: RetroDECK says OK but supersnes9x says size mismatch: got 524288, accepted [256] - supersnes9x [size] - Core files: 1495 in pack, 0 required missing, 1 optional missing -``` - -`--verbose` also lists the passing entries that an emulator validates, with -the check and the source lines behind it: - -``` -Recalbox: 351/351 OK [md5] - OK (required): macintosh/MacII.ROM - minivmac validates size=262144 [minivmac/cfg/CNFGRAPI.h:32, minivmac/src/OSGLUERETRO.c:781] -``` - -### Cross-reference section - -After per-file results, `verify.py` prints a cross-reference report. This lists -files that emulator cores need but that the platform YAML doesn't declare. These -files are still included in packs automatically, but the report helps identify -gaps in platform coverage data. - -The cross-reference uses `resolve_platform_cores()` to determine which emulator -profiles are relevant for each platform, then checks whether each profile's files -appear in the platform config. - -### Filtering output - -```bash -# By platform -python scripts/verify.py --platform batocera - -# By emulator core -python scripts/verify.py --emulator beetle_psx - -# By system -python scripts/verify.py --system sony-playstation - -# By hardware target -python scripts/verify.py --platform retroarch --target switch - -# JSON for scripted processing: the entries that need attention -python scripts/verify.py --platform retroarch --json -# every entry, passing ones included, with its ground truth -python scripts/verify.py --platform retroarch --json --verbose -``` - ## Installation script fails Problems with `install.py`, `install.sh`, or `download.sh`. @@ -317,7 +217,7 @@ checked before anything is downloaded, so a "No file list" message comes from the revision: `RETROBIOS_REF` naming a tag that does not exist, or one older than the platform. Unset it, or pick a tag from the releases page. What a tag holds and for how long is under -[environment overrides](installer.md#environment-overrides). +Details: [environment overrides](installer.md#environment-overrides). ### Manual pack not found in release @@ -346,3 +246,16 @@ many there are. it against the release's checksums before extracting. Both stage inside the destination directory, never in `/tmp`, which is a RAM disk on most of these systems. +## Checks of the collection itself + + + +`scripts/verify.py` checks this project's own collection against each +platform, from a clone of the repository. It never reads your BIOS folder. +Reference: [Reading verify.py output](verification-modes.md#hash-mismatch-untested). + +!!! tip "Next" + + - The page of your setup, from [Get your files](../which-pack.md): its folder + and how to check it + - [FAQ](faq.md), for the questions people ask before installing diff --git a/wiki/verification-modes.md b/wiki/verification-modes.md index 112cc9ba..30398dd0 100644 --- a/wiki/verification-modes.md +++ b/wiki/verification-modes.md @@ -1,4 +1,8 @@ -# Verification modes +--- +level: reference +--- + +# How each platform checks files Each platform verifies BIOS files differently. `verify.py` replicates the native behavior of each platform so that verification results match what the platform itself would report. @@ -413,3 +417,146 @@ name or any name the emulator's hashes reach. If found, it ships. If not, the Batocera-verified file ships and the discrepancy stays visible in verification and gap reporting. +## Hash mismatch / UNTESTED + +This section and the two after it are about this repository's own tools, run +from a clone. `scripts/verify.py` checks the collection in `bios/` against each +platform, never your BIOS folder: for that, use the installer's `--check` +([When a game does not start](troubleshooting.md)). + +`verify.py` reports `UNTESTED` for a file of the collection that was hashed +and does not match what the platform declares. The line gives the reason: + +``` + UNTESTED (required): nintendo-ds/dsifirmware.bin: declared hash contradicted by the local file +``` + +The system page on the site lists every known version of the file with its +expected MD5 and SHA1. + +### Existence-mode platforms never report UNTESTED + +On RetroArch, Lakka and RetroPie, the platform check only asks whether the file +is there, so the only outcomes are OK and MISSING. `verify.py` adds the emulator +ground truth from the profiles and flags a `DISCREPANCY` when a file passes the +platform check but fails the emulator's own size or hash validation. + +### The .variants/ directory + +When multiple versions of the same BIOS exist (different revisions, regions, or +dumps), the primary version lives in the main directory and alternatives live in +`.variants/`. `verify.py` checks the primary file first, then falls back to +variants when resolving by hash. + +If your file matches a variant hash but not the primary, it's a valid BIOS -- +just not the preferred version. Some cores accept multiple versions. + +## Pack is missing files + +A generated pack doesn't contain all the files you expected. + +### Severity levels + +`verify.py` assigns a severity to each issue, counted under `severity_counts` +in its `--json` output. Not all missing files are equally important: + +| Severity | Meaning | Action needed | +|----------|---------|---------------| +| CRITICAL | Required file missing on an MD5 or SHA1 platform | Must fix. Core won't function. | +| WARNING | Optional file missing on an MD5 or SHA1 platform, hash mismatch (UNTESTED) on any hash platform, or a required file missing on an existence platform | Core works but with reduced functionality, or runs on content nobody verified. | +| INFO | Optional file missing on an existence-only platform, or a missing file the core covers with an HLE fallback | Core works fine, BIOS improves accuracy. | +| OK | File present and verified | No action needed. | + +The full mapping, including how `hle_fallback` overrides the platform mode, is +in [severity matrix](#severity-matrix). + +Focus on CRITICAL issues first. WARNING files improve the experience but aren't +strictly necessary. INFO files are nice to have. + +### Large files (release assets) + +Files like PS3UPDAT.PUP, PSVUPDAT.PUP, and Switch firmware are too large for the +git repository. They are stored as GitHub release assets under the `large-files` +release and downloaded at build time. + +If a pack build fails to include these, check your network connection. In offline +mode (`--offline`), large files are only included if already cached locally in +`.cache/large/`. + +### Data directories + +Some cores need entire directory trees rather than individual files (e.g. Dolphin's +`Sys/` directory, PPSSPP's `assets/`). These are fetched by `refresh_data_dirs.py` +from upstream repositories. + +In offline mode, data directories are only included if already cached in `data/`. +Run `python scripts/refresh_data_dirs.py` to fetch them. + +## verify.py reports errors + +How to read and interpret `verify.py` output. + +### Status codes + +| Status | Meaning | +|--------|---------| +| `OK` | File present, hash matches (or existence check passed) | +| `UNTESTED` | File present, hash not confirmed against the expected value | +| `MISSING` | File not found in the repository | +| `DISCREPANCY` | The platform check passes, an emulator's own check fails | + +### Reading the output + +The first line counts the platform's files and names its check. Under it come +the entries that need attention, one per line as `STATUS (required|optional): +system/file`, followed by a reason where there is one, then the counts of the +files the emulator cores add. From `python scripts/verify.py --platform retrodeck`: + +``` +RetroDECK: 2007/2008 OK, 1 untested [md5] + UNTESTED (required): nintendo-ds/dsifirmware.bin: declared hash contradicted by the local file + DISCREPANCY: nintendo-snes/sgb2.boot.rom: RetroDECK says OK but supersnes9x says size mismatch: got 524288, accepted [256] + supersnes9x [size] + Core files: 1495 in pack, 0 required missing, 1 optional missing +``` + +`--verbose` also lists the passing entries that an emulator validates, with +the check and the source lines behind it: + +``` +Recalbox: 351/351 OK [md5] + OK (required): macintosh/MacII.ROM + minivmac validates size=262144 [minivmac/cfg/CNFGRAPI.h:32, minivmac/src/OSGLUERETRO.c:781] +``` + +### Cross-reference section + +After per-file results, `verify.py` prints a cross-reference report. This lists +files that emulator cores need but that the platform YAML doesn't declare. These +files are still included in packs automatically, but the report helps identify +gaps in platform coverage data. + +The cross-reference uses `resolve_platform_cores()` to determine which emulator +profiles are relevant for each platform, then checks whether each profile's files +appear in the platform config. + +### Filtering output + +```bash +# By platform +python scripts/verify.py --platform batocera + +# By emulator core +python scripts/verify.py --emulator beetle_psx + +# By system +python scripts/verify.py --system sony-playstation + +# By hardware target +python scripts/verify.py --platform retroarch --target switch + +# JSON for scripted processing: the entries that need attention +python scripts/verify.py --platform retroarch --json +# every entry, passing ones included, with its ground truth +python scripts/verify.py --platform retroarch --json --verbose +``` diff --git a/wiki/what-are-bios-files.md b/wiki/what-are-bios-files.md new file mode 100644 index 00000000..465df5df --- /dev/null +++ b/wiki/what-are-bios-files.md @@ -0,0 +1,76 @@ +--- +level: start +--- + +# What BIOS files are + +Many consoles have a small program built into them. It runs when the console +is switched on, sets up the hardware and then starts the game. That program is +called the BIOS, or the firmware. + +An emulator recreates a console in software. To start the games of many +consoles, it needs a copy of that built-in program. The copy is a file, and +that file is what people call a BIOS file. + + + +## How the pieces fit together + +1. You pick a game in the **frontend**, the menu you play from (EmulationStation, + the RetroArch menu, MinUI and others). +2. The frontend starts an **emulator** for that console. +3. The emulator reads the **BIOS file** from the **BIOS folder** of your setup. +4. The game starts. + +When the file is not in the folder, or is not the version the emulator expects, +step 3 fails. The emulator then says that a BIOS is missing, or the game stays +on a black screen. + +## Do I need them? + +It depends on the console. + +- **Usually yes** for consoles that start every game through their own + program: PlayStation, PlayStation 2, Saturn, Mega-CD, PC Engine CD, Neo Geo, + Dreamcast, and many home computers. +- **Usually not** for cartridge consoles such as the NES, the Super Nintendo, + the Mega Drive, the Game Boy and the Game Boy Advance. Their emulators start + the games without one. Some can use one to show the original start-up screen. +- **Add-ons need their own**, even on those consoles: the Famicom Disk System + and the Mega-CD each have a BIOS. + +Some emulators can stand in for the BIOS with code of their own. This is called +HLE, for high-level emulation. It works for many games, but some glitch or do +not start. The real file gives the best results, so the packs include it. + +Details: [the files each console needs](../systems/index.md), with the +emulators that read them. + +## Where they go + +Each setup keeps its BIOS files in a folder of its own: `system` for RetroArch, +`/userdata/bios` on Batocera, `Emulation/bios` with EmuDeck. The page of your +setup gives the exact folder. + +The name matters. An emulator looks for an exact name, such as `scph5501.bin` +for a PlayStation, and a file with another name is not found. + +## Why the right version matters + +Two files with the same name can hold different versions: one per region, or +a newer revision of the same chip. Some setups, among them Batocera, Recalbox +and EmuDeck, compare a fingerprint of the file's contents, called a hash, and +refuse a version they do not list. RetroBIOS gives each setup the versions it +expects. + +## Is it legal? + +Laws on copying console firmware differ by country, and the question is not +settled. The [project's position](faq.md#is-this-legal) explains where it +stands, where its arguments are weakest, and how to ask for a file to be +removed. + +!!! tip "Next" + + - [Get your files](../which-pack.md), for the setup you play on + - [Glossary](glossary.md), every word these pages use