kmoody/libretro
1
0
Fork 0
mirror of https://github.com/Abdess/retroarch_system.git synced 2026-04-13 12:22:33 -05:00
Code Issues Packages Projects Releases Wiki Activity
96 Commits 5 Branches 4 Tags
08f68e792de2b6ffe89b34f4f95a9be7612ae95e
Commit Graph

96 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Abdessamad Derraz
08f68e792d refactor: centralize hash logic, fix circular imports and perf bottlenecks 2026-03-18 11:51:12 +01:00
Abdessamad Derraz
becd0efb33 fix: relative links in readme, commit pending changes 2026-03-18 11:28:58 +01:00
Abdessamad Derraz
09ebaa9316 fix: retropie logo from github avatar, remove *.png gitignore 2026-03-18 11:26:46 +01:00
Abdessamad Derraz
81278bd2e4 fix: system icons (systematic theme), retropie logo 2026-03-18 11:25:14 +01:00
Abdessamad Derraz
a52ab19cf8 fix: full hashes, list format for system files 2026-03-18 11:15:11 +01:00
Abdessamad Derraz
300e5d7439 fix: redesign home page UX, fix broken retropie logo 2026-03-18 11:09:36 +01:00
Abdessamad Derraz
54c0f1d27e refactor: review fixes, DRY coverage, filter test nav 
- Extract compute_coverage to common.py (was duplicated)
- Filter test cores from nav and emulator index
- Use absolute URL for README download links
- Consistent page titles with site name suffix
- Safer mkdocs.yml nav rewrite with regex
- Build all_platform_names once in gap analysis
 2026-03-18 11:05:13 +01:00
Abdessamad Derraz
e218763500 feat: add emulator logos to profiles and site 2026-03-18 10:57:00 +01:00
Abdessamad Derraz
6885681c65 feat: add platform logos to registry and site 2026-03-18 10:55:47 +01:00
Abdessamad Derraz
21a50c992f feat: slim readme + ci site deployment 
README: 11141 -> 43 lines. Details on the MkDocs site.
generate_readme.py: 444 -> 164 lines. Slim coverage table only.
build.yml: adds mkdocs-material install, generate_site.py, gh-deploy.
Adds pages: write permission for GitHub Pages deployment.
 2026-03-18 10:44:13 +01:00
Abdessamad Derraz
32e4f6e580 fix: review fixes for generate_site.py 2026-03-18 10:39:23 +01:00
Abdessamad Derraz
0b1ed3cb1a feat: add gap analysis page + platform tracking 2026-03-18 10:31:02 +01:00
Abdessamad Derraz
883e153a62 fix: clean platform/emulator page layout 2026-03-18 10:27:08 +01:00
Abdessamad Derraz
b15b062782 feat: add mkdocs site generator, 332 pages 
generate_site.py reads database.json + platforms/*.yml + emulators/*.yml
and produces a complete MkDocs Material documentation site:
- Home: stats, downloads, coverage dashboard
- 7 platform pages with per-file verification status
- 60 system pages grouped by manufacturer with cross-references
- 260 emulator pages with source code analysis
- Contributing guide

mkdocs.yml with Material theme, system fonts, auto dark mode.
Generated docs/ in .gitignore (built in CI only).
 2026-03-18 10:22:00 +01:00
Abdessamad Derraz
dd9e59c8e3 chore: remove nested bios/Philips/CD-i/bios/ duplicate + empty TOS ghosts 
Nested bios/ directory inside CD-i was an agent artifact.
tos100uk.img and tos206us.img were 0-byte ghost files from failed git show.
 2026-03-18 08:25:56 +01:00
Abdessamad Derraz
3de4bf8190 refactor: extract _fetch_raw to BaseScraper (DRY) 
Identical _fetch_raw() implementation (URL fetch + cache + error handling)
was duplicated in 4 scrapers. Moved to BaseScraper.__init__ with url param.

Each scraper now passes url to super().__init__() and inherits _fetch_raw().
Eliminates ~48 lines of duplicated code.

DRY audit now clean: resolve logic in common.py, scraper CLI in base_scraper,
_fetch_raw in BaseScraper. Remaining duplications are justified (different
list_platforms semantics, context-specific hash computation).
 2026-03-18 08:22:21 +01:00
Abdessamad Derraz
2466fc4a97 refactor: extract scraper_cli() to base_scraper.py (DRY) 
Shared CLI boilerplate for all scrapers: argparse, dry-run, json, yaml output.
4 scrapers (libretro, batocera, retrobat, emudeck) reduced from ~58 lines
main() each to 3 lines calling scraper_cli().

~220 lines of duplicated boilerplate eliminated.
recalbox + coreinfo keep custom main() (extra flags: --full, --compare-db).
 2026-03-18 08:17:14 +01:00
Abdessamad Derraz
00700609d8 refactor: extract resolve_local_file to common.py (DRY) 
Single source of truth for file resolution logic:
- common.py:resolve_local_file() = 80 lines (core resolution)
- verify.py:resolve_to_local_path() = 3 lines (thin wrapper)
- generate_pack.py:resolve_file() = 20 lines (adds storage tiers + release assets)

Before: 103 + 73 = 176 lines of duplicated logic with subtle divergences
After: 80 lines shared + 23 lines wrappers = 103 lines total (-41%)

Resolution chain: SHA1 -> MD5 multi-hash -> truncated MD5 ->
zipped_file index -> name existence -> name composite -> name fallback
-> (pack only) release assets
 2026-03-18 08:11:10 +01:00
Abdessamad Derraz
0c367ca7c6 feat: restore deleted TOS UK images, regenerate database 
tos102uk.img, tos104uk.img, tos106uk.img were accidentally deleted
by a background agent. Restored from git history.
 2026-03-18 07:47:25 +01:00
Abdessamad Derraz
8c18638cd2 regenerate database after merge 2026-03-18 07:31:46 +01:00
Abdessamad Derraz
7b1c6a723e refactor: review fixes - resolve coherence + cleanup 
1. fetch_large_file moved to last resort (avoids HTTP before name lookup)
2. fetch_large_file receives first MD5 only (not comma-separated string)
3. verify.py MD5 lookup now splits comma-separated + lowercases (matches generate_pack)
4. seen_destinations simplified to set (stored hash was dead data)
5. Variable suffix shadowing renamed to file_ext
 2026-03-18 07:18:40 +01:00
Abdessamad Derraz
7ae995fb32 fix: resolve_file multi-MD5 + md5_composite for Recalbox packs 
Three fixes in resolve_file():
- Split comma-separated MD5 lists (Recalbox uses multi-hash)
- Add md5_composite check in name fallback (matches verify.py logic)
- Use ".zip" in basename instead of endswith for variant files

Recalbox pack: 346/346 verified (was 332/346 with 14 wrong hash)
Batocera pack: 359/359 verified (was 304/359 with 55 inner missing)
All 5 platforms now produce 0 untested, 0 missing packs.
 2026-03-18 07:18:40 +01:00
Abdessamad Derraz
a1dc6fa4ef fix: resolve_file prefers primary over variants for name fallback 
When resolving by name with no MD5 (existence check), prefer files
NOT in .variants/ directory. Fixes naomi2.zip resolving to the
Recalbox variant (15 files) instead of the primary (21 files).

Also applies to hash_mismatch fallback path.
 2026-03-18 07:18:40 +01:00
github-actions[bot]
b0dad7dcf3 regenerate database and docs 2026-03-18 05:29:59 +00:00
Abdessamad Derraz
046fb276b0 fix: case-insensitive MD5 lookup in resolve_file 
Recalbox uses uppercase MD5 hashes (6E3735FF...) but database index
is lowercase. Added .lower() to MD5 lookups in resolve_file().

Fixes scph101.bin wrong variant in Recalbox pack (was picking
.variants/ copy instead of primary due to MD5 case mismatch).
 2026-03-18 06:27:35 +01:00
Abdessamad Derraz
040ea9f217 fix: resolve_file skips MD5 lookup for zipped_file entries 
Same guard as verify.py: when zipped_file is set, the md5 is for the
inner ROM, not the container ZIP. Direct MD5 lookup resolved to the
standalone ROM file instead of the ZIP parent.

Fixes: ep64.zip/ep128.zip (Enterprise) written as raw ROM data instead
of ZIP archives in Batocera pack. Also fixes any other zipped_file entry
where the inner ROM MD5 matched a standalone file in the database.

Also: update Dinothawr.zip SHA1 in retroarch.yml to match actual file.
 2026-03-18 06:27:35 +01:00
Abdessamad Derraz
84ab0ea6d3 fix: revert verify dedup (breaks counts), optimize pack generation 
verify.py: removed destination dedup - verify counts ALL platform
entries (398 for RetroArch). Pack deduplicates at generation (395).
The delta (3 files: c52/g7400/jopac.bin) is correct behavior.

generate_pack.py: skip build_zip_contents_index() when no zipped_file
entries exist. RetroArch pack: 20s -> 11s (no ZIP scan needed).
 2026-03-18 06:27:35 +01:00
github-actions[bot]
06ea19ee20 regenerate database and docs 2026-03-18 04:48:32 +00:00
Abdessamad Derraz
97a25b17ff fix: CI restores large files before DB regen, fix coverage numbers 
build.yml now downloads large-files release assets and copies them
to their bios/ paths (matched via .gitignore) before regenerating
the database. This fixes Batocera showing 675/680 instead of 679/680
on the remote (PS3UPDAT.PUP, dsi_nand.bin, PSVUPDAT.PUP were missing).

Local: Batocera 679/680, all others 100%.
 2026-03-18 05:46:16 +01:00
Abdessamad Derraz
4faae161b4 feat: implement --include-extras with hybrid core detection 
generate_pack.py now merges Tier 2 emulator files into platform packs:
- Auto-detects cores from platform YAML "core:" fields (31 for RetroArch)
- Also reads manual "emulators:" list from _registry.yml (for Batocera etc)
- Union of both sources = complete emulator coverage per platform
- Files already in platform pack are skipped (Tier 1 wins)

Results with --include-extras:
  RetroArch: 395 -> 654 files (+259 emulator extras)
  Batocera:  359 -> 632 files (+273 emulator extras)

Pack naming: BIOS_Pack.zip (normal) vs Complete_Pack.zip (with extras)
 2026-03-18 05:39:13 +01:00
Abdessamad Derraz
58f3006d20 docs: add core_version, display_name to all 260 emulator profiles 
Every profile now has:
- profiled_date: date of source code analysis
- core_version: version from libretro-core-info .info files
- display_name: human-readable name from .info files

260/260 profiles complete. 294/294 libretro cores covered.
Standalone emulators (cemu, rpcs3, xemu, vita3k) versioned manually.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
d929424988 feat: add profiled_date to all YAMLs, create 56 alias profiles 
- Added profiled_date field to all 204 existing profiles for update tracking
- Created 56 alias profiles for cores that share BIOS with a parent
  (e.g., mednafen_psx -> beetle_psx, fbalpha2012 -> fbneo)

260 total profiles covering all 294 libretro cores (204 unique + 56 alias).
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
62262db6ac feat: complete 294/294 libretro core profiling 
Final batch: stonesoup, vitaquake3+vitavoyager, uw8, uxn, wasm4,
tempgba, retro8, moonlight, ffmpeg, imageviewer, mpv, remotejoy,
puzzlescript, lutro, mojozork, simcp (SAM Coupe, ROM built-in),
romcleaner, redbook, onsyuri, mcsoftserve, x64sdl (VICE alias)

204 total emulator/core profiles covering ALL 294 libretro cores.
Every core analyzed from source code. Complete museum documentation.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
b43d71e113 feat: add emulator profiles (119-series batch 6 partial) 
dolphin_launcher (stub launcher), oberon (RISC, bootloader embedded),
17 test cores (00_example through vidtest), 17 game/utility cores
(2048 through vaporspec), bennugd, boytacean, emux_chip8, fake08,
jaxe, lowresnx, opentyrian, openlara, squirreljme, vitaquake2, xrick

182 total profiles. 6 agents failed (API 500), will retry.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
a5553def94 feat: add 10 emulator profiles (119-series batch 5) 
boytacean (GB Rust, 6 boot ROMs embedded), emux_chip8 (no BIOS),
fake08 (PICO-8, Lua runtime compiled in), jaxe (CHIP-8, fonts hardcoded),
lowresnx (fantasy console), opentyrian (Tyrian freeware data),
openlara (Tomb Raider engine, game data), squirreljme (J2ME, 8 JARs),
vitaquake2 (Quake 2, 4 variants), xrick (data.zip sound pack)

145 total profiles.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
31501211de feat: add 10 emulator profiles (119-series batch 4) 
bennugd (game engine, no BIOS), cruzes (puzzle game, no BIOS),
dice (discrete arcade, no BIOS), ishiiruka (Dolphin fork, same BIOS),
mkxp_z (RPG Maker, 3 RTP dirs), onscripter (VN engine, no BIOS),
qemu (55 firmware files! SeaBIOS/VGA/iPXE/OpenBIOS/OpenSBI),
rustation (PS1 Rust, 22 BIOS by SHA-256), rvvm (RISC-V, OpenSBI),
reminiscence (Flashback engine, game data only)

135 total profiles.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
6a778a09a3 feat: add 10 emulator profiles (119-series batch 3) 
arduous (Arduboy AVR sim, no BIOS), boom3 (Doom 3, game data only),
directxbox (Xbox, mcpx + cromwell), doukutsu_rs (Cave Story, no BIOS),
meteor (GBA, full HLE), nxengine (Cave Story, Doukutsu.exe + data/),
tgbdual (Game Boy link, HLE), tic80 (fantasy console),
virtualxt (IBM XT, open BIOS embedded), sdlpal (Sword & Fairy, 13 .mkf)

125 total profiles.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
ebc8e4413f feat: add 10 emulator profiles (119-series batch 2) 
freeintv_ts_overlay (same BIOS as FreeIntv), bnes (NES, no BIOS),
clownmdemu (Mega Drive, open-source MCD boot ROM embedded),
m2000 (Philips P2000T, all embedded), mesen (NES, FDS + StudyBox BIOS),
race (NGP, HLE with disabled BIOS loader), vemulator (VMU, HLE only),
gw (Game & Watch, Lua simulator), pd777 (Cassette Vision, no BIOS),
play (PS2, full HLE, LoadBIOS commented out)

115 total profiles.
 2026-03-18 05:20:05 +01:00
Abdessamad Derraz
a77477cbe8 feat: add 10 emulator profiles (119-series batch 1) 
81 (ZX81, embedded), DoubleCherryGB (GB dual, no BIOS),
amiarcadia (Arcadia 2001, no BIOS), applewin (Apple II, 26 embedded),
ardens (Arduboy, 6 bootloaders embedded), daphne (LaserDisc, no BIOS),
pcem (IBM PC, 75+ machine/video/sound ROMs), panda3ds (3DS, optional keys),
easyrpg (RPG Maker, soundfont + RTP), frodo (C64, 4 ROMs embedded)

105 total profiles.
 2026-03-18 05:20:05 +01:00
github-actions[bot]
9a69b49eac regenerate database and docs 2026-03-17 19:10:52 +00:00
Abdessamad Derraz
ff82f64cb6 feat: complete all emulator profiles, 95 engines total 
Final batch: scummvm (55 files, MT-32/engine data/themes/shaders),
gearcoleco (colecovision.rom), geargrafx (4 syscard variants),
gearlynx (lynxboot.img), dosbox_core (MT-32 + BASS libs),
geolith (Neo Geo FPGA, aes.zip/neogeo.zip), retrodream (DC boot+flash),
noods (NDS, full HLE), freej2me (JAR), jollycv (coleco+creativision),
fixgb (gbc_bios), numero (TI-83), smsplus (SMS/GG/CV), vircon32

95 profiles covering every libretro core with firmware requirements.
Cross-reference: 803 undeclared files, 249 in repo, 554 to source.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
60d9d38d69 feat: complete emulator profiling, 81 engines (final batch) 
mame2003_plus (16 BIOS ZIPs, MAME 0.78 naming), mame2010 (51 ZIPs),
mame2016 (37 ZIPs incl Lindbergh), hbmame (MAME 0.220, no custom BIOS),
tyrquake (no BIOS), cannonball (29 OutRun ROMs with CRC32),
uzem (open source, no BIOS), vecx (ROM embedded), emuscv (UPD7801G),
b2 (BBC Micro standalone, ROMs bundled)

81 total emulator profiles covering all libretro cores with firmware.
Cross-reference: 731 undeclared files, 186 already in repo.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
99581d8aba feat: add 10 emulator profiles (batch 6) 
gpsp (GBA, open BIOS fallback), ecwolf (ecwolf.pk3),
prboom (prboom.wad), x1 (Sharp X1, 5 files in xmil/),
minivmac (MacII.ROM/MacIIx.ROM), theodore (Thomson, all embedded),
crocods (CPC 6128, all embedded), vba_next (GBA, HLE fallback),
same_cdi (CD-i, 3 MAME ZIPs + 2 undeclared), beetle_lynx (lynxboot.img)

71 total profiles. Cross-reference: 690 undeclared, 186 in repo.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
4bc7085c1c feat: add 10 emulator profiles (batch 5) 
snes9x (BS-X + STBIOS, HLE coprocessors), mupen64plus (64DD IPL),
pcsx_rearmed (7 BIOS + dynamic scan fallback), stella (no BIOS),
fmsx (12 MSX ROMs), virtualjaguar (6 BIOS embedded),
sameboy (8 boot ROM types, all built-in), beetle_vb (no BIOS),
handy (lynxboot.img 512B), pokemini (bios.min + FreeBIOS fallback)

61 total profiles. Cross-reference: 667 undeclared, 182 in repo.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
9eede1c957 feat: add 10 emulator profiles (batch 4) 
blastem (MD, no BIOS needed), dosbox_pure (14 MT-32 ROMs + SF2),
desmume (NDS, 3 files all HLE), cap32 (CPC, all embedded),
mu (Palm OS, 5 ROMs), bk (Elektronika BK, 9 Soviet ROMs),
gearsystem (SMS/GG, 2 optional boot ROMs),
beetle_ngp (Neo Geo Pocket, pure HLE), beetle_wswan (WonderSwan, pure HLE),
galaksija (Yugoslav computer, all embedded)

51 total profiles. Cross-reference: 658 undeclared, 177 in repo.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
8768aed7aa feat: add 10 emulator profiles (batch 3) 
freechaf (Channel F, 3 ROMs), freeintv (Intellivision, exec+grom),
ep128emu (Enterprise/TVC/CPC/ZX, 22 ROMs all built-in),
kronos (Saturn + 14 ST-V blobs), gambatte (GB/GBC boot ROMs),
mgba (GBA/GB/GBC/SGB, 10 CRC32 revisions), nestopia (FDS + NstDatabase),
px68k (X68000, 6 ROMs in keropi/), fuse (ZX Spectrum, 17 machines),
o2em (Odyssey2/Videopac, 4 BIOS by CRC32)

41 total profiles. Cross-reference: 644 undeclared, 167 in repo.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
639ff69d6c feat: add 10 emulator profiles (batch 2) 
a5200, prosystem (Atari 7800), atari800 (400/800/XL/XE),
vice (C64/C128/VIC-20/Plus4/PET - 10 cores, all ROMs embedded),
neocd (Neo Geo CD, 12 BIOS variants), beetle_pce (PC Engine, 6 files),
beetle_pcfx (PC-FX), np2kai (PC-9801, 14 files + YM2608 samples),
quasi88 (PC-8801, 13 ROMs + 3 hidden fonts), opera (3DO, 10 BIOS + 3 kanji)

31 total profiles. Cross-reference: 572 undeclared, 148 in repo.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
f6a44c9409 feat: add 10 emulator profiles (batch 1 P1) 
genesis_plus_gx (12 files), picodrive (18 files), fbneo (19 systems),
hatari (TOS ROMs), bluemsx (60+ MSX/ColecoVision/SG-1000),
fceumm (FDS/Game Genie), bsnes (20 coprocessor ROMs),
beetle_psx (5 BIOS + aliases), beetle_saturn (5 files),
puae (14 Kickstart ROMs, 3 naming conventions)

21 total emulator profiles. Cross-reference: 436 undeclared files,
120 already in repo, 316 to source.
 2026-03-17 20:08:27 +01:00
Abdessamad Derraz
9052a6b750 feat: add emulator profiles and cross-reference engine (tier 2) 
New two-tier architecture:
- Tier 1: Platform configs (what the UI checks) - unchanged
- Tier 2: Emulator profiles (what the code actually loads)

11 emulator profiles from source code analysis:
  cemu, citra, dolphin, duckstation, flycast,
  melonds, pcsx2, ppsspp, rpcs3, vita3k, xemu

Each profile documents every file the emulator loads with
source code references (file:line), hashes, and notes.

New scripts/cross_reference.py computes gaps between what
platforms declare and what emulators need.

Current gap: 200 undeclared files, 24 already in repo.
DuckStation alone recognizes 105 PS1/PS2 BIOS variants.

generate_pack.py gains --include-extras flag (future use).
_registry.yml maps platforms to their emulators.
 2026-03-17 20:08:27 +01:00
github-actions[bot]
a38c86cf1e regenerate database and docs 2026-03-17 16:53:26 +00:00