fix: hold every data tree, present or not

export_native and cross_reference read data/ without the shared hold, so
a refresh could swap a tree mid-walk. hold_data_dirs skipped a cache not
yet on disk, so a refresh could install it between verify and the packs
of one pipeline run. Both now hold every registered tree.
This commit is contained in:
Abdessamad Derraz committed 2026-10-21 09:56:49 +02:00
1 parent 563ce1ebf2
commit 9bb1a73f91
4 files changed
+41 -3

No files matched your search

+9
View File
@@ -959,6 +959,15 @@ def main() -> None:
if args.platform:
require_platform(args.platform, args.platforms_dir)
import contextlib
from refresh_data_dirs import hold_data_dirs
# Files resolve against data/: the trees stay put until the process
# ends, which releases the hold.
held = contextlib.ExitStack()
hold_data_dirs(held, load_data_dir_registry(args.platforms_dir))
profiles = load_emulator_profiles(args.emulators_dir)
emulator = args.emulator
if emulator:
+11
View File
@@ -898,6 +898,17 @@ def main() -> None:
require_platform(args.platform, args.platforms_dir)
platforms = [args.platform]
if not args.refresh_cache:
import contextlib
from refresh_data_dirs import hold_data_dirs
# Removal candidates are proved against data/: a refresh swapping a
# tree mid-walk listed entries only that tree holds as removable.
# The process exit releases the hold.
held = contextlib.ExitStack()
hold_data_dirs(held, load_data_dir_registry(args.platforms_dir))
sys.exit(
run(
platforms,
+4 -2
View File
@@ -99,11 +99,13 @@ def hold_data_dirs(stack: contextlib.ExitStack, registry: dict) -> None:
later. A refresh swapping the tree in between shipped bytes nobody
verified, or died on a file the new tree no longer has. A refresh waits
for the holders. Never called before a refresh in the same process:
the exclusive lock would wait on this one.
the exclusive lock would wait on this one. A cache not yet on disk is
held too: the lock sits beside the tree, and skipped, a refresh could
install the tree after verify ran and before the packs read it.
"""
for entry in registry.values():
cache = entry.get("local_cache", "")
if cache and os.path.isdir(cache):
if cache:
stack.enter_context(cache_lock(cache, shared=True))