Fix native CPU-env training hang (OMP regions + missing log keys) - #668
Open
IlyaasK wants to merge 1 commit into
Open
Fix native CPU-env training hang (OMP regions + missing log keys)#668IlyaasK wants to merge 1 commit into
IlyaasK wants to merge 1 commit into
Conversation
Two OpenMP regions in the CPU-env path (env_start reset loop and the per-buffer worker step loop) deadlock inside containerized processes (observed on Thunder Compute / CUDA 13 / Ubuntu 22.04 with every env including stock breakout). Serialize both: they are one-time init and per-buffer stepping, so serial execution is correct and cheap. Also guard dashboard/log-history reads (util/*, uptime, agent_steps) with a new dict_get_default so missing log keys no longer abort training. Repro: build.sh breakout && puffer train --train.total-timesteps=50000 -> hangs after env_setup; this branch completes epochs at ~80-110K sps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
puffer trainhangs indefinitely at init for every CPU-host env in the5.0branch when run in a containerized/virtualized environment (reproduced on Thunder Compute: NVIDIA A6000, CUDA 13.0, Ubuntu 22.04, stockbuild.sh breakout && puffer train --train.total-timesteps=50000; same withcartpoleand ourpinballenv).Symptoms:
env_startcompletes, then the process spins at ~107% of one core with GPU at 0%, no output, workers futex-waiting. Never reaches the train loop.Root cause
Two
#pragma omp parallel forregions in the CPU-env path:env_start()— the one-timepuf_resetof all envs runs in an OMP regionvec_thread_main()— the per-bufferpuf_steploop runs in an OMP regionBoth deadlock inside the sandboxed/containerized process (standalone OpenMP works; the deadlock is process-context-specific, e.g. interaction with injected monitoring threads/hardened seccomp). Since region 1 is one-time init and region 2 already runs per-buffer on its own worker thread, serial execution preserves behavior.
Fix
dict_get_default()toini.hutil/*,uptime,agent_steps) with defaults so missing log keys no longer abort training after the first epochVerification
Before: hang after env_setup (marker-traced; env-independent).
After: full epoch progress on Thunder Compute, live dashboard, ~80–110K SPS, losses (policy/value/entropy) updating, 200K-step sanity run completes without error.
Related: behaviors validated with stock
breakoutconfig too — same marker trace on both envs before the fix.