Skip to content

Prevent image-transform worker deadlocks - #252

Merged
clemsgrs merged 1 commit into
mainfrom
agent/issue-245
Jul 28, 2026
Merged

Prevent image-transform worker deadlocks#252
clemsgrs merged 1 commit into
mainfrom
agent/issue-245

Conversation

@clemsgrs

Copy link
Copy Markdown
Owner

Summary

  • resolve automatically selected dense- and pooled-image transform workers to zero after the model runtime is initialized
  • preserve explicit positive worker counts with a safe spawn DataLoader context
  • retain distributed sharding/output semantics and add bounded public-API subprocess regressions

Root cause

The image embedding paths loaded the encoder/runtime before constructing a DataLoader. Default worker selection then forked transform workers from a parent with initialized native threads, which could leave sleeping workers and deadlock the run.

User impact

Default dense-image and pooled-image embedding now complete without post-runtime worker forks. Explicit zero remains supported, while callers that explicitly request worker parallelism keep it through spawned workers.

Validation

  • 140 relevant tests passed across dense/pooled image stages, shards, batching, and core execution options
  • bounded public Model.embed_images_dense(...) and Model.embed_images(...) subprocess regressions passed and reported no active child workers
  • explicit spawned distributed dense and pooled worker tests passed with preserved shard outputs
  • git diff --check passed
  • two-axis main...HEAD review passed after both Standards judgement calls were resolved; Spec reported no findings

Closes #245

@clemsgrs
clemsgrs merged commit bd14aa6 into main Jul 28, 2026
3 checks passed
@clemsgrs
clemsgrs deleted the agent/issue-245 branch July 28, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent auto-worker deadlocks during dense-image embedding

1 participant