Skip to content

pr_runtime: per-candidate error boundary - #267

Open
rycerzes wants to merge 1 commit into
huggingface:mainfrom
rycerzes:fix/per-candidate-errors
Open

rycerzes wants to merge 1 commit into
huggingface:mainfrom
rycerzes:fix/per-candidate-errors

Conversation

@rycerzes

Copy link
Copy Markdown
Contributor

Summary

  • An exception from any one candidate (Docker, decoding, task building) ended the whole run. Each candidate now has its own error boundary. The traceback is logged with the PR, and the failure is counted in skip_reasons.
  • Infrastructure failures (DockerError, OSError, subprocess errors) count as infra_error and replace the validation sandbox. Other exceptions count as candidate_error. Task outcomes such as no_fail_to_pass are unchanged.

Test plan

  • New test: three candidates (infrastructure error, candidate error, normal skip) are all processed, the reasons are separate, and the sandbox is replaced only after the infrastructure error. It fails on main.
  • test_pipeline_pr_runtime, test_validate_stage_script, test_ungraded_sites (103 passed), ruff check, ruff format --check.

Out of scope

Most of the diff is reindentation (git diff -w: about 30 lines). Part of #253 (phase 0).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant