Skip to content

fix(xcat-dep): one dead mock chroot fails the perl package, the cell and the run - #96

Closed
dhilst wants to merge 2 commits into
xcat2:masterfrom
VersatusHPC:fix/dep-perl-build-retry
Closed

dhilst wants to merge 2 commits into
xcat2:masterfrom
VersatusHPC:fix/dep-perl-build-retry

Conversation

@dhilst

@dhilst dhilst commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

xcat-dep-build-el8-x86_64 #107 failed on perl-Digest-SHA1 alone, with ten of eleven packages passing beside it in the same run. Its build.log shows make test reporting Result: PASS and %doc ending + exit 0, and the next line is Child return code was: 255 with no rpm error at all: the Provides/Requires pass never ran, and systemd recorded the container as Deactivated successfully. The same srpm, with the same defines, alone on the same host, builds all four rpms and exits 0.

run_mock in mockbuild-perl-packages.pl retried exit 30 and nothing else, so a chroot that died without failing a build had no second attempt. One package then failed the cell and the whole EL family of the run. buildrpms.pl in xcat-core has carried sh_retry for this class since the systemd-nspawn ENOMEDIUM race; the perl builder is the copy that never got it.

mock_exit_is_transient names the exit codes that mean the RUN died rather than the BUILD failed -- 30 for the package manager, 255 for a chroot with no rpm error behind it -- and run_mock gives those another attempt after a backoff. Exit 1 is the build itself and is not retried, because three attempts at a broken spec is three times the wait for the same answer; an unlisted code is not retried either.

t/mock_transient_exit_is_retried.t holds both halves down. Dropping 255 from the list fails the assertion the test exists for; retrying every non-zero exit fails the two assertions that keep a real build failure fast. prove -j4 t/ passes 32 files and 1331 tests with upstream master merged in.

…and the run

xcat-dep-build-el8-x86_64 #107 failed on perl-Digest-SHA1 alone, with ten of
eleven packages passing beside it in the same run. Its build.log shows `make
test` reporting `Result: PASS` and %doc ending `+ exit 0`, and the next line is
`Child return code was: 255` with no rpm error: the Provides/Requires pass never
ran, and systemd recorded the container as `Deactivated successfully`. The same
srpm, with the same defines, alone on the same host, builds all four rpms and
exits 0. That one package failed the cell, the el family and the whole of
xcat-ci #146.

run_mock in mockbuild-perl-packages.pl retried exit 30 and nothing else, so a
chroot that died without failing a build had no second attempt.
xcat-core's buildrpms.pl has carried sh_retry for this class since the
systemd-nspawn ENOMEDIUM race; the perl builder is the copy that never got it.

mock_exit_is_transient in MockBuildUtils.pm names the exit codes that mean the
RUN died rather than the BUILD failed -- 30 for the package manager, 255 for a
chroot with no rpm error behind it -- and run_mock gives those another attempt
after a backoff. Exit 1 is the build itself and is not retried, because three
attempts at a broken spec is three times the wait for the same answer; an
unlisted code is not retried either.

mock_transient_exit_is_retried.t holds both halves down. Dropping 255 from the
list fails the assertion the test exists for, and retrying every non-zero exit
fails the two that keep a real build failure fast.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
One conflict, the MockBuildUtils export list: master gained packaged_path with
xcat-dep/xcat2#95 and this branch adds mock_exit_is_transient and mock_attempts.
Resolved as the union.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
@dhilst
dhilst requested a review from viniciusferrao October 23, 2026 18:10
@dhilst dhilst self-assigned this Oct 23, 2026

@viniciusferrao viniciusferrao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@dhilst dhilst closed this Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants