Skip to content

Transition KVM disk-only VM snapshot to Error state on delete failure - #14283

Draft
DaanHoogland wants to merge 1 commit into
apache:4.22from
shapeblue:ghi14019-kvmVmSnapshotDeleteErrorState
Draft

Transition KVM disk-only VM snapshot to Error state on delete failure#14283
DaanHoogland wants to merge 1 commit into
apache:4.22from
shapeblue:ghi14019-kvmVmSnapshotDeleteErrorState

Conversation

@DaanHoogland

@DaanHoogland DaanHoogland commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

deleteVMSnapshot in KvmFileBasedStorageVmSnapshotStrategy transitioned the snapshot to Expunging but had no exception handling, so any exception thrown by the merge/delete helpers (e.g. an agent command timeout) left the snapshot permanently stuck in Expunging. VMSnapshotManagerImpl.hasActiveVMSnapshotTasks treats Expunging as active work, so this permanently blocked reboot and further snapshot operations on the VM.

Wrap the delete body in a try/catch, mirroring the existing pattern already used by takeVMSnapshot and revertVMSnapshot in the same class: on any RuntimeException, transition the snapshot to Error via the existing Expunging -> OperationFailed state machine transition before rethrowing.

Test: KvmFileBasedStorageVmSnapshotStrategyTest#testDeleteVMSnapshotMarksSnapshotFailedWhenDeleteThrowsAfterExpungeRequested fails without this fix (no OperationFailed transition is ever fired) and passes with it.

Description

This PR...

Fixes: #14019

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

@codecov

codecov Bot commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 10.81081% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.02%. Comparing base (2974af8) to head (db74da6).

Files with missing lines Patch % Lines
...napshot/KvmFileBasedStorageVmSnapshotStrategy.java 10.81% 33 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff            @@
##               4.22   #14283   +/-   ##
=========================================
  Coverage     18.02%   18.02%           
- Complexity    16250    16253    +3     
=========================================
  Files          5936     5936           
  Lines        535823   535827    +4     
  Branches      65612    65612           
=========================================
+ Hits          96582    96594   +12     
+ Misses       428242   428236    -6     
+ Partials      10999    10997    -2
Flag Coverage Δ
uitests 4.04% <ø> (ø)
unittests 19.10% <10.81%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ai-fixes @DaanHoogland
CS-3474: Transition KVM disk-only VM snapshot to Error state on delet…
db74da6 
…e failure

deleteVMSnapshot in KvmFileBasedStorageVmSnapshotStrategy transitioned the
snapshot to Expunging but had no exception handling, so any exception thrown
by the merge/delete helpers (e.g. an agent command timeout) left the
snapshot permanently stuck in Expunging. VMSnapshotManagerImpl.hasActiveVMSnapshotTasks
treats Expunging as active work, so this permanently blocked reboot and
further snapshot operations on the VM.

Wrap the delete body in a try/catch, mirroring the existing pattern already
used by takeVMSnapshot and revertVMSnapshot in the same class: on any
RuntimeException, transition the snapshot to Error via the existing
Expunging -> OperationFailed state machine transition before rethrowing.

Test: KvmFileBasedStorageVmSnapshotStrategyTest#testDeleteVMSnapshotMarksSnapshotFailedWhenDeleteThrowsAfterExpungeRequested
fails without this fix (no OperationFailed transition is ever fired) and
passes with it.
@DaanHoogland
DaanHoogland force-pushed the ghi14019-kvmVmSnapshotDeleteErrorState branch from 678ed74 to db74da6 Compare October 7, 2026 14:19
@DaanHoogland

DaanHoogland commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

blueorangutan commented Oct 7, 2026

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Oct 7, 2026

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19451

@DaanHoogland

DaanHoogland commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

blueorangutan commented Oct 12, 2026

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

blueorangutan commented Oct 13, 2026

Copy link
Copy Markdown

[SF] Trillian test result (tid-17124)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 55423 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr14283-t17124-kvm-ol8.zip
Smoke tests completed. 147 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestClusterDRS>:setup Error 0.00 test_cluster_drs.py
test_list_system_vms_metrics_history Failure 0.25 test_metrics_api.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

component:storage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@DaanHoogland @blueorangutan