layer: Force bypass when requested format is unsupported on fallback - #2183
Open
matte-schwartz wants to merge 1 commit into
Open
layer: Force bypass when requested format is unsupported on fallback#2183matte-schwartz wants to merge 1 commit into
matte-schwartz wants to merge 1 commit into
Conversation
When canBypassXWayland returns false at swapchain creation and the requested format is not on the XCB fallback surface, the layer refuses the swapchain with VK_ERROR_INITIALIZATION_FAILED and the app has no way to recover. Star Citizen hits this during launch. It recreates its swapchain while its X11 toplevel and inner window are mid-resize, so the geometry check fails and the create lands on the fallback surface, which only advertises the B8G8R8A8 family. Without HDR exposed SC asks for VK_FORMAT_R8G8B8A8_UNORM, with HDR exposed it asks for VK_FORMAT_R16G16B16A16_SFLOAT, and either way the game stalls on a black screen after the refused recreate. Force bypass instead when the Wayland surface supports the format, and record that on the swapchain so QueuePresentKHR does not ask for a recreate that would only force bypass again. The Wine offscreen gate still wins, since flipping that content is wrong whatever its format, so canBypassXWayland caches whether the gate refused and both the force and the present-time check honor it. When the geometry check is what refused, forcing scans out the inner window and drops whatever obscured it, and the swapchain only returns to a normal bypass swapchain through the extent change path.
matte-schwartz
force-pushed
the
star-citizen-layerfix
branch
from
September 1, 2026 03:58
b77cda0 to
02893f2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When canBypassXWayland returns false at swapchain creation and the requested format isn't on the fallback surface, refusing the swapchain leaves the app with no recovery path. Instead, prefer bypass over INITIALIZATION_FAILED.
Star Citizen reproduces this during the game's launch process: it recreates its swapchain exactly when its X11 toplevel and inner window are mid-resize, so canBypass returns false (geometry mismatch) and the create lands on the XCB fallback surface. The XCB fallback only advertises B8G8R8A8 family formats, which means the recreate is refused for any other format. Without HDR exposed SC asks for VK_FORMAT_R8G8B8A8_UNORM, and with HDR exposed it instead tries to use VK_FORMAT_R16G16B16A16_SFLOAT. Either way, the game stalls on an infinite black screen after gamescope refuses the swapchain recreate.
Track the bypass override on the swapchain so QueuePresentKHR doesn't ask the app to recreate when the live canBypass disagrees.