fix(coderd): fix flake in TestAPI/ModifyAutostopWithRunningWorkspace
#18932
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
This PR fixes a flaky test TestAPI/ModifyAutostopWithRunningWorkspace caused by a race condition in AwaitWorkspaceBuildJobCompleted. The issue occurs when the API returns stale workspace build data due to non-transactional database queries between workspace builds and provisioner jobs.
- Adds a re-fetch of the workspace build before accessing its deadline to ensure fresh data
- Restructures the test flow to validate the deadline state before making TTL updates
Comments suppressed due to low confidence (1)
coderd/workspaces_test.go:2880
- The variable name 'build' is being reused and shadows the original build variable from the outer scope. Consider using a more descriptive name like 'freshBuild' or 'updatedBuild' to avoid confusion.
build, err := client.WorkspaceBuild(ctx, build.ID)
johnstcn
approved these changes
Jul 21, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes coder/internal#521
This happened due to a race condition present in how
AwaitWorkspaceBuildJobCompletedworks.AwaitWorkspaceBuildJobCompletedworks by waiting until/api/v2/workspacesbuilds/{workspacebuild}/returns a workspace build with.Job.CompletedAt != nil. The issue here is that sometimes the returnedcodersdk.WorkspaceBuildcan contain a build from before a provisioner job completed, but contain the provisioner job from after it completed.Let me demonstrate:
Here we query the database for
database.WorkspaceBuild.coder/coderd/coderd.go
Lines 1409 to 1415 in a3f64f7
Inside of the
workspaceBuildroute handler, we callworkspaceBuildsDatacoder/coderd/workspacebuilds.go
Line 54 in a3f64f7
This then calls
GetProvisionerJobsByIDsWithQueuePositioncoder/coderd/workspacebuilds.go
Lines 852 to 856 in a3f64f7
As these two calls happen outside of a transaction, the state of the world can change underneath. This can result in an in-progress workspace build having a completed provisioner job attached to it.
Note: The change in this PR only touches the flakey test. The underlying cause of the flake isn't being fixed. I'm happy to expand the scope of this PR to fix the cause of the flake as that might also fix other flakes.