Skip to content

Add additional protections for persistent resources #12705

@denbeigh2000

Description

@denbeigh2000

Background

Some elements of a Coder template are expected to be ephemeral (e.g., a container or VM), whereas some elements are persistent and will cause user disruption if they are lost between workspace restarts (e.g., a home disk).

While these resources can be persisted with the start_count or transition fields, sometimes drift occurs outside the template author's control, which causes terraform to recreate these resources without a confirmation.

Example

We recently experienced an issue where a change in the policy attachment used for backups was updated, causing both the policy attachment and the disk it was attached to to be replaced on the next start of the workspace, leading to the loss of their home disk.

Screenshot 2024-03-21 at 08 08 59

Existing approaches

There are a few ways to mitigate this in terraform itself, but all have various shortcomings:

  • adding lifecycle>ignore_changes to certain fields is reasonable, but not comprehensive: it can only guard against failure modes one is already aware of.
  • adding prevent_destroy to resources makes Coder unable to delete the resources when the workspace is intentionally deleted
  • adding destroy-time provisioners to resources isn't feasible, because destroy-time provisioners can only access attributes of the resource they're attached to, which excludes transition and start_count (and would also only provide this protection on workspace start, because start_count/transition cannot indicate a delete operation)

Request

Coder performs a terraform plan before any execution, and prints a summary of what will be added, changed and deleted.
These should be some way of annotating resources in a template, such that if Coder intends to delete them outside of a workspace deletion operation, it will either fail or require additional confirmation before continuing.

This would prevent an entire class of mistakes that are easy to run into, and would give operators confidence that even if there is some mistake in their template, Coder will avoid deleting resources that they do not want to lose.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions