Deprecating Terraform Variables and Outputs

Content

What’s changing?
Let’s talk code!

What’s changing?

In Terraform 1.15, we will allow you to set deprecated=<STRING> attributes on variables and outputs. You can now also set ignore_nested_deprecations=<BOOL> on module calls. Using these values will result in deprecation warnings being shown during validate, plan, and apply.

A full configuration example looks like this:

# main.tf

## The variable will show a deprecation warning if a value is passed into it.
variable "foo" {
  type = string
  deprecated = "Use bar: Moving from a comma separated list to a list of strings"
}

variable "bar" {
  type = list(string)
}

module "modA" {
  source = "./modules/moduleA"

  # This will show a deprecation warning because we are passing
  # a value into the deprecated variable "hello".
  hello = "a, b, c"

  # This will mute all deprecation warnings within this module or deeper.
  ignore_nested_deprecations = true
}

locals {
  # This will show a deprecation warning because we are 
  # referencing the deprecated output "goodbye".
  goodbye = module.modA.goodbye
}

output "foo" {
  value = var.foo
  # ⚠️⚠️⚠️ You can not deprecate root module outputs.
  # You should be able to change them right away.
  deprecated = "Oh no! This is not allowed!"
}

# modules/moduleA/main.tf
variable "hello" {
  type = string
  deprecated = "Use world: Moving from a comma separated list to a list of strings"
}

output "goodbye" {
  value = "goodbye"
  deprecated = "This output is deprecated"
}

Variables

When you set deprecated on a variable, Terraform will print a warning whenever a value is passed into the variable. So if you pass a value into a module call for a deprecated variable, you will get a warning with the deprecation message you set in the deprecated attribute. Also, if you set a deprecated variable on the root module (e.g. through -var or TF_VAR_), you will get the warning as well.

We won’t show errors where the value is references, since it’s the module caller that needs to do the work of adapting to the deprecation. When the deprecation grace period is over you can just release a new version of your module without the deprecated variable and replace all references to the old variable with the new one.

Outputs

When you set deprecated on an output, Terraform will print a warning whenever the output is referenced with the deprecation message. This includes using the output in a temporary value e.g. a local; we think deprecated values should be avoided in all places.

When a child module of your module deprecates an output, you will also get a warning when you reference that output. If you reference the child module output in an output of your own, you can also set a deprecation message on your output to propagate the deprecation up to your users.

Ignoring nested deprecations

When you set ignore_nested_deprecations on a module call, Terraform will not show any deprecation warnings for deprecated variables and outputs in that module or any of its child modules. This is useful when you are using a module that has deprecations but is outside your control to fix. It allows you to mute the warnings for now and still get them for your own code. We recommend using this as a temporary measure and fixing the deprecations in the module as soon as possible.

Caveats

Not all deprecations can be detected during validation, so it can happen that terraform validate has no warnings, but terraform plan and terraform apply do. This is the case for deprecations on outputs of child modules with for_each or count due to how we deal with unknown data during validation.

Want to have it today?

We already shipped an alpha with this feature, so feel free to give it a try! You can find the alpha releases here. We would love to get your feedback on the implementation and the user experience of this feature, so please give it a try and let us know what you think!

Let’s talk code!

If you are using Terraform and not interested in the implementation details, you can stop reading here. There won’t be any new, relevant information for you in the rest of this post. If you are interested in how this is implemented in Terraform, read on!

On your Marks

Marks are a way to annotate values in Terraform with additional information. I wrote a bit about them in Inside Terraform, but the TL;DR is you can attach a mark to a value and detect a mark being used during evaluation.

// Source: https://github.com/hashicorp/terraform/blob/a48e873790c07bd5d0a38e4ddb651ddd2d819219/internal/lang/marks/marks.go#L114-L134
// DeprecationMark is a mark indicating that a value is deprecated. It is a struct
// rather than a primitive type so that it can carry a deprecation message.
type DeprecationMark struct {
	Message string

	OriginDescription string // a human-readable description of the origin
}

func (d DeprecationMark) GoString() string {
	return "marks.deprecation<" + d.Message + ">"
}

// Empty deprecation mark for usage in marks.Has / Contains / etc
var Deprecation = NewDeprecation("", "")

func NewDeprecation(message string, originDescription string) DeprecationMark {
	return DeprecationMark{
		Message:           message,
		OriginDescription: originDescription,
	}
}

So far we only had string marks (e.g. sensitive or ephemeral), but deprecations are different since it’s not a simple “this is sensitive”, but it’s rather “this is deprecated with this message”. So it’s rather a class of marks with each instance being a different deprecation message. This added some extra complexity to the implementation, but it was worth it since it allows us to have a very flexible implementation of deprecations.

Marks are contagious in the sense that a value derived from a value with a mark will also have that mark (e.g. functions).

During the evaluation of a the result of a module we add the mark to the output values of the module:

// Source: https://github.com/hashicorp/terraform/blob/a48e873790c07bd5d0a38e4ddb651ddd2d819219/internal/terraform/evaluate.go#L419-L446
		atys := make(map[string]cty.Type, len(outputConfigs))
		as := make(map[string]cty.Value, len(outputConfigs))
		for name, c := range outputConfigs {
			atys[name] = cty.DynamicPseudoType // output values are dynamically-typed
			val := cty.UnknownVal(cty.DynamicPseudoType)
			if c.DeprecatedSet {
				accessor := "."
				switch {
				case callConfig.Count != nil:
					accessor = ".[*]."
				case callConfig.ForEach != nil:
					accessor = ".[*]."
				}
				val = val.Mark(marks.NewDeprecation(c.Deprecated, fmt.Sprintf("%s%s%s", addr.String(), accessor, name)))
			}
			as[name] = val
		}
		instTy := cty.Object(atys)

		switch {
		case callConfig.Count != nil:
			return cty.UnknownVal(cty.List(instTy)), diags
		case callConfig.ForEach != nil:
			return cty.UnknownVal(cty.Map(instTy)), diags
		default:
			val := cty.ObjectVal(as)
			return val, diags
		}

We also do this for resources and in all other phases, just had to pick one example here.

Then during the evaluation of a reference to an output, we check if the value has a deprecation mark and if it does, we add a warning to the diagnostics. In this example we are in the action node:

// Source: https://github.com/hashicorp/terraform/blob/a48e873790c07bd5d0a38e4ddb651ddd2d819219/internal/terraform/node_action_instance.go#L74-L76
		var deprecationDiags tfdiags.Diagnostics
		configVal, deprecationDiags = ctx.Deprecations().ValidateConfig(configVal, n.Schema.ConfigSchema, n.ModulePath())
		diags = diags.Append(deprecationDiags.InConfigBody(n.Config.Config, n.Addr.String()))

As you can see we run the ValidateConfig on a global deprecations object. We have this global object to keep track of the modules that suppress the deprecation warnings.

// Source: https://github.com/hashicorp/terraform/blob/a48e873790c07bd5d0a38e4ddb651ddd2d819219/internal/deprecation/deprecation.go#L108-L144
func (d *Deprecations) ValidateConfig(value cty.Value, schema *configschema.Block, module addrs.Module) (cty.Value, tfdiags.Diagnostics) {
	var diags tfdiags.Diagnostics
	unmarked, pvms := value.UnmarkDeepWithPaths()

	if d.IsModuleCallDeprecationSuppressed(module) {
		// Even if we don't want to get deprecation warnings we want to remove the marks
		return unmarked.MarkWithPaths(marks.RemoveAll(pvms, marks.Deprecation)), diags
	}

	for _, pvm := range pvms {
		for m := range pvm.Marks {
			if depMark, ok := m.(marks.DeprecationMark); ok {
				diag := tfdiags.AttributeValue(
					tfdiags.Warning,
					"Deprecated value used",
					depMark.Message,
					pvm.Path,
				)
				if depMark.OriginDescription != "" {
					diag = tfdiags.Override(
						diag,
						tfdiags.Warning, // We just want to override the extra info
						func() tfdiags.DiagnosticExtraWrapper {
							return &tfdiags.DeprecationOriginDiagnosticExtra{
								// TODO: Remove common prefixes from origin descriptions?
								OriginDescription: depMark.OriginDescription,
							}
						})
				}

				diags = diags.Append(diag)

			}
		}
	}

	return unmarked.MarkWithPaths(marks.RemoveAll(pvms, marks.Deprecation)), diags

Here we go through all the marks, find the deprecation marks and emit a diagnostic for each of them. We then remove just the deprecation marks from the value and return the diagnostics and the cleaned value. This allows us to keep the deprecation marks from being emitted multiple times for the same value, but still keep other marks (e.g. sensitive) intact.

Good error messages are a love language

As you can see in the code snippets above, we use a Diagnostic Extra to give you the information about where the deprecation is coming from (e.g. which module and which variable or output is deprecated). This is especially important for outputs since they can be referenced in multiple places and it’s not always clear where the deprecation is coming from.

An example deprecation message

Central place to track deprecation suppression

On the EvalContext we have a Deprecations field that keeps track of which modules have deprecation warnings suppressed. You can see the value being read in the snippet above.

When we evaluate a module call, we check if the ignore_nested_deprecations attribute is set and if it is, we add the module to the list of modules that have deprecation warnings suppressed:

// Source: https://github.com/hashicorp/terraform/blob/a48e873790c07bd5d0a38e4ddb651ddd2d819219/internal/terraform/node_module_expand.go#L110-L117
// GraphNodeExecutable
func (n *nodeExpandModule) Execute(globalCtx EvalContext, op walkOperation) (diags tfdiags.Diagnostics) {
	expander := globalCtx.InstanceExpander()
	_, call := n.Addr.Call()

	if n.ModuleCall.IgnoreNestedDeprecations {
		globalCtx.Deprecations().SuppressModuleCallDeprecation(n.Addr)
	}

Since we are sure that the module call will be evaluated before any of the outputs of the module are referenced, we can be sure that this will work as expected and that no deprecation warnings will be emitted for deprecated variables and outputs in that module or any of its child modules.