Inside Terraform: References - How Terraform Connects the Dots

Disclaimer: I am working at HashiCorp (now IBM) as part of the Terraform Core team. The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.
Since I am involved in Terraform my opinions can sometimes be (unconsciously) biased. I hope you enjoy the post anyway.

This is part of my Inside Terraform series where I deep dive into different parts of Terraform and explain how they work under the hood. This is the second post in the series, the first one is about the addrs package.

I’m still experimenting with the format of these posts, so please let me know if you have feedback or suggestions for improvement.

What are References?

When you write Terraform code you often need to refer to other entities in your configuration. For example, you might want to refer to an attribute of a resource in another resource’s argument:

resource "aws_instance" "example" {
  ami           = "ami-123456"
  instance_type = "t2.micro"
}

resource "aws_eip" "example_eip" {
  instance = aws_instance.example.id // <-- This is a reference
}

References allow you to connect different parts of your configuration together. Terraform needs to understand these references to build the dependency graph, plan changes, and apply them correctly.

References can be within the same module (e.g. local.aws_region / var.hello / aws_instance.example.arn) or reference module outputs (e.g. module.foo.output_name). References can point to a single attribute / resource or to multiple ones (e.g. aws_instance.example[*].id).

Why do we need References?

References are what we need to form the dependency graph of Terraform. Since we will evaluate all dependencies before evaluating a given entity, we will have all the data to evaluate the entity correctly.

How to find References

If you want to find references in an HCL expression or in an HCL Block you can use the langrefs package. It has the commonly used functions langrefs.ReferencesInExpr and langrefs.ReferencesInBlock that return all references found in the given expression or block. You pass in addrs.ParseRef as the parser of the references and the aforementioend functions will return a slice of addrs.Reference structs.

A common use-case is the References function we implement on nodes in the graph to ensure the right order of evaluation. This is an example for an action, but most nodes will have a similar implementation:

// Source: https://github.com/hashicorp/terraform/blob/cf047be4e4ad9417b2f40217a74a1f8386f69c98/internal/terraform/node_action_abstract.go#L68-L83
func (n *NodeAbstractAction) References() []*addrs.Reference {
	var result []*addrs.Reference
	c := n.Config

	refs, _ := langrefs.ReferencesInExpr(addrs.ParseRef, c.Count)
	result = append(result, refs...)
	refs, _ = langrefs.ReferencesInExpr(addrs.ParseRef, c.ForEach)
	result = append(result, refs...)

	if n.Schema != nil {
		refs, _ = langrefs.ReferencesInBlock(addrs.ParseRef, c.Config, n.Schema.ConfigSchema)
		result = append(result, refs...)
	}

	return result
}

We evaluate the count and for_each meta-argument and then find all references in the config { block of the action. We then return all found references as a slice.

The addrs.ParseRef function is normally used to parse the references from hcl.Traversal (which is basically an AST of a reference) to addrs.Reference structs. The addrs.Reference struct contains a Subject which is a addrs.Referencable address. Referencable is a marker interface that is implemented by all address types that can be referenced (e.g. InputVariable, LocalValue, ModuleCall, etc.). The struct also contains the SourceRange of the reference in the HCL file for error reporting purposes and a Remaining attribute in case the access to the reference is not fully consumed (e.g. if you have aws_instance.example.tags["Name"], the Subject will be the aws_instance.example resource and the Remaining will contain the tags["Name"] access as an hcl.Traversal).

So far I have encountered no use-case where I needed to pass something else than addrs.ParseRef to the langrefs functions.

How to make use of References

Aside from implementing the GraphNodeReferencer interface on graph nodes to ensure correct evaluation order, references are also used in the evaluation phase. We will use references to look up the value of the referenced entity and pass it to HCL for evaluation. This will be the content of the next post in this series when we look at go-cty and the evaluation of values.

// Source: https://github.com/hashicorp/terraform/blob/cf047be4e4ad9417b2f40217a74a1f8386f69c98/internal/moduletest/graph/transform_references.go#L16-L18
type GraphNodeReferencer interface {
	References() []*addrs.Reference
}

You can also use references for validation or other purposes. What is normally done then is a type switch or type cast on the Subject of the reference to check what kind of entity is being referenced. Then one can make assertions on the referenced entity or look up additional information about it. This is an example from the depends_on attribute of resources, which is not allowed to reference actions:

// Source: https://github.com/hashicorp/terraform/blob/cf047be4e4ad9417b2f40217a74a1f8386f69c98/internal/terraform/node_resource_validate.go#L814-L827
		// We don't allow depends_on on actions because their ordering is depending on the resource
		// that triggers them, therefore users should use a depends_on on the resource instead.

		if ref != nil {
			switch ref.Subject.(type) {
			case addrs.Action, addrs.ActionInstance:
				diags = diags.Append(&hcl.Diagnostic{
					Severity: hcl.DiagError,
					Summary:  "Invalid depends_on reference",
					Detail:   "Actions can not be referenced in depends_on. Use depends_on on the resource that triggers the action instead.",
					Subject:  traversal.SourceRange().Ptr(),
				})
			}
		}

One easy mistake to make is to not cover the expanded address and only use addrs.Action in the type switch. This would miss references to individual instances of an action, e.g. action.my_action.name[3].