Terraform Providers with Recursive Patterns

Disclaimer: I am working at HashiCorp (now IBM) as part of the Terraform Core team. The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.
Since I am involved in Terraform my opinions can sometimes be (unconsciously) biased. I hope you enjoy the post anyway.

Prerequisites

This blog posts expects you to be familiar with writing Terraform Providers using the Terraform Plugin Framework. If you need to update your provider to use the Plugin Framework, check out this guide.

What do I mean with recursive patterns?

Some APIs allow for patterns that repeat themselves. The aws_wafv2_rul_group resource has for example a statement block that can contain another statement block, which can again contain another statement block, and so on. (Fun fact: This resource caused a lot of trouble in the code generation of Terraform CDK due to the sheer size of the schema and to-be-generated code.)

Another example (and the one I will be using in this blog post since I just encountered it today) is an Ansible Inventory. An Ansible Inventory can contain groups, and groups can contain other groups. This allows for nesting groups within groups to create a hierarchy of hosts.

Escape Hatch

Before diving into how to deal with recursion within the provider code, I want to mention that there is a pragmatic escape hatch one can use instead of implementing recursion in the provider code: Using JSON or YAML strings (as mentioned in this discussion). If this provides a good enough developer experience for your users, this is a valid approach to take.

Recursive Schemas

When it comes to the resource / data source / action schema definition, we can abstract recursive patterns by defining a function that returns the schema definition for the recursive block. This function can then call itself to define the nested blocks.

For the ansible/ansible provider I defined a function like this within the Schema method of the ansible_inventory resource I have been implementing.

func (i *InventoryResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
	createGroupBlock := func(blockOverrides map[string]schema.Block) schema.ListNestedBlock {
	    blocks := map[string]schema.Block{
            // all the blocks aside from the nested ones
		}
	
		maps.Copy(blocks, blockOverrides)
		return schema.ListNestedBlock{
    		// ...
    		Blocks: blocks,
		}
	}

    // This schema is nested, we will support 3 levels of nesting.
	thirdLevelGroupBlock := createGroupBlock(map[string]schema.Block{})
	secondLevelGroupBlock := createGroupBlock(map[string]schema.Block{
		"group": thirdLevelGroupBlock,
	})
	firstLevelGroupBlock := createGroupBlock(map[string]schema.Block{
		"group": secondLevelGroupBlock,
	})
	resp.Schema = schema.Schema{
		// ...
		Blocks: map[string]schema.Block{
            "group": firstLevelGroupBlock,
        },
	}
}

As you can see we end the recursion after 3 levels of nesting since Terraform does not support truely recursive schemas. In most cases, 3 levels of nesting should be sufficient.

Recursive Parsing

Now comes the part that is a bit more tricky: Parsing data of (partially) recursive structures. I tried various things but since the Plugin Framework needs to parse the data into strongly typed Go structs where no field is optional and all fields of the model and the real data need to match there is no way to support true recursion on the parsing side either.

I would have loved to define a recursive function that just checks if the recursive groups block is set and supports arbitrary levels of nesting, but this is not possible with the Plugin Framework.

Instead, the best way to solve this is to keep track of the level of nesting and define separate models for the nested and unnested versions of the data structure. One thing that saves us a lot of effort here is that you can embed the common fields of the nested and unnested versions of the data structure into a common struct.

// Source: https://github.com/DanielMSchmidt/terraform-provider-ansible/blob/259c93fa1fdced54a59107734c4e19697f4483af/framework/resource_inventory.go#L31-L49
type InventoryResourceModel struct {
	Path   types.String `tfsdk:"path"`
	Groups types.List   `tfsdk:"group"`
}

type SharedGroupModel struct {
	Name  types.String `tfsdk:"name"`
	Vars  types.Map    `tfsdk:"vars"`
	Hosts types.List   `tfsdk:"host"`
}

type NestedGroupModel struct {
	SharedGroupModel
	Groups types.List `tfsdk:"group"`
}

type FinalGroupModel struct {
	SharedGroupModel
}

This way we can easily parse the common part in both cases. In this resource I parse the data and immediately convert it to JSON since I want to output a JSON ansible inventory file. So I defined a func sharedGroupToJson(group SharedGroupModel) (map[string]json.RawMessage, diag.Diagnostics) function and can use this for the final neseting level and in my nestedGroupToJson function:

// Source: https://github.com/DanielMSchmidt/terraform-provider-ansible/blob/259c93fa1fdced54a59107734c4e19697f4483af/framework/resource_inventory.go#L114-L132
func nestedGroupToJson(group NestedGroupModel, level int) (map[string]json.RawMessage, diag.Diagnostics) {
	jsonValue, diags := sharedGroupToJson(group.SharedGroupModel)
	if diags.HasError() {
		return nil, diags
	}

	groupsJson, diags := groupsToJson(group.Groups, level+1)
	if diags.HasError() {
		return nil, diags
	}
	b, err := json.Marshal(groupsJson)
	if err != nil {
		diags.Append(diag.NewErrorDiagnostic("Could not marshal nested groups to JSON", err.Error()))
		return nil, diags
	}
	jsonValue["children"] = b

	return jsonValue, diags
}

As you can see, I increase the nesting level each time I go into a deeper level, so let’s see how groupsToJson handles the different levels:

// Source: https://github.com/DanielMSchmidt/terraform-provider-ansible/blob/259c93fa1fdced54a59107734c4e19697f4483af/framework/resource_inventory.go#L67-L113
func groupsToJson(list types.List, level int) (map[string]json.RawMessage, diag.Diagnostics) {
	jsonValue := map[string]json.RawMessage{}
	var diags diag.Diagnostics

	if level < groupNestingLevel {
		// There is a deeper nesting level
		var nestedGroups []NestedGroupModel
		diags := list.ElementsAs(context.Background(), &nestedGroups, false)
		if diags.HasError() {
			return nil, diags
		}
		for _, group := range nestedGroups {
			groupJson, diags := nestedGroupToJson(group, level)
			if diags.HasError() {
				return nil, diags
			}
			b, err := json.Marshal(groupJson)
			if err != nil {
				diags.Append(diag.NewErrorDiagnostic("Could not marshal group to JSON", err.Error()))
			}

			jsonValue[group.Name.ValueString()] = b
		}
	} else {
		// This is the final nesting level
		var finalGroups []FinalGroupModel
		diags := list.ElementsAs(context.Background(), &finalGroups, false)
		if diags.HasError() {
			return nil, diags
		}

		for _, group := range finalGroups {
			groupJson, diags := sharedGroupToJson(group.SharedGroupModel)
			if diags.HasError() {
				return nil, diags
			}
			b, err := json.Marshal(groupJson)
			if err != nil {
				diags.Append(diag.NewErrorDiagnostic("Could not marshal group to JSON", err.Error()))
			}

			jsonValue[group.Name.ValueString()] = b
		}
	}

	return jsonValue, diags
}

If you want to see the full implementation of this resource, check out the pull request where I implemented it.

Conclusion

True recursion does not exist in Terraform Provider development, neither on the schema side nor on the parsing side. However, with some clever abstractions we can implement support for recursive patterns in our providers.

Please leave a comment

Do you have feedback around actions or do you want to hear about a specific topic around Terraform / Software Development / Language Design / Infrastructure as Code? Please let me know in the comments below.