Terraform Providers: Migrating from SDK v2 to the Framework

Disclaimer: I am working at HashiCorp (now IBM) as part of the Terraform Core team. The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.
Since I am involved in Terraform my opinions can sometimes be (unconsciously) biased. I hope you enjoy the post anyway.

Open Table of Contents

Why?
Where can I get more details?
Status Quo
Adding terraform-plugin-mux to the provider
Adding a framework provider
Aligning the provider schemas
Adding your first framework resource

Why?

As the SDKv2 documentation states

We recommend using the framework to develop new provider functionality because it offers significant advantages as compared to the SDKv2. We also recommend migrating existing providers to the framework when possible.

The framework has a ton of benefits that I won’t go through in detail, but all boil down to a better developer experience. The future of provider development lives in the Framework so it makes sense to move your provider over.

To facilitate this you can mux your provider so that parts of the implementation come from the Framework (and all new resources can be added through the Framework) and parts can stay in the SDK. After setting this up you can shift to the framework resource by resource.

Where can I get more details?

Selena Goods did a great workshop around Ephemeral Values in which she describes the process of muxing (start at minute 20 for that, but don’t the rest is also very interesting): Ephemeral Values Workshop

Status Quo

I will reference the ansible/ansible provider for this entire blog post since I just added muxing there (to get access to the cool new framework features). Let’s take a quick look at the main.go for this provider:

// Source: https://github.com/ansible/terraform-provider-ansible/blob/main/main.go#L1-L15
package main

import (
	"github.com/ansible/terraform-provider-ansible/provider"
	"github.com/hashicorp/terraform-plugin-sdk/v2/plugin"
)

// Generate the Terraform provider documentation using `tfplugindocs`:
//go:generate go run github.com/hashicorp/terraform-plugin-docs/cmd/tfplugindocs

func main() {
	plugin.Serve(&plugin.ServeOpts{
		ProviderFunc: provider.Provider,
	})
}

It is like in most providers very simple:

Import the provider from the local provider package
Import terraform-plugin-sdk/v2/plugin for the SKDv2 plugin functionality
Run plugin.Serve with the ProviderFunc being provider.Provider

Adding `terraform-plugin-mux` to the provider

To add terraform-plugin-mux to the provider, you need to install it via go get:

go get github.com/hashicorp/terraform-plugin-mux

Then you create a list of providers (called providers in the code snippet) we want to multiplex between, which is for now just the one SDK provider. Next we create a new mux server using these providers (saved into muxServer in the code snippet below) and use the ProviderServer to serve the muxed provider using tf5server.Serve. You need to add the registry URL of the muxed provider here as well.

// Source: https://github.com/ansible/terraform-provider-ansible/blob/43cb82a29b8f82f97c72bc194c165f0290fc83b7/main.go#L1-L36
package main

import (
	"context"
	"log"

	"github.com/ansible/terraform-provider-ansible/provider"
	"github.com/hashicorp/terraform-plugin-go/tfprotov5"
	"github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server"
	"github.com/hashicorp/terraform-plugin-mux/tf5muxserver"
	"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

// Generate the Terraform provider documentation using `tfplugindocs`:
//go:generate go run github.com/hashicorp/terraform-plugin-docs/cmd/tfplugindocs

func main() {
	ctx := context.Background()
	providers := []func() tfprotov5.ProviderServer{
		func() tfprotov5.ProviderServer {
			return schema.NewGRPCProviderServer(provider.Provider())
		},
	}

	muxServer, err := tf5muxserver.NewMuxServer(ctx, providers...)
	if err != nil {
		log.Fatal(err)
	}

	var serveOpts []tf5server.ServeOpt

	err = tf5server.Serve("registry.terraform.io/ansible/ansible", muxServer.ProviderServer, serveOpts...)
	if err != nil {
		log.Fatal(err)
	}
}

Adding a framework provider

Now we need to create a framework provider to mux to. For this we create a new package called framework and create it on the root level of the repository.

Please ignore the mention of "github.com/hashicorp/terraform-plugin-framework/action" in the import block of the code snippet, this hints to the reason I am muxing this provider, but is not relevant for the muxing process.

// Source: https://github.com/DanielMSchmidt/terraform-provider-ansible/blob/1e7be2e0a82308cbf0093c0473a2f896dc6ee975/framework/provider.go#L1-L53
package framework

import (
	"context"

	"github.com/hashicorp/terraform-plugin-framework/action"
	"github.com/hashicorp/terraform-plugin-framework/datasource"
	"github.com/hashicorp/terraform-plugin-framework/provider"
	"github.com/hashicorp/terraform-plugin-framework/provider/schema"
	"github.com/hashicorp/terraform-plugin-framework/resource"
)

var _ provider.Provider = &fwprovider{}

// New returns a new, initialized Terraform Plugin Framework-style provider instance.
// The provider instance is fully configured once the `Configure` method has been called.
func New(primary interface{ Meta() interface{} }) provider.Provider {
	return &fwprovider{
		Primary: primary,
	}
}

type fwprovider struct {
	Primary interface{ Meta() interface{} }
}

func (f *fwprovider) Metadata(ctx context.Context, request provider.MetadataRequest, response *provider.MetadataResponse) {
	response.TypeName = "ansible"
}

func (f *fwprovider) Schema(ctx context.Context, request provider.SchemaRequest, response *provider.SchemaResponse) {
	response.Schema = schema.Schema{
		Attributes: map[string]schema.Attribute{},
		Blocks:     map[string]schema.Block{},
	}
}

func (f *fwprovider) Configure(ctx context.Context, request provider.ConfigureRequest, response *provider.ConfigureResponse) {
	// Provider's parsed configuration (its instance state) is available through the primary provider's Meta() method.
	v := f.Primary.Meta()
	response.DataSourceData = v
	response.ResourceData = v
	response.EphemeralResourceData = v
	response.ActionData = v
}

func (f *fwprovider) DataSources(ctx context.Context) []func() datasource.DataSource {
	return nil
}

func (f *fwprovider) Resources(ctx context.Context) []func() resource.Resource {
	return nil
}

This snippet is basically the empty blueprint for a framework provider. Before we make changes to it let’s walk through the interesting part:

We create the framework provider based off of the plugin SDK provider (See New function). This helps us with forwarding the Metadata that the configure call in which Terraform sets the metadata in the SDK provider to the framework provider. In the Configure method we call the SDK provider’s Meta() method where this data is stored and pass it on to the configure response (again, ignore the ActionData).

Now let’s see how we initialize the framework provider in the main.go file.

// Source: https://github.com/DanielMSchmidt/terraform-provider-ansible/blob/1e7be2e0a82308cbf0093c0473a2f896dc6ee975/main.go#L1-L40
package main

import (
	"context"
	"log"

	"github.com/ansible/terraform-provider-ansible/framework"
	"github.com/ansible/terraform-provider-ansible/provider"
	"github.com/hashicorp/terraform-plugin-framework/providerserver"
	"github.com/hashicorp/terraform-plugin-go/tfprotov5"
	"github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server"
	"github.com/hashicorp/terraform-plugin-mux/tf5muxserver"
	"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

// Generate the Terraform provider documentation using `tfplugindocs`:
//go:generate go run github.com/hashicorp/terraform-plugin-docs/cmd/tfplugindocs

func main() {
	ctx := context.Background()
	primary := provider.Provider()
	providers := []func() tfprotov5.ProviderServer{
		func() tfprotov5.ProviderServer {
			return schema.NewGRPCProviderServer(primary)
		},
		providerserver.NewProtocol5(framework.New(primary)),
	}

	muxServer, err := tf5muxserver.NewMuxServer(ctx, providers...)
	if err != nil {
		log.Fatal(err)
	}

	var serveOpts []tf5server.ServeOpt

	err = tf5server.Serve("registry.terraform.io/ansible/ansible", muxServer.ProviderServer, serveOpts...)
	if err != nil {
		log.Fatal(err)
	}
}

We create the primary provider outside the providers list and our providers list got a new entry for the framework provider which is initialized with the sdk provider. That’s all we needed to do to set everything up.

Aligning the provider schemas

The last thing we need to do is to sync up the schemas for the only shared item between the SDK provider and the framework provider: The provider configuration schema.

In the case of the ansible/ansible provider there is no configuration schema:

// Source: https://github.com/DanielMSchmidt/terraform-provider-ansible/blob/3321a4ba57d73f2f6a9b041ef8b0ccba09653448/provider/provider.go#L7-L18
// Provider exported function.
func Provider() *schema.Provider {
	return &schema.Provider{
		ResourcesMap: map[string]*schema.Resource{
			"ansible_playbook": resourcePlaybook(),
			"ansible_vault":    resourceVault(),
			"ansible_host":     resourceHost(),
			"ansible_group":    resourceGroup(),
		},
	}
}

Let’s imagine this provider configuration schema:

func Provider() *schema.Provider {
	return &schema.Provider{
		Schema: map[string]*schema.Schema{
			"ansible_playbook_binary": {
				Type:        schema.TypeString,
				Required:    true,
				Optional:    false,
				Description: "ansible-playbook binary path.",
			},
		},
		ResourcesMap: map[string]*schema.Resource{

The equivalent would look like this in the framework provider:

func (f *fwprovider) Schema(ctx context.Context, request provider.SchemaRequest, response *provider.SchemaResponse) {
	response.Schema = schema.Schema{
		Attributes: map[string]schema.Attribute{
			"ansible_playbook_binary": schema.StringAttribute{
				Required:    true,
				Optional:    false,
				Description: "ansible-playbook binary path.",
			},
		},
		Blocks: map[string]schema.Block{},
	}
}

Adding your first framework resource

HashiCorp has great tutorials for how to create providers with the Terraform Plugin Framework, please take a look!