[Epic] Support bridging providers built with Terraform Plugin Framework

[t0yv0]

Overview

Enable building Pulumi providers from Terraform providers that are defined fully or in part using the new Terraform Plugin Framework by updating pulumi-terraform-bridge to understand it.

The work in progress is available as pf Go Module
which includes a README with instructions on
trying it out.

Support

#plugin-framework in Community Slack.

Timeline

2023 Q1

Releases:

• v0.1.0
• v0.2.0
• v0.2.1
• v0.3.0
• v0.4.0
• v0.5.0
• v0.6.0
• v0.6.1
• v0.7.0

Bridged Terraform Plugin Framework Providers ⚙️

Pulumi-maintained providers targeting an update soon:

• pulumi-random
• pulumi-tls

Other provider ideas, open for community suggestions and unblocking issues encountered on community ports:

• github.com/fly-apps/terraform-provider-fly
• github.com/hashicorp/terraform-provider-tls
• github.com/aiven/terraform-provider-aiven
• github.com/civo/terraform-provider-civo
• github.com/fortinetdev/terraform-provider-fortios
• github.com/tmeckel/terraform-provider-mssql

Work Items ⚙️

M0.79

• pulumi/home#2380
• pulumi/home#2387
• Investigate quick support options via muxing to v5 protocol

M0.80

• Read support
• Rewrote Diff/Create/Update with better understanding of state vs checkedInputs from Design syncs, remove TODOs
• Support more types on the wire
• Boolean
• List
• Map
• Support computing PackageSpec Pulumi schema from PF schema (by reusing tfgen)
• Resources
• Provider config
• Build new-style tfgen.Main
• Integration tests cover CRUD scenarios over random provider and test-only testbridge provider
• Unit tests replay gRPC logs from integration tests to check CRUD methods more quickly

M0.81

• Support Object types
• Ensure nested attributes work
• Ensure ID generation feature parity with prod tfbridge
• pulumi/home#2432
• pulumi/home#2452
  • Schema support for recognizing Int64 type #656
  • Fix race on ID allocation causing flaky tests #657
  • Keep language metadata #663
• Ensure provider bridge honors all tfgen-initiated property renames
  • Currently only handling pluralized List property names
  • Need to handle anything tfgen decides to do
  • Need to handle user-initiated name overrides set in ProviderInfo
  • Refine path model and path assumptions #661
  • Tabulate renaming decisions #649

M0.82

• pulumi/home#2470
• Ensure data sources schema generation support
• Recover Deprecated messages #697
• Fix lost descriptions when MarkdownDescription is not available #695

M0.83

• pulumi/home#2471
• pulumi/home#2472
• pulumi/home#2473

Review:

• Dedicated convert package #666
• Fix issues around confused nested block encoding #690
• Recognize Terraform blocks in the Plugin Framework #694
• Extract DataSource metadata to generate functions schema #696
• Fix a panic in renames module due to provider path confusion #698
• Refined ProposedNew  #703
• Flattened encoder support #707
• Propagate version into ProviderInfo when unset there #708
• Support invokes into Plugin Framework #715

M0.84

Dev:

• [Plugin Framework] No Major.Minor.Patch elements found #727 No Major.Minor.Patch
• [Plugin Framework] PropertyValue should be secret but is not #729 PropertyValue should be secret but is not
• pulumi/home#2570
• Implement State Upgrade in the Terraform Plugin Framework #758

Review:

https://github.com/pulumi/home/issues/2543 PF 1.0

M0.85

• [Plugin Framework] Support Tuple type #730
• Update pulumi-random to Plugin Framework v1.1.1 pulumi-random#257
• [Plugin Framework] Support State Upgrade #748
• [Plugin Framework] Support ignoreChanges correctly in the Diff method #751
• [Plugin Framework] Ensure CI runs tests #797
• Pulumi stack got stuck with random 4.9.0 version pulumi-random#265
• Apply callback on RandomInteger.result is executed in pulumi preview with string value pulumi-random#279
• Updating to version >=4.9.0 tries to recreate my certificates pulumi-tls#173
• Fix unknown value encoding in Plugin Framework #872
• [Plugin Framework] Verify defaults application #732
• Upgrade terraform-provider-tls to v4.0.4 pulumi-tls#149
• [Plugin Framework] Issues with output strings #812
• [Plugin Framework] Populate ReadResult.Inputs  #795
• [Plugin Framework] Support Resource Import #749

M0.86

• [1/3] [Plugin Framework] Init and connect Terraform logging to Pulumi logging #820
• [1/3] Implement stack upgrade testing for TLS provider pulumi-tls#175
• [1/3] [Plugin Framework] Implement MarshallableProviderInfo and GetMapping() #819
• [1] [Plugin Framework] Compute deleteBeforeReplace correctly #823
• [3] [Plugin Framework] Ensure that ProviderInfo customizations work properly or generate warnings #828
• [3] [Plugin Framework] Support Set type #731
• [5] [Plugin Framework] Support bridging providers which incorporate both TFPF and SDK resources #830
• [3] [Plugin Framework] Support CustomType #803
• [2] [Plugin Framework] Support Assets and Archives #750
• [1] [Plugin Framework] Support computing DetailedDiff in Diff method for best CLI experience #752
• [1] [Plugin Framework] Implement CheckConfig properly #821
• [1] [Plugin Framework] Validate resource configuration in Check #822
• [1] [Plugin Framework] Implement the DiffConfig method #753

To Triage

• [Plugin Framework] Support AutoName #917
• [Plugin Framework] Support v1.2.0 #918
• [Plugin Framework] IgnoreChanges only applies to Diff #920
• [Plugin Framework] Mark manually renamed AutoName resources as DeleteBeforeRelpace #930
• [Plugin Framework] Support UniqueNameFields #931
• [Plugin Framework] PreConfigureCallaback Support #935
• [Plugin Framework] Stable and ForceNew properties support #936
• [Plugin Framework] Support Transform Transformers #937
• [Plugin Framework] Support populating defaults from DefaultInfo #938

Remaining

• [Plugin Framework] Compute StableKeys in Diff #824
• [Plugin Framework] Ensure custom timeouts work properly #827
• [Plugin Framework] Propagate ProviderMeta correctly #794
• [Plugin Framework] Verify inherited client configuration support #754
• [Plugin Framework] Support Private State #747
• [Plugin Framework] Add a test for Read handling a not-found resource #793
• [Plugin Framework] Consider supporting running acc tests against bridged provider #829

[ryanrasti]

Wow -- what great timing! I was looking for this feature for writing our fly configuration with Pulumi (brought here by this thread).

Any idea on what the timeline for getting TPF support would be?

[t0yv0]

@ryanrasti it's my team's top priority for the quarter. What's available today is viable for small/simple providers and I'm working to itemize the list of known missing features and provides links to getting started. Several Pulumi partners are trying this out already and we will be dogfooding it in random and tls providers shortly.

Any feedback you might have or even just upvoting issues is super helpful!

[tmeckel]

@t0yv0 Actually I wanted to publish the PGSoft MSSQL Terraform Provider as a Pulumi provider using TF Bridge. pulumiverse/infra#59 But I wasn't aware of the fact that the provider is using the terraform-plugin-framework. So I suspect that I've to postpone my work until TF bridge will support terraform-plugin-framework.

That's a small bummer because I planned to use the Pulumi version of the provider in a customer project to fulfill compliance requirements. Sad that I could upvote this issue here only once 😁

[tmeckel]

@t0yv0 Any guidance on how to use the PF Version of TF Bridge? I could try to add/integrate the required steps to wrap a TF provider using the terraform-plugin-framework in my cookiecutter template to create new Pulumi providers based on TF Bridge.

[t0yv0]

@tmeckel sorry for late reply here, I have been keeping quite busy! The cookiecutter template support sounds quite exciting actually.

The README you link to is the best we have for the moment, but we will have some example repositories shortly. Our current goal is to start using it ourselves as soon as practical. The first attempt to release pulumi-random encountered a critical issue with unexpected replace plans on pf/v0.1.0, which is fixed by #748, and now I'm working to try to upgrade pulumi-random again.

There're some trade-offs with trying to use pf/tfbridge before it reaches GA - it may not be suitable for all providers yet based on incomplete feature set support. We try to keep the list in this issue up to date to help evaluate, and I'll start correlating it to versions as in a CHANGELOG shortly.

[t0yv0]

Sorry folks to ask for update the bookmarks, but according to our company process the remaining work rolls over to a continuation epic for Q2 2023: #956

[pulumi-bot]

Cannot close issue without required labels: resolution/