Enable Plugin Framework based resources and functions for pulumi-gcp #1075

iwahbe · 2023-05-01T18:08:43Z

Fixes #1051

Most of the diff in the PR comes from upgrading the bridge to the latest that can be done as a separate PR.

Here is also some context on deploying pulumi/pulumi-terraform-bridge#981 providers with a muxed SDKv2/PF design.

This currently does not have any known blockers and smaller providers are now in prod exercising the necessary code.

Per @iwahbe testing we are unable to test the new datasource due to test account limitations.

github-actions · 2023-05-01T19:19:06Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

iwahbe · 2023-05-01T22:43:00Z

This change adds a single datasource: getAndroidAppConfig. In an attempt to test the datasource, I tried to deploy an AndroidApp. This is the message I got:

Diagnostics:
  gcp:firebase:AndroidApp (app):
    error: 1 error occurred:
        * Error creating AndroidApp: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the firebase.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.
    Details:
    [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "domain": "googleapis.com",
        "metadata": {
          "consumer": "projects/764086051850",
          "service": "firebase.googleapis.com"
        },
        "reason": "SERVICE_DISABLED"
      }
    ]

github-actions · 2023-05-17T23:20:41Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

kpitzen · 2023-05-18T18:17:37Z

Hey @t0yv0 and @iwahbe - thanks for pushing this work forward! Does adding this change the upgrade / release process for the provider re: what's currently in our published playbooks?

t0yv0 · 2023-05-18T18:34:59Z

@kpitzen not intentionally. Can you point me to the runbooks I can work with you on that. One obvious change is the new dependency on pulumi-terraform-bridge/pf module, that needs to be kept up to date.

t0yv0 · 2023-05-18T18:35:41Z

FYI Ian is on PTO until May 30 but I can answer any questions in the meanwhile.

danielrbradley

Happy with all the implementation.

Let's split this PR so we've got a simple bridge upgrade PR first so we can see if there's any significant SDK changes.

Looking at we've reverted it looks like only data sources have been implemented in tfpf so far - so we're not seeing any added resources?

Once we're ready to merge, I'd suggest also including into this PR reverting framework revert patch from the fork as part of the upgrade so we don't carry it forward

danielrbradley · 2023-05-22T09:10:18Z

provider/resources.go

 // Provider returns additional overlaid schema and metadata associated with the gcp package.
 func Provider() tfbridge.ProviderInfo {
 	p := shimv2.NewProvider(google.Provider())
+	p = pf.MuxShimWithPF(context.Background(), p, google.New(""))


What's the significance of the google.New("") - why the empty string?

It's a version. Perhaps version.Version would be more appropriate here, something we could try.

Looks like the version is used by the pluginFrameworkProvider in particular in responses to GetMetadata function but I doubt we depend on that being accurate.

// New is a helper function to simplify provider server and testing implementation. func New(version string) provider.ProviderWithMetaSchema { return &frameworkProvider{ version: version, } }

t0yv0 · 2023-05-23T16:52:37Z

I'm curious if there's automation to keep the bridge reference up-to-date, it's a chore.

Trying this:

https://github.com/pulumi/pulumi-gcp/actions/runs/5059976598

t0yv0 · 2023-05-23T16:53:56Z

Looking at we've reverted it looks like only data sources have been implemented in tfpf so far - so we're not seeing any added resources?

Yes, this is a bit strange; do you know which resources we expect? Is this because some PF resources were poly-filled in a different way? Can you give some pointers?

Once we're ready to merge, I'd suggest also including into this PR reverting framework revert patch from the fork as part of the upgrade so we don't carry it forward

Any pointers?

t0yv0 · 2023-05-23T17:20:33Z

#1090 baseline bridge update

t0yv0 · 2023-05-23T19:40:44Z

Updated the PR. The diff is much smaller and highlights that's only one method is being added.

github-actions · 2023-05-23T19:45:15Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

github-actions · 2023-05-23T19:46:53Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

github-actions · 2023-05-23T19:49:21Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

t0yv0 · 2023-05-26T14:08:39Z

Per conv with @mikhailshilkov this provider has not been patching PF-based resources. Checks on upstream code. No PF resources:

// Resources defines the resources implemented in the provider.
func (p *frameworkProvider) Resources(_ context.Context) []func() resource.Resource {
	return nil
}

We should be seeing more data sources though:


// DataSources defines the data sources implemented in the provider.
func (p *frameworkProvider) DataSources(_ context.Context) []func() datasource.DataSource {
	return []func() datasource.DataSource{
		NewGoogleClientConfigDataSource,
		NewGoogleClientOpenIDUserinfoDataSource,
		NewGoogleDnsManagedZoneDataSource,
		NewGoogleDnsRecordSetDataSource,
		NewGoogleDnsKeysDataSource,
		NewGoogleFirebaseAndroidAppConfigDataSource,
		NewGoogleFirebaseAppleAppConfigDataSource,
		NewGoogleFirebaseWebAppConfigDataSource,
	}
}

I'l have a closer look as to why we are not seeing those.

t0yv0 · 2023-05-26T19:47:34Z

Thanks @danielrbradley for pointing out an upstream commit that patched upstream to serve the missing datasources from SDKv2 versions. I've tried reverting this commit on an upstream branch and building again, hoping this move all the datasources onto PF implementation now.

Revert framework conversions
- Leave the new framework versions in-place.
- Rename the reverted versions to have the filename suffix `_reverted` and the function name suffix `...Sdk`.
- Re-insert into old DataSourcesMap in provider.go

github-actions · 2023-05-26T19:55:00Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

t0yv0 · 2023-05-26T19:59:22Z

So the 7 functions are not new according to schema-tools check but they're switching over to PF implementations as evidenced in the edit to bridge-metadata.json dispatch mappings.

github-actions · 2023-05-26T20:50:33Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New functions:

firebase/getAndroidAppConfig.getAndroidAppConfig

mikhailshilkov

Good to go as far as I am concerned

kpitzen

Apologies for the late reply - agreed that this LGTM

t0yv0 · 2023-05-30T15:30:41Z

Couple more quick questions before I merge.

Does the pulumi/terraform-provider-google-beta@e22b067c5b69 change look good?

Should I tag this in the upstream somehow?

kpitzen · 2023-05-30T15:32:05Z

Couple more quick questions before I merge.

Does the pulumi/terraform-provider-google-beta@e22b067c5b69 change look good?

Should I tag this in the upstream somehow?

This LGTM - we had to do something similar as the upstream TF provider refactored some of the config functionality. Shouldn't be a need to tag upstream.

iwahbe self-assigned this May 1, 2023

iwahbe marked this pull request as draft May 1, 2023 18:08

iwahbe force-pushed the iwahbe/hybrid-sdk-pf-provider branch from f5936db to d7ce0f8 Compare May 1, 2023 18:12

iwahbe mentioned this pull request May 1, 2023

[Epic] Unblock provider use of Plugin Framework alongside SDKv2 pulumi/pulumi-terraform-bridge#981

Closed

9 tasks

iwahbe added 4 commits May 1, 2023 12:10

Convert provider

b25adc9

Run make tfgen

63f4c52

Run make build_sdks

fabb9b5

Allow no docs for new resource

86feb0e

iwahbe force-pushed the iwahbe/hybrid-sdk-pf-provider branch from 54c43f1 to 86feb0e Compare May 1, 2023 19:10

t0yv0 added 4 commits May 17, 2023 18:56

Merge branch 'master' into iwahbe/hybrid-sdk-pf-provider

c903183

Update dependencies

e7929de

make tfgen

94668f7

make build_sdks

4e24304

t0yv0 requested review from a team and kpitzen May 18, 2023 00:38

t0yv0 marked this pull request as ready for review May 18, 2023 00:40

t0yv0 changed the title ~~Hybrid sdk pf provider~~ Enable Plugin Framework based resources and functions May 18, 2023

t0yv0 changed the title ~~Enable Plugin Framework based resources and functions~~ Enable Plugin Framework based resources and functions for pulumi-gcp May 18, 2023

danielrbradley reviewed May 22, 2023

View reviewed changes

t0yv0 added 2 commits May 23, 2023 15:30

Merge master

95137a8

make build_sdks

431ddc0

t0yv0 added 2 commits May 23, 2023 15:38

pf/v0.9.2

4b077a2

Use version.Version

a122bfc

t0yv0 added 3 commits May 26, 2023 15:34

Link to upstream with reverted anti-PF datasource patch

7340fe1

make provider

5131d0d

make build_sdks

174374e

Regenerate code

7919a03

t0yv0 requested review from danielrbradley, mikhailshilkov and t0yv0 May 30, 2023 14:53

mikhailshilkov approved these changes May 30, 2023

View reviewed changes

kpitzen approved these changes May 30, 2023

View reviewed changes

t0yv0 merged commit 6e90538 into master May 30, 2023

t0yv0 deleted the iwahbe/hybrid-sdk-pf-provider branch May 30, 2023 15:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Plugin Framework based resources and functions for pulumi-gcp #1075

Enable Plugin Framework based resources and functions for pulumi-gcp #1075

iwahbe commented May 1, 2023 •

edited by t0yv0

Loading

github-actions bot commented May 1, 2023

iwahbe commented May 1, 2023

github-actions bot commented May 17, 2023

kpitzen commented May 18, 2023

t0yv0 commented May 18, 2023

t0yv0 commented May 18, 2023

danielrbradley left a comment

danielrbradley May 22, 2023

t0yv0 May 23, 2023

t0yv0 commented May 23, 2023

t0yv0 commented May 23, 2023

t0yv0 commented May 23, 2023

t0yv0 commented May 23, 2023

github-actions bot commented May 23, 2023

github-actions bot commented May 23, 2023

github-actions bot commented May 23, 2023

t0yv0 commented May 26, 2023

t0yv0 commented May 26, 2023

github-actions bot commented May 26, 2023

t0yv0 commented May 26, 2023

github-actions bot commented May 26, 2023

mikhailshilkov left a comment

kpitzen left a comment

t0yv0 commented May 30, 2023

kpitzen commented May 30, 2023

Enable Plugin Framework based resources and functions for pulumi-gcp #1075

Enable Plugin Framework based resources and functions for pulumi-gcp #1075

Conversation

iwahbe commented May 1, 2023 • edited by t0yv0 Loading

github-actions bot commented May 1, 2023

Does the PR have any schema changes?

New functions:

iwahbe commented May 1, 2023

github-actions bot commented May 17, 2023

Does the PR have any schema changes?

New functions:

kpitzen commented May 18, 2023

t0yv0 commented May 18, 2023

t0yv0 commented May 18, 2023

danielrbradley left a comment

Choose a reason for hiding this comment

danielrbradley May 22, 2023

Choose a reason for hiding this comment

t0yv0 May 23, 2023

Choose a reason for hiding this comment

t0yv0 commented May 23, 2023

t0yv0 commented May 23, 2023

t0yv0 commented May 23, 2023

t0yv0 commented May 23, 2023

github-actions bot commented May 23, 2023

Does the PR have any schema changes?

New functions:

github-actions bot commented May 23, 2023

Does the PR have any schema changes?

New functions:

github-actions bot commented May 23, 2023

Does the PR have any schema changes?

New functions:

t0yv0 commented May 26, 2023

t0yv0 commented May 26, 2023

github-actions bot commented May 26, 2023

Does the PR have any schema changes?

New functions:

t0yv0 commented May 26, 2023

github-actions bot commented May 26, 2023

Does the PR have any schema changes?

New functions:

mikhailshilkov left a comment

Choose a reason for hiding this comment

kpitzen left a comment

Choose a reason for hiding this comment

t0yv0 commented May 30, 2023

kpitzen commented May 30, 2023

iwahbe commented May 1, 2023 •

edited by t0yv0

Loading