Add gcp provider support #141

praveenkumar · 2023-09-15T12:41:44Z

No description provided.

adrianriobo · 2023-09-15T12:49:42Z

pkg/provider/gcp/create-instance.go

+			compute.FirewallAllowArgs{
+				Protocol: pulumi.String("tcp"),
+				Ports: pulumi.ToStringArray([]string{
+					"22", "443", "6443", "80",


Can we create some common pkg with this information, so each provider will get values from there and will create the resources it need based on those values... (aws sgs, gcp firewall...).

Otherwise if for whatever reason we want to add some other port in the future we need to go provider by provider

we can make port as part of constants and use it for each provider.

adrianriobo · 2023-09-15T12:51:14Z

pkg/provider/gcp/create-instance.go

+	return nil
+}
+
+func createKey(ctx *pulumi.Context) (*tls.PrivateKey, error) {


same here move to common pkg and reuse from all providers

for aws it is different signature func createKey(ctx *pulumi.Context) (*tls.PrivateKey, *ec2.KeyPair, error) but yes some of it can be consumed for each provider.

adrianriobo · 2023-09-15T12:53:06Z

pkg/provider/gcp/gcp.go

+func (a *Provider) GetPlugin() *providerAPI.PluginInfo {
+	return &providerAPI.PluginInfo{
+		Name:    "gcp",
+		Version: "v6.65.0"}


need this version at Containerfile as the other plugins
(i.e.

crc-cloud/oci/Containerfile

Line 40 in eb143b9

ARG PULUMI_AZURE_NATIVE_VERSION=v2.6.0

)

then also add the plugin installation cmd
(i.e.

crc-cloud/oci/Containerfile

Line 48 in eb143b9

pulumi plugin install resource azure-native ${PULUMI_AZURE_NATIVE_VERSION}

)

Ack, will update that.

adrianriobo · 2023-09-18T08:04:19Z

Can we add some README content on how to use the gcp, some code snippet on how to invoke the container or the binary? Envs required....

Also did you give a try to build a container and run from it AFAIK we will need to add the gcp cli to the container as well...may at https://github.com/crc-org/crc-cloud/blob/main/requirements.txt

praveenkumar · 2023-12-13T07:39:16Z

Can we add some README content on how to use the gcp, some code snippet on how to invoke the container or the binary? Envs required....

Yes I updated the README which have all the steps and usecase.

Also did you give a try to build a container and run from it AFAIK we will need to add the gcp cli to the container as well...may at https://github.com/crc-org/crc-cloud/blob/main/requirements.txt

gcp cli added as part of container image, I am not sure why requirements.txt is there ?

adrianriobo · 2023-12-13T10:18:28Z

gcp cli added as part of container image, I am not sure why requirements.txt is there ?

need to add both the pulumi plugin for gpc and the gcp cli I only see the the pulumi plugin, also that requirements is a way to manage tooling through pip if I am not wrong, so maybe you can just add a new line for the gcp cli there. AFAIK the benefit to handle it within the requirements.txt is that renovate acts on it out of the box.

praveenkumar · 2023-12-13T11:46:31Z

need to add both the pulumi plugin for gpc and the gcp cli I only see the the pulumi plugin

Why I need to add gcp cli, I created the image using this container and able to create cluster using this image.

that requirements is a way to manage tooling through pip if I am not wrong

But where are we using this requirements only for renovate?

In go lang `init` is a special function and we shouldn't use it in this context even `Init` so better to use `Create` which is what we are doing.

adrianriobo · 2023-12-13T14:08:59Z

But where are we using this requirements only for renovate?

Not only for renovate we are using it as an unified way for pkg management (+ it is supported by renovate).

So pros is we have a unified way to handle pkgs... cons is aws cli v2 is not supported there...on qenvs I use a different approach for each cli.

adrianriobo · 2023-12-13T14:13:52Z

As a side note I realize you are using gcp classic provisioner https://github.com/pulumi/pulumi-gcp recently pulumi shift the way they create the provisioners to a native way using the cloud providers API https://github.com/pulumi/pulumi-google-native

It seems pulumi recommendation is to move to native provisioners...but they are not mature enough as an example I was able to use the azure native but not the aws as it misses still lot of resources... so add this as a comment we may can create a following issue as spike to migrate the gcp to the native provisioner WDYT?

This patch will remove the use of getSupportedProviders() and able to use getProvider functionality for provider specific parameter.

``` $ crc-cloud create gcp --gcp-image-id crc --backed-url file:///tmp/gcp/ --project-name prkumar-test --output /tmp/gcp --key-filepath /home/prkumar/.crc/cache/crc_libvirt_4.13.6_amd64/id_ecdsa_crc --pullsecret-filepath /home/prkumar/pull-secret --gcp-disk-size 100 --gcp-instance-type n1-standard-8 ```

Right now it only contain the ports which is used across provider but we should add more common constants to this package in future.

It is going to used for all the provider so better to extract it and have different subpackage.

praveenkumar · 2023-12-14T05:51:57Z

It seems pulumi recommendation is to move to native provisioners...but they are not mature enough as an example I was able to use the azure native but not the aws as it misses still lot of resources... so add this as a comment we may can create a following issue as spike to migrate the gcp to the native provisioner WDYT?

As per https://github.com/pulumi/pulumi-google-native looks like it is not stable and development is in active state so better to have follow up issue in case we want to switch over in near future.

adrianriobo added the GCP label Sep 15, 2023

adrianriobo requested changes Sep 15, 2023

View reviewed changes

praveenkumar force-pushed the gcp_provider branch 5 times, most recently from c98de2d to 55e98ce Compare September 15, 2023 16:41

praveenkumar force-pushed the gcp_provider branch 4 times, most recently from 407c366 to a4c1e12 Compare December 13, 2023 07:36

praveenkumar force-pushed the gcp_provider branch from a4c1e12 to 48b1062 Compare December 13, 2023 13:52

context: Rename Init function to Create to avoid confusion

ebfa93c

In go lang `init` is a special function and we shouldn't use it in this context even `Init` so better to use `Create` which is what we are doing.

praveenkumar force-pushed the gcp_provider branch from 48b1062 to 5a27957 Compare December 13, 2023 13:58

praveenkumar added 6 commits December 13, 2023 20:38

aws: Avoid redundant type conversion

da4dfaf

Pass provider detail to CreateParams function

f138507

This patch will remove the use of getSupportedProviders() and able to use getProvider functionality for provider specific parameter.

provider/constants: Use constants package to use it across provider

2f193c1

Right now it only contain the ports which is used across provider but we should add more common constants to this package in future.

tls: Create separate pkg to create keypair

61ae6d9

It is going to used for all the provider so better to extract it and have different subpackage.

Update readme with gcp provider

55e0dd5

praveenkumar force-pushed the gcp_provider branch from 5a27957 to 55e0dd5 Compare December 14, 2023 05:46

adrianriobo self-requested a review December 14, 2023 11:13

adrianriobo approved these changes Dec 14, 2023

View reviewed changes

adrianriobo merged commit f4868b9 into crc-org:main Dec 14, 2023
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add gcp provider support #141

Add gcp provider support #141

praveenkumar commented Sep 15, 2023

adrianriobo Sep 15, 2023

praveenkumar Sep 15, 2023

praveenkumar Sep 15, 2023

adrianriobo Sep 15, 2023

praveenkumar Sep 15, 2023

praveenkumar Sep 15, 2023

adrianriobo Sep 15, 2023

praveenkumar Sep 15, 2023

praveenkumar Sep 15, 2023

adrianriobo commented Sep 18, 2023

praveenkumar commented Dec 13, 2023

adrianriobo commented Dec 13, 2023

praveenkumar commented Dec 13, 2023

adrianriobo commented Dec 13, 2023

adrianriobo commented Dec 13, 2023

praveenkumar commented Dec 14, 2023

Add gcp provider support #141

Add gcp provider support #141

Conversation

praveenkumar commented Sep 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adrianriobo commented Sep 18, 2023

praveenkumar commented Dec 13, 2023

adrianriobo commented Dec 13, 2023

praveenkumar commented Dec 13, 2023

adrianriobo commented Dec 13, 2023

adrianriobo commented Dec 13, 2023

praveenkumar commented Dec 14, 2023