SecurityGroup removes SecurityGroupRules added later

[jhsinger-klotho]

What happened?

We create a security group and 2 rules added in another component. On fresh deploy Pulumi sets it all up correctly, but then on the next up it resets the security group to the ingress and egress specified only on the groups component itself and does not realize there are other rules attached since those aws.ec2.SecurityGroupRules are not changing.

Expected Behavior

I would expect that pulumi would understand that the security group rules are attached to a security group managed in the deployment and ensure they are still configured on the security group

Steps to reproduce

include this code in the index.ts

• run pulumi up
• run pulumi up again
• check the aws console and notice the 2 added ingress rules no longer exist

const sg = new aws.ec2.SecurityGroup(sgName, {
            name: sgName,
            vpcId: vpc.id,
            egress: [
                {
                    cidrBlocks: ['0.0.0.0/0'],
                    description: 'Allows all outbound IPv4 traffic.',
                    fromPort: 0,
                    protocol: '-1',
                    toPort: 0,
                },
            ],
            ingress: [
                {
                    description:
                        'Allows inbound traffic from network interfaces and instances that are assigned to the same security group.',
                    fromPort: 0,
                    protocol: '-1',
                    self: true,
                    toPort: 0,
                },
                {
                    cidrBlocks: ['0.0.0.0/0'],
                    fromPort: 9443,
                    protocol: 'TCP',
                    self: true,
                    toPort: 9443,
                },
            ],
        })

        pulumi.output(vpc.privateSubnets).apply((ps) => {
            const cidrBlocks: any = ps.map((subnet) => subnet.subnet.cidrBlock)
            new aws.ec2.SecurityGroupRule(`-private-ingress`, {
                type: 'ingress',
                cidrBlocks: cidrBlocks,
                fromPort: 0,
                protocol: '-1',
                toPort: 0,
                securityGroupId: sg.id,
            })
        })

        pulumi.output(vpc.publicSubnets).apply((ps) => {
            const cidrBlocks: any = ps.map((subnet) => subnet.subnet.cidrBlock)
            new aws.ec2.SecurityGroupRule(`public-ingress`, {
                type: 'ingress',
                cidrBlocks: cidrBlocks,
                fromPort: 0,
                protocol: '-1',
                toPort: 0,
                securityGroupId: sg.id,
            })
        })

Output of pulumi about

CLI
Version 3.52.1
Go Version go1.19.5
Go Compiler gc

Plugins
NAME VERSION
aws 5.29.1
aws 5.10.0
command 0.5.2
docker 3.6.1
eks 0.42.7
kubernetes 3.24.0
kubernetes 3.20.2
kubernetes-cert-manager 0.0.3
nodejs unknown

Host
OS darwin
Version 12.5
Arch arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v19.7.0'
Dependencies:
NAME VERSION
@pulumi/kubernetesx 0.1.6
@pulumi/pulumi 3.55.0
@types/multimap 1.1.2
@types/uuid 8.3.4
@pulumi/aws 5.29.1
@pulumi/awsx 0.40.1
@pulumi/command 0.5.2
@pulumi/eks 0.42.7
@pulumi/kubernetes-cert-manager 0.0.3
requestretry 7.1.0
multimap 1.1.0
@pulumi/kubernetes 3.24.0
axios 0.27.2
cdk8s-cli 2.1.135
@types/node 16.18.12
patch-package 6.5.1
simple-sha256 1.1.0

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[danielrbradley]

Hi @jhsinger-klotho thanks for the detail of your issue. Unfortunately, managing a single resource declaritively from two separate sources does lead to these kinds of issues. I'm not aware of a resource that allows you to manage the ingress rules independently of a security group - are you modifying the security group resource directly to achive this separation?

If you really can't centralise the definition of your security group to include all ingress rules there's a two step process you could try as as workaround to the rules being removed:

1. Refresh the security group before each deployment. This can also be done by using the -r option when calling pulumi up.
2. Set the ingress and egress properties as ignored using resource options. This will then use the value from your state (which you refreshed in step 1) instead of the value in your program.

I'm going to close this as "won't fix" as this is expected behaviour - resetting a resource back to the declared state if it's been modified outside the program. Let me know you think this needs re-opening though if I haven't correctly understood the scenario.

[MMartyn]

Coming in a year later on this one, but this does seem like a big gotcha. For instance if I have a NewSecurityGroup that has the Ingress property filled in and then have some separate NewSecurityGroupRule's that add more rules then the ones added later get removed on the next run.

[MMartyn]

Coming in a year later on this one, but this does seem like a big gotcha. For instance if I have a NewSecurityGroup that has the Ingress property filled in and then have some separate NewSecurityGroupRule's that add more rules then the ones added later get removed on the next run.

I see it is documented and I must have missed it 😞