This message was deleted.

I have a “base infrastructure” stack that contains things that rarely changed and can be reused over and over (VPC, subnets, NAT gateways, etc.). In that stack were some security groups as well. I was finding that I needed to have “higher layer” stacks that built resources on top of the base infrastructure stack occasionally need to add a rule to these groups. Cool, no problem so far.

I'd probably opt for adding new security groups in downstream stacks, but adding the new groups to the upstream VPC. I feel like modifying or adding rules to an existing group is a leaky abstraction and removes the idempotency from each layer.

Removing the idempotency (particularly from the base infrastructure layer) is the key issue, yes. In this particular instance, there is a constraint that requires I modify existing groups, but that won’t always be the case (so your point about creating new groups instead of modifying existing is well taken). That being said, would separating the definition of the group and the definition(s) of the rules in the code fix the idempotency issue? I haven’t completely finished refactoring my code so I haven’t had the issue to test yet (but I certainly can).

having a single security group and adding the rules downstream feels you're treating security groups like a firewall and the rules as firewall rules. you should have a security group per application/program. the security groups themselves should always live directly wth the thing that needs it

In general I agree; however, when there’s a constraint to modify existing groups what’s the best approach?

I'm not sure your second approach will work. I believe Pulumi will still detect the changes in the base layer, because IIRC the security group -> rule relationship is bidirectional and I'm curious if upstream will be confused by the addition? I'm out of office today, but I can run tests on Thursday - or feel free to share how you get on. I'm mighty curious

I’ll do some additional testing and let you know. No need to spend any of your cycles on this, I’m sure you have more important things to do!

In general I agree; however, when there’s a constraint to modify existing groups what’s the best approach?

😞 I think adding the rules is going to be your only option then