I'm finding that repeated pulumi up and pulumi refresh is constantly editing and destroying rules in my AWS security groups, and then restoring them on the next cycle of refresh and up. From looking at the first refresh it seems like it is finding say 5 rules in the security group which match what the code generates but they are out of order. e.g. in the code 1,2,3,4,5 and then in aws 1,2,4,3,5 and then pulumi thinks it needs to update rules 3 and 4 essentially morphing each into the other. I think this update fails somehow (aws sg can be fussy about having rules added which match rules which already exist) and the result is that one rule is morphed into the other and the other is broken. The another round of refresh and update will fix it. As far as I know the order in which the AWS api returns the security group rules is based on their name which is sg- and a random hex string, as I can't know this before the sg are created I can't order the statements in my code to match the order in which they will appear in the api. Especially when you consider deploying the same code to multiple stacks. Is this a common issue? Can I solve it easily? Do I need to look at using AWS-Native over the AWS-Classic?

Why are you refreshing? And, why is one SG being managed by two pieces of code?

I am refreshing to ensure that the infra and state match up, and every time I do so they appear to not match up

If you lock down your target account so that no one except Pulumi has update permissions, then you can skip the refresh step. Refreshing is good for recovering from accidental changes, but typically it is not needed. Refreshing exposes you to the risk of state change due to unimportant changes made by your cloud service. Things like unpredictable ordering (e.g. security group rules, WAF rules), default values and others, which do not affect the normal preview-up-repeat cycle, become potentially destructive and certainly inconvenient problems when you switch to a refresh-preview-up cycle. To get this working, you need to extend your dev work to include manually refreshing after each dev up then updating your code to match whatever the refresh changed. If refreshing pulls an unimportant default value from the cloud into your state, you need to update your code to match the state, or else Pulumi will try to delete that default value. And removing some default values can be a destructive action, causing a replace rather than an update.

So the solution was to let the EKS module create the sg, and then to add rules to it so I could access 443 from my AdminVM with code like

// add a rule allowing the AdminVM in to kubectl the cluster
const adminClusterRule = new aws.ec2.SecurityGroupRule("adminClusterRule", {
	type: "ingress",
	fromPort: 443,
	toPort: 443,
	protocol: "tcp",
	securityGroupId: clusterSecurityGroupId,
	sourceSecurityGroupId: adminsg.ids[0],
	description: `Allow AdminVM to communicate with the ${nam}-eksCluster API Server`,
});

If you try to create the sg first with a rule like that and then pass that to the

new eks.cluster

function as clusterSecurityGroup then you get into the state I describe.

I don't think it's a bug, unfortunately. It functions as designed. An alternative design that works better for IaC would be good: this design appears to have been created for console users.