Then, we would like to set up cross-account roles/permissions so that the pulumi dev backend state is saved in an S3 bucket in the root account, so that developers in the dev account can't accidentally alter the state. Any tips or resources are appreciated!

This won't work as a block to making updates; it will only stop people from updating the shared understanding of what has been deployed. There's nothing stopping someone deploying from code: the state isn't a lock, it's a cache. To stop people making updates, ensure that they don't have access to the resources that the code describes. That is: use AWS/Azure/GCP/etc. controls to protect your resources.

Sorry, I may not have been 100% clear. The idea is that we have: • an s3 pulumi state bucket "pulumi_state_root" that only few selected have access to which manages groups and fundamental resources • an s3 pulumi state bucket "pulumi_state" that more developers can read/write to to cache the state of any IaC created Hence one pulumi backend would be strongly protected, the other one less so. But via AWS prefix-based permissions, we could manage the read/write access to different stacks for different user groups differently?

State is stored per project. You can put different projects' state in different files in one bucket, or in different buckets. However, no matter where you put the state or how protected you make it, it does not protect your in-cloud resources at all. Anyone with access to the resources can change those. And anyone with no access to the resources cannot use Pulumi to change the state describing those resources, since the cloud update will fail and therefore Pulumi won't even try to update the state. Obviously I don't know your use case, but at a glance, it seems like what you're trying to do cannot be done with the solution you're currently investigating.

OK, thanks for the feedback!

As tenwit said, the state is just a cache. If you want to prevent people from making changes to your infrastructure, you have to protect the infrastructure, not the state files.

If you have infrastructure that should only be changed by a select group of users, make sure that only these users have the necessary permissions to do so.

Yes, our thought was that AWS control tower, permission group definitions and s3 state files should be created in root account, such that only users that have access to the root account can alter them. Any other resources would be in another account and could thus be altered by those that have access to that account. However, that means that users need cross-account permission to the s3 state bucket (or at least parts of it) that is in the root account.

Yes, if you want to store all state files in the same bucket, you need to give the users access to that bucket

If you have infrastructure that should only be changed by a select group of users, make sure that only these users have the necessary permissions to do so.

How would you do that? I mean, one could go crazy and define for each group which kind of AWS resources they are allowed to alter? Or the other extreme would be that any user can do anything for a respective account?

Via AWS IAM. You create deployment roles and scope them as required. If I'm only allowed to change S3 buckets called

kilian-*

restrict my S3 permissions to these buckets

Yeah 😄

> Our concern was that if the state bucket is in the same account in which users happily pulumi up and destroy, the state bucket may accidentally be deleted or altered, causing huge pain This isn't a concern, with appropriate architecture. Your IaC infrastructure effectively cannot be dependent on (the same) IaC. For example, if you want account abc123 to be completely controlled by IaC, then your IaC state should not be in account abc123. This is the same concern as your logging bucket: you should not put your logging buckets in the accounts containing the resources being logged. Just as you should have an account reserved for logging / monitoring, which lasts long after all your other accounts are deleted (so that you can report to your auditors what happened in the past), you should have a state storage area for your IaC state files that will last long after any and all states are deleted. For most of us, that state storage area is the Pulumi app: after we delete all our AWS accounts, our Pulumi account will remain, with our auditable info. If you're storing your state in S3, then that S3 bucket should be in an account that Pulumi does not manage: it just writes files there.

OK, thanks a lot for the additional explanation!

No, the effort is small. It just shouldn't be solved with Pulumi. Create your bucket for keeping state, but do it manually, or via CFM or something. Put it somewhere "global", maybe in your existing logging account. Put all your state files in the one bucket (to reduce overhead), just change the prefix for each one as appropriate. Re: being locked in: lock-in to the Pulumi service isn't a thing. You can export your state at any time no matter what the backend is, you can copy that that anywhere (e.g. to S3), and then all you need to do is log each project into that backend, and you've moved.

Fair points!