This message was deleted.

sparse-intern-71089

07/27/2023, 8:59 PM

This message was deleted.

stocky-restaurant-98004

07/28/2023, 3:07 PM

This is actually the way you want to design it: Put all your state files in a single bucket (in a semi-dedicated account, e.g. the one where you keep your pipelines rather than actual running workloads). The principal logging into S3 for state file storage is not necessarily the same principal that is executing

pulumi up

. You can pass a profile to

pulumi login

for S3 state file storage, whereas the principal for

pulumi up

is determined either by either provider config or the standard AWS credentials chain (

AWS_PROFILE

, security keys, etc.)

stocky-restaurant-98004

07/28/2023, 3:08 PM

Details here: https://www.pulumi.com/docs/concepts/state/#aws-s3

stocky-restaurant-98004

07/28/2023, 3:09 PM

You may also want some cross-region backups on that S3 bucket just in case, since losing a state file is... not great, to put it mildly.

late-piano-64593

07/28/2023, 5:17 PM

Ahhh wonderful! That makes a lot of sense. thank you.

youre welcome 1

late-piano-64593

08/09/2023, 7:09 PM

Sorry for a later follow up but quick question, ideally I would have a profile-per-stack so I could correctly scope the policy to only allow dev account to access the 'dev/' sub directory on s3 and same for prod. So that would mean I would need to change the contents of

backend.url

Pulumi.yaml

when I switched between accounts correct? I could solve this with bucket policies and skip the assume-role but I feel like thats not the "right" way to do it. I might be missing something though.

late-piano-64593

08/09/2023, 7:18 PM

Actually I am not sure I can solve this with bucket policies either because the sub-directory in the bucket would have to to change in the

url

field when switching between accounts. Does that mean that the current design requires that my dev account have full permissions to accidentally overwrite the state bucket of prod?

stocky-restaurant-98004

08/09/2023, 8:08 PM

Configure the dev stack to use an AWS profile that can only access the

dev/

folder.

stocky-restaurant-98004

08/09/2023, 8:10 PM

You shouldn't be able to overwrite the state file very easily, tho. Double-check by creating a few dummy stacks, but I believe that that state files go to known paths based on the project and stack.

late-piano-64593

08/09/2023, 8:55 PM

Can you set the

backend.url

in the stack level settings ex:

Pulumi.dev.yaml

? TIL! Is there a documentation of that schema somewhere?

stocky-restaurant-98004

08/10/2023, 1:36 PM

I don't think you'd want to. I'd need to know exactly how Pulumi determines the state file's path, but I'm guessing you would want the same backend.url either for all Pulumi projects if it's

{{ backend.url}}/{{ project }}/ {{ stack }}

, or the a backend.url per-project if it's just

{{ backend.url}}/{{ stack }}

. I would guess it's the former, but IDK for sure.

late-piano-64593

08/10/2023, 5:38 PM

Ideally it would be a single

backend.url

for the project then if pulumi only touched specific sub directories in that root, per stack I could scope the S3 permissions correctly to prevent accountA (dev) from being able to Put/Get in AccountB (prod)'s stacks. But right now in s3 in the

backend.url

root is just a

.pulumi/

with some high level meta data and then the

stacks/

directory below that.

stocky-restaurant-98004

08/11/2023, 4:59 PM

If I am reading this blog post right, you should be able to use the same

backend.url

for everything: https://www.pulumi.com/blog/project-scoped-stacks-in-self-managed-backend/

stocky-restaurant-98004

08/11/2023, 4:59 PM

Specifically, this diagram shows that each project would get its own folder:

late-piano-64593

08/12/2023, 4:08 PM

I was digging through that post hoping for that structure, running

pulumi state upgrade

so far has not changed the structure in s3 to match that folder structure but I am gonna try migrating to a new bucket next week.

late-piano-64593

08/15/2023, 5:48 PM

So I dug into that a little bit, that works somewhat if your projects are broken into different account but not if your stacks are.

late-piano-64593

08/15/2023, 5:50 PM

I ended up having to allow all the different account principles read/write access to the whole bucket, because the only way to do least privs was to scope the Resources block in the IAM to all the different subfolders in the pulumi state bucket, things like

backups

history

stacks

etc, then encode the specific stack names into those resource rules. Its possible but it means that I would need to update the IAM everytime I need to add a new stack. (if for example the dev AWS account might have multiple stacks in it while prod only has 1)

👍 1

78 Views

Open in Slack

Previous Next

Pulumi Community

No matter how you like to participate in developer communities, Pulumi wants to meet you there. If you want to meet other Pulumi users to share use-cases and best practices, contribute code or documentation, see us at an event, or just tell a story about something cool you did with Pulumi, you are part of our community.