https://pulumi.com logo
Powered by
#aws
Title
l

late-piano-64593

07/27/2023, 8:59 PM
I am currently designing a multi-account layout were I will have Pulumi stacks in multiple AWS accounts and I would like to share a common s3 bucket using cross-account assumed roles: ex: https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-walkthroughs-managing-access-example4.html#access-policies-walk[…]gh-example4-overview to allow each account to manage their stack data in a centralized location. Is it possible to associate a role-to-assume with just the operations touching the cloud-url at: 
s3://...
?
s

stocky-restaurant-98004

07/28/2023, 3:07 PM
This is actually the way you want to design it: Put all your state files in a single bucket (in a semi-dedicated account, e.g. the one where you keep your pipelines rather than actual running workloads). The principal logging into S3 for state file storage is not necessarily the same principal that is executing 
pulumi up
. You can pass a profile to 
pulumi login
for S3 state file storage, whereas the principal for 
pulumi up
is determined either by either provider config or the standard AWS credentials chain (
AWS_PROFILE
, security keys, etc.)
You may also want some cross-region backups on that S3 bucket just in case, since losing a state file is... not great, to put it mildly.
l

late-piano-64593

07/28/2023, 5:17 PM
Ahhh wonderful! That makes a lot of sense. thank you.
Sorry for a later follow up but quick question, ideally I would have a profile-per-stack so I could correctly scope the policy to only allow dev account to access the 'dev/' sub directory on s3 and same for prod. So that would mean I would need to change the contents of 
backend.url
in 
Pulumi.yaml
when I switched between accounts correct? I could solve this with bucket policies and skip the assume-role but I feel like thats not the "right" way to do it. I might be missing something though.
Actually I am not sure I can solve this with bucket policies either because the sub-directory in the bucket would have to to change in the 
url
field when switching between accounts. Does that mean that the current design requires that my dev account have full permissions to accidentally overwrite the state bucket of prod?
s

stocky-restaurant-98004

08/09/2023, 8:08 PM
Configure the dev stack to use an AWS profile that can only access the 
dev/
folder.
You shouldn't be able to overwrite the state file very easily, tho. Double-check by creating a few dummy stacks, but I believe that that state files go to known paths based on the project and stack.
l

late-piano-64593

08/09/2023, 8:55 PM
Can you set the 
backend.url
in the stack level settings ex: 
Pulumi.dev.yaml
? TIL! Is there a documentation of that schema somewhere?
s

stocky-restaurant-98004

08/10/2023, 1:36 PM
I don't think you'd want to. I'd need to know exactly how Pulumi determines the state file's path, but I'm guessing you would want the same backend.url either for all Pulumi projects if it's 
{{ backend.url}}/{{ project }}/ {{ stack }}
, or the a backend.url per-project if it's just 
{{ backend.url}}/{{ stack }}
. I would guess it's the former, but IDK for sure.
l

late-piano-64593

08/10/2023, 5:38 PM
Ideally it would be a single 
backend.url
for the project then if pulumi only touched specific sub directories in that root, per stack I could scope the S3 permissions correctly to prevent accountA (dev) from being able to Put/Get in AccountB (prod)'s stacks. But right now in s3 in the 
backend.url
root is just a 
.pulumi/
with some high level meta data and then the 
stacks/
directory below that.
s

stocky-restaurant-98004

08/11/2023, 4:59 PM
If I am reading this blog post right, you should be able to use the same 
backend.url
for everything: https://www.pulumi.com/blog/project-scoped-stacks-in-self-managed-backend/
Specifically, this diagram shows that each project would get its own folder:
l

late-piano-64593

08/12/2023, 4:08 PM
I was digging through that post hoping for that structure, running 
pulumi state upgrade
so far has not changed the structure in s3 to match that folder structure but I am gonna try migrating to a new bucket next week.
So I dug into that a little bit, that works somewhat if your projects are broken into different account but not if your stacks are.
I ended up having to allow all the different account principles read/write access to the whole bucket, because the only way to do least privs was to scope the Resources block in the IAM to all the different subfolders in the pulumi state bucket, things like 
backups
history
stacks
etc, then encode the specific stack names into those resource rules. Its possible but it means that I would need to update the IAM everytime I need to add a new stack. (if for example the dev AWS account might have multiple stacks in it while prod only has 1)
Powered by