S3 backend issue: so for reasons I...do not partic...
# general
S3 backend issue: so for reasons I...do not particularly love, we're using multiple AWS accounts that access the same shared S3 bucket for stacks. When I run
pulumi stack ls
as the bucket owner, not every stack created by other accounts in that bucket show up. (Some do. Not all.) When I do
aws s3 ls
, however, the JSON files do appear in the list. I can't read them as the bucket owner, however, and get a 403 on HeadObject/GetObject. My initial expectation was an ACL problem, but I've since run
aws s3 cp
across the entire bucket setting everything to bucket-owner-full-control and while that has in the past worked occasionally (for reasons I am not fully sure of), it's not working now. 1. How can I unjam myself? 2. My intuition is that this is a problem with the S3 client used by Pulumi. What steps can I take to prevent it from happening again?
From your notes - sounds like you indeed don’t have permission to read these files in S3 - and I would expect Pulumi needs to read them to list details of the stacks. I don’t know precisely what to suggest without seeing all the policies you have applied to these buckets - but it does not sound like there is anything particularly Pulumi specific here. Unfortunately, cross-account bucket policy configuration in AWS is frequently very confusing.
So typically in this situation one would apply a condition to the bucket policy that requires bucket-owner-full-control, but Pulumi barfs at that.
The bucket policies are just global access delegated to the sub-accounts. It seems to be an object-level problem.
but Pulumi barfs at that
Are you saying that you can download the contents of a file with a set of credentials, but cannot see it with
pulumi stack ls
I'm saying that I can't upload to a bucket with an ACL restriction.
Which is why we don't have them
And which appears related, though I'm not yet certain, to having the bucket owner account unable to find the stack.