Hi All, Unsure if this is the correct channel but...
# getting-started
b
Hi All, Unsure if this is the correct channel but feel free to point me elsewhere. We are adopting Pulumi using python + aws and are trying to develop repeatable pkg classes starting with S3. The goal is to build a custom S3 class that support methods to attach bucket policies in an additive manner. See below code snippet. The code works fine when i call one method, but when i call both method the code complains of duplicate URN. I am expecting the resource to update not create a new one. Anyone have experience with this type of structure? main.py
custom_s3 = CustomS3('name')
custom_s3.attach_alb_policy()
custom_s3.attach_cloudfront_policy('someid')
customS3.py
class CustomS3(pulumi.ComponentResource):
def __init__():
...
self.s3_bucket= aws.s3.BucketV2 (self.name)
...
def attach_alb_policy(self):
...
configure_bucket_policy ('policy to add')
def attach_cloudfront_policy(self,origin_access_id):
...
configure_bucket_policy ('policy to add')
def configure_bucket_policy (self, policy_to_add):
#Try to pull teh current policy if exists on bucket
try:
current_policy = aws.s3.get_bucket_policy(bucket=self.s3_bucket)
except:
#Found no policy current attached
current_policy = None
generated_policy = json.dumps(collapse_policy_statements(json.loads(current_policy.policy),policy_to_add)) if current_policy else json.dumps(policy_to_add)
print (generated_policy)
self.bucket_policy = aws.s3.BucketPolicy(
f"s3-policy-{self.name}",
bucket=self.s3_bucket,
policy= generated_policy,
opts=pulumi.ResourceOptions(parent=self.s3_bucket)
)
m
The name of the
aws.s3.BucketPolicy
generated by your
configure_bucket_policy
method is always
f"s3-policy-{self.name}"
, where
self.name
is the name of the S3 bucket.
The resource names in a stack have to be unique. Also, you're overriding
self.bucket_policy
every time you call the method, so this will always be set to the latest policy object.
I think you'll rather want to store all the policy resources in a list or some other container so that you can access them later.
b
So the methods would manage a self.bucket_policy array and move the bucket_policy creation to the init potentially?
m
It all depends on what you want to achieve. I thought maybe something like this is what you intended:
Copy code
class CustomS3(pulumi.ComponentResource):
  def __init__(self, name: str):
    ...
    self.bucket_name = name
    self.bucket_policies = []

  def configure_policy(self, policy: str):
     ...
     
     new_policy = aws.s3.BucketPolicy(
                # generate a unique resource name
                f"s3-policy-for-{self.bucket_name}-{len(self.bucket_policies)}",
                ...)
     self.bucket_policies.append(new_policy)
This will enumerate your policies, so the resource names are unique, and you can potentially look up policy resources you've already created.
Perhaps
self.bucket_policies
could also be a dictionary for easier lookup, perhaps you have a set of standard policies that you frequently re-use that have proper names.
Ah, there can be just one BucketPolicy per S3 bucket, right?
b
Correct, i was just typing that out. So i am basically trying to augment the policy string and add more policies to it via method calls.
m
Yes, but then the problem is similar, you're still generating more than one BucketPolicy resource and run into the name duplication conflict before you ever encounter the problem that you're potentially attaching multiple policies to the same bucket
b
Right so ill try this path: • Make a new self.bucket_policy, all methods will simply update this resource • Use the init to make the policy if self.bucket_policy has a value This way the resource is the same just the string is being updated. In theory should work right?
m
I think it's best to create the
aws.s3.BucketPolicy
only after you've collected all the statements.
b
Right in that route i would have to make a more generic add policy method supporting all the policies i want to attach and a method to denote and combine them.
m
Keep in mind that your Pulumi program is not interacting with AWS while it is executed. It will run first and then hand over the desired infrastructure to the Pulumi engine, which then computes the diff and applies any changes necessary.
So you don't want to use
aws.s3.get_bucket_policy()
if you have the bucket resource in your stack, which you do.
b
Makes sense, ill give it a shot and let you know how it goes. Thanks!