Hi, during the pulumi destroy of a stack with an A...
# generalw
witty-helmet-74677
08/27/2021, 9:31 AMHi, during the pulumi destroy of a stack with an AWS S3 Bucket which has 15K objects takes around 45 minutes to delete the objects before deleting the bucket. Using AWS CLI the deletion takes 2 minutes. Is there any argument which could speed the stack destroy in this case or this is a know issue? Language: go.
b
busy-honey-73811
08/27/2021, 10:00 AMCan confirm, we have the same problem with very large S3 Buckets and it takes literally hours to delete them with pulumi. Usually those stacks fail (timeout) on destroy and we have to manually clean-up the S3 Buckets (using CLI or AWS Web Console) to speed things up.
w
witty-helmet-74677
08/27/2021, 11:34 AMOne more addition. The objects are not big 2-3 KB each. In total like 40MB.
b
busy-honey-73811
08/27/2021, 11:49 AMSame here, we have thousands (if not millions) of small to medium sized objects (KB to few MB).
g
great-sunset-355
08/27/2021, 2:04 PMDo you manage each object with pulumi or did you try
force_destroy=Trueb
billowy-army-68599
08/27/2021, 4:38 PM
could you run a performance trace here?
https://www.pulumi.com/docs/troubleshooting/#performance
b
busy-honey-73811
08/27/2021, 6:57 PMIn my case we only manage the S3 Bucket with Pulumi, the objects are created by applications (not Pulumi).
w
witty-helmet-74677
08/30/2021, 8:37 AMWe also maintain only the bucket with pulumi. The objects are created by apps. The pulumi uses force destroy, otherwise wouldn't be able to delete the bucket.
witty-helmet-74677
08/30/2021, 1:18 PMI executed the pulumi destroy now with the traces turned on suggested by @billowy-army-68599. The 45 minutes are spent in /pulumirpc.ResourceProvider/Delete.
b
billowy-army-68599
08/30/2021, 6:01 PM
@witty-helmet-74677 just to be sure I'm understanding, you're populating a bucket with an external program then doing a force delete?
billowy-army-68599
08/30/2021, 6:29 PM
as pointed out by @white-balloon-205 (thanks!) it seems we inherit this from the upstream AWS provider:
https://github.com/hashicorp/terraform-provider-aws/issues/12146
w
witty-helmet-74677
08/31/2021, 8:27 AMThanks @billowy-army-68599. Answer to your question is yes, bucket is populated with a external program (actually to reproduce I just synchronize a local folder into a bucket: aws sync - this is the fastest way I found) and then I destroy the cluster which will call the force_delete.
witty-helmet-74677
08/31/2021, 8:29 AMthese are the args used to create the bucket: s3.NewBucket(ctx, "my-bucket-12354698", &s3.BucketArgs{
AccelerationStatus: nil,
Acl: pulumi.String("private"),
Arn: nil,
Bucket: nil,
BucketPrefix: nil,
CorsRules: nil,
ForceDestroy: pulumi.Bool(true),
Grants: nil,
HostedZoneId: nil,
LifecycleRules: s3.BucketLifecycleRuleArray{&s3.BucketLifecycleRuleArgs{Enabled: pulumi.Bool(true), Expiration: &s3.BucketLifecycleRuleExpirationArgs{Days: pulumi.Int(10)}}},
Loggings: nil,
ObjectLockConfiguration: nil,
Policy: nil,
ReplicationConfiguration: nil,
RequestPayer: nil,
ServerSideEncryptionConfiguration: &s3.BucketServerSideEncryptionConfigurationArgs{
Rule: &s3.BucketServerSideEncryptionConfigurationRuleArgs{
ApplyServerSideEncryptionByDefault: &s3.BucketServerSideEncryptionConfigurationRuleApplyServerSideEncryptionByDefaultArgs{
KmsMasterKeyId: pulumi.String("<key_id>"),
SseAlgorithm: pulumi.String("aws:kms"),
},
BucketKeyEnabled: pulumi.Bool(true),
},
},
Tags: nil,
TagsAll: nil,
Versioning: nil,
Website: nil,
WebsiteDomain: nil,
WebsiteEndpoint: nil,
})