Channels
pulumi-ai
package-authoring
pulumi-cdk
oracle-cloud-infrastructure
learn-pulumi-events
linen
registry
built-with-pulumi
pulumi-cloud
contribex
testingtesting321
hacktoberfest
cloudengineering
yaml
pulumi-crosscode
content-share
finops
multi-language-hackathon
office-hours
workshops
blog-posts
localstack
welcome
gitlab
pulumi-kubernetes-operator
jobs
aws
pulumi-deployments
dotnet
golang
announcements
java
pulumiverse
python
install
getting-started
cloudengineering-support
testingtesting123
hackathon-03-19-2020
typescript
google-cloud
contribute
azure
kubernetes
docs
automation-api
status
general
Title
w
witty-helmet-74677
08/27/2021, 9:31 AMHi, during the pulumi destroy of a stack with an AWS S3 Bucket which has 15K objects takes around 45 minutes to delete the objects before deleting the bucket. Using AWS CLI the deletion takes 2 minutes. Is there any argument which could speed the stack destroy in this case or this is a know issue? Language: go.
b
busy-honey-73811
08/27/2021, 10:00 AMCan confirm, we have the same problem with very large S3 Buckets and it takes literally hours to delete them with pulumi. Usually those stacks fail (timeout) on destroy and we have to manually clean-up the S3 Buckets (using CLI or AWS Web Console) to speed things up.
w
witty-helmet-74677
08/27/2021, 11:34 AMOne more addition. The objects are not big 2-3 KB each. In total like 40MB.
b
busy-honey-73811
08/27/2021, 11:49 AMSame here, we have thousands (if not millions) of small to medium sized objects (KB to few MB).
g
great-sunset-355
08/27/2021, 2:04 PMDo you manage each object with pulumi or did you try
force_destroy=Trueb
billowy-army-68599
08/27/2021, 4:38 PMcould you run a performance trace here?
https://www.pulumi.com/docs/troubleshooting/#performance
b
busy-honey-73811
08/27/2021, 6:57 PMIn my case we only manage the S3 Bucket with Pulumi, the objects are created by applications (not Pulumi).
w
witty-helmet-74677
08/30/2021, 8:37 AMWe also maintain only the bucket with pulumi. The objects are created by apps. The pulumi uses force destroy, otherwise wouldn't be able to delete the bucket.
I executed the pulumi destroy now with the traces turned on suggested by @billowy-army-68599. The 45 minutes are spent in /pulumirpc.ResourceProvider/Delete.
b
billowy-army-68599
08/30/2021, 6:01 PM@witty-helmet-74677 just to be sure I'm understanding, you're populating a bucket with an external program then doing a force delete?
as pointed out by @white-balloon-205 (thanks!) it seems we inherit this from the upstream AWS provider:
https://github.com/hashicorp/terraform-provider-aws/issues/12146
w
witty-helmet-74677
08/31/2021, 8:27 AMThanks @billowy-army-68599. Answer to your question is yes, bucket is populated with a external program (actually to reproduce I just synchronize a local folder into a bucket: aws sync - this is the fastest way I found) and then I destroy the cluster which will call the force_delete.
these are the args used to create the bucket: s3.NewBucket(ctx, "my-bucket-12354698", &s3.BucketArgs{
AccelerationStatus: nil,
Acl: pulumi.String("private"),
Arn: nil,
Bucket: nil,
BucketPrefix: nil,
CorsRules: nil,
ForceDestroy: pulumi.Bool(true),
Grants: nil,
HostedZoneId: nil,
LifecycleRules: s3.BucketLifecycleRuleArray{&s3.BucketLifecycleRuleArgs{Enabled: pulumi.Bool(true), Expiration: &s3.BucketLifecycleRuleExpirationArgs{Days: pulumi.Int(10)}}},
Loggings: nil,
ObjectLockConfiguration: nil,
Policy: nil,
ReplicationConfiguration: nil,
RequestPayer: nil,
ServerSideEncryptionConfiguration: &s3.BucketServerSideEncryptionConfigurationArgs{
Rule: &s3.BucketServerSideEncryptionConfigurationRuleArgs{
ApplyServerSideEncryptionByDefault: &s3.BucketServerSideEncryptionConfigurationRuleApplyServerSideEncryptionByDefaultArgs{
KmsMasterKeyId: pulumi.String("<key_id>"),
SseAlgorithm: pulumi.String("aws:kms"),
},
BucketKeyEnabled: pulumi.Bool(true),
},
},
Tags: nil,
TagsAll: nil,
Versioning: nil,
Website: nil,
WebsiteDomain: nil,
WebsiteEndpoint: nil,
})