Channels
announcements
automation-api
aws
azure
blog-posts
built-with-pulumi
cloudengineering
cloudengineering-support
content-share
contribex
contribute
docs
dotnet
finops
general
getting-started
gitlab
golang
google-cloud
hackathon-03-19-2020
hacktoberfest
install
java
jobs
kubernetes
learn-pulumi-events
linen
localstack
multi-language-hackathon
office-hours
oracle-cloud-infrastructure
package-authoring
pulumi-ai
pulumi-cdk
pulumi-cloud
pulumi-crosscode
pulumi-deployments
pulumi-kubernetes-operator
pulumiverse
python
registry
status
testingtesting123
testingtesting321
typescript
welcome
workshops
yaml
Title
e
elegant-pager-5412
06/02/2021, 1:53 PMIs Pulumi having a maintenance now?
Deployments are AWFULLY slow for me 😕
Deploying just 4 lambdas and 8 queues is running now for over 2 hours without finishing (it also deletes 4 lambdas and 8 queues)
b
billowy-army-68599
06/02/2021, 2:04 PMThere’s no maintenance happening now. What type of queue are you deleting? Can you check the console to see the state of the deletions
e
elegant-pager-5412
06/02/2021, 2:04 PM@billowy-army-68599 Hey, yea sure what do you want me to check?
b
billowy-army-68599
06/02/2021, 2:05 PMAssuming it’s SQS queue, Check the queue status, ie if it is being deleted and if there are any messages in there
e
elegant-pager-5412
06/02/2021, 2:05 PMWhat I’m creating:
4 Lambdas
8 queues (4 queues - 4 DLQ)
Event source subscrption (automatically)
RolePolicyAttachment (default)
@billowy-army-68599 The queues were already deleted. It’s now finishing deleting the last Lambda. It took around 2:30 hours
I get that a lot that the update times are inconsistent. Sometimes I deploy lambda code changes in just 30-40 seconds, but sometimes it takes around 10 minutes. Really inconsistent
But 2:30 hours just for a few lambdas and queues? That’s insane
By the way, the queues were all empty
@billowy-army-68599 By the way, there is a huge data stream downloaded when visiting the online update screen. It fetches tens of MB and takes over a minute to display something. I think something is happening on your end that’s draining you
@billowy-army-68599 It finished now. Took 3 hours and 10 minutes. Just insane
Is there any data I can share with you so you can further investigate it?
b
billowy-army-68599
06/02/2021, 3:03 PMI'm sorry you feel it's "insane". Bearing in mind how the Pulumi model works, it seems very unlikely it's related to the service. You can do a performance log and attach it to a GitHub issue:
https://www.pulumi.com/docs/troubleshooting/#performance
are you attaching an ENI to your lambda?
e
elegant-pager-5412
06/02/2021, 3:04 PMWhat is ENI?
b
billowy-army-68599
06/02/2021, 3:04 PMelastic network interface
e
elegant-pager-5412
06/02/2021, 3:04 PMno, I’m not attaching an ENI then 😕
Sorry if it sounds offensive, but I’m just not used to have such high deployment times. It usually takes a few minutes
b
billowy-army-68599
06/02/2021, 3:09 PMbasically: while you're deleting your stack, you need to verify that the mode the lambda function is in when the deletion is in progress. If you could screenshot the pulumi deletion and the AWS console for the lambda function it would help. The Pulumi service does not in any way affect deployment times, all we do is store the checkpoint
If you break down how this works:
- Pulumi calls the AWS API to delete a lambda function
- it polls the AWS API periodically to check the state of the function (ie, is it deleted yet?)
- it writes that status to the checkpoint file, which gets written to the pulumi console
if your lambda is taking hours to delete, it's more likely that it's misconfigured somehow than it is likely to be the Pulumi service - especially considering our monitoring seems to show now issues and no other customers are reporting any delays or slowness at this time
and just to be entirely frank: this forum and community is based on best effort support. I do my best to respond in a timely manner - I understand your frustration at the slowness, but the language being used isn't helpful.
If you need more help, we have support plans available - please feel free to email sales@pulumi.com
e
elegant-pager-5412
06/02/2021, 3:12 PM@billowy-army-68599 Sorry if you found my language offensive, it was not my intent!
The slowness also happened when creating resources. According to your answer before, it means that the issue is on AWS side rather than Pulumi’s. Is there any log in AWS that you’re aware of that I can inspect?
b
billowy-army-68599
06/02/2021, 3:14 PMI think the best next step is to do a performance trace when creating:
https://www.pulumi.com/docs/troubleshooting/#performance
e
elegant-pager-5412
06/02/2021, 3:15 PMAlright, sounds good. I’ll do it for every deployment so I have the performance log for each deployment and will inspect it when I encounter a slow deployment again.
Thanks!