https://pulumi.com logo
Join Slack
Powered by
Is Pulumi having a maintenance now? Deployments ar...
# general
e
Is Pulumi having a maintenance now? Deployments are AWFULLY slow for me 😕 Deploying just 4 lambdas and 8 queues is running now for over 2 hours without finishing (it also deletes 4 lambdas and 8 queues)
b
There’s no maintenance happening now. What type of queue are you deleting? Can you check the console to see the state of the deletions
e
@billowy-army-68599 Hey, yea sure what do you want me to check?
b
Assuming it’s SQS queue, Check the queue status, ie if it is being deleted and if there are any messages in there
e
What I’m creating: 4 Lambdas 8 queues (4 queues - 4 DLQ) Event source subscrption (automatically) RolePolicyAttachment (default)
@billowy-army-68599 The queues were already deleted. It’s now finishing deleting the last Lambda. It took around 2:30 hours
I get that a lot that the update times are inconsistent. Sometimes I deploy lambda code changes in just 30-40 seconds, but sometimes it takes around 10 minutes. Really inconsistent
But 2:30 hours just for a few lambdas and queues? That’s insane
By the way, the queues were all empty
@billowy-army-68599 By the way, there is a huge data stream downloaded when visiting the online update screen. It fetches tens of MB and takes over a minute to display something. I think something is happening on your end that’s draining you
@billowy-army-68599 It finished now. Took 3 hours and 10 minutes. Just insane
Is there any data I can share with you so you can further investigate it?
b
I'm sorry you feel it's "insane". Bearing in mind how the Pulumi model works, it seems very unlikely it's related to the service. You can do a performance log and attach it to a GitHub issue: https://www.pulumi.com/docs/troubleshooting/#performance
are you attaching an ENI to your lambda?
e
What is ENI?
b
elastic network interface
e
no, I’m not attaching an ENI then 😕
Sorry if it sounds offensive, but I’m just not used to have such high deployment times. It usually takes a few minutes
b
basically: while you're deleting your stack, you need to verify that the mode the lambda function is in when the deletion is in progress. If you could screenshot the pulumi deletion and the AWS console for the lambda function it would help. The Pulumi service does not in any way affect deployment times, all we do is store the checkpoint If you break down how this works: - Pulumi calls the AWS API to delete a lambda function - it polls the AWS API periodically to check the state of the function (ie, is it deleted yet?) - it writes that status to the checkpoint file, which gets written to the pulumi console if your lambda is taking hours to delete, it's more likely that it's misconfigured somehow than it is likely to be the Pulumi service - especially considering our monitoring seems to show now issues and no other customers are reporting any delays or slowness at this time
and just to be entirely frank: this forum and community is based on best effort support. I do my best to respond in a timely manner - I understand your frustration at the slowness, but the language being used isn't helpful. If you need more help, we have support plans available - please feel free to email sales@pulumi.com
e
@billowy-army-68599 Sorry if you found my language offensive, it was not my intent! The slowness also happened when creating resources. According to your answer before, it means that the issue is on AWS side rather than Pulumi’s. Is there any log in AWS that you’re aware of that I can inspect?
b
I think the best next step is to do a performance trace when creating: https://www.pulumi.com/docs/troubleshooting/#performance
e
Alright, sounds good. I’ll do it for every deployment so I have the performance log for each deployment and will inspect it when I encounter a slow deployment again. Thanks!
5 Views
Open in Slack
PreviousNext
Powered by