https://pulumi.com logo
Join Slack
Powered by
Pulumi is really slow. Would using local state imp...
# general
r
Pulumi is really slow. Would using local state improve things? Is there a way to profile the 
pulumi up
?
l
Generally, the slow part is the API calls to the clouds. Turn up the verbosity (
--verbose 9
) for more output. You can also profile the CPU and memory usage if you think there might be a problem there (
--memprofilerate
, 
--profiling
, 
--tracing
), but generally it's the network requests and waiting for them to return that takes a while. You can check to see if you have unnecessary dependencies causing reduced parallelism.
To your original question: probably not. If it did, it would be milliseconds of a difference, and slow Pulumi projects can easily be measured in minutes, so the location of the state rarely makes a helpful difference.
r
I don't believe it is the API calls that are slowing Pulumi down the most. It takes about 25 seconds to even print the Type, Name, Plan, and Info headers
Additionally I found no additional output with verbose 9 so I am not sure why the startup is so slow
l
Are you storing your state on a floppy disc?
(That's a humourous conversation-starter, not a jibe 🙂 )
r
no haha. It looks like it is taking a long time to start the node language server and GetRequiredPlugins
I have cut out TSC and replaced with SWC so it should not be hanging on a TypeScript compilation step and I see no logs that that is called.
I don't know why that step would take so long, because the plugins should be cached locally
Copy code
❯ pulumi about
CLI
Version      3.127.0
Go Version   go1.22.5
Go Compiler  gc

Plugins
KIND      NAME    VERSION
resource  gcp     7.34.0
resource  gcp     7.33.0
language  nodejs  unknown
resource  random  4.16.3
resource  random  4.16.3

Host
OS       ubuntu
Version  22.04
Arch     x86_64

This project is written in nodejs: executable='/usr/bin/node' version='v20.16.0'

Backend
Name           code-server
URL            file://~
User           ubuntu
Organizations
Token type     personal

Pulumi locates its logs in /tmp by default
warning: Failed to get information about the Pulumi program's dependencies: could not find either /home/ubuntu/code/git.kalinow.ski/nimbus/infra/yarn.lock or /home/ubuntu/code/git.kalinow.ski/nimbus/infra/package-lock.json
warning: Failed to get information about the current stack: No current snapshot
I'm not sure why nodejs version is unknown, why gcp and random are both listed twice, or why it is failing to understand PNPM dependencies
Here are some logs:
Copy code
$ pulumi -v=9 --tracing file:pulumi.trace --logtostderr preview
I0806 06:05:59.734891  584610 util.go:617] errors detecting git metadata: 1 error occurred:
        * detecting Git remote URL: could not read origin information: remote not found

Previewing update (dev):
I0806 06:05:59.843070  584610 update.go:210] *** Starting Update(preview=true) ***
I0806 06:05:59.843511  584610 plugins.go:124] gatherPluginsFromProgram(): gathering plugins from language host
I0806 06:05:59.843709  584610 plugins.go:1809] GetPluginPath(language, nodejs, <nil>): found on $PATH /home/ubuntu/.pulumi/bin/pulumi-language-nodejs
I0806 06:05:59.843976  584610 plugins.go:1832] GetPluginPath(language, nodejs, <nil>): found next to current executable /home/ubuntu/.pulumi/bin/pulumi-language-nodejs
I0806 06:05:59.844194  584610 plugin.go:186] newPlugin(): Launching plugin 'nodejs' from '/home/ubuntu/.pulumi/bin/pulumi-language-nodejs' with args: -packagemanager=pnpm,-typescript=false,-root=/home/ubuntu/code/git.kalinow.ski/nimbus/infra,127.0.0.1:33579
I0806 06:05:59.908294  584610 langruntime_plugin.go:250] langhost[nodejs].GetPluginInfo() executing
I0806 06:05:59.909864  584610 langruntime_plugin.go:137] langhost[nodejs].GetRequiredPlugins(root=/home/ubuntu/code/git.kalinow.ski/nimbus/infra, program=/home/ubuntu/code/git.kalinow.ski/nimbus/infra, entryPoint=.) executing
I0806 06:06:22.452930  584610 langruntime_plugin.go:187] langhost[nodejs].GetRequiredPlugins(root=/home/ubuntu/code/git.kalinow.ski/nimbus/infra, program=/home/ubuntu/code/git.kalinow.ski/nimbus/infra, entryPoint=.) success: #versions=4
I0806 06:06:22.452990  584610 plugins.go:137] gatherPluginsFromProgram(): plugin nodejs <nil> () is required by language host
It seems the GetRequiredPlugins is taking 23 seconds and then it does more plugin work. This is quite painful as it slows the dev feedback loop a lot and it really should take no time at all (plugins already are installed). I wonder if this is Pulumi not understanding pnpm but it should from my understanding of the closed issues
f
Good news! There was a fix for this 4 days ago (in v3.128.0, https://github.com/pulumi/pulumi/pull/16865) and the perf metric will be closely watched at some point: https://github.com/pulumi/pulumi/issues/16866
r
Thank goodness! That is so much nicer
❤️ 1
71 Views
Open in Slack
PreviousNext
Powered by