pulumi is awesome but as my project is growing eac...
# general
w
pulumi is awesome but as my project is growing each update becomes slower and slower how can I profile the performance to understand what’s slowing things down?
👀 1
b
Are you using the oss backends? Which provider are you using for storage if so?
w
it was the same when I was using pulumi.com now I’m using s3
b
See the performance logging section there. It should definitely not happen with Pulumi.com
p
I can confirm that things can get slower once you have a lot of resources (I’m using Pulumi Service). One solution is to divide your code into smaller, independent pulumi projects.
I’m just interested how easy we can generate some profiling to determine where’s the bottleneck in all this.
w
I’m wondering if I have too many dependant outputs that’s causing the slowdown
I mean what else can slow down the flow?
p
I guess the execution must look something like this: • download the latest state from the backend (Pulumi Service or other custom backends) • execute all reads (either Stack References or reading existing infrastructure resources) • calculate the diff (it should be done locally)
@white-chef-55657 are you suspecting the diff calculation takes too much time due to the number of dependencies?
w
the preview, yea
g
do you have many Resource Output dependencies or also dependencies on other projects/stacks (stack reference)?
w
wait, I can have cross-stack/project dependencies? 😄
g
yes https://www.pulumi.com/docs/intro/concepts/stack/#stackreferences there are basically 3 types of references Resource Output - (within stack) Stack Reference (stack output) - another project/stack Lookup - of existing resource
w
interesting, will look into it I’m hoping to share the same k8s cluster between different stacks and deploy multiple stacks as different namespaces
g
take some time to play around as this decision will impact the overall architecture, I wonder if https://archive.pulumi.com/ managed to capture one very interesting conversation where we shared different approaches