https://pulumi.com logo
Title
w

white-chef-55657

05/11/2022, 12:27 PM
pulumi is awesome but as my project is growing each update becomes slower and slower how can I profile the performance to understand what’s slowing things down?
👀 1
b

billowy-army-68599

05/11/2022, 2:10 PM
Are you using the oss backends? Which provider are you using for storage if so?
w

white-chef-55657

05/11/2022, 2:11 PM
it was the same when I was using pulumi.com now I’m using s3
b

billowy-army-68599

05/11/2022, 2:13 PM
See the performance logging section there. It should definitely not happen with Pulumi.com
p

prehistoric-activity-61023

05/11/2022, 2:13 PM
I can confirm that things can get slower once you have a lot of resources (I’m using Pulumi Service). One solution is to divide your code into smaller, independent pulumi projects.
I’m just interested how easy we can generate some profiling to determine where’s the bottleneck in all this.
w

white-chef-55657

05/11/2022, 2:14 PM
I’m wondering if I have too many dependant outputs that’s causing the slowdown
I mean what else can slow down the flow?
p

prehistoric-activity-61023

05/11/2022, 2:16 PM
I guess the execution must look something like this: • download the latest state from the backend (Pulumi Service or other custom backends) • execute all reads (either Stack References or reading existing infrastructure resources) • calculate the diff (it should be done locally)
@white-chef-55657 are you suspecting the diff calculation takes too much time due to the number of dependencies?
w

white-chef-55657

05/11/2022, 2:19 PM
the preview, yea
g

great-sunset-355

05/11/2022, 2:20 PM
do you have many Resource Output dependencies or also dependencies on other projects/stacks (stack reference)?
w

white-chef-55657

05/11/2022, 2:20 PM
wait, I can have cross-stack/project dependencies? 😄
g

great-sunset-355

05/11/2022, 2:21 PM
yes https://www.pulumi.com/docs/intro/concepts/stack/#stackreferences there are basically 3 types of references Resource Output - (within stack) Stack Reference (stack output) - another project/stack Lookup - of existing resource
w

white-chef-55657

05/11/2022, 2:24 PM
interesting, will look into it I’m hoping to share the same k8s cluster between different stacks and deploy multiple stacks as different namespaces
g

great-sunset-355

05/11/2022, 2:27 PM
take some time to play around as this decision will impact the overall architecture, I wonder if https://archive.pulumi.com/ managed to capture one very interesting conversation where we shared different approaches