https://pulumi.com logo
#typescript
Title
# typescript
k

kind-room-82948

07/14/2022, 6:19 PM
Anyone have any performance tips for larger stacks? We've got a couple stacks with 1000+ objects with a mix of GCP and K8s objects and running a refresh+up can take 10-15 minutes at this point. Doesn't seem to be specific to certain objects either since all stacks are different. I've run the debug tracing and haven't been able to get much out of that yet.
Trace seems to point to calls mentioned in this issue: https://github.com/pulumi/pulumi/issues/7682
Running with GCS backend
w

worried-rain-74420

07/14/2022, 6:29 PM
Wow, this is pretty great timing. I was just looking into TypeScript performance improvements. I think you found one of the biggest issues. I’m going to be looking into lazy-loading modules in TypeScript this quarter, which should fix this issue.
🙌 1
k

kind-room-82948

07/14/2022, 6:30 PM
There does seem to be some runtime issues as well at least for my stack, though for whatever reason the
pulumi-up
trace doesn't actually split into sub-traces and is all in one block. Still indicates 11 minute runtime in my example
Excited for the compilation improvements though!
w

worried-rain-74420

07/14/2022, 6:33 PM
10-15 minutes is a long time! Which storage backend are you using?
k

kind-room-82948

07/14/2022, 6:33 PM
Google storage, though I have tried out pulumi service with what seemed like only marginal improvements
I previously have a stack with 3000+ objects that took up to 45 minutes that I split up to make it a little more efficient, but whatever the problem was it seems linear to object count
With I think the refresh portion being a big part of that
w

worried-rain-74420

07/14/2022, 6:36 PM
Interesting! I can imagine why refresh would be slow, since that introduces a lot of network transfer. But I haven’t looked into the perf of
refresh
yet.
k

kind-room-82948

08/03/2022, 8:34 PM
Updating here that we actually moved over to s3 and had some substantial performance improvements over gs. Looks like google storage limits single file writes to once per second, which when you have 1000 objects to refresh is 1000s (16 minutes)
w

worried-rain-74420

08/09/2022, 5:10 PM
Wow, that’s good to know! I’ll make a note to look into that!
Thanks for following up! Sorry for the slow response, I was on vacation.
2 Views