Anyone have any performance tips for larger stacks...
# typescript
Anyone have any performance tips for larger stacks? We've got a couple stacks with 1000+ objects with a mix of GCP and K8s objects and running a refresh+up can take 10-15 minutes at this point. Doesn't seem to be specific to certain objects either since all stacks are different. I've run the debug tracing and haven't been able to get much out of that yet.
Trace seems to point to calls mentioned in this issue:
Running with GCS backend
Wow, this is pretty great timing. I was just looking into TypeScript performance improvements. I think you found one of the biggest issues. I’m going to be looking into lazy-loading modules in TypeScript this quarter, which should fix this issue.
🙌 1
There does seem to be some runtime issues as well at least for my stack, though for whatever reason the
trace doesn't actually split into sub-traces and is all in one block. Still indicates 11 minute runtime in my example
Excited for the compilation improvements though!
10-15 minutes is a long time! Which storage backend are you using?
Google storage, though I have tried out pulumi service with what seemed like only marginal improvements
I previously have a stack with 3000+ objects that took up to 45 minutes that I split up to make it a little more efficient, but whatever the problem was it seems linear to object count
With I think the refresh portion being a big part of that
Interesting! I can imagine why refresh would be slow, since that introduces a lot of network transfer. But I haven’t looked into the perf of
Updating here that we actually moved over to s3 and had some substantial performance improvements over gs. Looks like google storage limits single file writes to once per second, which when you have 1000 objects to refresh is 1000s (16 minutes)
Wow, that’s good to know! I’ll make a note to look into that!
Thanks for following up! Sorry for the slow response, I was on vacation.