https://pulumi.com logo
#general
Title
# general
a

aloof-leather-66267

01/16/2024, 3:55 AM
Does Pulumi with a local backend storage have a practical limit on number of resources in a stack? I have a stack with around 5000 resources, and Pulumi is getting really slow.
pulumi preview
with no changes take 5 minutes. In addition, now when I try creating new nested resources, some of the child resources are created, and then the process fails with an error:
Copy code
error: post-step event returned an error: failed to save snapshot: .pulumi/stacks/live.json: snapshot integrity failure; it was already written, but is invalid (backup available at .pulumi/stacks/live.json.bak): child resource urn:CHILD_URN refers to missing parent urn:PARENT_URN
c

cuddly-computer-18851

01/16/2024, 4:21 AM
you could try setting ``PULUMI_SKIP_CHECKPOINTS` and see if any different?
e

echoing-dinner-19531

01/16/2024, 1:08 PM
If preview is slow then it's nothing to do with checkpoints, previews don't write checkpoints. It is probably the same issue that https://github.com/pulumi/pulumi/pull/14764 is trying to fix, but changing the engine to be parallel is far more involved than that PR suggests.
Copy code
child resource urn:CHILD_URN refers to missing parent urn:PARENT_URN
That looks like an engine bug. You should raise an issue at https://github.com/pulumi/pulumi/issues with more information on that.
a

aloof-leather-66267

01/16/2024, 5:19 PM
That looks like an engine bug. You should raise an issue at https://github.com/pulumi/pulumi/issues with more information on that.
Thanks, done here: https://github.com/pulumi/pulumi/issues/15155
o

orange-policeman-59119

01/17/2024, 7:01 PM
@steep-portugal-37539 great to see your issue addressed so quickly! Regarding filestate scaling, we do expect a practical limit in that IO (disk or network) will scale quadratically with the number of resources, the Pulumi Cloud backend uses a diffing protocol to send much smaller, proportional-to-change-in-state deltas. That may be something to keep in mind, especially if you're using filestate over a network filesystem (whether S3 or NFS/CIFS/etc mounted drives).
a

aloof-leather-66267

01/17/2024, 7:10 PM
I'm using the local filesystem. Shouldn't the only I/O for a preview be to load the stored state and code once? My state file is 15mb, but that's not so big.
o

orange-policeman-59119

01/17/2024, 7:11 PM
Oh yes, for a preview that's right. For up, there is that quadratic scaling factor in the amount of IO. The size state file is proportional to the # of resources, and the number of times it is written is also proportional to the # of resources.
a

aloof-leather-66267

01/17/2024, 8:35 PM
and the # of times it is written is also proportional to the # of resources.
Would that only apply if checkpoints are being written? If checkpoints are disabled, there should only be 1 write, for the final state.
e

echoing-dinner-19531

01/17/2024, 10:34 PM
If checkpoints are disabled, there should only be 1 write, for the final state.
Correct, but then you loose out on the safety of tracking operations in the case of a crash/power-outage/etc.