Hi all, We see a lot of requests to our state bac...
# general
a
Hi all, We see a lot of requests to our state backend when doing a
pulumi up
without any changes. Is this expected behavior? I would expect Pulumi to only make a snapshot if something changes, is that not the case? We have found the skip-checkpoints flag, but would like to better understand the underlying issue (if any) before we enable it. Our large >3000 resource stacks are currently taking a very long time (>40min) to
up
even if there are little or no changes and are looking to speed it up.
e
If you run with debug logs
--logtostderr -v9
you might see some "SnapshotManager: mustWrite() true because" lines which will tell you why they system still thought it had to do a write even though the inputs hadn't changed enough to trigger a provider update. Give that a try, we can see if that's whats causing the writes.
a
Sweet! Sounds good, will do. Just to preemptively clarify; it's not expected to see snapshot writes w/o actual changes?
e
It's not expected but we've had issues in the past that triggered it
a
When doing a
pulumi up
on a test stack I'm seeing:
Copy code
mustWrite() false
on all resources when parsing the logs. When doing
pulumi up --refresh
I see:
Copy code
Resources:
    39 unchanged
in stdout And a lot of:
Copy code
SnapshotManager: mustWrite() true because of Inputs
The logs are a bit hard to parse. I have added
-p 1
in the hopes log lines wouldn't interleave, but I'm still having trouble seeing where operations start and end. Any pointers on how to further debug, and/or what to look at are much appreciated : ) I am looking at the
.Diff()
lines and am seeing that the
.oldInputs
have 2 entries with
__defaults={[]}
and the
.newInputs
have 3. Not sure if that could have anything to do with the issue.
e
oh
__defaults again
yeh that makes sense
can you raise an issue about this, we'll need to get that fixed across a number of repos I think
a
Do you think the number of
__defaults={[]}
entries is indeed the culprit?
e
yeh probably, the engine isn't smart enough to know what inputs are meaningless vs important, so we depend on the providers to give good data here
a
Makes sense. If any other info is needed please let me know, I'll be monitoring the GH issue. Thanks for the quick response, much appreciated! cute pulumi heart