wide-parrot-89490
03/09/2023, 4:24 PMpulumi up
fails with pre-step event returned an error: failed to save snapshot: [403] The provided update token has expired.
I've seen the message mentioned here before but the only occurrence I see in search now is in a slightly different context. I also see this issue in pulumi tracker, but in our case it's definitely not triggered by resource pressure on the machine where we run pulumi.
As far as I can figure out this is a combination of project size (it happens only in our largest project, 1k+ resources) and effective network speed (less likely to happen with better connection, although "worse connections" in this context can still be fast FTTH links, just farther away from pulumi in network distance); when it happens it's reproducible for extended periods of time from that particular location, but the pulumi up
still succeeds when retried from a different location. Guessing from the outside, it feels like the sync with pulumi server just takes a long time and token refresh uses the same connection pool and request queue as everything else, timing out before the request runs.
We're considering splitting the project but since that's a high lift change I figured I'd ask: is this maybe something known and being worked on? Is there any simpler workaround we could try, even temporarily?many-telephone-49025
03/10/2023, 3:13 PMwide-parrot-89490
03/10/2023, 3:39 PMmany-telephone-49025
03/10/2023, 3:44 PM