https://pulumi.com logo
#general
Title
# general
j

jolly-window-25842

11/14/2023, 12:05 PM
hello, so I'm kind of in a bad situation: someone fatfingered the Google Cloud dashboard and deleted a StatefulSet from a k8s cluster managed by pulumi. So I thought I'd run
pulumi refresh
and then
pulumi up
to recreate it. The
refresh
detected a couple of small changes and so I confirmed it. Now, if I run
pulumi up
it fails because it wants to re-create all Cloud SQL database instances (which are protected), however the
refresh
didn't detect any changes to the dbs. What are my option? Right now, the pulumi state is completely messed up.
d

dry-keyboard-94795

11/14/2023, 12:07 PM
you run
pulumi preview --diff
to identify the changes pulumi thinks needs making to the dbs
From there, it might be easiest to update the code to reflect what's detected from gcp
j

jolly-window-25842

11/14/2023, 12:11 PM
I can't get the diff because of this error:
unable to replace resource "urn:......." as it is currently marked for protection
should I just unprotect it?
d

dry-keyboard-94795

11/14/2023, 12:12 PM
Sure, though remember to protect it again after finding out the diff
j

jolly-window-25842

11/14/2023, 12:17 PM
but why didn't
refresh
show me these changes? like there is a bucket that now shows
lifecycleRules
were deleted, but
refresh
didn't show it
d

dry-keyboard-94795

11/14/2023, 12:18 PM
It could be a bug, or could be a refresh happened inbetween the change in GCP and the refresh you've just done
j

jolly-window-25842

11/14/2023, 12:22 PM
it's showing strange things, quite sure nobody changed all this values and refresh didn't give any hints about this
still trying to get a full diff, though
d

dry-keyboard-94795

11/14/2023, 12:24 PM
It's worth checking the obvious things at this point. Are you pointing to the right gcp account, have you got the correct branch checked out, is there anything in the state history (if you're using Pulumi Cloud)
j

jolly-window-25842

11/14/2023, 12:26 PM
correct account: I've only got one. Correct branch. And how do I check the state history in Pulumi cloud?
d

dry-keyboard-94795

11/14/2023, 12:48 PM
Something that can end up showing more values in the diff is provider updates, though these should be harmless. pulumi-gcp recently updated to v7. What version are you on currently, is it the same as your last known good apply? An example of the diff (redacted how you see fit) would be helpful
j

jolly-window-25842

11/14/2023, 1:05 PM
I updated the cli, but pulumi-gcp is still at 6
s

stale-answer-34162

11/14/2023, 1:27 PM
I would love a webinar on the stack activity topic for SRE.
j

jolly-window-25842

11/14/2023, 2:50 PM
I don't see the activity link. Anyways, I was able to fix it: there were a few values that people manually changed without going through pulumi, plus a couple of defaults that weren't set in the pulumi, but were returned by
refresh
. And finally, GCP autoscales db disk size and since it was higher than what was set in the pulumi code, pulumi wanted to replace them. I still think the
refresh
command was off: it didn't show any of these things in the preview. Thanks for your help @dry-keyboard-94795, I really appreciate it!
s

stale-answer-34162

11/14/2023, 2:51 PM
image.png
d

dry-keyboard-94795

11/14/2023, 3:06 PM
Glad it was eventually resolved. For the db resizing, you can use the
ignoreChanges
option so the auto-sizing doesn't cause the same issue in future. The defaults in GCP, I believe they're not supposed to show in the refresh output, but they're also not supposed to show in the diff either. Does it still show changes to apply to the db?
2 Views