https://pulumi.com logo
Title
f

faint-dog-16036

07/27/2021, 7:09 PM
Man, absolute nightmare scenario using pulumi (this was probably my mistake): I just finished moving our dev cluster and vcs system to be managed by pulumi. I ran locally before landing into gitops to ensure things were clean, pulumi runs on gitops and wipes away our vcs hard disk, blob files, database and half of my cluster at random. The connection and state was fine just seconds before.
😞 1
🤔 1
b

billowy-army-68599

07/27/2021, 7:13 PM
this sounds scary, can you share more of the diff/output? feel free to DM me
are you using the OSS backends? did you run preview before you ran
pulumi up
?
f

faint-dog-16036

07/27/2021, 7:32 PM
Using the Azure blob backend.
The sadness of the whole story is I was just about to add protect to all of the important resources. I was just cleaning something up.
Edited title, as I'm sure some part of my system messed up here, I just don't know where.
b

billowy-army-68599

07/27/2021, 7:47 PM
I would love to see the diff from your CI to debug this, usually when I see this it’s because the connection to the object store isn’t corrrct
f

faint-dog-16036

07/27/2021, 8:19 PM
yeah, will definitely send it, just trying to restore everything first, it's been one of those days
@billowy-army-68599: Finally got everything in a good place, ended up being an out of order job against an old commit that retried AFTER I migrated everything over, blowing some pieces of imported infrastructure away. ... I didn't have resource locks or `protect`s on this infra yet as it was a dev tier under development. Luckily we had backups for everything, it just cost me a day of pain after blowing things away.
b

billowy-army-68599

07/28/2021, 4:27 PM
ah sorry to hear that. you might want to turn on state locking too:
PULUMI_SELF_MANAGED_STATE_LOCKING=1
🙌 1