https://pulumi.com logo
#general
Title
# general
t

thankful-optician-22583

01/29/2020, 5:04 PM
Hey guys I have a genuine question about pulumi and cicd, I currently use gitlab + pulumi and I was wondering how do you guys handle rollbacks, with your deploy. In helm, this is abit more straight forward. Have you guys come up with a good solution to this problem? Specifically with k8s. just an open discussion in a thread would be great.
👍 6
m

many-lock-25065

01/29/2020, 5:59 PM
We would also be interested to understand this.
t

thankful-optician-22583

01/29/2020, 6:07 PM
As of now, our solution is to replay the previous job. This makes sense but, maybe together we can discuss a more elegant solution.
g

green-school-95910

01/29/2020, 11:57 PM
I always considered replaying the previous job the best option for a rollback. In case of a failure what I want is for my service(s) to be exactly as they were on the last successful deploy. Since doing the deploy requires credentials it has to be done in the exact same environment as before, which might also have changed (might even be the cause of the failure), so rolling back from a job on the same commit may break things even more. What I usually do is a job to be executed when the deploy fails, it runs a script to list the pipelines of the project and create a new one on the same commit of the last successful one. Currently I'm running on Github Actions and doing the same thing.
👍 1
❤️ 1
t

thankful-optician-22583

01/30/2020, 6:17 PM
Just updating this thread with another solution
Guys this is what I did to handle rollback, I first create an output object called
Copy code
export const appTag ='your image tag';
before I run pulumi update, I query the output file like this
Copy code
export PREV_APP_TAG=$(pulumi stack output -j | jq '.appTag')
And then if my update fails, i set this
Copy code
pulumi config set app:tag $PREV_APP_TAG
and then run a pulumi update again.
👍 1
3 Views