Hello! Happy Friday! What should one do when they ...
# kubernetes
m
Hello! Happy Friday! What should one do when they encounter this error with a
helm.Release
?
Copy code
~  kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release> airflow **updating failed** [diff: ~values]; error: another operation (install/upgrade/rollback) is in progress
Couldn't find any documentation about it 🤔 For now I've destroyed the whole stack & rebuilt it, which is thankfully possible for now, but not later 😅
bump, still cannot find an answer to this.
bump again, pretty sure I'm about to have to nuke the entire stack & redeploy it again because it's unclear how to rollback these resources 🙈
b
do you have other processes doing helm releases?
m
nope
when they fail, the helm hooks fail in a loop
and it never exits 😞
b
do you have a reliable repro?
m
the only reason I'm using a
Release
instead of a
Chart
is because the chart I'm deploying (Apache Airflow) will only deploy with its hooks firing. however I've goofed some config a couple times, and been unable to correct the pulumi state.
ahhh, maybe? I could probably contrive one with the airflow chart. creating a minimal one is a bit out of my depth, I'd have to go learn more about helm hooks, haha
it's essentially this job crashlooping that causes the problem.
okay, I maaayyyy have a minimal repro with that chart
that'll hang on first deploy, pulumi will just wait forever (I've seen it hang in CI for >1hr). you'll see a
Job
called
fail-airflow-run-airflow-migrations
that will keep failing. using
pulumi cancel
or even manually removing the
pending_operations
from state won't help. next
pulumi up
the preview will "look okay" but it'll fail with the above error 👍