bump again, pretty sure I'm about to have to nuke the entire stack & redeploy it again because it's unclear how to rollback these resources 🙈

when they fail, the helm hooks fail in a loop

and it never exits 😞

ahhh, maybe? I could probably contrive one with the airflow chart. creating a minimal one is a bit out of my depth, I'd have to go learn more about helm hooks, haha

it's essentially this job crashlooping that causes the problem.

okay, I maaayyyy have a minimal repro with that chart

that'll hang on first deploy, pulumi will just wait forever (I've seen it hang in CI for >1hr). you'll see a

Job

called

fail-airflow-run-airflow-migrations

that will keep failing. using

pulumi cancel

or even manually removing the

pending_operations

from state won't help. next

pulumi up

the preview will "look okay" but it'll fail with the above error 👍