We are trying to upgrade our GKE cluster and the o...
# general
c
We are trying to upgrade our GKE cluster and the operations keeps timing out at the pulumi end with
10
mins timeout. We are stuck and not able to proceed.
w
Does re-running the
pulumi update
solve this? Or does
pulumi refresh
followed by re-running
pulumi update
work? Which change is it you are making that triggers this? Is it the addition of
networkPolicy
? Does the change eventually succeed in the GKE cluster? https://github.com/pulumi/pulumi/issues/2655 is tracking making timeouts configurable, but I would expect re-running
update
and/or
refresh
to address this if it is just a timeout issue and not a GKE issue.
g
@white-balloon-205 I had this issue occasionally when increasing the
version
property of a large gke nodepool. Since GKE updates each node in the pool one by one and it takes typically ~4-5 minutes per node to do so it can easily exceed the 10 minutes operation timeout. Rerunning pulumi update and refresh after the update completed solved the problem for me.
c
@white-balloon-205 Yes , it is working. But we would certainly like an option to override this issue. The problem is that we would like to run this from CD and we cannot have an
refresh
and
update
part of standard installation. So we end up running from our local box which is not desirable.