freezing-quill-32178
07/22/2021, 9:28 AMHelm
and pulumi.StackReference
, particularly slow pulumi up
… for example stack that usually took ~2 mins now take 5+ 10, some Helm stacks never finish so had to be killed as well (which lost the actual stack.json state and had to be recreated from backup and cleaned up)
The Helm resources are marked as deleted (and actually deleted on the k8s cluster) but the pulumi up
is stuck and never finishes), inspecting the stack json the resources are still present in the pending_operations
.
I’m using cert-manager
helm chart for example, failed during install and during uninstall as well
Also seeing a lot of
I0722 11:14:27.696360 74884 request.go:655] Throttling request took 1.07776852s, request: GET:https://[REDACTED]/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s
I0722 11:14:38.496586 74884 request.go:655] Throttling request took 1.098435921s, request: GET:https://[REDACTED]/apis/admissionregistration.k8s.io/v1?timeout=32s
pulumi up
installing cert-manager
v1.4.0 through the helm chart took 45 minutes.billowy-army-68599
freezing-quill-32178
07/22/2021, 6:55 PMTF_LOG=TRACE pulumi up --logtostderr -v=9
for better description