stocky-student-96739
03/29/2021, 9:29 PMkubernetes:apps/v1:Deployment (web):
error: 2 errors occurred:
* the Kubernetes API server reported that "my-application/web-v845050y" failed to fully initialize or become live: 'web-v845050y' timed out waiting to be Ready
* Attempted to roll forward to new ReplicaSet, but minimum number of Pods did not become live
If I watch the cluster I can see all of the pods (2) in the RS stand up and become Ready, and the ReplicaSet/Deployment reports them all as up + Ready/Up-to-Date inside of 2 minutes. This is happening for every Deployment I have configured on this cluster.
Tried latest @pulumi/kubernetes
Node module, I’m on latest Pulumi CLI binary, on EKS 1.19. I tried blowing everything up and redeploying. Nothing of note when describing the Deployment or ReplicaSet. It’s like the Pulumi client is just ignoring the state of the ReplicaSet.
Any assistance would be appreciated, there’s very little out there on the Googles other than what I’ve already tried.billowy-army-68599
03/29/2021, 9:38 PMstocky-student-96739
03/29/2021, 9:38 PMgorgeous-egg-16927
03/29/2021, 10:56 PMstocky-student-96739
03/30/2021, 2:19 AMI0330 15:01:42.143571 1 deployment_controller.go:490] "Error syncing deployment" deployment="my-application/web-rov2l78t" err="Operation cannot be fulfilled on deployments.apps \"web-rov2l78t\": the object has been modified; please apply your changes to the latest version and try again"
gorgeous-egg-16927
03/31/2021, 3:33 PM<http://kubectl.kubernetes.io/restartedAt|kubectl.kubernetes.io/restartedAt>
annotation. I haven’t tested a workaround, but I expect some combination of removing that annotation and using the skipAwait
annotation (https://www.pulumi.com/blog/improving-kubernetes-management-with-pulumis-await-logic/#new-annotations-to-customize-kubernetes-await-logic) would get you unblocked for now.stocky-student-96739
03/31/2021, 4:32 PMfaint-table-42725
04/01/2021, 12:01 AMstocky-student-96739
04/01/2021, 12:59 AM<http://kubectl.kubernetes.io/restartedAt|kubectl.kubernetes.io/restartedAt>
on the Deployment, ReplicaSet, or Pods. Adding the skipAwait
notation got us past the part where we were timing out, but I don’t feel it’s a proper solution since it’s just firing and forgetting and not concerned with the ultimate state of the deployment.able-afternoon-73080
05/12/2021, 2:08 PMstocky-student-96739
07/01/2021, 3:44 PMsparse-park-68967
07/07/2021, 10:59 PMpulumi --logflow --verbose=9 --debug --logtostderr up --yes >& /tmp/logs
) would be very useful. Happy to setup time with you if you are concerned about sharing the detailed logs.stocky-student-96739
07/07/2021, 11:01 PMsparse-park-68967
07/08/2021, 8:20 PMstocky-student-96739
07/08/2021, 8:25 PMsparse-park-68967
07/08/2021, 8:44 PM