No matter how you like to participate in developer communities, Pulumi wants to meet you there. If you want to meet other Pulumi users to share use-cases and best practices, contribute code or documentation, see us at an event, or just tell a story about something cool you did with Pulumi, you are part of our community.

Pulumi Community

Sorry for vagueness here, but I am having an issue with pulumi and k8s, where replicasets fail to update.  It seems like maybe pulumi is waiting for the "old" replicaset of a deployment to have the expected number of replicas, which of course never happens.

Anything around this a known issue?

<@UFM2CJXA5> that should not be true. why do you think that?

in other words, what makes you think it’s waiting on old replicas

I wish I had more detail and something reproducible here, but

Occasionally I get into a state where `pulumi up` will fail, timing out because of an error like this:

```
error: Preview failed: 2 errors occurred:

    * Resource 'ops-view-kube-ops-view' was created but failed to initialize
    * Minimum number of Pods to consider the application live was not attained
```

when I do a `ktl get replicasets` I will see that two of them exist for the given deployment, the new one, which is fine and has the expected number of replicas.  And the old one with zero replicas

The only way I've been able to fix is to manually delete the old replicaset from the cluster, perform manual state surgery on pulumi to remove the deployment, and then do `pulumi refresh`

<@UFM2CJXA5> we never check the old replica count, we are basically waiting for the deployment status to be updated with a condition that says the new replicaset is marked available.

next time it happens `kubectl get -o yaml` on it and let’s take a look at the status field.

Gotcha, hmm yeah I have no idea what is really happening. I'll grab that next time it occurs.

we have seen some issues where the deployment controller is not populating that status condition, which might be a bug upstream.

Maybe. :slightly_smiling_face: Maybe not.

Frustrating in any case.  Especially since state surgery is not very friendly yet, and my json file is growing large haha

this is not good, we’ll get to the bottom of it for sure though.

the deployment stuff has been stable for a reasonably long time though, and these issues are only very recently popping up