Hi. We are using managed kubernetes on AKS.

azure.containerservice.KubernetesCluster

resource provides the property

kubeAdminConfigRaw

which we feed directly in the k8s provider like so:

let cluster = new k8s.Provider(name, {kubeconfig: aks.kubeAdminConfigRaw,})

This works perfectly fine except due to this conservative diffing https://github.com/pulumi/pulumi-kubernetes/blob/master/provider/pkg/provider/provider.go#L270 every change on the AKS resource will trigger a complete recreation of all resources that use this k8s provider instance. Note, that the kubeconfig is mostly a means of authentication, not actually something stateful itself ... Does anyone have the same problem? Any solutions? 🙂

also had the same issue lots of times - i think we ended up just putting 'kubeConfigRaw' in the ignorechanges: [] block for the CustomResourceOptions

@better-rainbow-14549 okay, thanks for your response 🙂 that we tried. Has the downside that if you actually recreate your cluster, you must somehow manually trigger the k8s provider recreation

yeah

Would you solve that with a code change then? Or can you somehow "force" recreation of the provider from the command line for a single run?

yeah take off the ignore changes and run it manually outside of CI i guess

Guess you could

pulumi state delete

it 🤔

ah probably yeah not messed with it too much tbh

Fair enough 😊 thank you

but i expect that sometime soon our service principal credentials will rotate and it'll be somthing we have to fix

cc @gorgeous-egg-16927

I’ve encountered this issue on this exact use-case as well as other use-cases involving creating a provider with the

kubeconfig

input. If the update to

kubeconfig

would result in a different namespace/cluster being targeted then the provider update and dependent resource updates is actually a desirable behaviour; but when the

kubeconfig

update doesn’t effect the namespace/cluster target then it’s quite problematic and actually causes pulumi to crash when replacing dependent resources - pulumi will attempt to replace existing resources and create-then-delete semantics result in “resource already exists” problems… ugh. I don’t know how pulumi can better solve this issue, but it seems like the provider resource needs to do a smarter diff of the credentials to determine if dependent resources actually need to be marked for recreation.