if they're connected to a deployment or stateset you could in principle scale it down to 0 and then back up again.. but that's a bit wonky. I might be using that in combination with automation for tasks related to scaling up persistent volume claims

Running into this issue with a new EKS deployment:

kubernetes:apps/v1:Deployment (web):
    error: 2 errors occurred:
    	* the Kubernetes API server reported that "my-application/web-v845050y" failed to fully initialize or become live: 'web-v845050y' timed out waiting to be Ready
    	* Attempted to roll forward to new ReplicaSet, but minimum number of Pods did not become live

If I watch the cluster I can see all of the pods (2) in the RS stand up and become Ready, and the ReplicaSet/Deployment reports them all as up + Ready/Up-to-Date inside of 2 minutes. This is happening for every Deployment I have configured on this cluster. Tried latest

@pulumi/kubernetes

Node module, I’m on latest Pulumi CLI binary, on EKS 1.19. I tried blowing everything up and redeploying. Nothing of note when describing the Deployment or ReplicaSet. It’s like the Pulumi client is just ignoring the state of the ReplicaSet. Any assistance would be appreciated, there’s very little out there on the Googles other than what I’ve already tried.

Hey peeps... I am tryping to upgrade an AKS cluster and add a VMSS (node pool) to the cluster. When I do the pulumi up, I'm getting this scary "I'm going to replace your resource group" message/warning.

As far as I know, replacing a rg would be a delete/create operation in Azure, and deleting an RG deletes everything inside of it. This particular RG contains all of our lower environment control planes. I've tried to do a non-

import

and an

import

of the resource group and and the cli always shows as wanting to replace it. The

details

(when trying to

import

)doesn't seem to indicate why it wants to replace it.

=>azure:core/resourceGroup:ResourceGroup: (import-replacement) 🔒
        [urn=urn:pulumi:dpts-shared::KubernetesCluster::azure:core/resourceGroup:ResourceGroup::rg-kubernetes]
        [provider=urn:pulumi:dpts-shared::KubernetesCluster::pulumi:providers:azure::default_3_52_1::04da6b54-80e4-46f7-96ec-b56ff0331ba9]      
        name: "rg-kubernetes"
    +-azure:core/resourceGroup:ResourceGroup: (replace) 🔒
        [id=/subscriptions/ad73ec2e-0337-4de5-983c-3944fcb68be8/resourceGroups/rg-kubernetes]
        [urn=urn:pulumi:dpts-shared::KubernetesCluster::azure:core/resourceGroup:ResourceGroup::rg-kubernetes]
        [provider: urn:pulumi:dpts-shared::KubernetesCluster::pulumi:providers:azure::default_3_6_1::09b02d81-f05d-4347-a5d2-831be11283e0 => urn:pulumi:dpts-shared::KubernetesCluster::pulumi:providers:azure::default_3_52_1::output<string>]
        id      : "/subscriptions/ad73ec2e-0337-4de5-983c-3944fcb68be8/resourceGroups/rg-kubernetes"
        location: "centralus"
        name    : "rg-kubernetes"

Do you have any insights as to how I can get my pulumi app to not touch this resource-group? I've added the

protect

statement to it and I've also added a

lock

in the Azure Portal. I do NOT want to accidentally delete all of our control planes and subsequently delete all of the clusters.

This is the details output when I do not try to import the existing ResourceGroup:

If i wanted to ignore changes to the image of a container in a deployment, which is a part of a config group, what would i have to do? I've read this article: https://www.pulumi.com/docs/intro/concepts/resources/#ignorechanges. I am unsure what shape the "root" is in the scenario where i am passing multiple yaml files in a config group. Would the root be an array? 🤔

pulumi up:

error: resource default/azure-secret was not successfully created by the Kubernetes API server : secrets "azure-secret" already exists

but:

$ KUBECONFIG=./kube.yaml k get secret azure-secret -n default   
Error from server (NotFound): secrets "azure-secret" not found

any leads for debugging this? kube.yaml is from:

p stack output kubeconfig --show-secrets > kube.yaml

with an azure-native ManagedCluster and pulumi_kubernetes, how do I: 1. create a namespace conditionally (check if it exists, create if if doesn't) 2. create a secret scoped to the namespace

Anyone ever experience a refresh on a helm chart causing pulumi to think

apiVersion

on an ingress is different than what it actually is. Basically in reality my ingress is

extensions/v1beta1

but when I refresh pulumi changes state to

<http://networking.k8s.io/v1|networking.k8s.io/v1>

and then when I preview it shows it wanting to change it to

extensions/v1beta1

which it already is... I'm also seeing this

Ingress has at least one rule that does not target any Service. Field '.spec.rules[].http.paths[].backend.serviceName' may not match any active Service

which is also around this helmchart and ingress. I found an issue form 2020 around this. Basically then it was simply a misconfigured helmchart. However, I have confirmed that my labels are correct and my label selectors all line up.

How can I tell Pulumi to overwrite all resources that are already there?

--replace

requires that you specify each resource, but there are quite a lot, I just want it to clobber everything.

I would like to automate our deployments, but want our abstractions to take into account the changes between versions of Kubernetes, e.g. APIs being promoted which leads to other resource versions, different annotations etc. What is the best way to find out against which version of Kubernetes we are working? So far I looked in the Kubernetes

Provider

class but that doesn’t expose the info. Is there a standard way of getting this using the

pulumi-kubernetes

provider?

Hey all, I recently deployed the aws-load-balancer-controller chart into an EKS cluster with Pulumi. The actual deploy worked great, but now every

pulumi preview

is convinced that it needs to delete all the resources deployed by that chart:

└─ kubernetes:<http://helm.sh/v3:Chart|helm.sh/v3:Chart>                                                   aws-load-balancer-controller                                                 
 -      ├─ kubernetes:<http://rbac.authorization.k8s.io/v1:Role|rbac.authorization.k8s.io/v1:Role>                               default/aws-load-balancer-controller-leader-election-role         delete     
 -      ├─ kubernetes:core/v1:Secret                                                  default/aws-load-balancer-tls                                     delete     
 -      ├─ kubernetes:<http://rbac.authorization.k8s.io/v1:ClusterRoleBinding|rbac.authorization.k8s.io/v1:ClusterRoleBinding>                 aws-load-balancer-controller-rolebinding                          delete     
 -      ├─ kubernetes:core/v1:ServiceAccount                                          default/aws-load-balancer-controller                              delete     
 -      ├─ kubernetes:<http://rbac.authorization.k8s.io/v1:RoleBinding|rbac.authorization.k8s.io/v1:RoleBinding>                        default/aws-load-balancer-controller-leader-election-rolebinding  delete     
 -      ├─ kubernetes:core/v1:Service                                                 default/aws-load-balancer-webhook-service                         delete     
 -      ├─ kubernetes:<http://admissionregistration.k8s.io/v1:ValidatingWebhookConfiguration|admissionregistration.k8s.io/v1:ValidatingWebhookConfiguration>  aws-load-balancer-webhook                                         delete     
 -      ├─ kubernetes:<http://rbac.authorization.k8s.io/v1:ClusterRole|rbac.authorization.k8s.io/v1:ClusterRole>                        aws-load-balancer-controller-role                                 delete     
 -      ├─ kubernetes:<http://admissionregistration.k8s.io/v1:MutatingWebhookConfiguration|admissionregistration.k8s.io/v1:MutatingWebhookConfiguration>    aws-load-balancer-webhook                                         delete     
 -      ├─ kubernetes:apps/v1:Deployment                                              default/aws-load-balancer-controller                              delete     
 -      └─ kubernetes:<http://apiextensions.k8s.io/v1beta1:CustomResourceDefinition|apiextensions.k8s.io/v1beta1:CustomResourceDefinition>           targetgroupbindings.elbv2.k8s.aws                                 delete

If you actually go through with the

pulumi up

though, it (correctly) leaves all those resources alone. Has anyone seen anything like this recently? I saw a few old issues about

pulumi preview

being wrong, but none that were presently open.

We're having some problems trying to perform an

up

on an existing cluster in EKS. I determined that we needed to associate an AWS IAM Role with a Kubernetes group in order for us to connect to each other's clusters. Now however after changing the CI from the original IAM user to a service account that assumes the role we get the following error on

up

.

Configured Kubernetes cluster is unreachable: unable to load schema information from the API server: the server has asked for the client to provide credentials

We have our Pulumi code outputting the kubeconfig file and its the same one I am currently connected with so it can't be that the cert is expired or the kubeconfig is invalid. Any help is appreciated.

how can i prevent auto-naming for created namespaces? tried to follow links via FAQ, but haven't been able to figure it out yet with python, azure-native, pulumi_kubernetes:

i'm trying to create a docker registry (private) secret in a namespace, and then patch the default service account for that namespace to use that secret, but i get the following error:

error: resource ****/serviceAccount-****-ge0e5qf8 was not successfully created by the Kubernetes API server : ServiceAccount in version "v1" cannot be handled as a ServiceAccount: v1.ServiceAccount.ImagePullSecrets: []v1.LocalObjectReference: readObjectStart: expect { or n, but found ", error found in #10 byte of ...|ecrets":["****/|..., bigger context ...|{"apiVersion":"v1","imagePullSecrets":["****/regcred"],"kind":"ServiceAccount","metad|...

any insights? code as follows:

What is the Pulumi equivalent of

kubectl -n $NAMESPACE patch serviceaccount default -p '{"imagePullSecrets": [{"name": "regcred"}]}'

? (azure-natice, python, pulumi_kubernetes)

For kubernetes-helm stuff - do you need to have the target repository cached locally first? Trying to run this:

const keyvaultCSI = new k8s.helm.v3.Chart("keyVaultCSI",{
        chart: "csi-secrets-store-provider-azure",
        version: "0.0.17",
        fetchOpts: {
            repo: "<https://raw.githubusercontent.com/Azure/secrets-store-csi-driver-provider-azure/master/charts//>"
        },
        values: {
            logFormatJSON: true,            
        }
    },{
        provider: cluster
    })

And getting this as an error:

pulumi:pulumi:Stack baseline-k8s-dev.k8s.infratesting create error: Unhandled exception: Error: invocation of kubernetes:helm:template returned an error: failed to generate YAML for specified Helm chart: failed to pull chart: no cached repo found. (try 'helm repo update'):

Basically following this instruction for installation: https://azure.github.io/secrets-store-csi-driver-provider-azure/getting-started/installation/

Anyone that has experienced a diff between what is in

last-applied-configuration

compared to what is actually deployed? I have a case where an environment variable, with a reference to a secret is defined in

last-applied-configuration

, but it is not under the actual spec. What's weird is that it is just this one variable, another variable defined exactly the same way on the line above in pulumi is there in both places.

So - I'm hitting the first trial of "getting started with pulumi for kubernetes" 🙂 I've created a namespace - with pulumi - and now it wants to recreate it, even though it exists. How can I ensure that it doesn't try to do that again (since it currently fails because it already exists). Basically, it seems to have gotten deleted from the stack (i've done a few destroys and ups and backs and forths), thus is not populated in refresh, and attempts to create it fail because it already exists.

also going against AKS - so getting lots of apiserver "throttling request" responses - any chance of adding some rate limiting to the kubernetes provider, or do I just hope that this will resolve itself and not ending up blocking my provider at some point? 😉

Hi, I use helm and would like to pass the values to my chart as a yaml file. The Chart class takes a json structure as input for values. Is there a utility in pulumi to read a yaml file and feed it to the chart constructor ? Or do you use something like https://www.npmjs.com/package/js-yaml? Actually with this, I am not sure if pulumi is able to know if there is a diff to apply regarding the file.

I've used js-yaml, seems like what pulumi also uses for node.

In other news.. I've got a few helm executions in my code, and of course I hadn't run it for a long stint while I added stuff - now I hit this issue, getting a whole bunch of these:

unhandled rejection: CONTEXT(1168): Invoking function: tok=kubernetes:helm:template asynchronously
STACK_TRACE:
Error
    at Object.debuggablePromise (c:\<path>\node_modules\@pulumi\pulumi\runtime\debuggable.js:69:75)
    at c:\<path>\node_modules\@pulumi\pulumi\runtime\invoke.js:126:45
    at Generator.next (<anonymous>)
    at fulfilled (c:\<path>\node_modules\@pulumi\pulumi\runtime\invoke.js:18:58)
    at processTicksAndRejections (node:internal/process/task_queues:94:5)

Searching around has shown similar messages when helm is unable to output something - but there is no error message outside of this. Also - this is on the stack.preview() step (using automation) edit: grmbl.. may have found the culprit - by going through each helm resource one by one and making sure its the only one commented out, I finally got a proper error message. So strange that I otherwise only get a bunch of these and otherwise nothing else..

after updating python pulumi_kubernetes from 2.8.4 to 2.9.0, i'm getting a diff in my state it's not clear what the diff is, and selecting yes does not make it go away the next time (update succeeds, and the diagnostic warning is a deliberate placeholder) diff as follows in thread:

Hi, I create an helm chart with Pulumi and need to access one LB service created from it. How should I go about this? In general, is it possible to get a k8s service in a Pulumi program? Actually I do this:

export const hostname = ambassador.getResourceProperty("v1/Service", "ambassador", "status")

but status is not found. Where as with

kubectl get svc ambassador -o yaml

I get an output with a status. Is it normal that pulumi does not find the info juste after the creation of the chart?

EDIT: False alarm 😞 Leaving it up though as deleted posts just make people curious for no reason 😉 There's something weird about diffs with one helm chart in particular.. every time I run the stack, it wants to replace some certificates.. with the exact identical certificate..

├─ kubernetes:<http://helm.sh/v3:Chart|helm.sh/v3:Chart>                                                 akv2k8s
 ~   │  ├─ kubernetes:<http://admissionregistration.k8s.io/v1:MutatingWebhookConfiguration|admissionregistration.k8s.io/v1:MutatingWebhookConfiguration>  akv2k8s/akv2k8s-envinjector       update      [diff: ~webhooks]
 +-  │  ├─ kubernetes:core/v1:Secret                                                akv2k8s/akv2k8s-envinjector-tls   replace     [diff: ~data]
 +-  │  ├─ kubernetes:core/v1:Secret                                                akv2k8s/akv2k8s-envinjector-ca    replace     [diff: ~data]
 ~   │  └─ kubernetes:apps/v1:Deployment                                            akv2k8s/akv2k8s-envinjector       update      [diff: ~spec]

Edit: neeevermind! there is a tiny diff in the certificates generated.. a bit frustrating, but of no consequence. Was almost certain there might have been some encoding issues triggering the diff or something, but no - new certs are generated by the chart every time it's touched. oh well.

This does lead me to another thing though.. is there any way of ignoring such changes? I know it's possible to do ignoreChanges on a resource - but how do I match the underlying components in the helm chart? Basically I want to set a little flag that will state that typically I do not want to cycle these certs every single time I operate against the stack.. i.e. ignore akv2k8s/akv2k8s-envinjector-ca.data, etc..

├─ kubernetes:<http://helm.sh/v3:Chart|helm.sh/v3:Chart>                                                 akv2k8s
 ++  │  ├─ kubernetes:core/v1:Secret                                                akv2k8s/akv2k8s-envinjector-ca    created replacement     [diff: ~data];
 ++  │  ├─ kubernetes:core/v1:Secret                                                akv2k8s/akv2k8s-envinjector-tls   created replacement     [diff: ~data];
 ~   │  ├─ kubernetes:apps/v1:Deployment                                            akv2k8s/akv2k8s-envinjector       updated                 [diff: ~spec]; Deployment initialization
 ~   │  └─ kubernetes:<http://admissionregistration.k8s.io/v1:MutatingWebhookConfiguration|admissionregistration.k8s.io/v1:MutatingWebhookConfiguration>  akv2k8s/akv2k8s-envinjector       updated                 [diff: ~webhooks]

How do I match that?

is there any way of getting intermediate output from the helm resource? I'm getting errors along the lines of `invocation of kubernetes:helm:template returned an error: failed to generate YAML for specified Helm chart: failed to create chart from template: YAML parse error on akv2k8s/templates/env-injector-apiservice.yaml: error converting YAML to JSON: yaml: line 15: could not find expected ':'`and it would be great to inspect what it's trying to do along the way..

does pulumi offer any functions to generate kubeconfig's ? I have an existing kubernetes cluster, and I want to try and connect to it using pulumi + typescript, the provider can accept a kubeconfig (https://www.pulumi.com/docs/reference/pkg/kubernetes/provider/) which is easy if i'd created the cluster using 

pulumi-eks

 . what am I missing?