I'm talking with someone about https://cdk8s.io/ and it sounds a lot like pulumi - but pulumi requires storing somewhere state, whereas this doesn't. What / why does pulumi need to store in case of kubernetes, if "kubernetes already stores state in etcd"?

Hi. We are using managed kubernetes on AKS.

azure.containerservice.KubernetesCluster

resource provides the property

kubeAdminConfigRaw

which we feed directly in the k8s provider like so:

let cluster = new k8s.Provider(name, {kubeconfig: aks.kubeAdminConfigRaw,})

This works perfectly fine except due to this conservative diffing https://github.com/pulumi/pulumi-kubernetes/blob/master/provider/pkg/provider/provider.go#L270 every change on the AKS resource will trigger a complete recreation of all resources that use this k8s provider instance. Note, that the kubeconfig is mostly a means of authentication, not actually something stateful itself ... Does anyone have the same problem? Any solutions? 🙂

Hi there, I have a newbie question regarding Helm deployments with Pulumi. I created a helm resource with the official spinnaker helm chart. I adapt the values and the with the values I create Configmaps. The first deployment with Pulumi was successfully. When I make changes in the Configmaps I see the proper Pulumi resources are being replaced. But it seems after the replace there is no upgrade to the helm chart executed. I need this because there is a K8s job triggered by post-upgrade and this job isn’t running but is strongly required. Can anyone help me out?

Question unrelated to Pulumi, but does anyone here have any experience with the pure Java Kubernetes API? (https://javadoc.io/doc/io.kubernetes/client-java-api/7.0.0/index.html). Context: I’m trying to create a PersistentVolume and PersistentVolumeClaim that I can use later on as a volume in a pod running Pulumi for local state storage. I could use some examples… (It’s the maximum I want to do without Pulumi. 😉 )

Hi, i'm trying to import a

Deployment

which I believe has the same spec, but pulumi is giving me a diff where the entire

containers

array is going to be deleted. When I dump out my deployment object in the code, it has the containers array filled in. I'm guessing i'm wrong and the state doesn't match for the import. Is this a pulumi bug where it can't diff the array contents?

@white-balloon-205 or whoever has most recently touched pulumi/kubernetes, I think there might be a regression or bug in Pulumi 1.14 (Kubernetes provider 1.6.0) where: When changing env vars in a Deployment as in this thread: https://pulumi-community.slack.com/archives/CRFURDVQB/p1589939594223900?thread_ts=1585519243.027400&cid=CRFURDVQB 1. Deployment resources are marked for "replace", not "update" despite having no changes incompatible with an update/PATCH 2.

deleteBeforeReplace

is not used, causing the deployment to fail because deployment names are unique within a namespace and the creation fails I have two questions: 1. Is there a way to force the Kubernetes provider to perform an "update"? 2. Are there best practices for maintaining large lists of env vars to avoid bizarre updates like the one shown in the link above, where inserting an env var "out of order" resulted in a bizarre diff?

General kubernetes question, kubeconfig should be treated as a secret right? e.g. stored encrypted, restrict access etc..

Anyone else have deprecation warnings disappear after an update of the kubernetes plugin? It's as though

suppressDeprecationWarnings

is on by default now.

I try to modify the default namespace in my kubernetes cluster using

ConfigFile

and an actual manifest. Running the manifest with

kubectl apply -f

works, but it fails through pulumi. The error message is: error: resource default was not successfully created by the Kubernetes API server : namespaces "default" already exists So I guess pulumi does something else than

kubectl apply

. Is there a way to run a manifest that updates a namespace? The part in the code I have now looks like this (F#)

ConfigFile("proxy_inject",
        ConfigFileArgs(
            File = input "manifests/proxy_inject.yaml"
        )) |> ignore

More questions coming 🙂. Is there a way to a service name that doesn't add the extra unique characters to the string? Making a POC in .NET and here I would like to set the Uri for the service in the

appsettings.json

file in the frontend app, but that doesn't work since pulumi adds some characters. I guess one way could be to get the name from the environment somehow instead.

When deploying to kubernetes I've had a couple of deploys sort of being stuck with "[1/2] Waiting for app ReplicaSet be marked available" and then failing. Is this a known issue?

Just FYI, I created a twitter thread that’s all about our k8s provider. Should be a good primer if you’re new to Pulumi: https://twitter.com/levi_blackstone/status/1265798769242550272?s=20

Is the recommended way of managing migrations with Kubernetes still

initContainers

? Would something like https://github.com/pulumi/pulumi-kubernetes/pull/633 help with waiting for a migration

Job

to finish before deploying other resources?

Does anyone know if I need to install the AWS EFS CSI driver to be able to create an EFS storage class? EBS seems to work without installing the EBS CSI driver.

This appears to indicate that you to: https://www.pulumi.com/blog/persisting-kubernetes-workloads-with-amazon-efscsi-volumes-using-pulumi-sdks/.

Also curious why that article doesn't use the CSI driver helm better for long term maintainance. Does that Pulumi code do something different than the efs csi driver helm chart?

it's almost a year old, so practices have probably changed since then

That is frustrating... 1 year is not very long. Is there up to date documentation?

unfortunately 1 year seems to be a long time in the k8s community. Regarding your original question, EBS works without a driver because it's (currently) in tree: https://github.com/kubernetes/kubernetes/tree/323f34858de18b862d43c40b2cced65ad8e24052/pkg/volume/awsebs although there are tentative plans to remove it at some point in the future. So yes, you will have to install the EFS CSI driver

Hi, I followed the basic tutorial to start a k8s cluster using Pulumi (creating the cluster and then adding a NGINX deployment on it). This works fine (except when using a non-default Amazon credentials, although I'm not sure why). The problem I have is that after all the steps complete, I couldn't access the public IP of the nginx deployment from my browser. I tried updating the code using that of another tutorial (which sets up an nginx to redirect the pulumi.github.io page), and all the steps worked correctly but I still couldn't see the page from outside. Could someone indicate what permissions I should change?

Similarly, I'm having bugs (on osX, latest pulumi) with a simple example like this https://github.com/pulumi/examples/tree/master/kubernetes-ts-configmap-rollout First, there are a few small bugs (missing a

selector

in the code, and the README shows

svc/frontend

for the port forwarding which does not exist there), but even deploying locally on minikube I can't access the nginx forward there

Anyone run into issues with

pulumi preview

giving different behavior to

pulumi up

when using GKE? I'm finding that

preview

fails to properly use

gcloud

to authenticate with the cluster, causing the preview to not see any existing kubernetes resources.

up

on the other hand works as expected

Is there a way to add helm repos from pulumi? Feels a bit awkward that pulumi handles everything else, but I still have to go to the terminal to do

helm repo add ...

Is it possible to override files inside helm charts? 🤔 I'm trying to use chart

bitnami/phpmyadmin

but the maximum upload size is too low for me and they don't offer a way to set it. So I would like to override the config file with a different one

not sure if this is the correct channel vs gcp but with the Istio addon on GKE, how can I get the external IP of the ingress gateway? It’s automatically created by GCP so I don’t have the resource in Pulumi to be able to output the load balancer IP. Any ideas?

been a while since using pulumi on this repo, and now it doesn't work. i keep getting

configured Kubernetes cluster is unreachable: unable to load schema information from the API server: the server has asked for the client to provide credentials

. any idea why this might be the case? i'm explicitly getting the kubeconfig and providing it as the

provider

, so it's not like some sort of ambient credential problem.

I'm deploying a nginx-ingress helm chart on AKS to provision a private loadbalancer.

new k8s.helm.v3.Chart(name, {
    chart: "nginx-ingress",
    version: "1.36.3",
    fetchOpts: {
        repo: "<https://kubernetes-charts.storage.googleapis.com>",
    },
    values: {
        controller: {
            service: {
                loadBalancerIP: props.backendIpAddress,
                annotations: {
                    "<http://service.beta.kubernetes.io/azure-load-balancer-internal|service.beta.kubernetes.io/azure-load-balancer-internal>": "true"
                }
            },
        }
    },
})

In my case the cluster gets recreated and the Kubernetes provider tries to deploy the helm chart on the new cluster with the same IP address. Because the IP address is still allocated by the old loadbalancer, it will fail. Do you got any ideas on how to delete the old service/loadbalancer before the new one gets created?

helm chart transformations can come in pretty handy, using it to get around apiVersion deprecation warnings when using cert-manager on k8s 1.18

following this https://www.pulumi.com/docs/guides/crosswalk/kubernetes/apps/#create-a-deployment-with-a-secret. I have noticed if I add

metadeta.name

to my secret. Then my deployment diff shows

[secret]

and does a delete-replace instead of simply updated the deployment spec. If I do not include

metadata.name

in my secret and update the

stringData

then the deployent does an update instead of delete replace. It also, does not show the

[secret]

diff. Is this expect?