todays puzzle…. working with kustomize directories like :-

# setup EFS CSI driver
    k8s_k_efscsi = k8s.kustomize.Directory(
        "itplat-kust-efs-driver",
        directory="<https://github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/ecr/?ref=release-1.1>",
        resource_prefix="itplat-kust-efs-driver",
        opts=pulumi.ResourceOptions(provider=k8s_use1_provider),
    )

works fine the first time but every subsequent

pulumi up

i get

Diagnostics:
  pulumi:pulumi:Stack (aws_eks-itplat-aws-eks):
    error: update failed

  <kubernetes:storage.k8s.io/v1beta1:CSIDriver> (<http://itplat-kust-efs-driver-efs.csi.aws.com|itplat-kust-efs-driver-efs.csi.aws.com>):
    warning: This resource contains Helm hooks that are not currently supported by Pulumi. The resource will be created, but any hooks will not be executed. Hooks support is tracked at <https://github.com/pulumi/pulumi-kubernetes/issues/555>
    warning: <http://storage.k8s.io/v1beta1/CSIDriver|storage.k8s.io/v1beta1/CSIDriver> is deprecated by <http://storage.k8s.io/v1/CSIDriver|storage.k8s.io/v1/CSIDriver>.
    error: resource <http://efs.csi.aws.com|efs.csi.aws.com> was not successfully created by the Kubernetes API server : <http://csidrivers.storage.k8s.io|csidrivers.storage.k8s.io> "<http://efs.csi.aws.com|efs.csi.aws.com>" already exists

Hi gang! I’m running into some trouble with a Pulumi project which manages multiple kubernetes clusters (via AWS EKS, in case it’s relevant). In short, I am trying to deploy a helm chart (metrics-server) into each cluster and am getting the error:

Duplicate resource URN 'urn:pulumi:prod::submission_strategy::<kubernetes:helm.sh/v3:Chart$kubernetes:apiregistration.k8s.io/v1:APIService::v1beta1.metrics.k8s.io>'; try giving it a unique name

Note that I am already using the

resource_prefix

argument in my “ChartOpts” object to separate the resources URI’s for each cluster, and except for the one “APIService” resource this appears to work as expected. Looking into the docs, I don’t see a clear way to specify the “APIService” resource URN 😕. Anyone have any suggestions ?

What’s the go-to approach for looking up a Kubernetes resource? I tried

k8s.core.v1.Secrets.get

as an example but get my hand slapped by Pulumi when it runs in an update.

/* Returns auth from a `ServiceAccount`'s associated secret */
export const getServiceAccountAuth = (serviceAccount: ServiceAccount): pulumi.Output<{ token: string, caCert: string}> =>
    pulumi.output(serviceAccount.secrets[0].name).apply(secretName => {
        const secret = k8s.core.v1.Secret.get(secretName, secretName)
        const { token, ['ca.crt']: caCert } = secret.data.get()
        return { token, caCert }
    })

I guess I could just use the k8s client, but didn’t know if there’s a Pulumi-native way of doing it

what is the right way to write an inline k8s config file yaml ? in

<https://www.pulumi.com/docs/reference/pkg/kubernetes/yaml/configfile/>

it describes the option of

Using a literal string containing YAML, or a list of such strings:

and i was expecting a k8s.ConfigFile(“myconfig”, config=“”"<inline yaml>“”") but cant see anything like that, only

file=

Is there a way to figure out why pulumi is deleting and replacing some resources instead of updating them? I'm changing the

data

of a configMap and it wants to delete-replace the configMap, and the deployment that refers to it, causing downtime.

--kubernetes:core/v1:ConfigMap: (delete-replaced)
    [id=default/project-internal-tools-service-common]
    [urn=urn:pulumi:internal-tools.dev::project-internal-tools::kubernetes:core/v1:ConfigMap::common]
    [provider=urn:pulumi:internal-tools.dev::project-internal-tools::pulumi:providers:kubernetes::eks::...]
+-kubernetes:core/v1:ConfigMap: (replace)
    [id=default/project-internal-tools-service-common]
    [urn=urn:pulumi:internal-tools.dev::project-internal-tools::kubernetes:core/v1:ConfigMap::common]
    [provider=urn:pulumi:internal-tools.dev::project-internal-tools::pulumi:providers:kubernetes::eks::...]
    ~ data: {
        + FOO: "bar"
    }
++kubernetes:core/v1:ConfigMap: (create-replacement)
    [id=default/project-internal-tools-service-common]
    [urn=urn:pulumi:internal-tools.dev::project-internal-tools::kubernetes:core/v1:ConfigMap::common]
    [provider=urn:pulumi:internal-tools.dev::project-internal-tools::pulumi:providers:kubernetes::eks::...]
    ~ data: {
        + FOO: "bar"
    }

(this configMap was not initially created by Pulumi, it was imported)

@many-psychiatrist-74327 did you try using the resource_prefix option? https://www.pulumi.com/docs/reference/pkg/kubernetes/yaml/configfile/#resource_prefix_pythob

hey guys, does pulumi work with vmware tanzu?

How's everyone deploying ingress like

ingress-nginx

? I'd prefer not to rely on helm, as it adds another layer of complexity that I'm trying to avoid, but using

kube2pulumi

on ingress backend yml feels fairly clunky.

General question, why does it take so long for Pulumi to apply some k8s resources? For example, when installing cert-manager the

cert-manager/cert-manager-cainjector

resource has been

creating...

for several minutes already. Why isn't it essentially instant?

if i created a namespace without a

provider

set and then later want to set the provider, is there a way to do this without having to recreate the namespace? I've tried updating the statefile directly with the provider I want from the

pulumi preview

diff, but Pulumi does like my edits

For

k8s.rbac.v1.ClusterRole

, in

k8s.rbac.v1.PolicyRuleArgs

, if I omit setting

api_groups

, is it equivalent to:

rules:
- apiGroups: [""] # "" indicates the core API group

Hello peeps.. I just posted this to twitter and wanted to also ask here...

Twitter peeps, I need help.

I have an http://ASP.NET Core Host serving gRPC endpoints up. This host is hosted in Kubernetes behind a nginx-ingress-controller. Ingress Controller terminates HTTPS. For the life of me, I cannot get to the gRPC service endpoints.

I'm hoping to find someone whom I can talk to, or point me at examples. My google-foo for this is failing terribly.

Thanks in advance.

Using

k8s.rbac.v1.ClusterRole

with Azure AKS, I get this even while recreating a stack from scratch (destroy, then up):

Diagnostics:
  <kubernetes:rbac.authorization.k8s.io/v1:ClusterRole> (clusterRole-storage):
    error: resource system:azure-cloud-provider was not successfully created by the Kubernetes API server : <http://clusterroles.rbac.authorization.k8s.io|clusterroles.rbac.authorization.k8s.io> "system:azure-cloud-provider" already exists

Is this expected? What should I be doing ideally?

Is there a way to suppress a k8s Provider from putting the kubeconfig in the logs in plaintext?

Anyone else experience extreme slowness when deploying larger Helm charts? For example, I’m deploying gloo edge overnight and it took 2h47m. I looked into adding pulumi.com/skipAwait annotations but I’m not sure it’s speeding it up (still running). Deploying the same helm chart with terraform only takes 37 seconds.

This message was deleted.

Hi guys 👋 I’m investigating moving away from an in-house deployment tool that uses google-go-gcp SDK and k8s SDK to using Pulumi as one-library-to-rule-them-all solution.

I have a question which is the reason why we are making this move and why we are considering Pulumi. In our current solution, we need to use different k8s APIs / go packages to deal with the different k8s API changes/deprecations (e.g. Ingress in networking/v1beta vs Ingress in networking/v1) and this is causing us to have different code branches based on k8s server versions.

Is this something that Pulumi can internalise? Is Pulumi clever enough to determine which k8s APIs to use based on k8s server version?

Can I clone a running stack from one cluster to another? What would be the Pulumi API to attempt this? Could I clone a deployed running stack in the Pulumi service, and rewire everything? Is there a negative(?) to

pulumi refresh

which would presumably refresh the cluster state based off the Pulumi stack state? I want to copy over a bunch of running pods from my toy cluster to a live cluster, and skip the deployment-from-scratch workflow altogether; what's interesting to me is the actual state of the app, I want to keep what I was tinkering with in the toy cluster (memory state, PVCs). Kind of like freezing a VM and moving it to another hypervisor, but for an already deployed k8s app... A non-me use-case could be to clone the state of an k8s app and send it as-is to a friend to run on their cluster (many assumptions withstanding) Maybe sething at the container/pod/resource layer can be used? I suppose I could keep searching for DRP, whole cluster import/export/backup patterns, not sure if this is a valid use-case in other contexts... Maybe I need to think differently about all this; feedback appreciated :-)

I’ve kinda been avoiding the point and just making sure I pass all my K8s resources a

provider

, but is there an easier way?

I’m thinking a transformation?

Just a 10PM thought without checking the code; I assumed this would be possible as I already have apiVersion transforms

Does the Pulumi Operator work without the Pulumi hosted service?

Hi everybody. Does anybody have a success story of using Pulumi as an ArgoCD plugin for pure Yaml (k8s objects) generation? This comment on Github got some attention, but I haven’t been able to find any working example 😪

👋 I’ve been seeing pulumi dependency graph problems when using

k8s.yaml.ConfigFile

. In simplified terms, I have two yaml files:

foo.yaml

and

bar.yaml

, each of which defines multiple resources. The resources in

bar.yml

depend on those in

foo.yaml

. Thus, my pulumi (typescript) code looks something like:

const foo = new k8s.yaml.ConfigFile("foo", { file: "foo.yaml" });
const bar = new k8s.yaml.ConfigFile("bar", { file: "bar.yaml" }, { dependsOn: foo });

however, when pulumi runs the update, it starts creating the resources under

bar

first.. and of course they fail. it actually starts retrying them 5 times, and sometimes they’ll eventually succeed because the resources in

foo

got created in the meantime, but the behavior is non-deterministic and fails very often. Do you know why pulumi isn’t waiting on

foo

resources to be created before creating the resources in

bar

?

is it correct that if i establish a k8s provider using an eks cluster output lifted as input to the provider, this should create implicit dependency chain ? and then everything i use that provider as an option for also inherits that implicit dependency ??? ie -

# create cluster resource
    eks_cluster = aws.eks.Cluster("itplat-eks-cluster", opts=provider_opts, **eks_cluster_config)
 
    k8s_use1_provider = k8s.Provider(
        k8s_use1_provider_name,
        cluster=eks_cluster.arn,
        context=eks_cluster.arn,
        enable_dry_run=None,
        namespace=None,
        render_yaml_to_directory=None,
        suppress_deprecation_warnings=None,
    )

    # lets have a go at creating a "crossplane-system" namespace
    crossplane_namespace = k8s.core.v1.Namespace(
        "crossplane-system", opts=pulumi.ResourceOptions(provider=k8s_use1_provider), metadata=k8s.meta.v1.ObjectMetaArgs(name="crossplane-system")
    )

this makes namespace dependent on provider which is dependent on eks_cluster ??

Not technically a Pulumi question, but maybe someone here has an idea; I have 

nginx-ingress

 deployed with Pulumi (helm chart) in an EKS cluster sitting in front of Node.js pods, and every so often I get a 

upstream prematurely closed connection while reading response header from upstream

 and I have no clue why. Does that ring a bell to anyone?