I have a situation I’m not quite sure what’s causing it. I have the following update diff if I do preview/up:

+-kubernetes:core/v1:Secret: (replace)
        [id=itay9/odas-secrets-o46s05yb]
        [urn=urn:pulumi:tenant-itay9::okera-trial-tenants::kubernetes:core/v1:Secret::odas-secrets]
        [provider=urn:pulumi:tenant-itay9::okera-trial-tenants::pulumi:providers:kubernetes::k8s-ssa-provider::b438617b-1a9e-4bd4-94f4-9ab2a01529a2]
      ~ stringData: {
          ~ SYSTEM_TOKEN: "[secret]" => "[secret]"
        }

In the code, the secret is created like this:

...

		StringData: pulumi.StringMap{
            ...,
			"SYSTEM_TOKEN":                 systemToken,

and

systemToken

is defined like this:

systemToken := pulumi.All(jwtKey.PrivateKeyPem).ApplyT(
		func(args []interface{}) (string, error) {
           ...
		},
	).(pulumi.StringOutput)

This would all make sense if there was an update in

systemToken

, but there isn’t - that’s not one of the resources that Pulumi is saying requires an update. Any idea why this might be the case? How do I debug something like this?

I’m wondering on using Security Groups for Pods on EKS — I’ve got most of it automated with a Managed Node Group + using the EKS Addons. — general instructions are here: https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html One spot I’ve ran into an issue is with patching the

ENABLE_POD_ENI

environment variable? Any ideas on how to best do this with Pulumi? The AWS documentation says to run ”

kubectl set env daemonset aws-node -n kube-system ENABLE_POD_ENI=true

How do people manage the sometimes complex web of dependencies so that resource deletion works without timing out? ("timed out waiting to be Ready"); e.g. • install aws load balancer controller (albc) and kube prometheus stack (kps) via helm charts • configure kps values for ingress with alb annotations, so ingress creates an aws alb (and adds a finalizer) • kps effectively depends on albc and this must be explicit via pulumi otherwise you get the timeouts above Do people just carefully wire these dependencies explicitly, in which case how do you handle the inevitable circular dependencies?

I've reordered resources and taken dependencies on helm charts rather than just crds (in separate config groups) and destroy got much further but was hanging for ages until finally failing with:

Diagnostics:
  pulumi:pulumi:Stack (k8s-dev):
    could not get token: RequestError: send request failed
    caused by: Post <https://sts.amazonaws.com/>: dial tcp: lookup <http://sts.amazonaws.com|sts.amazonaws.com> on 192.168.65.5:53: no such host
    could not get token: RequestError: send request failed
    caused by: Post <https://sts.amazonaws.com/>: dial tcp: lookup <http://sts.amazonaws.com|sts.amazonaws.com> on 192.168.65.5:53: no such host

    error: update failed

  kubernetes:<http://apiextensions.k8s.io/v1:CustomResourceDefinition|apiextensions.k8s.io/v1:CustomResourceDefinition> (appmesh-system/appmesh-controller-selfsigned-issuer):
    error: Delete "https://[REDACTED].<http://gr7.us-west-2.eks.amazonaws.com/apis/cert-manager.io/v1alpha2/namespaces/appmesh-system/issuers/appmesh-controller-selfsigned-issuer|gr7.us-west-2.eks.amazonaws.com/apis/cert-manager.io/v1alpha2/namespaces/appmesh-system/issuers/appmesh-controller-selfsigned-issuer>": getting credentials: exec: executable
aws-iam-authenticator failed with exit code 1
    error: post-step event returned an error: failed to save snapshot: performing HTTP request: Patch "<https://api.pulumi.com/api/stacks/pharos/k8s/dev/update/27f96f11-ba00-4a38-b812-1a87d668ec8b/checkpoint>": dial tcp: lookup
<http://api.pulumi.com|api.pulumi.com> on 192.168.65.5:53: no such host

Resources:
    - 199 deleted

Duration: 2h44m7s

During the hang I checked resources and the aws appmesh controller and cert manager controllers had already been deleted. I didn't think it would matter since I didn't see any finalizers or owner refs associated with the

<http://cert-manager.io/v1/Issuer|cert-manager.io/v1/Issuer>

resource being deleted, but maybe I'm missing something? Looks like this is due to https://github.com/pulumi/pulumi-kubernetes/issues/861

So yeah.. back in single-cluster-duplicate-name-limbo.. A while ago I had to endure this: https://github.com/pulumi/pulumi/discussions/7382 Now I'm stuck with https://artifacthub.io/packages/helm/kong/kong where it complains that:

error: Duplicate resource URN 'urn:pulumi:prod.k8s.devsecops::baseline-k8s::app:k8s:foundation$kubernetes:<http://helm.sh/v3:Chart$kubernetes:apiextensions.k8s.io/v1beta1:CustomResourceDefinition::{uniquevar}-kongconsumers.configuration.konghq.com';|helm.sh/v3:Chart$kubernetes:apiextensions.k8s.io/v1beta1:CustomResourceDefinition::{uniquevar}-kongconsumers.configuration.konghq.com';> try giving it a unique name

where {uniquevar} is an alphanumeric string

anyone successfully use kong through helm with pulumi? This is a single cluster deployment so it should be unique regardless what I do..

So, I just ran

pulumi refresh

as I hadn’t used this repo/stack in a while and it seems to have untracked all of my k8s resources

and when I try to

pulumi preview

/

pulumi up

, it’s trying to create them from fresh

is there, uh, any way to undo this?

Hi - I have a strange issue where by when a deployment completes, the ingress temporarily returns 503 for services which are running - does anyone else see this? It clears after a few refreshes but that's obviously not ideal

Anyone noticed wrong ordering on delete? I make a PVC, then use the PVC in a deployment, creation works perfectly. But when deleting, it tires to delete the PVC first, which “stalls” because it is still mounted in the Deployment. Both the PVC and the deployment are in separate custom resources., the resource containing the PVC is passed intot he constructor of the custom resource that has the deployment.

I have argo events and argo workflows installed in EKS, but pulumi doesn't provide any interface for creating argo customer resources, such as workflows, event sources, sensors and triggers etc. I know pulumi provide a tool called crd2pulumi to generates APIs based on any customer defined resources YAML files. Just wondering if anybody has tried that with argo events/argo workflows CRDs. Any knowledge sharing will be appreciated!

Is there a way to get an object not created by Pulumi in code?, I'm trying to get the token for an SA created via Pulumi (the secret is automatically created by kubernetes), but the method (Go code) requires a Pulumi ID, I can provde some code samples if required

Anybody encountered this issue with "pulumi destroy"? Diagnostics: pulumi😛ulumi:Stack (infra-devops): error: update failed aws:eks:Cluster (devops-eks-eksCluster): error: post-step event returned an error: failed to save snapshot: [403] The provided update token has expired.

How can I recover from this?

It says: Once you have confirmed the status of the interrupted operations, you can repair your stack using 'pulumi stack export' to export your stack to a file. For each operation that succeeded, remove that operation from the "pending_operations" section of the file. Once this is complete, use 'pulumi stack import' to import the repaired stack.

How do I know which operation succeed? Do I just check each resources in the AWS console? Cannot pulumi do that automatically?

Okay, answer my own question. I checked the aws console and the pending resources are gone. So, just removed from the pending_operations section and import. It recovered. Still it might make senses for pulumi to automatically detect that and still not sure how got the update token has expired in the first place.

message has been deleted

Anybody used argo workflows and argo events here?

I’m trying to follow the EKS tutorial https://www.pulumi.com/docs/guides/crosswalk/aws/eks/ and running into this error right out of they gate when I try to provision the cluster. Isn’t it supposed to install into the default VPC? What am I missing?

(venv) a  devops git:(master) ✗ pulumi up
Previewing update (dev):
     Type                                   Name                                Plan       Info
     pulumi:pulumi:Stack                    devops-dev                        4 errors
 +   └─ eks:index:Cluster                   test-cluster                        create
 +      ├─ eks:index:ServiceRole            test-cluster-eksRole                create
 +      │  ├─ aws:iam:Role                  test-cluster-eksRole-role           create
 +      │  ├─ aws:iam:RolePolicyAttachment  test-cluster-eksRole-90eb1c99       create
 +      │  └─ aws:iam:RolePolicyAttachment  test-cluster-eksRole-4b490823       create
 +      ├─ eks:index:ServiceRole            test-cluster-instanceRole           create
 +      │  ├─ aws:iam:Role                  test-cluster-instanceRole-role      create
 +      │  ├─ aws:iam:RolePolicyAttachment  test-cluster-instanceRole-03516f97  create
 +      │  ├─ aws:iam:RolePolicyAttachment  test-cluster-instanceRole-e1b295bd  create
 +      │  └─ aws:iam:RolePolicyAttachment  test-cluster-instanceRole-3eb088f2  create
 +      ├─ eks:index:RandomSuffix           test-cluster-cfnStackName           create
 +      └─ aws:iam:InstanceProfile          test-cluster-instanceProfile        create

Diagnostics:
  pulumi:pulumi:Stack (devops-dev):
    error: Error: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
        * no matching VPC found
          
# ...... Long stacktace continues

I used Pulumi to create an EKS cluster in another AWS account using an assumed role. But

pulumi up

failed when trying to create kubernetes objects afterward, as part of the same stack. How can I access the EKS cluster I just created?

Hi folks, i’m seeking to understand a pulumi behaviour. I deploy both the “aws efs csi driver” and “aws load balancer controller” via helm (pulumi wrapped helm) and every time i “pulumi up” these resources get replaced/updated regardless of the fact the code is unchanged…. all thoughts and learning welcome, code snippets in thread

is there a pulumi equivalent means of getting the output of

helm template --debug

to generate the template final state yaml before deploying ?

So I'm using a utility to convert a

tls.PrivateKey

pem

into

openssh

format and throwing it into a k8s secret using

stringData

The issue is that pulumi thinks the secret changes every single update (despite it remaining stable), and so it recreates the secret every time. Anyone run into this?

Okay, first pulumi experience, I have EKS building with my index.ts, I wanted to also deploy a helm chart as part of my base platform IaC. Can

k8s.helm.v3.Chart

inherit the credentials for the cluster it built? How do I go from building the cluster to also deploying stuff to it? if I manually pull the kubeconfig (future will be oidc) and re-run

pulumi up

everything deploys fine. index.ts LINK

I recently started getting this error, haven't updated anything; not the cluster nor any dependecies:

getting credentials: exec plugin is configured to use API version <http://client.authentication.k8s.io/v1alpha1|client.authentication.k8s.io/v1alpha1>, plugin returned version <http://client.authentication.k8s.io/v1beta1|client.authentication.k8s.io/v1beta1>

. Can it be pulumi related?

Using digital ocean I keep getting the following error when attempting

pulumi up

: 404 could not find version. Any thoughts on what might be causing this? Would it be pulumi or digital ocean?

Hoping someone has some ideas, trying to deploy EKS using python based off some of the examples provided in the pulumi docs. Running into an issue when manually creating a nodegroup.

Diagnostics:
  pulumi:pulumi:Stack (eks-test-dev):
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 519, in do_rpc_call
        return monitor.RegisterResource(req)
      File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
        return _end_unary_response_blocking(state, call, False, None)
      File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
        raise _InactiveRpcError(state)
    grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    	status = StatusCode.UNKNOWN
    	details = "'dependsOn' was passed a value that was not a Resource."
    	debug_error_string = "{"created":"@1628011197.637119000","description":"Error received from peer ipv4:127.0.0.1:62720","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"'dependsOn' was passed a value that was not a Resource.","grpc_status":2}"