https://pulumi.com logo
Docs
Join the conversationJoin Slack
Channels
announcements
automation-api
aws
azure
blog-posts
built-with-pulumi
cloudengineering
cloudengineering-support
content-share
contribex
contribute
docs
dotnet
finops
general
getting-started
gitlab
golang
google-cloud
hackathon-03-19-2020
hacktoberfest
install
java
jobs
kubernetes
learn-pulumi-events
linen
localstack
multi-language-hackathon
office-hours
oracle-cloud-infrastructure
plugin-framework
pulumi-cdk
pulumi-crosscode
pulumi-deployments
pulumi-kubernetes-operator
pulumi-service
pulumiverse
python
registry
status
testingtesting123
testingtesting321
typescript
welcome
workshops
yaml
Powered by Linen
kubernetes
  • b

    bored-table-20691

    07/09/2021, 8:47 PM
    I have a situation I’m not quite sure what’s causing it. I have the following update diff if I do preview/up:
    +-kubernetes:core/v1:Secret: (replace)
            [id=itay9/odas-secrets-o46s05yb]
            [urn=urn:pulumi:tenant-itay9::okera-trial-tenants::kubernetes:core/v1:Secret::odas-secrets]
            [provider=urn:pulumi:tenant-itay9::okera-trial-tenants::pulumi:providers:kubernetes::k8s-ssa-provider::b438617b-1a9e-4bd4-94f4-9ab2a01529a2]
          ~ stringData: {
              ~ SYSTEM_TOKEN: "[secret]" => "[secret]"
            }
    In the code, the secret is created like this:
    ...
    
    		StringData: pulumi.StringMap{
                ...,
    			"SYSTEM_TOKEN":                 systemToken,
    and
    systemToken
    is defined like this:
    systemToken := pulumi.All(jwtKey.PrivateKeyPem).ApplyT(
    		func(args []interface{}) (string, error) {
               ...
    		},
    	).(pulumi.StringOutput)
    This would all make sense if there was an update in
    systemToken
    , but there isn’t - that’s not one of the resources that Pulumi is saying requires an update. Any idea why this might be the case? How do I debug something like this?
  • i

    important-sandwich-62391

    07/12/2021, 6:15 PM
    I’m wondering on using Security Groups for Pods on EKS — I’ve got most of it automated with a Managed Node Group + using the EKS Addons. — general instructions are here: https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html One spot I’ve ran into an issue is with patching the
    ENABLE_POD_ENI
    environment variable? Any ideas on how to best do this with Pulumi? The AWS documentation says to run ”
    kubectl set env daemonset aws-node -n kube-system ENABLE_POD_ENI=true
    s
    • 2
    • 3
  • w

    worried-city-86458

    07/13/2021, 3:51 AM
    How do people manage the sometimes complex web of dependencies so that resource deletion works without timing out? ("timed out waiting to be Ready"); e.g. • install aws load balancer controller (albc) and kube prometheus stack (kps) via helm charts • configure kps values for ingress with alb annotations, so ingress creates an aws alb (and adds a finalizer) • kps effectively depends on albc and this must be explicit via pulumi otherwise you get the timeouts above Do people just carefully wire these dependencies explicitly, in which case how do you handle the inevitable circular dependencies?
  • w

    worried-city-86458

    07/13/2021, 5:11 PM
    I've reordered resources and taken dependencies on helm charts rather than just crds (in separate config groups) and destroy got much further but was hanging for ages until finally failing with:
    Diagnostics:
      pulumi:pulumi:Stack (k8s-dev):
        could not get token: RequestError: send request failed
        caused by: Post <https://sts.amazonaws.com/>: dial tcp: lookup <http://sts.amazonaws.com|sts.amazonaws.com> on 192.168.65.5:53: no such host
        could not get token: RequestError: send request failed
        caused by: Post <https://sts.amazonaws.com/>: dial tcp: lookup <http://sts.amazonaws.com|sts.amazonaws.com> on 192.168.65.5:53: no such host
    
        error: update failed
    
      kubernetes:<http://apiextensions.k8s.io/v1:CustomResourceDefinition|apiextensions.k8s.io/v1:CustomResourceDefinition> (appmesh-system/appmesh-controller-selfsigned-issuer):
        error: Delete "https://[REDACTED].<http://gr7.us-west-2.eks.amazonaws.com/apis/cert-manager.io/v1alpha2/namespaces/appmesh-system/issuers/appmesh-controller-selfsigned-issuer|gr7.us-west-2.eks.amazonaws.com/apis/cert-manager.io/v1alpha2/namespaces/appmesh-system/issuers/appmesh-controller-selfsigned-issuer>": getting credentials: exec: executable
    aws-iam-authenticator failed with exit code 1
        error: post-step event returned an error: failed to save snapshot: performing HTTP request: Patch "<https://api.pulumi.com/api/stacks/pharos/k8s/dev/update/27f96f11-ba00-4a38-b812-1a87d668ec8b/checkpoint>": dial tcp: lookup
    <http://api.pulumi.com|api.pulumi.com> on 192.168.65.5:53: no such host
    
    Resources:
        - 199 deleted
    
    Duration: 2h44m7s
    During the hang I checked resources and the aws appmesh controller and cert manager controllers had already been deleted. I didn't think it would matter since I didn't see any finalizers or owner refs associated with the
    <http://cert-manager.io/v1/Issuer|cert-manager.io/v1/Issuer>
    resource being deleted, but maybe I'm missing something? Looks like this is due to https://github.com/pulumi/pulumi-kubernetes/issues/861
    • 1
    • 3
  • b

    better-shampoo-48884

    07/14/2021, 10:48 AM
    So yeah.. back in single-cluster-duplicate-name-limbo.. A while ago I had to endure this: https://github.com/pulumi/pulumi/discussions/7382 Now I'm stuck with https://artifacthub.io/packages/helm/kong/kong where it complains that:
    error: Duplicate resource URN 'urn:pulumi:prod.k8s.devsecops::baseline-k8s::app:k8s:foundation$kubernetes:<http://helm.sh/v3:Chart$kubernetes:apiextensions.k8s.io/v1beta1:CustomResourceDefinition::{uniquevar}-kongconsumers.configuration.konghq.com';|helm.sh/v3:Chart$kubernetes:apiextensions.k8s.io/v1beta1:CustomResourceDefinition::{uniquevar}-kongconsumers.configuration.konghq.com';> try giving it a unique name
    where {uniquevar} is an alphanumeric string
    b
    • 2
    • 2
  • b

    better-shampoo-48884

    07/14/2021, 10:49 AM
    anyone successfully use kong through helm with pulumi? This is a single cluster deployment so it should be unique regardless what I do..
  • l

    lemon-monkey-228

    07/14/2021, 2:29 PM
    So, I just ran
    pulumi refresh
    as I hadn’t used this repo/stack in a while and it seems to have untracked all of my k8s resources
  • l

    lemon-monkey-228

    07/14/2021, 2:29 PM
    and when I try to
    pulumi preview
    /
    pulumi up
    , it’s trying to create them from fresh
  • l

    lemon-monkey-228

    07/14/2021, 2:29 PM
    is there, uh, any way to undo this?
    b
    s
    • 3
    • 15
  • b

    bland-cat-29878

    07/16/2021, 1:07 PM
    Hi - I have a strange issue where by when a deployment completes, the ingress temporarily returns 503 for services which are running - does anyone else see this? It clears after a few refreshes but that's obviously not ideal
    b
    s
    • 3
    • 8
  • p

    proud-pizza-80589

    07/18/2021, 1:55 PM
    Anyone noticed wrong ordering on delete? I make a PVC, then use the PVC in a deployment, creation works perfectly. But when deleting, it tires to delete the PVC first, which “stalls” because it is still mounted in the Deployment. Both the PVC and the deployment are in separate custom resources., the resource containing the PVC is passed intot he constructor of the custom resource that has the deployment.
    p
    • 2
    • 4
  • r

    ripe-shampoo-80285

    07/19/2021, 3:05 PM
    I have argo events and argo workflows installed in EKS, but pulumi doesn't provide any interface for creating argo customer resources, such as workflows, event sources, sensors and triggers etc. I know pulumi provide a tool called crd2pulumi to generates APIs based on any customer defined resources YAML files. Just wondering if anybody has tried that with argo events/argo workflows CRDs. Any knowledge sharing will be appreciated!
  • n

    numerous-thailand-80976

    07/20/2021, 8:45 PM
    Is there a way to get an object not created by Pulumi in code?, I'm trying to get the token for an SA created via Pulumi (the secret is automatically created by kubernetes), but the method (Go code) requires a Pulumi ID, I can provde some code samples if required
    g
    • 2
    • 3
  • r

    ripe-shampoo-80285

    07/21/2021, 2:50 PM
    Anybody encountered this issue with "pulumi destroy"? Diagnostics: pulumi😛ulumi:Stack (infra-devops): error: update failed aws:eks:Cluster (devops-eks-eksCluster): error: post-step event returned an error: failed to save snapshot: [403] The provided update token has expired.
    c
    • 2
    • 2
  • r

    ripe-shampoo-80285

    07/21/2021, 2:51 PM
    How can I recover from this?
  • r

    ripe-shampoo-80285

    07/21/2021, 2:51 PM
    It says: Once you have confirmed the status of the interrupted operations, you can repair your stack using 'pulumi stack export' to export your stack to a file. For each operation that succeeded, remove that operation from the "pending_operations" section of the file. Once this is complete, use 'pulumi stack import' to import the repaired stack.
  • r

    ripe-shampoo-80285

    07/21/2021, 2:53 PM
    How do I know which operation succeed? Do I just check each resources in the AWS console? Cannot pulumi do that automatically?
  • r

    ripe-shampoo-80285

    07/21/2021, 3:04 PM
    Okay, answer my own question. I checked the aws console and the pending resources are gone. So, just removed from the pending_operations section and import. It recovered. Still it might make senses for pulumi to automatically detect that and still not sure how got the update token has expired in the first place.
    👍 1
  • f

    freezing-quill-32178

    07/22/2021, 10:53 AM
    message has been deleted
  • r

    ripe-shampoo-80285

    07/22/2021, 2:38 PM
    Anybody used argo workflows and argo events here?
    • 1
    • 1
  • a

    ambitious-afternoon-55254

    07/22/2021, 5:19 PM
    I’m trying to follow the EKS tutorial https://www.pulumi.com/docs/guides/crosswalk/aws/eks/ and running into this error right out of they gate when I try to provision the cluster. Isn’t it supposed to install into the default VPC? What am I missing?
    (venv) a  devops git:(master) ✗ pulumi up
    Previewing update (dev):
         Type                                   Name                                Plan       Info
         pulumi:pulumi:Stack                    devops-dev                        4 errors
     +   └─ eks:index:Cluster                   test-cluster                        create
     +      ├─ eks:index:ServiceRole            test-cluster-eksRole                create
     +      │  ├─ aws:iam:Role                  test-cluster-eksRole-role           create
     +      │  ├─ aws:iam:RolePolicyAttachment  test-cluster-eksRole-90eb1c99       create
     +      │  └─ aws:iam:RolePolicyAttachment  test-cluster-eksRole-4b490823       create
     +      ├─ eks:index:ServiceRole            test-cluster-instanceRole           create
     +      │  ├─ aws:iam:Role                  test-cluster-instanceRole-role      create
     +      │  ├─ aws:iam:RolePolicyAttachment  test-cluster-instanceRole-03516f97  create
     +      │  ├─ aws:iam:RolePolicyAttachment  test-cluster-instanceRole-e1b295bd  create
     +      │  └─ aws:iam:RolePolicyAttachment  test-cluster-instanceRole-3eb088f2  create
     +      ├─ eks:index:RandomSuffix           test-cluster-cfnStackName           create
     +      └─ aws:iam:InstanceProfile          test-cluster-instanceProfile        create
    
    Diagnostics:
      pulumi:pulumi:Stack (devops-dev):
        error: Error: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
            * no matching VPC found
              
    # ...... Long stacktace continues
    b
    • 2
    • 4
  • s

    steep-toddler-94095

    07/23/2021, 12:29 AM
    I used Pulumi to create an EKS cluster in another AWS account using an assumed role. But
    pulumi up
    failed when trying to create kubernetes objects afterward, as part of the same stack. How can I access the EKS cluster I just created?
    b
    b
    • 3
    • 41
  • p

    purple-plumber-90981

    07/24/2021, 10:38 PM
    Hi folks, i’m seeking to understand a pulumi behaviour. I deploy both the “aws efs csi driver” and “aws load balancer controller” via helm (pulumi wrapped helm) and every time i “pulumi up” these resources get replaced/updated regardless of the fact the code is unchanged…. all thoughts and learning welcome, code snippets in thread
    b
    w
    +2
    • 5
    • 56
  • p

    purple-plumber-90981

    07/26/2021, 4:18 AM
    is there a pulumi equivalent means of getting the output of
    helm template --debug
    to generate the template final state yaml before deploying ?
    • 1
    • 1
  • f

    faint-dog-16036

    07/26/2021, 6:26 PM
    So I'm using a utility to convert a
    tls.PrivateKey
    pem
    into
    openssh
    format and throwing it into a k8s secret using
    stringData
    The issue is that pulumi thinks the secret changes every single update (despite it remaining stable), and so it recreates the secret every time. Anyone run into this?
    b
    b
    • 3
    • 6
  • b

    busy-journalist-6936

    07/29/2021, 8:22 PM
    Okay, first pulumi experience, I have EKS building with my index.ts, I wanted to also deploy a helm chart as part of my base platform IaC. Can
    k8s.helm.v3.Chart
    inherit the credentials for the cluster it built? How do I go from building the cluster to also deploying stuff to it? if I manually pull the kubeconfig (future will be oidc) and re-run
    pulumi up
    everything deploys fine. index.ts LINK
    b
    • 2
    • 23
  • d

    dazzling-sundown-39670

    08/02/2021, 11:59 AM
    I recently started getting this error, haven't updated anything; not the cluster nor any dependecies:
    getting credentials: exec plugin is configured to use API version <http://client.authentication.k8s.io/v1alpha1|client.authentication.k8s.io/v1alpha1>, plugin returned version <http://client.authentication.k8s.io/v1beta1|client.authentication.k8s.io/v1beta1>
    . Can it be pulumi related?
    c
    p
    b
    • 4
    • 7
  • q

    quiet-state-42882

    08/03/2021, 3:33 AM
    Using digital ocean I keep getting the following error when attempting
    pulumi up
    : 404 could not find version. Any thoughts on what might be causing this? Would it be pulumi or digital ocean?
    s
    • 2
    • 2
  • d

    delightful-gigabyte-39334

    08/03/2021, 5:22 PM
    Hoping someone has some ideas, trying to deploy EKS using python based off some of the examples provided in the pulumi docs. Running into an issue when manually creating a nodegroup.
    Diagnostics:
      pulumi:pulumi:Stack (eks-test-dev):
        error: Program failed with an unhandled exception:
        error: Traceback (most recent call last):
          File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 519, in do_rpc_call
            return monitor.RegisterResource(req)
          File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
            return _end_unary_response_blocking(state, call, False, None)
          File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
            raise _InactiveRpcError(state)
        grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        	status = StatusCode.UNKNOWN
        	details = "'dependsOn' was passed a value that was not a Resource."
        	debug_error_string = "{"created":"@1628011197.637119000","description":"Error received from peer ipv4:127.0.0.1:62720","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"'dependsOn' was passed a value that was not a Resource.","grpc_status":2}"
    b
    • 2
    • 16
  • q

    quiet-architect-91246

    08/03/2021, 7:20 PM
    Hi everyone! When I setup a eks cluster with pulumi in aws I get
    "Container runtime not ready: cni not initialized".
    This can be solved by running
    export kubever=$(kubectl version | base64 | tr -d '\n')
    and
    kubectl apply -f "<https://cloud.weave.works/k8s/net?k8s-version=$kubever>"
    . This obviously requires manual input which doesnt work for a automatic cicd pipeline, so Id love to do it in the declaration of the cluster with pulumi. I could hand in a recreation of the used .yaml file as
    ClusterRole
    and
    ClusterRoleBinding
    , but that would lead to very large, hardly readable codeblock and would require manual changes when the kubernetes version changes. Which leads me to my question: Is there a better way to handle the network initialization?
    b
    • 2
    • 4
Powered by Linen
Title
q

quiet-architect-91246

08/03/2021, 7:20 PM
Hi everyone! When I setup a eks cluster with pulumi in aws I get
"Container runtime not ready: cni not initialized".
This can be solved by running
export kubever=$(kubectl version | base64 | tr -d '\n')
and
kubectl apply -f "<https://cloud.weave.works/k8s/net?k8s-version=$kubever>"
. This obviously requires manual input which doesnt work for a automatic cicd pipeline, so Id love to do it in the declaration of the cluster with pulumi. I could hand in a recreation of the used .yaml file as
ClusterRole
and
ClusterRoleBinding
, but that would lead to very large, hardly readable codeblock and would require manual changes when the kubernetes version changes. Which leads me to my question: Is there a better way to handle the network initialization?
b

billowy-army-68599

08/03/2021, 7:22 PM
how are you defining your EKS cluster? EKS uses the AWS VPC CNI by default and it should have been installed
can you share your code?
q

quiet-architect-91246

08/03/2021, 7:42 PM
Nothing out of the ordinary I think, could the issue be related to the default vpc thats used?
View count: 5