pulumi-community #kubernetes

Channels

kubernetes

fresh-judge-73477

09/28/2021, 7:44 AM

@here I am trying to create mapping type custom resource using pulumi v2 k8 SDK, below is the code snippet , but when i see debug logs of pulumi i see Spec object going as empty, not able to figure out whats goiing on, please help #kubernetes #golang #pulumi-kubernetes-operator

func CreateCustomResource(ctx *pulumi.Context, applicationName string, apiKey string, urlPrefix string, apiVersion string, kind string, labels pulumi.StringMap, timeout int64,k8sProvider *k8s.Provider,serviceAccount *v1.ServiceAccount)(*k8sApiExt.CustomResource, error){


   type SpecType struct {
      Prefix pulumi.String
      Service pulumi.String
      Headers pulumi.StringMap
      TimeOut <http://pulumi.Int|pulumi.Int>
   }

   //headersData := pulumi.StringMap{"x-api-key":pulumi.String(apiKey)}

   spec := SpecType{
      Prefix:  pulumi.String(fmt.Sprintf("/%s/",urlPrefix)),
      Service: pulumi.String(applicationName),
      TimeOut: <http://pulumi.Int|pulumi.Int>(timeout),
   }

   var otherfields k8s.UntypedArgs
   otherfields = make(k8s.UntypedArgs, 1)
   otherfields["Spec"] = spec


   customResourceDefnArgs := k8sApiExt.CustomResourceArgs{
      ApiVersion: pulumi.String(apiVersion),
      Kind:       pulumi.String(kind),
      Metadata:   metav1.ObjectMetaArgs{
         Labels:                     labels,
         Name:                       pulumi.String(applicationName),
         Namespace:                  pulumi.String(applicationName),

      },

      OtherFields: otherfields,

   }
   data, err := json.Marshal(customResourceDefnArgs)
   if(err == nil){
      logger.Infof("datataa= %s-%s",data,apiKey)
   }

   return k8sApiExt.NewCustomResource(ctx,GetResourceName(applicationName,"mapping"),&customResourceDefnArgs,pulumi.DependsOn([]pulumi.Resource{serviceAccount}),pulumi.Provider(k8sProvider))


}

rapid-soccer-18092

09/28/2021, 3:53 PM

Since deploying the Datadog helm chart, I get the following

pulumi preview

diff even if nothing has changed. This ultimately causes my datadog pods to get recreated, which is not desirable. I am deploying to AKS K8s (1.21.2) using Pulumi 3.12.0

pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:preprod::aks::pulumi:pulumi:Stack::aks-preprod]
            [provider=urn:pulumi:preprod::aks::pulumi:providers:kubernetes::k8s-provider::046e67e2-9780-4010-9c77-9999999ebefd]
          ~ spec: {
              ~ template: {
                  ~ metadata: {
                      ~ annotations: {
                          ~ checksum/clusteragent_token: "32a656c3c7aeb06e5c36xxx" => "8027d4026d2e72484f1xxx"
                        }
                    }
                }
            }
        +-kubernetes:core/v1:Secret: (replace)
            [id=default/datadog-chart-cluster-agent]
            [urn=urn:pulumi:preprod::aks::kubernetes:<http://helm.sh/v3:Chart$kubernetes:core/v1:Secret::default/datadog-chart-cluster-agent|helm.sh/v3:Chart$kubernetes:core/v1:Secret::default/datadog-chart-cluster-agent>]
            [provider=urn:pulumi:preprod::aks::pulumi:providers:kubernetes::k8s-provider::046e67e2-9780-4010-9c77-9999999ebefd]
          ~ data: {
            }
        ~ kubernetes:apps/v1:DaemonSet: (update)
            [id=default/datadog-chart]
            [urn=urn:pulumi:preprod::aks::kubernetes:<http://helm.sh/v3:Chart$kubernetes:apps/v1:DaemonSet::default/datadog-chart|helm.sh/v3:Chart$kubernetes:apps/v1:DaemonSet::default/datadog-chart>]
            [provider=urn:pulumi:preprod::aks::pulumi:providers:kubernetes::k8s-provider::046e67e2-9780-4010-9c77-9999999ebefd]
          ~ spec: {
              ~ template: {
                  ~ metadata: {
                      ~ annotations: {
                          ~ checksum/clusteragent_token: "bcc328b0b69baa07a7fae32a6baxxx" => "5200e78e7733904901f9511a094e8xxx"
                        }
                    }
                }
Resources:
    ~ 2 to update
    +-1 to replace
    3 changes. 103 unchanged

And the pruned logs from the

pulumi up

look like this:

-- kubernetes:core/v1:Secret default/datadog-chart-cluster-agent deleting original 
 ~  kubernetes:apps/v1:Deployment default/datadog-chart-cluster-agent updating [diff: ~spec]
 ~  kubernetes:apps/v1:DaemonSet default/datadog-chart updating [diff: ~spec]
 -- kubernetes:core/v1:Secret default/datadog-chart-cluster-agent deleting original 
 -- kubernetes:core/v1:Secret default/datadog-chart-cluster-agent deleted original
 ~  kubernetes:apps/v1:Deployment default/datadog-chart-cluster-agent updating [diff: ~spec]; [1/2] Waiting for app ReplicaSet be marked available
 ~  kubernetes:apps/v1:Deployment default/datadog-chart-cluster-agent updating [diff: ~spec]; Deployment initialization complete
 ~  kubernetes:apps/v1:Deployment default/datadog-chart-cluster-agent updated [diff: ~spec]; Deployment initialization complete 
 ~  kubernetes:apps/v1:DaemonSet default/datadog-chart updated [diff: ~spec] 
 +- kubernetes:core/v1:Secret default/datadog-chart-cluster-agent replacing [diff: ~data];
 +- kubernetes:core/v1:Secret default/datadog-chart-cluster-agent replaced [diff: ~data]; 
 ++ kubernetes:core/v1:Secret default/datadog-chart-cluster-agent creating replacement [diff: ~data]; 
 ++ kubernetes:core/v1:Secret default/datadog-chart-cluster-agent creating replacement [diff: ~data]; 
 ++ kubernetes:core/v1:Secret default/datadog-chart-cluster-agent created replacement [diff: ~data];

Can someone help point me in the right direction?

worried-city-86458

09/30/2021, 2:08 AM

@sparse-park-68967 I think there's an issue with the new helm releases, their dependencies and uninstall / delete AFAICT, under the hood the call to helm uninstall does not wait, so dependencies do not unwind in the correct order. When this deletes a resource with a finalizer then other resources are orphaned and it cascades from there...

worried-city-86458

09/30/2021, 7:33 PM

PS. helm releases are much faster than helm charts - shaves ~15-20m off times to create and destroy the k8s stack Presumably due to fewer pulumi api roundtrips for hundreds of fewer resources since helm releases does not look through and enumerate chart resources

rhythmic-rain-31941

09/30/2021, 9:29 PM

Hi Team, does pulumi kubernetes provider allows us to do

exec

as we can do in terraform provider? https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#exec-plugins #kubernetes

square-energy-17741

09/30/2021, 10:14 PM

Moving it from #general to #kubernetes Does anyone know an example of installing and configuring kubernetes "cluster-autoscaler" to use ManagedNodeGroup with pulumi/eks and pulumi/kubernetesx?

rhythmic-rain-31941

10/01/2021, 1:23 AM

Hi Team, I'm using rancher2 and kubernetes plugins for setting up my cluster. below is how my python looks,

rancher_ci_cluster = rancher2.Cluster()

k8s_provider = k8s.Provider("cluster-access",
    kubeconfig=rancher_ci_cluster.kube_config
)

sa = k8s.core.v1.ServiceAccount("microservice",
    metadata=k8s.meta.v1.ObjectMetaArgs(
      namespace="monitoring",
      name="microservice"
    )
)

this keep throwing below error

error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: Get "<https://992E92B1DDDD40F85E620599B78C3F9C.sk1.us-east-1.eks.amazonaws.com/openapi/v2?timeout=32s>": dial tcp: lookup <http://992E92B1DDDD40F85E620599B78C3F9C.sk1.us-east-1.eks.amazonaws.com|992E92B1DDDD40F85E620599B78C3F9C.sk1.us-east-1.eks.amazonaws.com> on 10.0.4.55:53: no such host

not sure if the provider is getting set correctly or not, can someone help me here please? p.s: that's not my cluster api server endpoint too

breezy-bear-50708

10/04/2021, 12:56 PM

Hello, wonder if there is any way to get more visibility on https://github.com/pulumi/pulumi/issues/4996 . It stops us from automating destroy/rebuild of clusters

👍 1

dry-football-2639

10/04/2021, 2:26 PM

Hey! Any1 able to help me here? https://stackoverflow.com/questions/69437619/how-to-modify-the-aws-auth-config-map-after-creating-eks-with-pulumi

dry-autumn-28966

10/04/2021, 6:09 PM

It appears that changing an Helm release

values

(Go automation API) doesn’t trigger an update during Up() operations. I can see the new value in the stack state file but not when I run

helm get values <release>

. I have tried setting Replace=true / ForceUpdate=true / ResetValues=true but no luck. Am I doing something wrong?

brash-quill-35776

10/05/2021, 5:58 PM

Hi, how can I use Pulumi to get istio configmap and to update the data? Istio configmap is created by

istioctl

, rather than Pulumi itself I know for resources that created by pulumi, I can definitely update that by running

pulumi up

, but how about resource that not created by

pulumi

average-market-57523

10/06/2021, 7:27 PM

Hi, is there any setting in pulumi that makes a Job Resource run on each deployment?

square-car-84996

10/07/2021, 1:40 PM

how do you set the default provider? i've been successfully passing it into helm.v3.Release, but a crd2pulumi'd resource is throwing

Cannot read properties of undefined (reading 'promise')

and the provider is not

undefined

square-energy-17741

10/08/2021, 3:31 AM

Hello friends, I have been trying to install kubernetes metrics-server using

helm.v3.Release

but it always fails with the following error -

error: timed out waiting for the condition

My code to install the helm chart looks like this -

const metricsServerChart = new k8s.helm.v3.Release (
  'kubesys-ms',
  {
    chart: 'metrics-server',
    version: '3.5.0',
    namespace: 'kube-system',
    name: 'kubesys-ms',
    repositoryOpts: {
      repo: '<https://kubernetes-sigs.github.io/metrics-server/>'
    },
    values: {},
  },
  { 
    dependsOn: [cluster], 
    provider: cluster.provider 
  }
);

I am able to install the same chart easily using helm command line -

helm -n kube-system install kubesys-ms metrics-server/metrics-server

Am I missing something?

square-energy-17741

10/08/2021, 3:43 PM

Has anyone used/tried using Pulumi to install JenkinsX on K8s?

dry-sugar-63293

10/11/2021, 8:00 AM

Any help would be much appreciated. thanks!

sparse-spring-91820

10/11/2021, 10:47 AM

https://pulumi-community.slack.com/archives/CRFURDVQB/p1630615849033000 any updates on this one? I have the same issue

dry-sugar-63293

10/11/2021, 3:25 PM

message has been deleted

square-energy-17741

10/12/2021, 3:36 AM

Has anyone faced issue with typescript

helm.v3.Release

not honoring

namespace

with certain charts. I have tried with

helm

CLI and

helm.v3.Chart

and they both work correctly. This is happening with only certain Charts - • jx-git-operator from https://jenkins-x-charts.github.io/repo • cluster-autoscaler from https://kubernetes.github.io/autoscaler

full-camera-24872

10/13/2021, 2:53 PM

Hello, is it possible to deploy local helm charts using the

k8s.helm.v3.Release

I have tried a number of different ways and it currently appears that you need a repo ?

limited-rain-96205

10/13/2021, 6:38 PM

In the situation where you’re deploying an Operator which adds CRDs to the cluster, and then have Pulumi deploy resources from those CRDs, is there a way to have Pulumi wait until the CRDs are present before creating those resources?

future-window-78560

10/14/2021, 11:11 AM

Hey! My Kubernetes cluster is taking 100GB disk storage by default. I am trying to specify storage spec in the code. On adding these lines of code, " resources=ResourceRequirementsArgs( requests={ "storage": "8Gi", }, ), " I am having an error "TypeError: __init__() got an unexpected keyword argument 'resources'"

prehistoric-kite-30979

10/14/2021, 10:51 PM

We’ve found that installing crds via pulumi config groups massively bloats the size of the state and makes all operations really slow. Has anyone found a way around this?

rapid-soccer-18092

10/15/2021, 7:40 AM

Hi. Pulumi preview/up doesnt seem to recognise changes to the helm chart values data. I've tried setting

ReplaceOnChanges = { "*" }

but that doesn't seem to kick in either. See thread for the code. How should I go about this?

steep-toddler-88913

10/16/2021, 1:54 PM

Hi! I am just starting to use Pulumi. I am using Python, DigitalOcean, and now trying to add some Kubernetes pieces (specifically, I want to add some Secrets to my cluster). Is it possible to create a cluster, then in the same run/file hook into that cluster and create some secrets? I don't already have the kubeconfig, so I'm unclear on if I can use the pulumi_kubernetes package to do this. I speculate that the Provider might help since it takes a kubeconfig argument, but I don't know how to use the provider or if that'd even work. Also my apologies if this isn't the right spot for this question, happy to be redirected.

import pulumi_digitalocean as do

k8s = do.KubernetesCluster("main-k8s",
                           region="nyc3",
                           version="1.21.3-do.0",
                           node_pool=do.KubernetesClusterNodePoolArgs(
                               name="main-pool",
                               size="s-1vcpu-2gb",
                               node_count=2
                           ))

# now create a secret in this^ cluster

most-lighter-95902

10/16/2021, 4:09 PM

Hi I’ve been trying to get Knative setup in Kubernetes with Pulumi, but running into an issue trying to

pulumi destroy

most-lighter-95902

10/16/2021, 4:09 PM

'knative-serving' timed out waiting to be Ready

most-lighter-95902

10/16/2021, 4:09 PM

There seems to be no pods running so I’m not sure what to do about this

most-lighter-95902

10/16/2021, 4:10 PM

I can’t complete the destroy process

able-translator-73668

10/18/2021, 8:49 AM

Hi guys, could you tell when we can expect GA for Helm Release Resource for Kubernetes? @billions-librarian-83807 any ideas? 🙂 🍻