https://pulumi.com logo
Join Slack
Powered by
Hello, in our team we are using Pulumi and we enco...
# kubernetes
b
Hello, in our team we are using Pulumi and we encountered a strange behaviour. One new team mate cloned the code from the repository, has all the same versions as us, the same state file (stored in S3 bucket) etc, but Pulumi prompts on 
pulumi up
to delete some resources. Here is the output of 
pulumi preview
Copy code
Type                              Name                   Plan       Info
     pulumi:pulumi:Stack               our-product-stuff
 ~   ├─ kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release>  linkerd-crds           update     [diff: -resourceNames]
 ~   ├─ kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release>  linkerd-control-plane  update     [diff: -resourceNames]
 ~   └─ kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release>  linkerd-multicluster   update     [diff: -resourceNames]
Where should we start searching and debugging this problem? All others of us don’t have the problem. Thank you in advance
b
can you share your code and the expanded diff?
b
Copy code
Previewing update (prod):
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:prod::infra::pulumi:pulumi:Stack::infra-prod]
    > pulumi:pulumi:StackReference: (read)
        [id=infra]
        [urn=urn:pulumi:prod::infra::pulumi:pulumi:StackReference::infra]
        name: "infra"
    ~ kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release>: (update)
        [id=linkerd/linkerd-crds]
        [urn=urn:pulumi:prod::infra::kubernetes:<http://helm.sh/v3:Release::linkerd-crds|helm.sh/v3:Release::linkerd-crds>]
        [provider=urn:pulumi:prod::infra::pulumi:providers:kubernetes::default_3_21_4::086e4490-9af1-4939-8bc6-613a867d731d]
      - resourceNames: {
          - <http://CustomResourceDefinition.apiextensions.k8s.io/apiextensions.k8s.io/v1|CustomResourceDefinition.apiextensions.k8s.io/apiextensions.k8s.io/v1>: [
          -     [0]: "<http://authorizationpolicies.policy.linkerd.io|authorizationpolicies.policy.linkerd.io>"
          -     [1]: "<http://httproutes.policy.linkerd.io|httproutes.policy.linkerd.io>"
          -     [2]: "<http://meshtlsauthentications.policy.linkerd.io|meshtlsauthentications.policy.linkerd.io>"
          -     [3]: "<http://networkauthentications.policy.linkerd.io|networkauthentications.policy.linkerd.io>"
          -     [4]: "<http://serverauthorizations.policy.linkerd.io|serverauthorizations.policy.linkerd.io>"
          -     [5]: "<http://servers.policy.linkerd.io|servers.policy.linkerd.io>"
          -     [6]: "<http://serviceprofiles.linkerd.io|serviceprofiles.linkerd.io>"
            ]
        }
    ~ kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release>: (update)
        [id=linkerd/linkerd-control-plane]
        [urn=urn:pulumi:prod::infra::kubernetes:<http://helm.sh/v3:Release::linkerd-control-plane|helm.sh/v3:Release::linkerd-control-plane>]
        [provider=urn:pulumi:prod::infra::pulumi:providers:kubernetes::default_3_21_4::086e4490-9af1-4939-8bc6-613a867d731d]
      - resourceNames: {
          - <http://ClusterRole.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|ClusterRole.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>                         : [
          -     [0]: "linkerd-heartbeat"
          -     [1]: "linkerd-linkerd-destination"
          -     [2]: "linkerd-linkerd-identity"
          -     [3]: "linkerd-linkerd-proxy-injector"
          -     [4]: "linkerd-policy"
            ]
          - <http://ClusterRoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|ClusterRoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>                  : [
          -     [0]: "linkerd-destination-policy"
          -     [1]: "linkerd-heartbeat"
          -     [2]: "linkerd-linkerd-destination"
          -     [3]: "linkerd-linkerd-identity"
          -     [4]: "linkerd-linkerd-proxy-injector"
            ]
          - ConfigMap/v1                                                                               : [
          -     [0]: "linkerd/linkerd-config"
          -     [1]: "linkerd/linkerd-identity-trust-roots"
            ]
          - CronJob.batch/batch/v1                                                                     : [
          -     [0]: "linkerd/linkerd-heartbeat"
            ]
          - Deployment.apps/apps/v1                                                                    : [
          -     [0]: "linkerd/linkerd-destination"
          -     [1]: "linkerd/linkerd-identity"
          -     [2]: "linkerd/linkerd-proxy-injector"
            ]
          - <http://MutatingWebhookConfiguration.admissionregistration.k8s.io/admissionregistration.k8s.io/v1|MutatingWebhookConfiguration.admissionregistration.k8s.io/admissionregistration.k8s.io/v1>  : [
          -     [0]: "linkerd-proxy-injector-webhook-config"
            ]
          - <http://Role.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|Role.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>                                : [
          -     [0]: "linkerd/linkerd-heartbeat"
            ]
          - <http://RoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|RoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>                         : [
          -     [0]: "linkerd/linkerd-heartbeat"
            ]
          - Secret/v1                                                                                  : [
          -     [0]: "linkerd/linkerd-identity-issuer"
          -     [1]: "linkerd/linkerd-policy-validator-k8s-tls"
          -     [2]: "linkerd/linkerd-proxy-injector-k8s-tls"
          -     [3]: "linkerd/linkerd-sp-validator-k8s-tls"
            ]
          - Service/v1                                                                                 : [
          -     [0]: "linkerd/linkerd-dst"
          -     [1]: "linkerd/linkerd-dst-headless"
          -     [2]: "linkerd/linkerd-identity"
          -     [3]: "linkerd/linkerd-identity-headless"
          -     [4]: "linkerd/linkerd-policy"
          -     [5]: "linkerd/linkerd-policy-validator"
          -     [6]: "linkerd/linkerd-proxy-injector"
          -     [7]: "linkerd/linkerd-sp-validator"
            ]
          - ServiceAccount/v1                                                                          : [
          -     [0]: "linkerd/linkerd-destination"
          -     [1]: "linkerd/linkerd-heartbeat"
          -     [2]: "linkerd/linkerd-identity"
          -     [3]: "linkerd/linkerd-proxy-injector"
            ]
          - <http://ValidatingWebhookConfiguration.admissionregistration.k8s.io/admissionregistration.k8s.io/v1|ValidatingWebhookConfiguration.admissionregistration.k8s.io/admissionregistration.k8s.io/v1>: [
          -     [0]: "linkerd-policy-validator-webhook-config"
          -     [1]: "linkerd-sp-validator-webhook-config"
            ]
        }
    ~ kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release>: (update)
        [id=linkerd-multicluster/linkerd-multicluster]
        [urn=urn:pulumi:prod::infra::kubernetes:<http://helm.sh/v3:Release::linkerd-multicluster|helm.sh/v3:Release::linkerd-multicluster>]
        [provider=urn:pulumi:prod::infra::pulumi:providers:kubernetes::default_3_21_4::086e4490-9af1-4939-8bc6-613a867d731d]
      - resourceNames: {
          - <http://ClusterRole.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|ClusterRole.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>       : [
          -     [0]: "linkerd-multicluster/linkerd-service-mirror-remote-access-default"
            ]
          - <http://ClusterRoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|ClusterRoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>: [
          -     [0]: "linkerd-multicluster/linkerd-service-mirror-remote-access-default"
            ]
          - <http://CustomResourceDefinition.apiextensions.k8s.io/apiextensions.k8s.io/v1|CustomResourceDefinition.apiextensions.k8s.io/apiextensions.k8s.io/v1>    : [
          -     [0]: "<http://links.multicluster.linkerd.io|links.multicluster.linkerd.io>"
            ]
          - Job.batch/batch/v1                                                       : [
          -     [0]: "namespace-metadata"
            ]
          - <http://Role.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|Role.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>              : [
          -     [0]: "namespace-metadata"
            ]
          - <http://RoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1|RoleBinding.rbac.authorization.k8s.io/rbac.authorization.k8s.io/v1>       : [
          -     [0]: "namespace-metadata"
            ]
          - Secret/v1                                                                : [
          -     [0]: "linkerd-multicluster/linkerd-service-mirror-remote-access-default-token"
            ]
          - <http://Server.policy.linkerd.io/policy.linkerd.io/v1beta1|Server.policy.linkerd.io/policy.linkerd.io/v1beta1>                       : [
          -     [0]: "linkerd-multicluster/gateway-proxy-admin"
          -     [1]: "linkerd-multicluster/service-mirror"
          -     [2]: "linkerd-multicluster/service-mirror-proxy-admin"
            ]
          - <http://ServerAuthorization.policy.linkerd.io/policy.linkerd.io/v1beta1|ServerAuthorization.policy.linkerd.io/policy.linkerd.io/v1beta1>          : [
          -     [0]: "linkerd-multicluster/proxy-admin"
          -     [1]: "linkerd-multicluster/service-mirror"
          -     [2]: "linkerd-multicluster/service-mirror-proxy-admin"
            ]
          - ServiceAccount/v1                                                        : [
          -     [0]: "linkerd-multicluster/linkerd-service-mirror-remote-access-default"
          -     [1]: "namespace-metadata"
            ]
        }
Resources:
    ~ 3 to update
    10 unchanged
Copy code
import * as k8s from "@pulumi/kubernetes";
import { Release } from "@pulumi/kubernetes/helm/v3";
import { Input } from "@pulumi/pulumi";
import * as tls from "@pulumi/tls";

export const privateKeyCa = new tls.PrivateKey("linkerd-mtls-private-key-ca", {
  algorithm: "ECDSA",
  ecdsaCurve: "P256",
});

export const certCa = new tls.SelfSignedCert("linkerd-mtls-ca-cert", {
  privateKeyPem: privateKeyCa.privateKeyPem,
  allowedUses: [
    "digital_signature",
    "cert_signing",
    "client_auth",
    "server_auth",
    "any_extended",
  ],
  validityPeriodHours: 24 * 7 * 52 * 2, // 2 years
  isCaCertificate: true,
  subject: {
    commonName: "cendo-linkerd-root",
  },
});

export const privateKeyIntermediate = new tls.PrivateKey(
  "linkerd-mtls-private-key-intermediate",
  {
    algorithm: "ECDSA",
    ecdsaCurve: "P256",
  }
);

export const certRequestIntermediate = new tls.CertRequest(
  "linkerd-mtls-intermediate-certrequest",
  {
    privateKeyPem: privateKeyIntermediate.privateKeyPem,
    subject: {
      commonName: "cendo-linkerd-intermediate",
    },
  }
);

export const certIntermediate = new tls.LocallySignedCert(
  "linkerd-mtls-intermediate-cert",
  {
    allowedUses: [
      "digital_signature",
      "cert_signing",
      "client_auth",
      "server_auth",
      "any_extended",
    ],
    caCertPem: certCa.certPem,
    caPrivateKeyPem: privateKeyCa.privateKeyPem,
    certRequestPem: certRequestIntermediate.certRequestPem,
    validityPeriodHours: 24 * 7 * 52 * 2, // 2 years
    isCaCertificate: true,
  }
);

const linkerdNamespace = new k8s.core.v1.Namespace("linkerd-namespace", {
  metadata: {
    name: "linkerd",
  },
});

const linkerdCrds = () =>
  new k8s.helm.v3.Release(
    "linkerd-crds",
    {
      chart: "linkerd-crds",
      version: "1.4.0",
      repositoryOpts: {
        repo: "<https://helm.linkerd.io/stable>",
      },
      namespace: "linkerd",
      name: "linkerd-crds",
      atomic: true,
      timeout: 60,
      skipAwait: false,
    },
    { dependsOn: linkerdNamespace }
  );

const linkerdCP = (
  crds: k8s.helm.v3.Release,
  caCertPem: Input<string>,
  intCertPem: Input<string>,
  intPrivkeyPem: Input<string>
) =>
  new k8s.helm.v3.Release(
    "linkerd-control-plane",
    {
      chart: "linkerd-control-plane",
      version: "1.9.3",
      repositoryOpts: {
        repo: "<https://helm.linkerd.io/stable>",
      },
      namespace: "linkerd",
      name: "linkerd-control-plane",
      atomic: true,
      timeout: 150,
      values: {
        imagePullSecrets: [{ name: "regcred" }],
        cniEnabled: false,
        identityTrustAnchorsPEM: caCertPem,
        identity: {
          issuer: {
            tls: {
              crtPEM: intCertPem,
              keyPEM: intPrivkeyPem,
            },
          },
        },
        proxyInit: {
          runAsRoot: true,
          // iptablesMode: "nft", // legacy (default) or nft
        },
        //   controlPlaneTracingNamespace: "linkerd",
      },
    },
    { dependsOn: crds }
  );

const linkerdMulticlusterNs = new k8s.core.v1.Namespace(
  "linkerd-multicluster-namespace",
  {
    metadata: {
      name: "linkerd-multicluster",
    },
  }
);

const linkerdMulticluster = (controlPlane: Release, elbId?: string) =>
  new k8s.helm.v3.Release(
    "linkerd-multicluster",
    {
      chart: "linkerd-multicluster",
      version: "30.2.3",
      repositoryOpts: {
        repo: "<https://helm.linkerd.io/stable>",
      },
      namespace: "linkerd-multicluster",
      name: "linkerd-multicluster",
      atomic: true,
      timeout: 150,
      values: {
        gateway: {
          enabled: elbId ? true : false,
          serviceAnnotations: {
            "<http://kubernetes.io/elb.id|kubernetes.io/elb.id>": elbId,
          },
        },
      },
    },
    { dependsOn: [linkerdMulticlusterNs, controlPlane] }
  );

export const linkerd = (
  caCertPem: Input<string>,
  intCertPem: Input<string>,
  intPrivkeyPem: Input<string>,
  elbId?: string
) => {
  const crds = linkerdCrds();
  const cp = linkerdCP(crds, caCertPem, intCertPem, intPrivkeyPem);
  const mc = linkerdMulticluster(cp, elbId);
};
b
my guess is you’re using different versions of the providers
you can check with 
pulumi plugin ls
b
Copy code
❯ pulumi plugin ls
NAME        KIND      VERSION  SIZE   INSTALLED  LAST USED
kubernetes  resource  3.21.4   82 MB  1 day ago  1 day ago
tls         resource  4.6.1    33 MB  1 day ago  1 day ago
We all have the same output
b
huh. can you file an issue?
b
We also compared all together 
pulumi about
and except obvious things like the name of the user under backend and the architecture (arm64 and x86) for MacOS there was no real difference. We manage our project also with the nix package manager, to get the same development environment for our developers.
4 Views
Open in Slack
PreviousNext
Powered by