https://pulumi.com logo
#kubernetes
Title
# kubernetes
g

great-byte-67992

12/12/2023, 10:13 PM
Hello, I'm wondering if a
k8s.Provider
exposes it's kubeconfig in any usable way. I'm trying to use
@kubernetes/client-node
to implement a readiness check for cert-manager for example
Copy code
import * as k8s from '@pulumi/pulumi';
import { KubeConfig } from '@kubernetes/client-node';

const provider = new k8s.Provider('k8s', {
  // ...
});

provider.kubeconfig.apply(kubeconfig => {
  const kc = new KubeConfig();
  kc.loadFromString(kubeconfig);
  // ...
});
the
provider.kubeconfig
isn't actually a thing, is there anyway to do this though? EDIT: turns out the property does exist at runtime; and here's how i'm using it to ready check cert-manager's admission controller webhooks:
Copy code
/**
 * This function takes a kubernetes provider and uses it to check if
 * a cert-manager installation is ready to be used.
 *
 * cert-manager can be slow to be ready because it uses kubernetes admission
 * controllers (webhooks) which take time to reach a ready state.
 *
 * This function checks cert-manager by attempting (dry-run) to create a ClusterIssuer
 * customer resource.
 *
 * The dry-run attempt is actually processed by kubernetes and all admission controllers
 * so it allows us to know if the webhooks used by cert-manager are available or not.
 */
function certManagerReadinessCheck(name: string, provider: k8s.Provider): pulumi.Output<string> {
  if (!('kubeconfig' in provider)) {
    return pulumi.output('skipped');
  }

  const kubeconfig = provider.kubeconfig as pulumi.Output<string>;

  return kubeconfig.apply(async (kubeconfig) => {
    if (pulumi.runtime.isDryRun()) {
      return 'dry-run';
    }

    const kc = new k8sc.KubeConfig();
    kc.loadFromString(kubeconfig);
    const client = kc.makeApiClient(k8sc.CustomObjectsApi);

    await withRetry(() =>
      client.createClusterCustomObject(
        'cert-manager.io',
        'v1',
        'clusterissuers',
        {
          apiVersion: 'cert-manager.io/v1',
          kind: 'ClusterIssuer',
          metadata: {
            name: `${name}-ready-check-test`,
          },
          spec: {
            selfSigned: {},
          },
        },
        undefined,
        // This is the important parameter for our readiness check.
        // This is the 'dryRun' parameter and we use 'All' so that
        // the manifest is submitted to kubernetes and validated against
        // things like cert-manager's webhooks.
        'All'
      )
    );

    return 'success';
  });
}

/**
 * WithRetry runs the given `fn` function and retries if it throws.
 *
 * The default configuration will attempt 60 times with a 5 second backoff between
 * attempts.
 *
 * TODO the default configuration will attempt over 5 minutes (60 * 5seconds) but in
 * the case that the `fn` is very slow this can be much longer so we need to support
 * a deadline/timeout.
 */
async function withRetry<T>(fn: () => Promise<T>, maxAttempts = 60, backoffSeconds = 5): Promise<T> {
  try {
    return await fn();
  } catch (error) {
    console.log(`withRetry: ${JSON.stringify(error)}`);
    if (maxAttempts > 0) {
      return withRetry(fn, maxAttempts - 1, backoffSeconds);
    }
    throw error;
  }
}
usage
Copy code
new k8s.apiextensions.CustomResource(
      'cluster-isser',
      {
        apiVersion: 'cert-manager.io/v1',
        kind: 'ClusterIssuer',
        metadata: {
          annotations: {
            'ready-check': certManagerReadinessCheck(name, props.provider),
          },
        },
        spec: {
          // ...
        },
      }
    );
g

glamorous-teacher-20849

12/13/2023, 12:49 AM
You would set kubeconfig on the provider vs reading it. Eg.
Copy code
var k8sProvider = new Provider("k8s", new ProviderArgs
        {
            KubeConfig = kubeConfig
        });
I construct mine like so:
Copy code
ListManagedClusterAdminCredentials.Invoke(new ListManagedClusterAdminCredentialsInvokeArgs
        {
            ResourceGroupName = resourceGroupName,
            ResourceName = clusterName
        }).Apply(credentials => {
            var encoded = credentials.Kubeconfigs[0].Value;
            var data = Convert.FromBase64String(encoded);
            return Encoding.UTF8.GetString(data);
        });
Sorry for the c#
thats azure k8s
it will be implementation specific
g

great-byte-67992

12/13/2023, 12:51 AM
I've already got a provider; i'm implementing a custom component for cert-manager which receives a k8s.Provider for deploying a chart to. But i need to use a k8s sdk to ready check the admission controllers So i need to use the kubeconfig again but want to get it back out of the provider instead of passing another input to the component if that makes sense
I actually found the
provider.kubeconfig
is available at runtime, it's just not in the type declarations for some reason.
g

glamorous-teacher-20849

12/13/2023, 12:52 AM
can you not pass it the entire provider instance and reference it in the chart resource?
g

great-byte-67992

12/13/2023, 12:53 AM
I did, and the chart deploys. That's not the question 😬
g

glamorous-teacher-20849

12/13/2023, 12:53 AM
Sure sure
Reading the docs kubeConfig is a documented field of the provider. Maybe an oversight in the typescript offering
I've been totally unhelpful, i'll leave you to it 👍
g

great-byte-67992

12/13/2023, 12:56 AM
No worries, confusing question
2 Views