glamorous-printer-66548
10/02/2018, 6:03 AMconst getK8sProviderByClusterName: (clusterName: string) => k8s.Provider
. It basically creates a k8s provider simply by passing a clustername (of an existing cluster - it will not magically create a new one) to it (and project id and zone are taken from the gcp config). The cluster is typically not defined itself in the stacks which use that utility function, so in order to get the cluster I’m using Cluster.get(...)
. The idea was to avoid having my team members to “correctly” configure their kubeconfig, set the right context etc. etc. but instead make the deployment target explicitly configured in code by using only the clusterName
. The problem I’m suddenly facing is that when anything non-substantial in the cluster changes, pulumi also wants to recreate (replace) the k8s provider and ALL the resources that have been previously created by that provider. Non-substantial changes in my case for example is that one of the node-pools scaled down by 1 instance. Even though I don’t use the node pool count at all to generate the k8s provider, pulumi wants to recreate it from scratch. Any advise on how to avoid this kind of problem? From my understanding Pulumi tracks here the dependency between the cluster and the k8s provider and because the k8s provider depends on the cluster any cluster changes will recreate the k8s provider. Is there any way to disable this auto-magic dependency tracking in this instance?import * as gcp from '@pulumi/gcp';
import * as pulumi from '@pulumi/pulumi';
import * as k8s from '@pulumi/kubernetes';
import { Cluster } from '@pulumi/gcp/container';
import { defaultZone, defaultProject } from './gcp-constants';
import { memoize } from 'ramda';
import { packageConfig } from './misc';
const gcpConfig = new pulumi.Config('gcp');
// most of the functions in this module are memoized. This is to improve lookup speed in bigger pulumi programs like solvvy-apis and
// to avoid resource id conflicts in pulumi when importing the same cluster / resource multiple times.
export const lookupClusterByName = memoize(
(clusterName: string): Cluster => {
return Cluster.get(clusterName, clusterName, {
name: clusterName,
zone: gcpConfig.get('zone') || defaultZone,
project: gcpConfig.get('project') || defaultProject
});
}
);
export function kubectlConfigFromGkeCluster(cluster: Cluster) {
return pulumi.all([cluster.name, cluster.endpoint, cluster.masterAuth]).apply(([name, endpoint, auth]) => {
const context = `${gcp.config.project}_${gcp.config.zone}_${name}`;
return `apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ${auth.clusterCaCertificate}
server: https://${endpoint}
name: ${context}
contexts:
- context:
cluster: ${context}
user: ${context}
name: ${context}
current-context: ${context}
kind: Config
preferences: {}
users:
- name: ${context}
user:
auth-provider:
config:
cmd-args: config config-helper --format=json
cmd-path: gcloud
expiry-key: '{.credential.token_expiry}'
token-key: '{.credential.access_token}'
name: gcp
`;
});
}
export const getK8sProviderByClusterName: (clusterName: string) => k8s.Provider = memoize((clusterName: string) => {
return new k8s.Provider(clusterName, {
kubeconfig: kubectlConfigFromGkeCluster(lookupClusterByName(clusterName))
});
});
export const getK8sProviderByCluster = memoize((providerInstanceName: string, cluster: Cluster) => {
return new k8s.Provider(providerInstanceName, {
kubeconfig: kubectlConfigFromGkeCluster(cluster)
});
});
export const getK8sProviderFromInferredCluster = memoize(() => {
const clusterName = packageConfig.require('cluster');
return new k8s.Provider(clusterName, {
kubeconfig: kubectlConfigFromGkeCluster(lookupClusterByName(clusterName))
});
});
creamy-potato-29402
10/02/2018, 8:29 PMmicroscopic-florist-22719
white-balloon-205
glamorous-printer-66548
10/02/2018, 8:41 PMgetCluster
instead of Cluster.get(...)
to get the values I need, but I opted to use Cluster.get because it’s API is nicer. Now the thing is one big difference between Cluster.get and getCluster seems to be that in the former pulumi keeps the output of the previous .get in the state, whereas with getCluster it does not.
In conclusion I think if I would have used getCluster I wouldn’t have seen this surprising behaviour. but I’m still wondering why there’s the existence of two potentially overlapping functions Cluster.get
and getCluster
and why they must behave so differently.creamy-potato-29402
10/02/2018, 8:42 PMwhite-balloon-205
new ks8.Provider
) in the Pulumi engine. We believe that in general you shouldn't see this behaviour - but we'll need to reproduce to understand exactly what is triggering this in the specific case here.glamorous-printer-66548
10/02/2018, 8:50 PMget<Resource>(...)
and Resource.get(...)
and why do those two overlapping methods with different underlying behaviour exist? What was the idea / philosophy for having this distinction?
We discussed this to some extent here https://pulumi-community.slack.com/archives/C84L4E3N1/p1538172290000200 having now faced the gke / k8s problem I’m still confused about the intention.white-balloon-205
getXYZ
functions are derived from a Terraform concept of "data sources". They are 1:1 with a Terraform data source. They do not return full fledged Pulumi resources, they are just code that runs to make certain API calls against the target cloud.
The Resource.get()
methods are a Pulumi concept that lets you read in the state of a resource and persist it to the checkpoint. They return true Pulumi resources that can be used like any other. We are expecting these to be used as a key path to how resources can be adopted into a Pulumi program.
For now, the former is generally more "battle tested" for the cases where it exists.glamorous-printer-66548
10/02/2018, 8:58 PMResource.get(...)
is persisted to the checkpoint? The resource is anyways not under management of pulumi, so what’s the point?incalculable-sundown-82514
10/02/2018, 8:59 PMThe resource is anyways not under management of pulumi, so what’s the point?Pulumi keeps track of the full resource dependency graph generated by your program. Since you can use output properties on a resource that you got via
Resource.get
, Pulumi must record the dependency between the .get
resource and a non-.get
resource created using a .get
resource’s outputs..get
resource must be in the checkpoint.glamorous-printer-66548
10/02/2018, 9:05 PM.get
resource that is not under management should be dependency tracked in the same full fledged fashion as a regular source at all? In my case the GKE cluster is actually under management in another project A and I just need to use the clusterCaCertificate
and endpoint
properties of that cluster in project B. So it’d like to establish a dependency on those two properties only and not the whole cluster resource as such.