https://pulumi.com logo
#aws
Title
# aws
m

modern-zebra-45309

01/16/2024, 8:22 PM
Hello everyone, I'm trying to pass the reference to an EKS cluster between stacks. I would like to potentially add resources to the cluster in a different stack. I'm already successful in passing the kubeconfig to other stacks. The goal is to be able to deploy the core cluster and to then be able to later add and remove extra node groups from it. We've structured our application into several micro-stacks, and this is the first time I'm running into the issue that I cannot (reliably, see below) pass a reference to the cluster around. I do have a stack where I create the cluster:
Copy code
import pulumi_eks as eks

def cluster_program():
    ...
  
    cluster = eks.Cluster(
        cluster_name,
        vpc_id=vpc_id,
        public_subnet_ids=public_subnet_ids,
        private_subnet_ids=private_subnet_ids,
        create_oidc_provider=True,
        # <https://github.com/pulumi/pulumi-eks/issues/95#issuecomment-588347991>
        skip_default_node_group=True,
        instance_role=cluster_instance_role,
    )

    pulumi.export("kubeconfig", cluster.kubeconfig)
    pulumi.export("cluster", cluster.core)
And I have a second stack where I would like to add nodegroups to it:
Copy code
def node_program():
    ...

    cluster_stack = pulumi.StackReference(
        f"{org}/{project_name}/{config.get('cluster_stack')}"
    )
    cluster_name = cluster_stack.get_output("cluster_name")
    cluster = cluster_stack.get_output("cluster")

    node_group = eks.ManagedNodeGroup(
            "training-cpu-node-group",
            cluster=cluster,
            node_group_name="training-cpu",
            node_role_arn=cluster_instance_role_arn,
            instance_types=["m3.large"],
            subnet_ids=private_subnet_ids,
            scaling_config=aws.eks.NodeGroupScalingConfigArgs(
                desired_size=config.get_int("training_cpu_node_group_desired_size"),
                min_size=config.get_int("training_cpu_node_group_min_size"),
                max_size=config.get_int("training_cpu_node_group_max_size"),
            ),
        )
This doesn't work, because the check whether the provided instance role is part of the cluster's instance roles fails:
Copy code
Exception: Cannot read properties of undefined (reading 'map')
    error: TypeError: Cannot read properties of undefined (reading 'map')
        at /snapshot/eks/bin/nodegroup.js:894:32
I've traced it back to here:
Copy code
// Check that the nodegroup role has been set on the cluster to
    // ensure that the aws-auth configmap was properly formed.
    const nodegroupRole = pulumi.all([core.instanceRoles, roleArn]).apply(([roles, rArn]) => {
        // Map out the ARNs of all of the instanceRoles.
        const roleArns = roles.map((role) => {
            return role.arn;
        });
        // Try finding the nodeRole in the ARNs array.
        return pulumi.all([roleArns, rArn]).apply(([arns, arn]) => {
            return arns.find((a) => a === arn);
        });
    });
The weird thing is, that somehow I was able to add node groups to the cluster from a different stack a couple of days ago, using this approach (or so I thought...) but this might very well be a glitch/side-effect of the various trial-and-error attempts I've made. I'd like to understand how I can get a reference to the cluster in a different stack. The "cluster" output is a dictionary that is compatible with CoreDataArgs. I can parse it into CoreDataArgs, but if I pass that to the ManagedNodeGroup, I get an error that "dependsOn" can only take resources. I don't define any explicit dependencies, but I assume that something inside of the ManagedNodeGroup construct is depending on the cluster that's passed in. I'm fine with re-constructing the cluster resource, importing it, or learning that what I'm trying to do is impossible or an anti-pattern.
2 Views