https://pulumi.com logo
Powered by
#aws
Title
p

prehistoric-kite-30979

01/25/2021, 3:12 PM
Hi all, when I create an eks cluster in a non-default VPC using the crosswalk library it fails because the security groups are still being created in the default VPC. Is this a bug or am I not configuring something I should have?
b

brave-planet-10645

01/25/2021, 3:22 PM
Can you share some code please? That sounds like a bug but want to check first
p

prehistoric-kite-30979

01/25/2021, 3:24 PM
Copy code
constructor(name: string, args: ClusterArgs) {
        super("tetrate:aws:eks", name)
        this.name = name
        const roles = this.createNodeGroupRoles(args.nodeGroups)
        this.eksCluster = new eks.Cluster(name, {
            version: args.version,
            vpcId: args.vpcId,
            publicSubnetIds: args.publicSubnetIds,
            privateSubnetIds: args.privateSubnetIds,
            
            enabledClusterLogTypes: ["api", "audit", "authenticator", "controllerManager", "scheduler"],
            createOidcProvider: true,
            
            skipDefaultNodeGroup: true,
            instanceRoles: roles,
        },{parent: this})
        this.createNodeGroups(args.nodeGroups, roles)
    }
b

brave-planet-10645

01/25/2021, 3:25 PM
And that's from a custom resource you're using?
p

prehistoric-kite-30979

01/25/2021, 3:25 PM
yes (well component)
Copy code
aws:eks:Cluster (cloudops-eksCluster):
    error: 1 error occurred:
    	* error creating EKS Cluster (cloudops-eksCluster-ff5ae24): InvalidParameterException: Security group(s) [sg-xxx] are not in the same VPC as the subnets. Please specify a security group that is associated with the VPC: vpc-xxx.
    {
      RespMetadata: {
        StatusCode: 400,
        RequestID: "be90565c-6cda-4e87-9d56-d86e32600c4d"
      },
      ClusterName: "cloudops-eksCluster-ff5ae24",
      Message_: "Security group(s) [sg-xxx] are not in the same VPC as the subnets. Please specify a security group that is associated with the VPC: vpc-xxx."
    }
b

brave-planet-10645

01/25/2021, 3:30 PM
Can email some more code over to support@pulumi.com? It'll be useful to see how you're calling the component in the surrounding code
p

prehistoric-kite-30979

01/25/2021, 3:32 PM
happy to post it here, it's not that large yet
Copy code
const networkStack = new pulumi.StackReference("aws.network.global")
const vpc = (networkStack.requireOutput("cloudops") as pulumi.Output<VpcImport>)
const eksCluster = new Eks("cloudops", {
    version: "1.18",
    vpcId: vpc.id,
    publicSubnetIds: vpc.publicSubnetIds,
    privateSubnetIds: vpc.privateSubnetIds,
})
b

brave-planet-10645

01/25/2021, 3:32 PM
that should be enough... thanks
p

prehistoric-kite-30979

01/25/2021, 3:32 PM
this is the project index.ts (minus the imports)
version is 
"@pulumi/eks": "^0.21.0",
b

brave-planet-10645

01/25/2021, 3:44 PM
Are you actually creating security groups as resources or are you letting the crosswalk library create them for you?
p

prehistoric-kite-30979

01/25/2021, 3:45 PM
letting the cw do it for me
I'm pretty sure I could fix it by creating them myself
b

brave-planet-10645

01/25/2021, 3:46 PM
in the aws.network.global stack?
p

prehistoric-kite-30979

01/25/2021, 3:46 PM
no that stack just creates vpc + subnets
its basically just the awsx vpc class
all it does is provice the vpcIds and subnetIds for cluster creation
the export is...
Copy code
get output(): VpcExport {
        return {
            vpcId: this.id,
            publicSubnetIds: this.publicSubnetIds,
            privateSubnetIds: this.privateSubnetIds,
            isolatedSubnetIds: this.isolatedSubnetIds,
            internetGatewayId: this.internetGateway.then((igw) => {return igw?.internetGateway.id}),
            natGatewayIds: this.natGateways.then((natgws) => {return natgws.map((natgw) => {return natgw.natGateway.id})})
        }
    }
b

brave-planet-10645

01/25/2021, 3:49 PM
Yeah that's odd because you're not really doing anything too complicated. Let me go and mess around a bit and get back to you
p

prehistoric-kite-30979

01/25/2021, 3:49 PM
thanks
yeah its a rewrite from go so is really barebones atm
b

brave-planet-10645

01/25/2021, 4:15 PM
So I think that it's because you're exporting the id into a 
vpcId
property but when you pass in the argument you're passing it in as 
vpc.id
. Can you do an export of the vpc object before you initialise the cluster... so something like this:
Copy code
const networkStack = new pulumi.StackReference("aws.network.global")
const vpc = (networkStack.requireOutput("cloudops") as pulumi.Output<VpcImport>)

export const clusterVpc = vpc;

const eksCluster = new Eks("cloudops", {
    version: "1.18",
    vpcId: vpc.id,
    publicSubnetIds: vpc.publicSubnetIds,
    privateSubnetIds: vpc.privateSubnetIds,
})
p

prehistoric-kite-30979

01/25/2021, 4:18 PM
same problem
Copy code
+ clusterVpc: {
      + internetGatewayId: "igw-0f295e01a8f642779"
      + isolatedSubnetIds: [
      +     [0]: "subnet-0e380d25a405c9d29"
      +     [1]: "subnet-0add20b1f6e3c152d"
      +     [2]: "subnet-0a1593001387e5003"
        ]
      + natGatewayIds    : [
      +     [0]: "nat-0c90334400f5f0829"
      +     [1]: "nat-0d73b08c5793a0d40"
      +     [2]: "nat-0aa02eb6feb839121"
        ]
      + privateSubnetIds : [
      +     [0]: "subnet-065c56466b6531f31"
      +     [1]: "subnet-0bf778d758cc65322"
      +     [2]: "subnet-0fb73ccbf5f130d85"
        ]
      + publicSubnetIds  : [
      +     [0]: "subnet-0cb9dea16c8c7f43f"
      +     [1]: "subnet-016267fc4f9031836"
      +     [2]: "subnet-07292ebfa06a1384f"
        ]
      + vpcId            : "vpc-xxx"
    }
was your theory that the vpcID isnt set so the cluster was getting the vpc from the subnets but the sg didn't?
b

brave-planet-10645

01/25/2021, 4:21 PM
ok, can you update how you're calling the Eks class to this:
Copy code
const eksCluster = new Eks("cloudops", {
    version: "1.18",
    vpcId: vpc.vpcId,
    publicSubnetIds: vpc.publicSubnetIds,
    privateSubnetIds: vpc.privateSubnetIds,
})
p

prehistoric-kite-30979

01/25/2021, 4:21 PM
ah, I see what you're getting at
pesky loose typing in outputs... 🙂
let me verify it works
b

brave-planet-10645

01/25/2021, 4:22 PM
so I suspect that it's passing the public and private subnet ids, but because vpc.id doesn't exist it's falling back to the default vpc
p

prehistoric-kite-30979

01/25/2021, 4:25 PM
🤦 Yup, that appears to have fixed it. Thanks for your help!
b

brave-planet-10645

01/25/2021, 4:26 PM
that's ok
Glad we got it working 😄
p

prehistoric-kite-30979

01/25/2021, 4:27 PM
the perils of a bolted-on type system 😬
b

brave-planet-10645

01/25/2021, 4:40 PM
FYI, I've just managed to reproduce it like this:
Copy code
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";

const vpc = new awsx.ec2.Vpc("vpc", {
    subnets: [{type: "private"}, {type: "public"}],
    numberOfAvailabilityZones: 2
});

const cluster = new eks.Cluster("cluster", {
    version: "1.18",
    publicSubnetIds: vpc.publicSubnetIds,
    privateSubnetIds: vpc.privateSubnetIds,
    enabledClusterLogTypes: ["api", "audit", "authenticator", "controllerManager", "scheduler"],
    createOidcProvider: true,
    skipDefaultNodeGroup: true,
})
(so not passing in the id of the VPC I created)
Powered by