https://pulumi.com logo
Title
a

acceptable-lawyer-72941

03/12/2023, 6:53 PM
Hi, I am having difficulty understanding how to create a SubnetGroup where the subnet ids are created in the call to
new awsx.ec2.Vpc("vpc-name", ...)
I am passing in a list of subnetSpecs:
const subnetSpecs = [
        {
            type: awsx.ec2.SubnetType.Public,
            name: "public",
            cidrMask: 24,
            tags: {
                ...publicSubnetTags,
            },
        },
        {
            type: awsx.ec2.SubnetType.Private,
            name: "kubernetes",
            cidrMask: 24,
            tags: {
                ...privateSubnetTags,
            },
        },
        {
            type: awsx.ec2.SubnetType.Private,
            name: "rds",
            cidrMask: 24,
            tags: {
                ...rdsSubnetTags,
            },
        },
    ];
rdsSunetTags is:
const rdsSubnetTags : {
        Name: "rds",
    };
Finally, and this is where I am stuck, after the call to
new awsx.ec2.Vpc("vpc-name", ...)
I want to create a subnet group for the rds subnets and attempt to do so as follows:
// Create RDS subnet group
    aws.ec2
        .getSubnets({
            tags: {
                Name: "rds",
            },
        })
        .then((rdsSubnets) => {
            console.log("rdsSubnets: ", rdsSubnets);

            // create a subnet group for the RDS subnets
            const rdsSubnetGroup = new aws.rds.SubnetGroup("rds-subnet-group", {
                subnetIds: rdsSubnets.ids,
                tags: {
                    Environment: env,
                },
            });
        });
Not surprisingly, rdsSubnets contains an empty list of subnets because new awsx.ec2.Vpc hasn't completed provisioning resources yet and there, in fact, are no subnets in the VPC. What is the right approach? Is there a way to get a Subnet promise that resolves when the the VPC and all of the subnets have been created? Thank you.
Figured it out.
const rdsSubnets = ptech_vpc.subnets.apply((subnets) => subnets
        .filter((subnet) =>
            subnet.tags.apply((tags) => tags?.Name === "rds")
        )
        .map((subnet) => subnet.id)
    );

    const rdsSubnetGroup = new aws.rds.SubnetGroup("rds-subnet-group", {
        subnetIds: rdsSubnets,
        tags: {
            Environment: env,
        },
    });
Is this an acceptable approach or should I be doing this differently? Thanks again.
b

billowy-army-68599

03/12/2023, 8:05 PM
@acceptable-lawyer-72941 the subnets should be available as a
privateSubnetIds
and
publicSubnetIds
output https://www.pulumi.com/registry/packages/awsx/api-docs/ec2/vpc/#privatesubnetids_nodejs You can just use that, no need to do an apply
a

acceptable-lawyer-72941

03/12/2023, 10:27 PM
I want the RDS instances on a separate private subnet. I'm creating two private subnets (in each az) and want to create a subnet group specifically for the rds subnet. I assume privateSubnetIds will return all the private subnetIds, not just those that belong to the RDS subnets.
b

billowy-army-68599

03/12/2023, 10:32 PM
ah yes, makes sense
a

acceptable-lawyer-72941

03/12/2023, 11:01 PM
Whew! 🙂 Thanks.
c

careful-family-14644

04/14/2023, 9:27 PM
@acceptable-lawyer-72941 Did this end up working for you? The filtering for me still doesn't seem to filter out the subnet I'm looking for.
a

acceptable-lawyer-72941

04/15/2023, 1:05 PM
Hi @careful-family-14644 - Thank you for checking. Yes. The following worked for me. Was just an Inputs/Outputs learning curve thing, but makes total sense now.
const rdsSubnets = ptech_vpc.subnets.apply((subnets) => subnets
        .filter((subnet) =>
            subnet.tags.apply((tags) => tags?.Name === "rds")
        )
        .map((subnet) => subnet.id)
    );

    const rdsSubnetGroup = new aws.rds.SubnetGroup("rds-subnet-group", {
        subnetIds: rdsSubnets,
        tags: {
            Environment: env,
        },
    });
c

careful-family-14644

04/17/2023, 2:11 PM
@acceptable-lawyer-72941 Very interesting because I used the same code you are using and it's not filtering my subnets. Even if I use a filter that would return nothing, it still returns all subnets. I've also tried several other forms that don't seem to work either.
const awsRegion = aws.getRegion({}).then(region => region.name);
export const eksSubnets = vpc.subnets.apply((subnets) => subnets
    .filter((subnet) =>
        subnet.tags.apply((tags) => tags?.Name === "dev-vpc-eks-1")
    )
);
export const eksSubnets2 = vpc.subnets.apply((subnets) => subnets
    .filter((subnet) =>
        subnet.tags.apply((tags) => tags?.Purpose === "eks")
    )
);
export const eksSubnetsIds = vpc.subnets.apply((subnets) => subnets
    .filter((subnet) =>
        subnet.tags.apply((tags) => tags?.Purpose === "eks")
    )
    .map((subnet) => subnet.id)
);

export const rdsSubnets = vpc.subnets.apply(subnets =>
    subnets.filter((subnet) =>
        subnet.tags.apply((tags) => {
            if ((tags?.Name.indexOf("rds") ?? -1) >= 0) { return true } return false;
        })
    ).map((subnet) => subnet.id)
);
b

billowy-army-68599

04/17/2023, 2:18 PM
do you subnets have those tags on them?
c

careful-family-14644

04/17/2023, 2:49 PM
tags : { Name : "dev-vpc-rds-2" Purpose : "rds" SubnetType : "Private" kubernetes.io/role/internal-elb: "1" } tagsAll : { Env : "dev" Name : "dev-vpc-rds-2" Purpose : "rds" SubnetType : "Private" kubernetes.io/role/internal-elb: "1" }
Yeah there's a handful associated within
@billowy-army-68599 There's others that have eks instead of rds for all tags
a

acceptable-lawyer-72941

04/17/2023, 5:09 PM
Apologies @careful-family-14644 - I went back to look at the code and it appears I did do some more monkeying around before I got it to work. Here's a cleaned up version of what I currently have. I haven't executed this exact code because I had to extract and rename a few details that can't be shared. If it doesn't work, it should be close. Hope this helps. Let me know how it goes.
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

const MAX_AZS = 2;

// Cluster tags for public subnets
const publicSubnetClusterTags: { [key: string]: string } = {};
publicSubnetClusterTags[`<http://kubernetes.io/cluster/clusterName`|kubernetes.io/cluster/clusterName`>] =
    "shared";
publicSubnetClusterTags["<http://kubernetes.io/role/elb|kubernetes.io/role/elb>"] = "1";

// Cluster tags for private subnets
const privateSubnetClusterTags: { [key: string]: string } = {};
privateSubnetClusterTags[`<http://kubernetes.io/cluster/clusterName`|kubernetes.io/cluster/clusterName`>] =
    "shared";
privateSubnetClusterTags["<http://kubernetes.io/role/internal-elb|kubernetes.io/role/internal-elb>"] = "1";

// Subnet specs
const subnetSpecs: awsx.types.input.ec2.SubnetSpecArgs[] = [
    {
        type: awsx.ec2.SubnetType.Public,
        name: "public",
        cidrMask: 24,
        tags: {
            Name: "public",
            ...publicSubnetClusterTags,
        },
    },
    {
        type: awsx.ec2.SubnetType.Private,
        name: "corporate",
        cidrMask: 24,
        tags: {
            Name: "corp-k8s",
            ...privateSubnetClusterTags,
        },
    },
    {
        type: awsx.ec2.SubnetType.Private,
        name: "corp-rds",
        cidrMask: 24,
        tags: {
            Group: "corp-rds",
        },
    },
];

export const vpc = new awsx.ec2.Vpc("corp-vpc", {
    numberOfAvailabilityZones: MAX_AZS,
    cidrBlock: "10.10.0.0/16",
    enableDnsHostnames: true,
    subnetSpecs,
    tags: {
        Environment: "dev",
    },
});

const corpRdsSubnets = vpc.subnets.apply((subnets) =>
    subnets
        .filter((subnet) =>
            subnet.tags.apply((tag) => tag?.Group === "corp-rds")
        )
        .map((subnet) => subnet.id)
);

export const corpRdsSubnetGroup = new aws.rds.SubnetGroup(
    "corp-rds-subnet-group",
    {
        subnetIds: corpRdsSubnets,
        tags: {
            Environment: "dev",
        },
    }
);
c

careful-family-14644

04/17/2023, 5:36 PM
export const vpc = new awsx.ec2.Vpc(resourcePrefix + "vpc", {
    ...networkArgs,
    numberOfAvailabilityZones: cfgNetwork.requireNumber("num-azs"),
    subnetSpecs: [
        {
            name: "pub",
            type: awsx.ec2.SubnetType.Public,
            cidrMask: cfgNetwork.requireNumber("public-subnetmask"),
            tags: {
                "<http://kubernetes.io/role/elb|kubernetes.io/role/elb>": "1"
            }
        },
        {
            name: "eks",
            type: awsx.ec2.SubnetType.Private,
            cidrMask: cfgNetwork.requireNumber("private-subnetmask"),
            tags: {
                "<http://kubernetes.io/role/internal-elb|kubernetes.io/role/internal-elb>": "1",
                Purpose: "eks"
            }
        },
        {
            name: "rds",
            type: awsx.ec2.SubnetType.Private,
            cidrMask: Number(cfgNetwork.requireNumber("rds-subnetmask")),
            tags: {
                "<http://kubernetes.io/role/internal-elb|kubernetes.io/role/internal-elb>": "1",
                Purpose: "rds"
            }
            
        }
    ],
    tags: {
        Name: resourcePrefix + "vpc",
    }
});
export const eksSubnetsIds = vpc.subnets.apply((subnets) =>
    subnets
        .filter((subnet) =>
            subnet.tags.apply((tag) => tag?.Purpose === "eks")
        )
        .map((subnet) => subnet.id)
);
With this code I get the following returned eksSubnetsIds : [ [0]: "subnet-0dc................................" [1]: "subnet-0ac................................" [2]: "subnet-0fe................................" [3]: "subnet-05b................................" [4]: "subnet-05d................................" [5]: "subnet-03b................................" ] All 6 subnets are being returned as opposed to just the 2 I'm wanting (Only 2 AZs configured in this case).
a

acceptable-lawyer-72941

04/17/2023, 5:58 PM
From just looking at it, it seems correct. Weird. What do you see if you console.log the tags in the filter lambda before calling subnet.tags.apply? I’m away from my computer atm so can’t test things myself. I’ll have another look tonight.
@careful-family-14644 - Thanks for your patience. I can confirm that this is also NOT working for me. I swear it was, but it's been some time since I have used this particular stack and it is no longer in use. However, there does appear to be a bug (or perhaps I am misunderstanding something). Here's my output:
corpRdsSubnetGroup: {
        arn                  : "arn:aws:rds:ca-central-1:888002373489:subgrp:corp-rds-subnet-group-3b94ada"
        description          : "Managed by Pulumi"
        id                   : "corp-rds-subnet-group-3b94ada"
        name                 : "corp-rds-subnet-group-3b94ada"
        namePrefix           : ""
        subnetIds            : [
            [0]: "subnet-0458e38521bb9608a"
            [1]: "subnet-0c902192e9f676853"
            [2]: "subnet-0d21a2f225347805e"
            [3]: "subnet-07a7cee228eaee44f"
            [4]: "subnet-0c0d64a8837f9df68"
            [5]: "subnet-0212cb68b71c5f704"
        ]
        supportedNetworkTypes: [
            [0]: "IPV4"
        ]
        tags                 : {
            Environment: "dev"
        }
        tagsAll              : {
            Environment: "dev"
        }
        urn                  : "urn:pulumi:test::help::aws:rds/subnetGroup:SubnetGroup::corp-rds-subnet-group"
    }
This is the output I get when I run the above script. I am also using only 2 AZs. and 2 AZs x 3 Subnets = 6 and all 6 are being included for some reason. The filtering is not working the way I would expect.
c

careful-family-14644

04/18/2023, 1:01 PM
@witty-candle-66007 @billowy-army-68599 Any thoughts on the issues Zach and I are talking about?
w

witty-candle-66007

04/18/2023, 1:43 PM
And to confirm, looking in the AWS console only two of the subnets have the
Purpose
tag set to
eks
?
c

careful-family-14644

04/18/2023, 2:10 PM
Correct
b

billowy-army-68599

04/18/2023, 2:14 PM
@careful-family-14644 might just be easier to schedule a call at this point, you can use this link https://zcal.co/leebriggs58
c

careful-family-14644

04/18/2023, 2:48 PM
Scheduled: @acceptable-lawyer-72941 you're welcome to join if you're interested.
a

acceptable-lawyer-72941

04/18/2023, 2:56 PM
Thanks. It’s less pressing for me, so I’m going to skip out, but curious to know if you figure it out.
c

careful-family-14644

04/19/2023, 6:51 PM
Just a follow-up to this thread. @billowy-army-68599 and I met up to research this problem and there's an issue to be created to allow for easier access to individual subnets and to be able to access them individually. In the meantime he is going to provide a way to make this operate based on filtering the tag values.
b

billowy-army-68599

04/19/2023, 7:13 PM
okay, I figured out why the code wasn’t working, and instead of trying to fix, just went down a route I know works:
export const eksSubnetsIds = vpc.privateSubnetIds.apply(ids => {
    let subnets = aws.ec2.getSubnetsOutput({
        filters: [{
            name: "tag:Purpose",
            values: ["eks"]
        }]
    })
    return subnets.ids
})
This will return the correct subnet ids
the reason this:
export const eksSubnetsIds = vpc.subnets.apply((subnets) =>
    subnets
        .filter((subnet) =>
            subnet.tags.apply((tag) => tag?.Purpose === "eks")
        )
        .map((subnet) => subnet.id)
);
Doesn’t work is because this:
subnet.tags.apply((tag) => tag?.Purpose === "eks")
returns an Output, so you can’t run a
map
operation on it, so it returns everything
c

careful-family-14644

04/19/2023, 7:17 PM
@billowy-army-68599 Awesome! Let met give this a shot on my end. Is there still the opinion of adding functionality to obtain this in a "cleaner" way?
b

billowy-army-68599

04/19/2023, 7:26 PM
yes going to add that to the issues board shortly