https://pulumi.com logo
Powered by
Title
r

ripe-vase-9728

02/27/2023, 4:54 PM
Hello, I am starting with Pulumi and I am having a very beginner-ish issue for the past 3 days that drives me crazy. I want to deploy a Vpc on AWS and then a RDS database inside of it. I have two custom components: One for my Vpc, and one for my RDS database. In my RDS custom component I am creating a Security Group using the VPC id. Whenever I try to run 
pulumi up
, it says I have hundreds of Promises leaks. When I change the VPC id used by the Security Group by any literal string though, all the leaks disappear. Do you have any idea what could be the source of the leaks ? I find it very opaque and hard to debug the leaks and I am completely stuck. I am pasting my Security Group code in the thread below.
const rdsSecurityGroup = new aws.ec2.SecurityGroup(
            `${name}-rds-security-group`,
            {
                vpcId: config.vpc.vpcId,
                description: `Enables access on ${name} RDS`,
                ingress: [
                    {
                        protocol: "tcp",
                        fromPort: 5432,
                        toPort: 5432,
                        cidrBlocks: ["0.0.0.0/0"],
                    },
                ],
                egress: [
                    {
                        protocol: "tcp",
                        fromPort: 5432,
                        toPort: 5432,
                        cidrBlocks: ["0.0.0.0/0"],
                    },
                ],
            },
            { parent: this }
        );
Here is my custom component for creating the VPC : 
import * as pulumi from "@pulumi/pulumi";
import { ComponentResourceOptions } from "@pulumi/pulumi/resource";
import * as awsx from "@pulumi/awsx";

const DEFAULT_NUMBER_OF_AVAILABILITY_ZONES = 1;

export class AubeNetwork extends pulumi.ComponentResource {
    vpc: awsx.ec2.Vpc;

    constructor(
        name: string,
        networkCidr: string,
        numberOfAvailabilityZones?: number,
        opts?: ComponentResourceOptions
    ) {
        super("pkg:aube:network", name, {}, opts);
        this.vpc = new awsx.ec2.Vpc(
            `vpc-${name}`,
            {
                enableDnsHostnames: true,
                enableDnsSupport: true,
                numberOfAvailabilityZones:
                    numberOfAvailabilityZones ||
                    DEFAULT_NUMBER_OF_AVAILABILITY_ZONES,
                cidrBlock: networkCidr,
                subnetSpecs: [
                    {
                        type: "Private",
                        tags: {
                            "<http://kubernetes.io/role/internal-elb|kubernetes.io/role/internal-elb>": "1", //<https://aws.amazon.com/premiumsupport/knowledge-center/eks-vpc-subnet-discovery/>
                        },
                        cidrMask: 19,
                    },
                    {
                        type: "Public",
                        tags: {
                            "<http://kubernetes.io/role/elb|kubernetes.io/role/elb>": "1",
                        },
                        cidrMask: 20,
                    },
                ],
                tags: {
                    Name: `vpc-${name}`,
                },
            },
            { parent: this }
        );
        this.registerOutputs({
            vpcId: this.vpc.vpcId,
            vpcPublicSubnetIds: this.vpc.publicSubnetIds,
            vpcPrivateSubnetIds: this.vpc.privateSubnetIds,
        });
    }
}
m

miniature-musician-31262

02/27/2023, 7:14 PM
Can you share a bit of the actual error/feedback you’re seeing? And also the version of Node you’re using?
r

ripe-vase-9728

02/27/2023, 7:21 PM
Hello, thanks for answering. My Node version is v19.2.0. One sample Promise error I have is this: 
I0227 17:34:35.178583   34632 log.go:71] eventSink::Infoerr(<{%reset%}>Promise leak detected:
<{%reset%}>)
I0227 17:34:35.178613   34632 log.go:71] eventSink::Infoerr(<{%reset%}>CONTEXT(155): rpcKeepAlive
<{%reset%}>)
I0227 17:34:35.178618   34632 log.go:71] eventSink::Infoerr(<{%reset%}>STACK_TRACE:
<{%reset%}>)
I0227 17:34:35.178624   34632 log.go:71] eventSink::Infoerr(<{%reset%}>Error: 
<{%reset%}>)
I0227 17:34:35.178633   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at Object.debuggablePromise (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/debuggable.ts:74:75)
<{%reset%}>)
I0227 17:34:35.178640   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at Object.rpcKeepAlive (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/settings.ts:413:25)
<{%reset%}>)
I0227 17:34:35.178644   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at /Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/resource.ts:483:30
<{%reset%}>)
I0227 17:34:35.178647   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at Generator.next (<anonymous>)
<{%reset%}>)
I0227 17:34:35.178651   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at /Users/nex/dev/aube/aube-infra/node_modules/@pulumi/pulumi/runtime/resource.js:21:71
<{%reset%}>)
I0227 17:34:35.178654   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at new Promise (<anonymous>)
<{%reset%}>)
I0227 17:34:35.178659   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at __awaiter (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/pulumi/runtime/resource.js:17:12)
<{%reset%}>)
I0227 17:34:35.178662   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at prepareResource (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/pulumi/runtime/resource.js:394:12)
<{%reset%}>)
I0227 17:34:35.178666   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at Object.registerResource (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/resource.ts:333:24)
<{%reset%}>)
I0227 17:34:35.178670   34632 log.go:71] eventSink::Infoerr(<{%reset%}>    at new Resource (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/resource.ts:398:13)
<{%reset%}>)
This error is the output of the command 
pulumi up --logtostderr --logflow -v=9 2> out.txt
Here are my plugins versions as stated in the out.txt file: 
I0227 17:34:31.731516   34632 log.go:71]   awsx            = 1.0.2
I0227 17:34:31.731518   34632 log.go:71]   docker          = 3.6.1
I0227 17:34:31.731521   34632 log.go:71]   random          = 4.11.2
I0227 17:34:31.731523   34632 log.go:71]   cloudflare      = 4.15.0
I0227 17:34:31.731526   34632 log.go:71]   aws             = 5.30.0
This is my index.ts file if it helps: 
import { Config } from "@pulumi/pulumi";
import * as pulumi from "@pulumi/pulumi";
import { AubeNetwork } from "./src/network";
import { AubeDatabase } from "./src/rds";
import { AubeRabbitMq } from "./src/rabbitmq";
import { AubeVpn } from "./src/vpn";

let stack = pulumi.getStack();

export const name = `aube-${stack}`;

/**
 * Deploy infrastructure VPC
 */
const vpcConfig = new Config("network");
const vpcNetworkCidr = vpcConfig.get("cidr") || "10.0.0.0/16";
const network = new AubeNetwork(name, vpcNetworkCidr, 1);


/**
 * Configure default RDS
 */
const dbConfig = new Config("db");
const db = new AubeDatabase(
    name,
    {
        vpc: network.vpc,
        username: dbConfig.get("username"),
        dbName: dbConfig.get("name"),
        allocatedStorage: 20, //I would like 256 GB but we must remain in the free tier for now
        instanceClass: dbConfig.get("instanceClass"),
        engineVersion: dbConfig.get("engineVersion"),
        multiAz: false, //Free tier is only for Single AZ
    },
    { parent: network.vpc }
);
m

miniature-musician-31262

02/27/2023, 8:00 PM
I’m not reproducing this on my end so far, but out of curiousity, where is 
config.vpc.vpcId
coming from? I see 
vpcConfig
and 
dbConfig
, but no 
config
.
I’d have expected based on your description something like 
vpcId: vpc.vpcId
in the SecurityGroup declaration, since it looks like you’re passing in 
network.vpc
.
e.g., something like this: https://gist.github.com/cnunciato/ae73657396224cb0f34165707c154f45#file-index-ts-L70
r

ripe-vase-9728

02/27/2023, 8:05 PM
I actually have something very close to the example you provided. 
config
is a structure parameter regrouping a lot of different parameters, it includes the 
network.vpc
. Here is my custom resource for RDS : 
import * as pulumi from "@pulumi/pulumi";
import { ComponentResourceOptions } from "@pulumi/pulumi/resource";
import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";
import * as random from "@pulumi/random";

const DEFAULT_ENGINE_VERSION = "15.2";
const DEFAULT_USERNAME = "aube";
const DEFAULT_MAINTENANCE_WINDOW = "Mon:03:00-Mon:04:00";
const DEFAULT_BACKUP_WINDOW = "22:00-03:00";
const DEFAULT_INSTANCE_CLASS = "db.t4g.micro"; //"db.m6g.large"
const DEFAULT_ALLOCATED_STORAGE = 20;
const DEFAULT_BACKUP_RETENTION_PERIOD = 7;
const DEFAULT_DB_NAME = "aube";
const DEFAULT_PERFORMANCE_INSIGHTS_RETENTION_PERIOD = 7;

let stack = pulumi.getStack();

export interface AubeRdsParams {
    vpc: awsx.ec2.Vpc;
    username?: string;
    engineVersion?: string;
    instanceClass?: string;
    allocatedStorage: number;
    backupRetentionPeriod?: number;
    dbName?: string;
    multiAz?: boolean;
    allowMajorVersionUpgrade?: boolean;
}

export class AubeDatabase extends pulumi.ComponentResource {
    rds: aws.rds.Instance;

    constructor(
        name: string,
        config: AubeRdsParams,
        opts?: ComponentResourceOptions
    ) {
        super("pkg:aube:rds", name, {}, opts);

        const rdsSecurityGroup = new aws.ec2.SecurityGroup(
            `${name}-rds-security-group`,
            {
                vpcId: "wrwerwer", //config.vpc.vpcId,
                description: `Enables access on ${name} RDS`,
                ingress: [
                    {
                        protocol: "tcp",
                        fromPort: 5432,
                        toPort: 5432,
                        cidrBlocks: ["0.0.0.0/0"],
                    },
                ],
                egress: [
                    {
                        protocol: "tcp",
                        fromPort: 5432,
                        toPort: 5432,
                        cidrBlocks: ["0.0.0.0/0"],
                    },
                ],
            },
            { parent: this }
        );

        const subnetGroup = new aws.rds.SubnetGroup(
            `${name}-rds-subnet-group`,
            {
                subnetIds: config.vpc.privateSubnetIds,
                tags: {
                    Name: `${name} RDS subnet group`,
                },
            },
            { parent: this }
        );

        const password = new random.RandomPassword(
            `${name}-rds-password`,
            { length: 16, special: false },
            { parent: this }
        ).result;

        this.rds = new aws.rds.Instance(
            `${name}-rds`,
            {
                engine: "postgres",
                engineVersion: config.engineVersion || DEFAULT_ENGINE_VERSION,
                username: config.username || DEFAULT_USERNAME,
                password: password,
                maintenanceWindow: DEFAULT_MAINTENANCE_WINDOW,
                backupWindow: DEFAULT_BACKUP_WINDOW,
                instanceClass: config.instanceClass || DEFAULT_INSTANCE_CLASS,
                allocatedStorage:
                    config.allocatedStorage || DEFAULT_ALLOCATED_STORAGE,
                backupRetentionPeriod:
                    config.backupRetentionPeriod ||
                    DEFAULT_BACKUP_RETENTION_PERIOD,
                dbName: config.dbName || DEFAULT_DB_NAME,
                multiAz: config.multiAz || true,
                performanceInsightsEnabled: true,
                performanceInsightsRetentionPeriod:
                    DEFAULT_PERFORMANCE_INSIGHTS_RETENTION_PERIOD,
                storageEncrypted: true,
                storageType: "gp2",
                skipFinalSnapshot: true,
                publiclyAccessible: false,
                dbSubnetGroupName: subnetGroup.name,
                vpcSecurityGroupIds: [rdsSecurityGroup.id],
                allowMajorVersionUpgrade:
                    config.allowMajorVersionUpgrade || false,
            },
            { parent: subnetGroup }
        );

        const secret = new aws.secretsmanager.Secret(
            `database/main`,
            {
                name: `database/main`,
                forceOverwriteReplicaSecret: true,
            },
            { parent: this.rds }
        );

        const secretVersion = new aws.secretsmanager.SecretVersion(
            `database/main`,
            {
                secretId: secret.id,
                secretString: pulumi
                    .all([
                        this.rds.address,
                        this.rds.port,
                        this.rds.username,
                        this.rds.password,
                        this.rds.dbName,
                    ])
                    .apply(([dbHost, dbPort, dbUser, dbPassword, dbName]) => {
                        return JSON.stringify({
                            DB_HOST: dbHost,
                            DB_PORT: dbPort,
                            DB_USER: dbUser,
                            DB_PASSWORD: dbPassword,
                            DB_NAME: dbName,
                        });
                    }),
            },
            { parent: secret }
        );

        this.registerOutputs();
    }
}
Can you please tell me what versions are you using for your packages ?
What surprises me most is that by replacing the `vpcId`by a random string the pulumi up command works. Only when using my vpc provided in the parameters it fails
m

miniature-musician-31262

02/27/2023, 9:50 PM
Ah! thanks for that. Ok, I have a local repro now. Lemme see.
(Incidentally I’m using the same package versions as you)
Ok so to me, it looks like it’s this line that’s the core problem: 
{ parent: network.vpc }
. Try removing that line (from your 
AubeDatabase
declaration) and running again.
Specifically what that line is saying is that you want your component resource (
AubeDatabase
) to be parented by an 
awsx.ec2.Vpc
resource that’s being passed into (and then parented by) the component you’re creating. This is technically allowed, at least today, and apparently works up to a certain point, but it’s not common practice to use 
parent
in this way; rather, it’s used mainly to parent the resources managed by a component resource (
{ parent: this }
) and that’s about it. I think you’ll find if you switch things up to use 
parent
to associate only the direct children of a component resource, and adjust all other 
parent
references accordingly, the errors will go away. (Alternatively, it looks like you could also change the 
parent
of 
AubeDatabase
to be 
network
, rather than 
network.vpc
, and have that work, too. I do think this qualifies as two kinds of bug, though — one, in the documentation, which makes no mention of this behavior, and two, of the CLI, which should probably warn or fail on this rather than kick out an error you can’t really make sense of.
I thought we might have a bug on this already, but so far, I’m not seeing one. I’ll get one filed if that’s the case and post back here — but do give that a try and let me know if that clears things up for you.
I went ahead and filed this issue to get some better documentation around this: https://github.com/pulumi/pulumi-hugo/issues/2519
r

ripe-vase-9728

02/28/2023, 2:49 PM
I did try to make `AubeDatabase`a children of 
network
instead of `network.vpc`and it indeed solved my issue!! I must say a very big THANK YOU for helping me on this. I spent literally a week modifying a lot of things, trying to debug and search all over internet without any success and without having any clue about what's happening. I was starting to consider giving up on Pulumi. You really saved me there. That's very kind of yours to take time to reproduce my bug and help me. I am feeling very grateful towards you! Have a good day 😄!
One thing that confuses me a bit still though is that I can add a `AubeVPN`that is a child of `network.vpc`without having any trouble. 
const cloudflareConfig = new Config("cloudflare-config");
const vpn = new AubeVpn(
    name,
    {
        vpc: network.vpc,
        cidr: vpcNetworkCidr,
        cloudflare: {
            zoneId: cloudflareConfig.require("zoneId"),
        },
    },
    { parent: network.vpc }
);
Ideally I would like my `AubeDatabase`to be a child of 
network.vpc
instead of 
network
because if the VPC disappear, the Database should logically as well. Why can't the `AubeDatabase`be a child of 
network.vpc
? I can't see any circular dependency. If it works for my `AubeVpn`then it should as well for the Database
m

miniature-musician-31262

03/01/2023, 6:53 PM
Sorry for the delayed response — yeah, I noticed as well that it wasn’t until adding the second reference to 
network
from within the Database component that the promise errors began. One reference, no problem — two, and the compilation fails. I’m afraid I don’t have a good answer for you as to the “why”, unfortunately; something about this particular component (
awsx.ec2.Vpc
) seems to be the root cause, but someone with deeper knowledge of that component will need to investigate. I’ll file another issue once I have a good repro and can explain the behavior. Will tag you on that so you have it as well.
#typescriptJoin Slack
Powered by