ripe-vase-9728
02/27/2023, 4:54 PMpulumi up
, it says I have hundreds of Promises leaks.
When I change the VPC id used by the Security Group by any literal string though, all the leaks disappear.
Do you have any idea what could be the source of the leaks ? I find it very opaque and hard to debug the leaks and I am completely stuck.
I am pasting my Security Group code in the thread below.const rdsSecurityGroup = new aws.ec2.SecurityGroup(
`${name}-rds-security-group`,
{
vpcId: config.vpc.vpcId,
description: `Enables access on ${name} RDS`,
ingress: [
{
protocol: "tcp",
fromPort: 5432,
toPort: 5432,
cidrBlocks: ["0.0.0.0/0"],
},
],
egress: [
{
protocol: "tcp",
fromPort: 5432,
toPort: 5432,
cidrBlocks: ["0.0.0.0/0"],
},
],
},
{ parent: this }
);
import * as pulumi from "@pulumi/pulumi";
import { ComponentResourceOptions } from "@pulumi/pulumi/resource";
import * as awsx from "@pulumi/awsx";
const DEFAULT_NUMBER_OF_AVAILABILITY_ZONES = 1;
export class AubeNetwork extends pulumi.ComponentResource {
vpc: awsx.ec2.Vpc;
constructor(
name: string,
networkCidr: string,
numberOfAvailabilityZones?: number,
opts?: ComponentResourceOptions
) {
super("pkg:aube:network", name, {}, opts);
this.vpc = new awsx.ec2.Vpc(
`vpc-${name}`,
{
enableDnsHostnames: true,
enableDnsSupport: true,
numberOfAvailabilityZones:
numberOfAvailabilityZones ||
DEFAULT_NUMBER_OF_AVAILABILITY_ZONES,
cidrBlock: networkCidr,
subnetSpecs: [
{
type: "Private",
tags: {
"<http://kubernetes.io/role/internal-elb|kubernetes.io/role/internal-elb>": "1", //<https://aws.amazon.com/premiumsupport/knowledge-center/eks-vpc-subnet-discovery/>
},
cidrMask: 19,
},
{
type: "Public",
tags: {
"<http://kubernetes.io/role/elb|kubernetes.io/role/elb>": "1",
},
cidrMask: 20,
},
],
tags: {
Name: `vpc-${name}`,
},
},
{ parent: this }
);
this.registerOutputs({
vpcId: this.vpc.vpcId,
vpcPublicSubnetIds: this.vpc.publicSubnetIds,
vpcPrivateSubnetIds: this.vpc.privateSubnetIds,
});
}
}
miniature-musician-31262
02/27/2023, 7:14 PMripe-vase-9728
02/27/2023, 7:21 PMI0227 17:34:35.178583 34632 log.go:71] eventSink::Infoerr(<{%reset%}>Promise leak detected:
<{%reset%}>)
I0227 17:34:35.178613 34632 log.go:71] eventSink::Infoerr(<{%reset%}>CONTEXT(155): rpcKeepAlive
<{%reset%}>)
I0227 17:34:35.178618 34632 log.go:71] eventSink::Infoerr(<{%reset%}>STACK_TRACE:
<{%reset%}>)
I0227 17:34:35.178624 34632 log.go:71] eventSink::Infoerr(<{%reset%}>Error:
<{%reset%}>)
I0227 17:34:35.178633 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at Object.debuggablePromise (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/debuggable.ts:74:75)
<{%reset%}>)
I0227 17:34:35.178640 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at Object.rpcKeepAlive (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/settings.ts:413:25)
<{%reset%}>)
I0227 17:34:35.178644 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at /Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/resource.ts:483:30
<{%reset%}>)
I0227 17:34:35.178647 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at Generator.next (<anonymous>)
<{%reset%}>)
I0227 17:34:35.178651 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at /Users/nex/dev/aube/aube-infra/node_modules/@pulumi/pulumi/runtime/resource.js:21:71
<{%reset%}>)
I0227 17:34:35.178654 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at new Promise (<anonymous>)
<{%reset%}>)
I0227 17:34:35.178659 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at __awaiter (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/pulumi/runtime/resource.js:17:12)
<{%reset%}>)
I0227 17:34:35.178662 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at prepareResource (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/pulumi/runtime/resource.js:394:12)
<{%reset%}>)
I0227 17:34:35.178666 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at Object.registerResource (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/runtime/resource.ts:333:24)
<{%reset%}>)
I0227 17:34:35.178670 34632 log.go:71] eventSink::Infoerr(<{%reset%}> at new Resource (/Users/nex/dev/aube/aube-infra/node_modules/@pulumi/resource.ts:398:13)
<{%reset%}>)
pulumi up --logtostderr --logflow -v=9 2> out.txt
I0227 17:34:31.731516 34632 log.go:71] awsx = 1.0.2
I0227 17:34:31.731518 34632 log.go:71] docker = 3.6.1
I0227 17:34:31.731521 34632 log.go:71] random = 4.11.2
I0227 17:34:31.731523 34632 log.go:71] cloudflare = 4.15.0
I0227 17:34:31.731526 34632 log.go:71] aws = 5.30.0
import { Config } from "@pulumi/pulumi";
import * as pulumi from "@pulumi/pulumi";
import { AubeNetwork } from "./src/network";
import { AubeDatabase } from "./src/rds";
import { AubeRabbitMq } from "./src/rabbitmq";
import { AubeVpn } from "./src/vpn";
let stack = pulumi.getStack();
export const name = `aube-${stack}`;
/**
* Deploy infrastructure VPC
*/
const vpcConfig = new Config("network");
const vpcNetworkCidr = vpcConfig.get("cidr") || "10.0.0.0/16";
const network = new AubeNetwork(name, vpcNetworkCidr, 1);
/**
* Configure default RDS
*/
const dbConfig = new Config("db");
const db = new AubeDatabase(
name,
{
vpc: network.vpc,
username: dbConfig.get("username"),
dbName: dbConfig.get("name"),
allocatedStorage: 20, //I would like 256 GB but we must remain in the free tier for now
instanceClass: dbConfig.get("instanceClass"),
engineVersion: dbConfig.get("engineVersion"),
multiAz: false, //Free tier is only for Single AZ
},
{ parent: network.vpc }
);
miniature-musician-31262
02/27/2023, 8:00 PMconfig.vpc.vpcId
coming from? I see vpcConfig
and dbConfig
, but no config
.vpcId: vpc.vpcId
in the SecurityGroup declaration, since it looks like you’re passing in network.vpc
.ripe-vase-9728
02/27/2023, 8:05 PMconfig
is a structure parameter regrouping a lot of different parameters, it includes the network.vpc
.
Here is my custom resource for RDS :
import * as pulumi from "@pulumi/pulumi";
import { ComponentResourceOptions } from "@pulumi/pulumi/resource";
import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";
import * as random from "@pulumi/random";
const DEFAULT_ENGINE_VERSION = "15.2";
const DEFAULT_USERNAME = "aube";
const DEFAULT_MAINTENANCE_WINDOW = "Mon:03:00-Mon:04:00";
const DEFAULT_BACKUP_WINDOW = "22:00-03:00";
const DEFAULT_INSTANCE_CLASS = "db.t4g.micro"; //"db.m6g.large"
const DEFAULT_ALLOCATED_STORAGE = 20;
const DEFAULT_BACKUP_RETENTION_PERIOD = 7;
const DEFAULT_DB_NAME = "aube";
const DEFAULT_PERFORMANCE_INSIGHTS_RETENTION_PERIOD = 7;
let stack = pulumi.getStack();
export interface AubeRdsParams {
vpc: awsx.ec2.Vpc;
username?: string;
engineVersion?: string;
instanceClass?: string;
allocatedStorage: number;
backupRetentionPeriod?: number;
dbName?: string;
multiAz?: boolean;
allowMajorVersionUpgrade?: boolean;
}
export class AubeDatabase extends pulumi.ComponentResource {
rds: aws.rds.Instance;
constructor(
name: string,
config: AubeRdsParams,
opts?: ComponentResourceOptions
) {
super("pkg:aube:rds", name, {}, opts);
const rdsSecurityGroup = new aws.ec2.SecurityGroup(
`${name}-rds-security-group`,
{
vpcId: "wrwerwer", //config.vpc.vpcId,
description: `Enables access on ${name} RDS`,
ingress: [
{
protocol: "tcp",
fromPort: 5432,
toPort: 5432,
cidrBlocks: ["0.0.0.0/0"],
},
],
egress: [
{
protocol: "tcp",
fromPort: 5432,
toPort: 5432,
cidrBlocks: ["0.0.0.0/0"],
},
],
},
{ parent: this }
);
const subnetGroup = new aws.rds.SubnetGroup(
`${name}-rds-subnet-group`,
{
subnetIds: config.vpc.privateSubnetIds,
tags: {
Name: `${name} RDS subnet group`,
},
},
{ parent: this }
);
const password = new random.RandomPassword(
`${name}-rds-password`,
{ length: 16, special: false },
{ parent: this }
).result;
this.rds = new aws.rds.Instance(
`${name}-rds`,
{
engine: "postgres",
engineVersion: config.engineVersion || DEFAULT_ENGINE_VERSION,
username: config.username || DEFAULT_USERNAME,
password: password,
maintenanceWindow: DEFAULT_MAINTENANCE_WINDOW,
backupWindow: DEFAULT_BACKUP_WINDOW,
instanceClass: config.instanceClass || DEFAULT_INSTANCE_CLASS,
allocatedStorage:
config.allocatedStorage || DEFAULT_ALLOCATED_STORAGE,
backupRetentionPeriod:
config.backupRetentionPeriod ||
DEFAULT_BACKUP_RETENTION_PERIOD,
dbName: config.dbName || DEFAULT_DB_NAME,
multiAz: config.multiAz || true,
performanceInsightsEnabled: true,
performanceInsightsRetentionPeriod:
DEFAULT_PERFORMANCE_INSIGHTS_RETENTION_PERIOD,
storageEncrypted: true,
storageType: "gp2",
skipFinalSnapshot: true,
publiclyAccessible: false,
dbSubnetGroupName: subnetGroup.name,
vpcSecurityGroupIds: [rdsSecurityGroup.id],
allowMajorVersionUpgrade:
config.allowMajorVersionUpgrade || false,
},
{ parent: subnetGroup }
);
const secret = new aws.secretsmanager.Secret(
`database/main`,
{
name: `database/main`,
forceOverwriteReplicaSecret: true,
},
{ parent: this.rds }
);
const secretVersion = new aws.secretsmanager.SecretVersion(
`database/main`,
{
secretId: secret.id,
secretString: pulumi
.all([
this.rds.address,
this.rds.port,
this.rds.username,
this.rds.password,
this.rds.dbName,
])
.apply(([dbHost, dbPort, dbUser, dbPassword, dbName]) => {
return JSON.stringify({
DB_HOST: dbHost,
DB_PORT: dbPort,
DB_USER: dbUser,
DB_PASSWORD: dbPassword,
DB_NAME: dbName,
});
}),
},
{ parent: secret }
);
this.registerOutputs();
}
}
miniature-musician-31262
02/27/2023, 9:50 PM{ parent: network.vpc }
. Try removing that line (from your AubeDatabase
declaration) and running again.AubeDatabase
) to be parented by an awsx.ec2.Vpc
resource that’s being passed into (and then parented by) the component you’re creating. This is technically allowed, at least today, and apparently works up to a certain point, but it’s not common practice to use parent
in this way; rather, it’s used mainly to parent the resources managed by a component resource ({ parent: this }
) and that’s about it. I think you’ll find if you switch things up to use parent
to associate only the direct children of a component resource, and adjust all other parent
references accordingly, the errors will go away. (Alternatively, it looks like you could also change the parent
of AubeDatabase
to be network
, rather than network.vpc
, and have that work, too.
I do think this qualifies as two kinds of bug, though — one, in the documentation, which makes no mention of this behavior, and two, of the CLI, which should probably warn or fail on this rather than kick out an error you can’t really make sense of.ripe-vase-9728
02/28/2023, 2:49 PMnetwork
instead of `network.vpc`and it indeed solved my issue!!
I must say a very big THANK YOU for helping me on this. I spent literally a week modifying a lot of things, trying to debug and search all over internet without any success and without having any clue about what's happening.
I was starting to consider giving up on Pulumi.
You really saved me there. That's very kind of yours to take time to reproduce my bug and help me.
I am feeling very grateful towards you!
Have a good day 😄!const cloudflareConfig = new Config("cloudflare-config");
const vpn = new AubeVpn(
name,
{
vpc: network.vpc,
cidr: vpcNetworkCidr,
cloudflare: {
zoneId: cloudflareConfig.require("zoneId"),
},
},
{ parent: network.vpc }
);
Ideally I would like my `AubeDatabase`to be a child of network.vpc
instead of network
because if the VPC disappear, the Database should logically as well.
Why can't the `AubeDatabase`be a child of network.vpc
? I can't see any circular dependency.
If it works for my `AubeVpn`then it should as well for the Databaseminiature-musician-31262
03/01/2023, 6:53 PMnetwork
from within the Database component that the promise errors began. One reference, no problem — two, and the compilation fails.
I’m afraid I don’t have a good answer for you as to the “why”, unfortunately; something about this particular component (awsx.ec2.Vpc
) seems to be the root cause, but someone with deeper knowledge of that component will need to investigate. I’ll file another issue once I have a good repro and can explain the behavior. Will tag you on that so you have it as well.