I’m using pulumi.StackReference and it seems that ...
# generalb
brash-plastic-65797
09/27/2019, 6:35 PMI’m using pulumi.StackReference and it seems that when I add calls to getOutput I’m getting Javascript Heap errors (out of memory) when running “pulumi up”, any smoking guns?
brash-plastic-65797
09/27/2019, 6:36 PMits hanging at “running read pulumipulumiStackReference” for a couple minutes before crashing
w
white-balloon-205
09/27/2019, 6:37 PM
Can you share details of the error - either here or in a GitHub bug report?
b
brash-plastic-65797
09/27/2019, 6:39 PMbrash-plastic-65797
09/27/2019, 6:43 PMimport * as pulumi from “@pulumi/pulumi”;
const env = pulumi.getStack();
const infraStackReference = new pulumi.StackReference(“username/pulumi-infra/“+env);
const vpcOutput = infraStackReference.getOutput(“vpc”);
const clusterOutput = infraStackReference.getOutput(“cluster”);
const albOutput = infraStackReference.getOutput(“internalAlb”);
brash-plastic-65797
09/27/2019, 6:44 PMjust doing that is enough to crash it, if I comment out ANY one of the getOutput calls it seems to work. but each getOutput call I add seems to add 1-2 minutes to the build time
brash-plastic-65797
09/27/2019, 6:50 PMhere is the log including the node/pulumi version info
brash-plastic-65797
09/27/2019, 7:58 PM@white-balloon-205 gonna try -v=9 to see if I can see whats happening during the stall at “running read pulumipulumiStackReference”
w
white-balloon-205
09/27/2019, 8:01 PM
Interesting - never seen this before. Could you open an issue in
pulumi/pulumiwhite-balloon-205
09/27/2019, 8:02 PM
Do you happen to have a particularly low memory limit on the environment you are running these deployments?
b
brash-plastic-65797
09/27/2019, 8:05 PMwe are running on aws codebuild and even bumped up to “7 GB memory, 4 vCPUs”
brash-plastic-65797
09/27/2019, 8:07 PMwe have an infra stack and a services stack, as you can see above I cut down the services index.ts to be very minimal
brash-plastic-65797
09/27/2019, 8:08 PMfor some reason each getOutput call is adding minutes to the build process
w
white-balloon-205
09/27/2019, 8:08 PM
And you have no reason to believe that the outputs of your stack are extremely large do you?
b
brash-plastic-65797
09/27/2019, 8:10 PMinfra is pretty much a vpc, 2 load balancers, s3 bucket, dynamodb table, and an ec2 instance
brash-plastic-65797
09/27/2019, 8:10 PMim prob missing a few details but no, not extremely large
brash-plastic-65797
09/27/2019, 8:10 PMthe infra deploys quickly with no issues
brash-plastic-65797
09/27/2019, 8:12 PMwe export the vpc for example, and we also export the ecs cluster (which is in the vpc), not sure if that causes any weird reference loop or un-necessary data
brash-plastic-65797
09/27/2019, 8:14 PMit just sits here until it fails: pulumipulumiStack pulumi-services-project-staging running read pulumipulumiStackReference username/pulumi-infra/project-staging
brash-plastic-65797
09/27/2019, 8:14 PMeven with -v=9 thats the last operation before it fails
brash-plastic-65797
09/27/2019, 8:15 PMIf I remove the 3 getOutput calls it completes quickly
brash-plastic-65797
09/27/2019, 8:15 PMeach one i add back slows it down by ~2 minutes until it eventually crashes
w
white-balloon-205
09/27/2019, 8:15 PM
Okay. Could you open an issue to track? We can look into it. This does sound like a Pulumi bug in
StackReferenceb
brash-plastic-65797
09/27/2019, 8:18 PMwill do
w
white-balloon-205
09/27/2019, 8:18 PM
We did make one change in this space recently - https://github.com/pulumi/pulumi/pull/3071 - which could in theory be triggering an issue here.
2 Views