https://pulumi.com logo
Join Slack
Powered by
Hello, I have a TS pulumi program that is using a ...
# typescript
l
Hello, I have a TS pulumi program that is using a dynamic nodejs provider defined within the project. We are seeing a strange issue in our CI system (Gitlab) only. This one's a bit of a headscratcher because I cannot reproduce it outside of CI. It runs preview/up just fine on my local dev env and in my local docker instance (we use the same image in CI) -- however in CI we see this (See thread)
Copy code
Diagnostics:
  pulumi:pulumi:Stack (pulumi-eks-cluster-pulumi-eks-cluster-useast1-isolated-dev):
    I0525 15:30:31.594439     277 main.go:498] Language host launching process:  /opt/bitnami/node/bin/node /program/node_modules/@pulumi/pulumi/cmd/run/index.js --monitor 127.0.0.1:36205 --engine 127.0.0.1:36803 --sync /tmp/pulumi-node-pipes915941137 --project pulumi-eks-cluster --stack pulumi-eks-cluster-useast1-isolated-dev --pwd /program --dry-run --query-mode false --parallel 2147483647 .
    I0525 15:30:32.196696     277 ignore.go:44] Explicitly ignoring and discarding error: sync /dev/stdout: invalid argument
    I0525 15:30:32.196745     277 ignore.go:44] Explicitly ignoring and discarding error: sync /dev/stderr: invalid argument
 
    unhandled rejection: CONTEXT(3): resource:pulumi-eks-cluster-pulumi-eks-cluster-useast1-isolated-dev[pulumi:pulumi:Stack]
    STACK_TRACE:
    Error:
        at Object.debuggablePromise (/program/node_modules/@pulumi/pulumi/runtime/debuggable.js:69:75)
        at Object.registerResource (/program/node_modules/@pulumi/pulumi/runtime/resource.js:219:18)
        at new Resource (/program/node_modules/@pulumi/pulumi/resource.js:215:24)
        at new ComponentResource (/program/node_modules/@pulumi/pulumi/resource.js:386:9)
        at new Stack (/program/node_modules/@pulumi/pulumi/runtime/stack.js:62:9)
        at Object.runInPulumiStack (/program/node_modules/@pulumi/pulumi/runtime/stack.js:48:23)
        at Object.run (/program/node_modules/@pulumi/pulumi/cmd/run/run.js:235:20)
        at /program/node_modules/@pulumi/pulumi/cmd/run/index.js:139:42
This is the pulumi command:
Copy code
TF_LOG=TRACE pulumi up --refresh --config-file "${PULUMI_CONFIG_FILE}" --non-interactive --yes --logflow --logtostderr --debug -v=9
l
I wonder if those two stdout/stderr errors are relevant? Maybe the GitLab-CI docker env has permissions set on those that the app isn't prepared for? The --non-interactive code changes a few then when outputting to consoles; maybe try without that flag and see if the regular code handles things better?
It should automatically detect that stdout/stderr can't handle colours; I don't know what else the flag does.
l
Hi @little-cartoon-10569 thanks for the suggestions. Changing the non-interactive flag didn't do anything. However, we think there's an issue with the grpc-js and usage of *_PROXY vars. When we unset all our proxy variables the pipeline started running. We haven't pinpointed the exact issue yet but this was a big revelation.
Another fun fact, not sure if it's documented anywhere, but using both 
--verbose=
flag with 
--debug
seems to cancel the 
--debug
output
It was indeed the corporate proxy vars. grpc-js v0.7.0 and above introduce proxy handling and their 
no_proxy
configuration only accepts a comma separeted list of host names. regular no_proxy conventions allow CIDR ranges and hostname prefixes. in our case we had 
127.0.0.0/8
in no_proxy but it was not respected by the grpc library. so it was trying to send 
127.0.0.1
traffic over the corp proxy.
34 Views
Open in Slack
PreviousNext
Powered by