Hello - I've been using pulumi v3.75 with python t...
# python
b
Hello - I've been using pulumi v3.75 with python that's been working for a while now. But suddenly it's now throwing this error:
Diagnostics:
pulumi:pulumi:Stack (vmtep-catalog.ndev):
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x58 pc=0x15b271e]
goroutine 33 [running]:
main.(*pythonLanguageHost).GetRequiredPlugins(0x0?, {0x1ef98b0, 0xc0006a1ec0}, 0xc00007f560)
/home/runner/work/pulumi/pulumi/sdk/python/cmd/pulumi-language-python/main.go:221 +0x3e
<http://github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler.func1({0x1ef98b0|github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler.func1({0x1ef98b0>, 0xc0006a1ec0}, {0x18d6340?, 0xc00007f560})
/home/runner/work/pulumi/pulumi/sdk/proto/go/language_grpc.pb.go:295 +0x72
<http://github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1ef98b0|github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1ef98b0>, 0xc0006a1650}, {0x18d6340, 0xc00007f560}, 0xc00065ba60, 0xc000655668)
/home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3d0
<http://github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler({0x195ea60|github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler({0x195ea60>?, 0xc0006a0330}, {0x1ef98b0, 0xc0006a1650}, 0xc0006eb260, 0xc00065b5c0)
/home/runner/work/pulumi/pulumi/sdk/proto/go/language_grpc.pb.go:297 +0x135
<http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc0006945a0|google.golang.org/grpc.(*Server).processUnaryRPC(0xc0006945a0>, {0x1f00df0, 0xc00052b380}, 0xc0006ba360, 0xc0006a03c0, 0x2bca460, 0x0)
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:1358 +0xe15
<http://google.golang.org/grpc.(*Server).handleStream(0xc0006945a0|google.golang.org/grpc.(*Server).handleStream(0xc0006945a0>, {0x1f00df0, 0xc00052b380}, 0xc0006ba360, 0x0)
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:1735 +0x9e7
<http://google.golang.org/grpc.(*Server).serveStreams.func1.1()|google.golang.org/grpc.(*Server).serveStreams.func1.1()>
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:970 +0xbb
created by <http://google.golang.org/grpc.(*Server).serveStreams.func1|google.golang.org/grpc.(*Server).serveStreams.func1> in goroutine 29
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:981 +0x145
Any advice as to why it suddenly broke?
a
Hey Seth, sorry you hit this error. This looks like a bug on our side. It'd be amazing if you could raise an issue with reproduction steps so we can fix it!
b
Hello, I've raised the issue here. https://github.com/pulumi/pulumi/issues/15022 Thank you
a
Thanks for that Seth! Could you also include some information on how to reproduce the error? Do you know what in your pulumi program causes the issue? It'd help us a ton if we can reproduce the problem.
b
Unfortunately I don't know exactly where in the code it's breaking. I actually just tried to do "pulumi about" with pulumi v3.75 and it gets on the same error with pulumi about which I assume is not related to the python program? The python program was also working about 1-2 weeks ago, but suddenly broke.
We haven't made changes to the python program in a couple months.
a
I see, I'm sorry to hear that. Could I ask you to add some verbose logs so we get an idea of what's going on? I think
pulumi about
crashing is particularly unexpected. There's instructions on how to enable that here https://www.pulumi.com/docs/support/troubleshooting/#verbose-logging
b
Here's the verbose logs to when "pulumi up" fails.
W0103 18:23:10.686558    2001 log.go:99] A new version of Pulumi is available. To upgrade from version '3.75.0' to '3.99.0', visit <https://pulumi.com/docs/install/> for manual instructions and release notes.
I0103 18:23:11.901958    2001 log.go:81] *** Starting Update(preview=true) ***
I0103 18:23:11.902230    2001 log.go:81] gatherPluginsFromProgram(): gathering plugins from language host
I0103 18:23:11.902430    2001 log.go:81] GetPluginPath(language, python, <nil>): found on $PATH /usr/local/bin/pulumi-language-python
I0103 18:23:11.902469    2001 log.go:81] Launching plugin 'python' from '/usr/local/bin/pulumi-language-python' with args: -virtualenv=venv,-root=/home/vsts/work/r1/a/_catalog/pulumi/deploy,127.0.0.1:46121
`I0103 182311.946561 2001 log.go:81] eventSink::Infoerr(<{%reset%}>I0103 182311.946480 2009 log.go:81] language host identified executor from path: `/usr/local/bin/pulumi-language-python-exec``
<{%reset%}>)
I0103 18:23:11.948218    2001 log.go:81] langhost[python].GetPluginInfo() executing
I0103 18:23:11.948564    2001 log.go:81] langhost[python].GetRequiredPlugins(proj=vmtep,pwd=/home/vsts/work/r1/a/_catalog/pulumi/deploy,program=.) executing
I0103 18:23:11.951277    2001 log.go:81] eventSink::Infoerr(<{%reset%}>panic: runtime error: invalid memory address or nil pointer dereference
<{%reset%}>)
I0103 18:23:11.951297    2001 log.go:81] eventSink::Infoerr(<{%reset%}>[signal SIGSEGV: segmentation violation code=0x1 addr=0x58 pc=0x15b271e]
<{%reset%}>)
I0103 18:23:11.951310    2001 log.go:81] eventSink::Infoerr(<{%reset%}>goroutine 34 [running]:
<{%reset%}>)
I0103 18:23:11.951316    2001 log.go:81] eventSink::Infoerr(<{%reset%}>main.(*pythonLanguageHost).GetRequiredPlugins(0x0?, {0x1ef98b0, 0xc000aabb00}, 0xc00003fb00)
<{%reset%}>)
I0103 18:23:11.951322    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/work/pulumi/pulumi/sdk/python/cmd/pulumi-language-python/main.go:221 +0x3e
<{%reset%}>)
I0103 18:23:11.951331    2001 log.go:81] eventSink::Infoerr(<{%reset%}><http://github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler.func1({0x1ef98b0|github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler.func1({0x1ef98b0>, 0xc000aabb00}, {0x18d6340?, 0xc00003fb00})
<{%reset%}>)
I0103 18:23:11.951338    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/work/pulumi/pulumi/sdk/proto/go/language_grpc.pb.go:295 +0x72
<{%reset%}>)
I0103 18:23:11.951346    2001 log.go:81] eventSink::Infoerr(<{%reset%}><http://github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1ef98b0|github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1ef98b0>, 0xc000aab290}, {0x18d6340, 0xc00003fb00}, 0xc000696380, 0xc000011938)
<{%reset%}>)
I0103 18:23:11.951353    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3d0
<{%reset%}>)
I0103 18:23:11.951358    2001 log.go:81] eventSink::Infoerr(<{%reset%}><http://github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler({0x195ea60|github.com/pulumi/pulumi/sdk/v3/proto/go._LanguageRuntime_GetRequiredPlugins_Handler({0x195ea60>?, 0xc0005ca7b0}, {0x1ef98b0, 0xc000aab290}, 0xc0006b4460, 0xc0001815e0)
<{%reset%}>)
I0103 18:23:11.951364    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/work/pulumi/pulumi/sdk/proto/go/language_grpc.pb.go:297 +0x135
<{%reset%}>)
I0103 18:23:11.951370    2001 log.go:81] eventSink::Infoerr(<{%reset%}><http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc000194780|google.golang.org/grpc.(*Server).processUnaryRPC(0xc000194780>, {0x1f00df0, 0xc000152ea0}, 0xc0004b6240, 0xc0005ca840, 0x2bca460, 0x0)
<{%reset%}>)
I0103 18:23:11.951376    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:1358 +0xe15
<{%reset%}>)
I0103 18:23:11.951383    2001 log.go:81] eventSink::Infoerr(<{%reset%}><http://google.golang.org/grpc.(*Server).handleStream(0xc000194780|google.golang.org/grpc.(*Server).handleStream(0xc000194780>, {0x1f00df0, 0xc000152ea0}, 0xc0004b6240, 0x0)
<{%reset%}>)
I0103 18:23:11.951389    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:1735 +0x9e7
<{%reset%}>)
I0103 18:23:11.951394    2001 log.go:81] eventSink::Infoerr(<{%reset%}><http://google.golang.org/grpc.(*Server).serveStreams.func1.1()|google.golang.org/grpc.(*Server).serveStreams.func1.1()>
<{%reset%}>)
I0103 18:23:11.951412    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:970 +0xbb
<{%reset%}>)
I0103 18:23:11.951418    2001 log.go:81] eventSink::Infoerr(<{%reset%}>created by <http://google.golang.org/grpc.(*Server).serveStreams.func1|google.golang.org/grpc.(*Server).serveStreams.func1> in goroutine 15
<{%reset%}>)
I0103 18:23:11.951424    2001 log.go:81] eventSink::Infoerr(<{%reset%}> /home/runner/go/pkg/mod/google.golang.org/grpc@v1.57.1/server.go:981 +0x145
<{%reset%}>)
I0103 18:23:11.954157    2001 log.go:81] langhost[python].GetRequiredPlugins(proj=vmtep,pwd=/home/vsts/work/r1/a/_catalog/pulumi/deploy,program=.) failed: err=error reading from server: EOF
I0103 18:23:11.954274    2001 log.go:81] *** Update(preview=true) complete ***
I0103 18:23:12.024407    2001 log.go:81] defaultSink::Error(error: failed to discover plugin requirements: error reading from server: EOF
<http://github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin.GetRequiredPlugins|github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin.GetRequiredPlugins>
/home/runner/work/pulumi/pulumi/sdk/go/common/resource/plugin/host.go:593
<http://github.com/pulumi/pulumi/pkg/v3/engine.gatherPluginsFromProgram|github.com/pulumi/pulumi/pkg/v3/engine.gatherPluginsFromProgram>
/home/runner/work/pulumi/pulumi/pkg/engine/plugins.go:123
<http://github.com/pulumi/pulumi/pkg/v3/engine.installPlugins|github.com/pulumi/pulumi/pkg/v3/engine.installPlugins>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:227
<http://github.com/pulumi/pulumi/pkg/v3/engine.newUpdateSource|github.com/pulumi/pulumi/pkg/v3/engine.newUpdateSource>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:378
<http://github.com/pulumi/pulumi/pkg/v3/engine.newDeployment|github.com/pulumi/pulumi/pkg/v3/engine.newDeployment>
/home/runner/work/pulumi/pulumi/pkg/engine/deployment.go:189
<http://github.com/pulumi/pulumi/pkg/v3/engine.update|github.com/pulumi/pulumi/pkg/v3/engine.update>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:454
<http://github.com/pulumi/pulumi/pkg/v3/engine.Update|github.com/pulumi/pulumi/pkg/v3/engine.Update>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:194
<http://github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).apply|github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).apply>
/home/runner/work/pulumi/pulumi/pkg/backend/filestate/backend.go:1059
<http://github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPrompt|github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPrompt>
/home/runner/work/pulumi/pulumi/pkg/backend/apply.go:116
<http://github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPromptThenExecute|github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPromptThenExecute>
/home/runner/work/pulumi/pulumi/pkg/backend/apply.go:248
<http://github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).Update|github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).Update>
/home/runner/work/pulumi/pulumi/pkg/backend/filestate/backend.go:933
<http://github.com/pulumi/pulumi/pkg/v3/backend.UpdateStack|github.com/pulumi/pulumi/pkg/v3/backend.UpdateStack>
/home/runner/work/pulumi/pulumi/pkg/backend/stack.go:95
<http://github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localStack).Update|github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localStack).Update>
/home/runner/work/pulumi/pulumi/pkg/backend/filestate/stack.go:89
main.newUpCmd.func1
/home/runner/work/pulumi/pulumi/pkg/cmd/pulumi/up.go:170
main.newUpCmd.func3
/home/runner/work/pulumi/pulumi/pkg/cmd/pulumi/up.go:491
<http://github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil.RunResultFunc.func1|github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil.RunResultFunc.func1>
/home/runner/work/pulumi/pulumi/sdk/go/common/util/cmdutil/exit.go:112
<http://github.com/spf13/cobra.(*Command).execute|github.com/spf13/cobra.(*Command).execute>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:944
<http://github.com/spf13/cobra.(*Command).ExecuteC|github.com/spf13/cobra.(*Command).ExecuteC>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068
<http://github.com/spf13/cobra.(*Command).Execute|github.com/spf13/cobra.(*Command).Execute>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
main.main
/home/runner/work/pulumi/pulumi/pkg/cmd/pulumi/main.go:56
runtime.main
/opt/hostedtoolcache/go/1.20.5/x64/src/runtime/proc.go:250
runtime.goexit
/opt/hostedtoolcache/go/1.20.5/x64/src/runtime/asm_amd64.s:1598
)
error: failed to discover plugin requirements: error reading from server: EOF
<http://github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin.GetRequiredPlugins|github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin.GetRequiredPlugins>
/home/runner/work/pulumi/pulumi/sdk/go/common/resource/plugin/host.go:593
<http://github.com/pulumi/pulumi/pkg/v3/engine.gatherPluginsFromProgram|github.com/pulumi/pulumi/pkg/v3/engine.gatherPluginsFromProgram>
/home/runner/work/pulumi/pulumi/pkg/engine/plugins.go:123
<http://github.com/pulumi/pulumi/pkg/v3/engine.installPlugins|github.com/pulumi/pulumi/pkg/v3/engine.installPlugins>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:227
<http://github.com/pulumi/pulumi/pkg/v3/engine.newUpdateSource|github.com/pulumi/pulumi/pkg/v3/engine.newUpdateSource>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:378
<http://github.com/pulumi/pulumi/pkg/v3/engine.newDeployment|github.com/pulumi/pulumi/pkg/v3/engine.newDeployment>
/home/runner/work/pulumi/pulumi/pkg/engine/deployment.go:189
<http://github.com/pulumi/pulumi/pkg/v3/engine.update|github.com/pulumi/pulumi/pkg/v3/engine.update>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:454
<http://github.com/pulumi/pulumi/pkg/v3/engine.Update|github.com/pulumi/pulumi/pkg/v3/engine.Update>
/home/runner/work/pulumi/pulumi/pkg/engine/update.go:194
<http://github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).apply|github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).apply>
/home/runner/work/pulumi/pulumi/pkg/backend/filestate/backend.go:1059
<http://github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPrompt|github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPrompt>
/home/runner/work/pulumi/pulumi/pkg/backend/apply.go:116
<http://github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPromptThenExecute|github.com/pulumi/pulumi/pkg/v3/backend.PreviewThenPromptThenExecute>
/home/runner/work/pulumi/pulumi/pkg/backend/apply.go:248
<http://github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).Update|github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localBackend).Update>
/home/runner/work/pulumi/pulumi/pkg/backend/filestate/backend.go:933
<http://github.com/pulumi/pulumi/pkg/v3/backend.UpdateStack|github.com/pulumi/pulumi/pkg/v3/backend.UpdateStack>
/home/runner/work/pulumi/pulumi/pkg/backend/stack.go:95
<http://github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localStack).Update|github.com/pulumi/pulumi/pkg/v3/backend/filestate.(*localStack).Update>
/home/runner/work/pulumi/pulumi/pkg/backend/filestate/stack.go:89
main.newUpCmd.func1
/home/runner/work/pulumi/pulumi/pkg/cmd/pulumi/up.go:170
main.newUpCmd.func3
/home/runner/work/pulumi/pulumi/pkg/cmd/pulumi/up.go:491
<http://github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil.RunResultFunc.func1|github.com/pulumi/pulumi/sdk/v3/go/common/util/cmdutil.RunResultFunc.func1>
/home/runner/work/pulumi/pulumi/sdk/go/common/util/cmdutil/exit.go:112
<http://github.com/spf13/cobra.(*Command).execute|github.com/spf13/cobra.(*Command).execute>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:944
<http://github.com/spf13/cobra.(*Command).ExecuteC|github.com/spf13/cobra.(*Command).ExecuteC>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068
<http://github.com/spf13/cobra.(*Command).Execute|github.com/spf13/cobra.(*Command).Execute>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
main.main
/home/runner/work/pulumi/pulumi/pkg/cmd/pulumi/main.go:56
runtime.main
/opt/hostedtoolcache/go/1.20.5/x64/src/runtime/proc.go:250
runtime.goexit
/opt/hostedtoolcache/go/1.20.5/x64/src/runtime/asm_amd64.s:1598
a
ok, amazing I'll post the information in the issue and someone from the team will get back to you there. Thanks!
b
Great, thank you!!