https://pulumi.com logo
Join Slack
Powered by
It says this for every pending update to the clust...
# general
c
It says this for every pending update to the cluster, even though it can somehow detect that there are diffs?...
l
Diffs are detected between the state and the code. It doesn't need remote access for that.
c
Ah, that makes sense
Think it might have something to do with DO rotating kubeconfigs: https://github.com/pulumi/examples/blob/master/digitalocean-ts-k8s/index.ts
Although now unfortunately I swapped out the DO token and even that won't authenticate haha
So pulumi just refuses to authenticate to digtialocean, even though using the exact same token in curl works. Not sure what's going on I've tried setting the DIGITALOCEAN_TOKEN environment variable, setting pulumi config, and using an explicit provider Wish I could see what HTTP requests it's making
l
strace?
c
I'll try now, but that's a bit granular haha
Is there any way you can actually see network requests with strace?
l
Yes, but you need to know the kernel api call and grep for that. I'm not that low-level. There's bound to be an easier solution...
c
Yeah that's what I imagined hehe
Some kinda write syscall
But in a program like this, there's bound to be a lot of them
l
This looks like the same use case? https://unix.stackexchange.com/a/293032/57957
nethogs. Never heard of it...
c
Was gonna install wireshark, this seems useful
Doesn't give me any content I'm afraid 😕, just shows usage
l
Wireshark is a good option too. Generally most useful when there's an expert around..
c
I happen to be that expert, so that's convenient
👍 1
Hmm, it's TLS traffic, which I forgot about. This is gonna take a while 😕.
I actually don't know how I would MITM this, since it's the pulumi CLI connecting directly and not e.g. my browser.
@little-cartoon-10569 I got it to print the api requests with TF_LOG=trace pulumi preview -v=3 --debug. For some reason, either it's not attaching the credentials in the Authorization header, or it's redacting them from the output. Do you know how I can find out which it is? Or what I would need to check if it were the former?
Copy code
================================================================================
The Pulumi CLI encountered a fatal error. This is a bug!
We would appreciate a report: <https://github.com/pulumi/pulumi/issues/>
Please provide all of the below text in your report.
================================================================================
Pulumi Version:   v2.22.0
Go Version:       go1.15.8
Go Compiler:      gc
Architecture:     amd64
Operating System: linux
Panic:            fatal: An assertion has failed: Expected to find parent node 'urn:pulumi:global::main-shared::kubernetes:core/v1:Namespace::cert-manager' in checkpoint tree nodes

goroutine 1 [running]:
runtime/debug.Stack(0xc00057d210, 0x197a4c0, 0xc000de3060)
        /Users/runner/hostedtoolcache/go/1.15.8/x64/src/runtime/debug/stack.go:24 +0x9f
main.panicHandler()
        /Users/runner/work/pulumi/pulumi/pkg/cmd/pulumi/main.go:29 +0x73
panic(0x197a4c0, 0xc000de3060)
        /Users/runner/hostedtoolcache/go/1.15.8/x64/src/runtime/panic.go:969 +0x1b9
<http://github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)|github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)>
        /Users/runner/work/pulumi/pulumi/sdk/go/common/util/contract/failfast.go:23
<http://github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.Assertf|github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.Assertf>(0xc0011a0500, 0x1d31729, 0x3a, 0xc00057d4f8, 0x1, 0x1)
        /Users/runner/work/pulumi/pulumi/sdk/go/common/util/contract/assert.go:33 +0x1a5
<http://github.com/pulumi/pulumi/pkg/v2/operations.makeResourceTreeMap(0xc0000a8a00|github.com/pulumi/pulumi/pkg/v2/operations.makeResourceTreeMap(0xc0000a8a00>, 0xf, 0x10, 0xc0011bacc0, 0x0)
        /Users/runner/work/pulumi/pulumi/pkg/operations/resources.go:77 +0x449
<http://github.com/pulumi/pulumi/pkg/v2/operations.NewResourceTree(...)|github.com/pulumi/pulumi/pkg/v2/operations.NewResourceTree(...)>
        /Users/runner/work/pulumi/pulumi/pkg/operations/resources.go:45
<http://github.com/pulumi/pulumi/pkg/v2/backend/filestate.GetLogsForTarget|github.com/pulumi/pulumi/pkg/v2/backend/filestate.GetLogsForTarget>(0xc0011bac90, 0xc0004ded20, 0x0, 0x0, 0xc0004e6500, 0xc0004daf00, 0x1fef140, 0xc0004daf30, 0xc0011bac90)
        /Users/runner/work/pulumi/pulumi/pkg/backend/filestate/backend.go:672 +0x9e
<http://github.com/pulumi/pulumi/pkg/v2/backend/httpstate.(*cloudBackend).GetLogs(0xc000cdd500|github.com/pulumi/pulumi/pkg/v2/backend/httpstate.(*cloudBackend).GetLogs(0xc000cdd500>, 0x2022b60, 0xc000130010, 0x203fc60, 0xc0000a8000, 0xc0004daf00, 0x1fef140, 0xc0004daf30, 0xc0004ded20, 0x0, ...)
        /Users/runner/work/pulumi/pulumi/pkg/backend/httpstate/backend.go:1201 +0x10d
<http://github.com/pulumi/pulumi/pkg/v2/backend.GetStackLogs(0x2022b60|github.com/pulumi/pulumi/pkg/v2/backend.GetStackLogs(0x2022b60>, 0xc000130010, 0x203fc60, 0xc0000a8000, 0xc0004daf00, 0x1fef140, 0xc0004daf30, 0xc0004ded20, 0x0, 0x0, ...)
        /Users/runner/work/pulumi/pulumi/pkg/backend/stack.go:118 +0xdf
<http://github.com/pulumi/pulumi/pkg/v2/backend/httpstate.(*cloudStack).GetLogs(0xc0000a8000|github.com/pulumi/pulumi/pkg/v2/backend/httpstate.(*cloudStack).GetLogs(0xc0000a8000>, 0x2022b60, 0xc000130010, 0xc0004daf00, 0x1fef140, 0xc0004daf30, 0xc0004ded20, 0x0, 0x0, 0x0, ...)
        /Users/runner/work/pulumi/pulumi/pkg/backend/httpstate/stack.go:170 +0xa6
main.newLogsCmd.func1(0xc000cdf8c0, 0x2c3ddc0, 0x0, 0x0, 0x11, 0x11)
        /Users/runner/work/pulumi/pulumi/pkg/cmd/pulumi/logs.go:100 +0x446
<http://github.com/pulumi/pulumi/sdk/v2/go/common/util/cmdutil.RunFunc.func1(0xc000cdf8c0|github.com/pulumi/pulumi/sdk/v2/go/common/util/cmdutil.RunFunc.func1(0xc000cdf8c0>, 0x2c3ddc0, 0x0, 0x0, 0x0, 0x0)
        /Users/runner/work/pulumi/pulumi/sdk/go/common/util/cmdutil/exit.go:96 +0x51
<http://github.com/pulumi/pulumi/sdk/v2/go/common/util/cmdutil.RunResultFunc.func1(0xc000cdf8c0|github.com/pulumi/pulumi/sdk/v2/go/common/util/cmdutil.RunResultFunc.func1(0xc000cdf8c0>, 0x2c3ddc0, 0x0, 0x0)
        /Users/runner/work/pulumi/pulumi/sdk/go/common/util/cmdutil/exit.go:112 +0x6b
<http://github.com/spf13/cobra.(*Command).execute(0xc000cdf8c0|github.com/spf13/cobra.(*Command).execute(0xc000cdf8c0>, 0x2c3ddc0, 0x0, 0x0, 0xc000cdf8c0, 0x2c3ddc0)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x2c2
<http://github.com/spf13/cobra.(*Command).ExecuteC(0xc000346dc0|github.com/spf13/cobra.(*Command).ExecuteC(0xc000346dc0>, 0x15, 0x0, 0x0)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x375
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main()
        /Users/runner/work/pulumi/pulumi/pkg/cmd/pulumi/main.go:48 +0x4d
^ upon running pulumi logs
I think my stack is cursed
l
Apparently it's a bug, it says so!
I think you'll need someone with more knowledge than me. I wonder if @billowy-army-68599 is still online? It's after working hours where he is..
c
I can't continue working until I figure out what's wrong with this stack 😞
l
I'd try to clone to new directory and remove (or move) ~/.pulumi. Maybe there's something cached that you can flush by doing that.
c
I went and imported an old pulumi stack version and it will literally 401 on pulumi refresh, but pulumi up works just fine.
Ok, I think the issue is fixed. I did a series of state-rollbacks, and ups+refreshes, and somehow I ended up with a working stack
Unfortunately I can't tell you what I did, it was kinda just random stuff
l
Ah. I'm sorry to have to tell you, but you have switched from "engineering" to "programming".
c
😜
I have another new problem after fixing these series of issues actually I copied the code from the example to handle rotating kubeconfigs:
Copy code
I have a problem; I'm doing this:
export const kubeConfig = cluster.status.apply(status => {
    if (status === "running") {
        const clusterDataSource = cluster.name.apply(name => doc.getKubernetesCluster({name}));
        return clusterDataSource.kubeConfigs[0].rawConfig;
    } else {
        return cluster.kubeConfigs[0].rawConfig;
    }
});
This works (as far as I can tell), but exports a non-secret output
I can see the kubeconfig in the logs
Not sure if this is a problem with the digitalocean datasource ro somehow the operations cause that
12 Views
Open in Slack
PreviousNext
Powered by