Any idea how am I supposed to debug this?

panic: interface conversion: interface {} is nil, not map[string]interface {}
    goroutine 44 [running]:
    <http://github.com/hashicorp/go-cty/cty.Value.GetAttr({{{0x10ea25960|github.com/hashicorp/go-cty/cty.Value.GetAttr({{{0x10ea25960>?, 0x1400a4c5210?}}, {0x0?, 0x0?}}, {0x10a4719ce, 0x8})
        /home/runner/go/pkg/mod/github.com/hashicorp/go-cty@v1.4.1-0.20200414143053-d3edf31b6320/cty/value_ops.go:711 +0x258
    <http://github.com/hashicorp/terraform-provider-aws/internal/provider.tagsResourceInterceptor.run({0x1400138b200|github.com/hashicorp/terraform-provider-aws/internal/provider.tagsResourceInterceptor.run({0x1400138b200>?, 0x10e8c6c40?, 0x10e8c6c38?}, {0x10ea25488, 0x1400e55ced0}, {0x10ea36290, 0x1400a1b5980}, {0x10e866300?, 0x1400220cb60?}, 0x1, ...)
        /home/runner/work/pulumi-aws/pulumi-aws/upstream/internal/provider/intercept.go:250 +0xea4
    <http://github.com/hashicorp/terraform-provider-aws/internal/provider.New.(*wrappedResource).Update.interceptedHandler[...].func10(0x0|github.com/hashicorp/terraform-provider-aws/internal/provider.New.(*wrappedResource).Update.interceptedHandler[...].func10(0x0>?, {0x10e866300?, 0x1400220cb60?})
        /home/runner/work/pulumi-aws/pulumi-aws/upstream/internal/provider/intercept.go:100 +0x11c
    <http://github.com/hashico|github.com/hashico>

I had a bunch of that last night; i was using

TF_LOG=TRACE pulumi up --logtostderr --logflow -v=10 2> out.txt

no I do not have anything large

so what I did was go to bottom and work up; it's quite noisy but at somme point there start to be error trace looking things, you keep working up and you'll run into this panic

the thing is that I've used the same code 2 days ago without issues so now I do not know if it's something on my Mac or something else

damn... i had "fun" like that today

thanks @gentle-application-59272 I managed to spot the problem, it has something to do with

OIDC

connector for IAM, I have a

tls

module to get the cert thumbprints but somehow it wanted to change the thumbprint and failed at that.

Folks I am sorry you are affected by this issue. This seems to be the same as https://github.com/pulumi/pulumi-aws/issues/2904 - we generally treat panics as P1s and are trying to make progress here but it has been difficult to get a reliable repro. @bright-wall-54528 has been trying to narrow down a repro but has not succeeded yet.

I will try to recreate it. I am quite positive I should be able to. Since Gitlab provides 2 thumbprints and OIDC provider needs only one. The panic likely came from aws provider trying to replace the thumbprint. https://www.pulumi.com/registry/packages/aws/api-docs/iam/openidconnectprovider/#inputs

@great-sunset-355 I happen to be working on GL OIDC for a KubeCon demo. Where did you see 2 certs? (I'm working against the GL SaaS, not self-hosted.)

@stocky-restaurant-98004 This is a piece of my OIDC component where I use the

tls

to get the cert. Certificates are always an array and 90% of internet examples take the first item of the array. IMO you should get the same result for

<https://gitlab.com/oauth/discovery/keys>

as well as

<tls://gitlab.com:443>

(haven't tested this) Then during the provider / package update (not sure where) pulumi wanted to replace 2nd item in the array but to my surprise, the values were the same, perhaps the order was swapped or who knows what went wrong. In the end that triggered a resource update which led to the panic. But I haven't verified that yet.

this.gitlabUrl = "<https://gitlab.com>"
// maybe use tls url "<tls://gitlab.com:443>" -> `tls://${this.gitlabUrl.host}:443`
    const cert = tls.getCertificateOutput({
      url: new URL(`/oauth/discovery/keys`, this.gitlabUrl).toString(),
    });
    const audiences: string[] = [this.gitlabUrl.origin, ...(args.audiences ?? [])];

    // this is always like this in the for IAM condition
    const cond = "gitlab.com";



    const gitlabOidcProvider = new aws.iam.OpenIdConnectProvider(
      `gitlab-oidc-provider`,
      {
        url: this.gitlabUrl.origin,
        clientIdLists: audiences,
        thumbprintLists: cert.certificates.apply((x) => x.map((o) => o.sha1Fingerprint)),
      },
      { deleteBeforeReplace: true, parent }
    );

thumbprints

[
  "b3dd7606d2b5a8b4a13771dbecc9ee1cecafa38a",
  "a88ed77cf52a6b6185a436f2dec4270e9d456721"
]

@great-sunset-355 The backend in AWS will lowercase the thumbprints if you did not enter them that way, which can cause a spurious diff.

Oh, interesting observation, I think the original thumbprints were lowercase