I am trying out the new Pulumi 3.23.0 capability of disabling the default providers, and have this in my stack:

pulumi:disable-default-providers:
  - aws
  - kubernetes

This seems to work well for AWS, but I am having an odd issue with the Kubernetes one. Specifically, I have the following resource:

_, err = yaml.NewConfigFile(ctx, "certmanager-deploy-file", &yaml.ConfigFileArgs{
		File: "./cert-manager.yaml",
		Transformations: []yaml.Transformation{
			// We need to make two modifications:
			// 1. Add the role ARN for IRSA
			// 2. Set the fsGroup for IRSA token mapping
			// Docs here: <https://cert-manager.io/docs/configuration/acme/dns01/route53/#eks-iam-role-for-service-accounts-irsa>
			func(state map[string]interface{}, opts ...pulumi.ResourceOption) {
				metadata := state["metadata"].(map[string]interface{})
				name := metadata["name"]
				if state["kind"] == "ServiceAccount" && name == "cert-manager" {
					var annotations map[string]interface{}
					if v, ok := metadata["annotations"]; !ok {
						annotations = make(map[string]interface{})
						metadata["annotations"] = annotations
					} else {
						annotations = v.(map[string]interface{})
					}
					annotations["<http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>"] = irsaRole.Arn
				}
				if state["kind"] == "Deployment" && name == "cert-manager" {
					deploymentSpec := state["spec"].(map[string]interface{})
					template := deploymentSpec["template"].(map[string]interface{})
					podSpec := template["spec"].(map[string]interface{})
					podSpec["securityContext"] = map[string]interface{}{
						"fsGroup": 1001,
					}
				}

			},
		},
	}, pulumi.DependsOn([]pulumi.Resource{irsaRole}), pulumi.Provider(eksConfig.Provider))
	if err != nil {
		return nil, err
	}

Where

eksConfig.Provider

is constructed as the result of an

eks.Cluster

creation:

k8sProvider, err := providers.NewProvider(ctx, "k8s-ssa-provider", &providers.ProviderArgs{
		Kubeconfig: kubeconfig,
	})
	if err != nil {
		return nil, err
	}

When I run this with the default Kubernetes one disabled, I get this error:

error: program failed: 1 error occurred:
    	* decoding YAML: rpc error: code = Unknown desc = unknown provider ''
    exit status 1

There is not any more info in the logs even if I set logging to 9. If I enable the Kubernetes default provider, it works just fine, even though I am passing an explicit provider here. Is this a bug or am I doing something unexpected here?

@bored-table-20691 👋 I saw this & commented as well. looks like we have two ways we could implement this

@orange-policeman-59119 hi! Thank you for responding. Do you still think a repro is needed? I don’t have one handy as it’s tied to our actual stack, but I can give it a shot.

I don't think so, I see exactly in the code where we stopped passing the provider down. Unintended consequence of disabling default providers is discovering this edge case where some of our resources are actually collections-of-resources, and we need to figure out which options should cascade down (all? some?)

Indeed.

I think I'm getting the hang of navigating our codebase a few weeks in though, will keep that ticket updated and next week will chat with @ancient-policeman-24615 about a solution. He just shipped the disable-default-providers option, so I imagine he'll know which route we should go. The team has been looking at refactoring providers in general to get rid of some tech debt, this is a good example for that work. I'll flag it to them

There was an interesting side effect of disabling the default provider, which is

iam.GetPolicyDocument

errors in the same weird way if you don’t pass in a provider and the default provider is disabled. This is even though this call conceptually does not need a provider