Hi there! I am using Pulumi to provision my GCP C...
# general
f
Hi there! I am using Pulumi to provision my GCP Cloud DNS zone, records and service accounts. Then I am using the command.remote library to bootstrap a K3s cluster on my rpi cluster. When provisioning a k3s cluster, the kubeconfig is written on the control node in a directory. I am using command.remote to cat the contents out to stdout. I am then using the stdout output to build a Kubernetes.Provider. I then pass that provider to kubernetes commands to apply yaml files. When I run "pulumi up" the preview fails with the following error:
Copy code
error: an unhandled error occurred: program failed:
    1 error occurred:
        * decoding YAML: rpc error: code = Unknown desc = invocation of kubernetes:yaml:decode returned an error: failed to initialize discovery client: The gcp auth plugin has been removed.
    Please use the "gke-gcloud-auth-plugin" kubectl/client-go credential plugin instead.
    See <https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke> for further details
I figured out this is caused because during "pulumi preview" the kubeconfig does not exist on the server it returns a empty string. So the provider cannot create with an empty kubeconfig. I have used DependsOn with all the previous commands required for it to work, but it seems that it always executes the command and tries to build the provider off an empty string. If I bypass preview with "pulumi up -f", everything works without issue. Sample Code:
Copy code
func (k *K3sCluster) exportKubeProvider(node K3sClusterNode, dependsOn ...pulumi.Resource) (*remote.Command, *kubernetes.Provider, error) {
	t := remote.ConnectionArgs{}
	t.Host = pulumi.String(node.IP)
	t.Password = pulumi.String(nodePassword)
	t.User = pulumi.String(nodeUsername)

	c := remote.CommandArgs{
		Connection: t.ToConnectionOutput(),
		Create:     pulumi.String("sudo cat /etc/rancher/k3s/k3s.yaml").ToStringPtrOutput(),
	}

	a, err := remote.NewCommand(k.ctx, fmt.Sprintf("get-k3s-kubeconfig-%s", node.Name), &c, pulumi.DependsOn(dependsOn))

	if err != nil {
		return nil, nil, err
	}

	dependsOn = append(dependsOn, a)

	provider, err := kubernetes.NewProvider(k.ctx, "k3s-provider", &kubernetes.ProviderArgs{
		Kubeconfig: a.Stdout.ApplyT(func(config string) string {
			config = strings.Replace(config, "<https://127.0.0.1:6443>", fmt.Sprintf("<https://%s:6443>", node.IP), -1)

			return config
		}).(pulumi.StringOutput).ToStringPtrOutput(),
	}, pulumi.DependsOn(dependsOn))

	if err != nil {
		return nil, nil, err
	}

	//k.ctx.Export("kubeconfig", config)

	dependsOn = append(dependsOn, provider)

	ingress, err := yaml.NewConfigFile(k.ctx, "k8s-ingress-nginx", &yaml.ConfigFileArgs{
		File: "pkg/kube/yaml/ingress-nginx.yaml",
	}, pulumi.Providers(provider), pulumi.DependsOn(dependsOn))
	if err != nil {
		return nil, nil, err
	}

	dependsOn = append(dependsOn, ingress)

	_, err = yaml.NewConfigFile(k.ctx, "k8s-ingress-nginx-load-balancer", &yaml.ConfigFileArgs{
		File: "pkg/kube/yaml/loadbalancer-nginx.yaml",
	}, pulumi.Providers(provider), pulumi.DependsOn(dependsOn))

	if err != nil {
		return nil, nil, err
	}

	return a, provider, nil
}
g
you may get a better answer in #google-cloud or #golang
f
Thanks