I’m missing Pulumi datasources in the kubernetes provider. Is there any specific reason why this provider doesn’t have any

get

functions? E.g.

getService

that I can feed some labels or annotations?

Is there a good way to wait for services to be ready when they are deployed using

k8s.yaml.ConfigFile

? I tried using dependsOn but that doesnt wait long enough.

Hi! I am trying to do this to fetch a chart from https://helm.dask.org/.

dask = helm.v3.Chart(
        f'dask-helm-{resource_suffix}',
        config=k8s.helm.v3.ChartOpts(
            chart='dask',
            repo='dask',
            version='4.5.7',
            fetch_opts=k8s.helm.v3.FetchOpts(
                repo='<https://helm.dask.org>',
            ),
        ),
        opts=pulumi.ResourceOptions(
            providers={ 'kubernetes': k8s_provider },
        ),
    )

But I get this error:

error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/home/alexandre/.pyenv/versions/daks/lib/python3.9/site-packages/pulumi/runtime/invoke.py", line 110, in do_invoke
        return monitor.Invoke(req)
      File "/home/alexandre/.pyenv/versions/daks/lib/python3.9/site-packages/grpc/_channel.py", line 923, in __call__
        return _end_unary_response_blocking(state, call, False, None)
      File "/home/alexandre/.pyenv/versions/daks/lib/python3.9/site-packages/grpc/_channel.py", line 826, in _end_unary_response_blocking
        raise _InactiveRpcError(state)
    grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    	status = StatusCode.UNKNOWN
    	details = "invocation of kubernetes:helm:template returned an error: failed to generate YAML for specified Helm chart: failed to pull chart: chart "dask/dask" version "4.5.7" not found in <https://helm.dask.org> repository"
    	debug_error_string = "{"created":"@1615537561.526221208","description":"Error received from peer ipv4:127.0.0.1:38703","file":"src/core/lib/surface/call.cc","file_line":1067,"grpc_message":"invocation of kubernetes:helm:template returned an error: failed to generate YAML for specified Helm chart: failed to pull chart: chart "dask/dask" version "4.5.7" not found in <https://helm.dask.org> repository","grpc_status":2}"

I tried many variations of the

Chart

parameters without success. Any idea? Thank you!

Hi. I'm trying to add an annotation on a service account. The code seems to work fine, at least I can see that the annotation gets added by

setIamRoleArn

method but Pulumi doesn't detect any changes. I wonder if annotations attribute is being ignores as there is a high change of it being updated outside of Pulumi? If so is there a way to force adding specific annotation? Thank you!

private deployCloudWatchAgentDaemonset(): k8s.yaml.ConfigFile {
    let serviceAccounts = this.serviceAccounts;

    return new k8s.yaml.ConfigFile('cloudwatch-agent-setup', {
        file: ContainerInsights.CW_AGENT_TEMPLATE,
        transformations: [(obj: any, _opts: pulumi.CustomResourceOptions) => {
            if (typeof serviceAccounts !== 'undefined') {
                ContainerInsights.setIamRoleArn(obj, serviceAccounts);
            }
        }],
    },
    { providers: { kubernetes: this.k8sProvider } });
}

private static setIamRoleArn(obj: any, serviceAccounts: pulumi.Output<any>): void {
    if (obj !== undefined && obj.kind == 'ServiceAccount') {
        serviceAccounts.apply(serviceAccounts => {
            if (typeof serviceAccounts !== 'undefined' && Object.keys(serviceAccounts).includes(obj.metadata.name)) {
                if (!obj.metadata.annotations) {
                    obj.metadata['annotations'] = {}
                }
                obj.metadata.annotations['<http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>'] = serviceAccounts[obj.metadata.name].role.arn;
            }
        });
    }
}

Is there any correlation between the memory taken up by

pulumi up

process and the number of objects in the K8s cluster? I have a situation where

pulumi up

was crawling to halt on a 8GB worker in CircleCI, and when I tried it locally I saw it taking up 20+GB of memory. Any tips on troubleshooting this further?

Hello! How to do something like:

kubectl --kubeconfig ${PATH_TO_KUBECONFIG} --cluster ${CLUSTER_NAME} --token ${TOKEN}

?I'm confused on how to set the

--token

part with Pulumi. From here, seems like it is

id_token

https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens

Hi! Not sure if this is the appropriate channel, but since it is related to the

pulumi-kubernetes

package I figured to try here first. I'm trying to write some unit tests in C# for my stacks and I'm having trouble with a stack which contains a

Helm.V3.Chart

. The problem can be produced by having the following stack:

public class FailingHelmStack : Pulumi.Stack
{
    public FailingHelmStack()
    {
        var chart = new Pulumi.Kubernetes.Helm.V3.Chart("chart"
            , new Pulumi.Kubernetes.Helm.ChartArgs()
            {
                Chart = "ingress-nginx",
                Namespace = "kube-system",
                FetchOptions = new Pulumi.Kubernetes.Helm.ChartFetchArgs
                {
                    Repo = "<https://kubernetes.github.io/ingress-nginx>",
                },
            });
    }
}

Using

Deployment.TestAsync

with this stack results in the following exception being thrown.

Pulumi.RunException : Running program '<path>\bin\Debug\net5.0\testhost.dll' failed with an unhandled exception:
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Collections.Immutable.ImmutableArray.CreateRange[TSource,TResult](ImmutableArray`1 items, Func`2 selector)
   at Pulumi.Extensions.SelectAsArray[TItem,TResult](ImmutableArray`1 items, Func`2 map)
   at Pulumi.InputList`1.op_Implicit(ImmutableArray`1 values)
   at Pulumi.Kubernetes.Helm.V3.Chart.<>c__DisplayClass3_0.<ParseTemplate>b__0(ImmutableArray`1 objs)
   at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
   at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
   at Pulumi.Output`1.Pulumi.IOutput.GetDataAsync()
   at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop, Boolean keepResources)
   at Pulumi.Deployment.SerializeFilteredPropertiesAsync(String label, IDictionary`2 args, Predicate`1 acceptKey, Boolean keepResources)
   at Pulumi.Deployment.SerializeAllPropertiesAsync(String label, IDictionary`2 args, Boolean keepResources)
   at Pulumi.Deployment.RegisterResourceOutputsAsync(Resource resource, Output`1 outputs)
   at Pulumi.Deployment.Runner.<>c__DisplayClass9_0.<<WhileRunningAsync>g__HandleCompletion|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Pulumi.Deployment.Runner.WhileRunningAsync()
    Stack Trace:
       at Pulumi.Deployment.TestAsync(IMocks mocks, Func`2 runAsync, TestOptions options)
    <snip>
--- End of stack trace from previous location ---

I think I might be able to workaround this problem by mocking some necessary outputs perhaps. But I'm having trouble figuring out what exactly is required. Any help or suggestions would be greatly appreciated.

hi room….python/eks question… im trying to create an EKS cluster with a nodegroup that uses spot instances. in trying to create my nodegroup :-

eks_nodegroup = eks.NodeGroup("my-eks-nodegroup", opts=eks_opts, **eks_node_group_config)

File "/Users/bmeehan/repos/itplat-pulumi-infrastructure/venv/lib/python3.7/site-packages/pulumi_eks/node_group.py", line 145, in __init__
        raise TypeError("Missing required property 'cluster'")
    TypeError: Missing required property 'cluster'

however in trying to create my cluster it seems to need nodegroup to pre-exist :-

# node_group_options: Optional[pulumi.Input[pulumi.InputType['ClusterNodeGroupOptionsArgs']]] = None,
so in my eks_cluster_config
"node_group_options": eks_node_group_config,
eks_cluster = eks.Cluster("myt-eks-cluster", opts=eks_opts, **eks_cluster_config)

so which should come first? chicken or egg ?

Hi ! i have multiple stacks that are using the same k8s namespace. is there a good way to do a “get or create” namespace ? or do i to need create a stack with the namespace and then pull it in with “requireOutput” in all the other stacks ?

Hi ppl, I'm using a Helm Chart to deploy part of my stack and in the Chart there's a Secrete I update in an init Container. Is there a way to get the Secret with its updated data after creating the Chart instance (I'm using Typescript)?

I have a stacks, one with 3 clusters exporting their kubeconfig like

providerregionkubeconfig

. A second stack which uses a stack reference and the predicable name to fetch the output, and create a provider to deploy stuff on the cluster. Since we are messing about with the clusters sometimes they need to be recreated. At that point the second stack is broken, i cannot destroy nor up since the cluster is replaced (error:

configured Kubernetes cluster is unreachable: unable to load schema information from the API server

) The fix is to manually edit the stack export json files are remove all references to stuff deployed on the cluster so i can destroy it. I’m looking for a) a way to not depend on the specific k8s api server endpoint but on what it gets from the stack reference or b) a --ignore-errors and delete whatever you can find option removing the things you cannot. Any hints?

oh, and i know i could have destroyed them in the right order, but i wanted to update something on the cluster and messed them up in an inrecoverable state as well.

Hi, How would you go about adding flux into the cluster? It requires a cli to be run as indicated here.

Can anyone give my an idea how to debug this error message. I'm trying to deploy

kube-prometheus-stack

from

<https://prometheus-community.github.io/helm-charts>

. I get the same error for a few of the resources

error: 2 errors occurred:
        * the Kubernetes API server reported that "kube-system/kube-prometheus-stack-kube-controller-manager" failed to fully initialize or become live: 'kube-prometheus-stack-kube-controller-manager' timed out waiting to be Ready
        * Service does not target any Pods. Selected Pods may not be ready, or field '.spec.selector' may not match labels on any Pods

Using AWS EKS and Typescript. Is there any way to enumerate a list of IAM users from an existing IAM group? I can use the 

aws.iam.Group.get

 method to get the group, but I'm not sure how to actually access the existing group membership. The ultimate goal is to pull in a list of users to apply specific RBAC permissions to. Is there a better way?

Trying to get the hang of the relationships between k8s and kx - I don't see how kx can set the context for the cluster in the kubeconfig things should be applied to, is the intention that I set up k8s.Provider first to lock down the context and then pass that as a provider: argument to kx?

Hello. Have anyone experienced any issues with deleting kubernetes namespaces? When I remove a namespace that has been provisioned by pulumi it ends up in a "terminating" state, like described here: https://stackoverflow.com/questions/52369247/namespace-stuck-as-terminating-how-do-i-remove-it#:~:text=This%20is%20caused%20by%20resources,controller%20is%20unable%20to%20remove.&text=You%20can%20edit%20namespace%20on,enter%20or%20save%2Fapply%20changes.

Does anyone have advice on silencing helm warnings? Every time I run I get 1000 warnings about CRD fields that will be deprecated in 3 versions and all the unsupported hook stuff.

General question not really related to pulumi but I have to build images periodically at runtime to be deployed via k8s in Azure This has to be optimized for speed I could a) spin up virtual machine(s) and run the build there (costly, slow to start up, hard to maintain, hard to scale) b) build (using buildah?) in a container possibly using KEDA or just a scaleset or similar Has anyone else faced this problem, any thoughts? Sorry for OT, I can remove this if needed :)

If you're using ACR you can use its build tasks? 😉

And just for good measure - this is also exposed via pulumi 😉 https://www.pulumi.com/docs/reference/pkg/azure-native/containerregistry/buildtask/

And lastly - if you're wanting to build those images as well - and you're running pulumi regardless - why not try https://www.pulumi.com/docs/reference/pkg/docker/

Is there any way to force the Kubernetes provider to overwrite instead of ‘create’ if I have no prior state?

Running this in CI and I’m currently not tracking state

Only option.I have at the minute is to delete the old resources prior (Which is fine as it’s a staging env)

Probably going to add some state management in GCS later

is there a way to kick of a k8s.batch.v1.Job only wait for lets say 2 min and then mark the job as succesfull eventhough it is still processing ? if that’s not possible, is there away to extend the time pulumi waits for just this job to complete?

is there a way to force a restart of all pods in a namespace with pulumi ?