https://pulumi.com logo
#kubernetes
Title
# kubernetes
b

boundless-petabyte-41580

12/28/2023, 8:38 PM
Hello 👋 I've got a question about a k8s use case: I'm running a proof of concept, where I want to ease my team into Pulumi. As a first stage, I would like to define k8s resources in Pulumi, render them into YAML and use Helm to manage them from there
so far, I tried a Provider with the following:
Copy code
const provider = new k8s.Provider("k8s", {
    renderYamlToDirectory: "my-chart/templates",
    skipUpdateUnreachable: true,
    kubeClientSettings: {
      timeout: 1
    }
    
})
since I don't actually want any communication with a k8s cluster, just YAMLs
but not files actually get written. Am I on the right track, or is there a better way to do this?
b

billions-river-87988

12/28/2023, 9:38 PM
you are on the right track- keep in mind this feature is in beta and not well supported. IMO this would make Pulumi more of a replacement to Helm, since it will output raw manifests, not helm templates.
b

boundless-petabyte-41580

12/28/2023, 10:17 PM
thats what I'm aiming for - to replace Helm/helmfile in my org. The worst part about it is the go templates, so I'm trying to eliminate them first and keep the Helm state machine
we also have AWS IAC via terraform, but thats a different beast for a different day
b

billions-river-87988

12/28/2023, 10:20 PM
yeah im not sure why you dont see templates generated- it works for me, i did a similar thing with ArgoCD and Pulumi generating the manifests. My main issue was that it doesn't seem to recreate the manifests if they get deleted (state thinks they still exist)
b

boundless-petabyte-41580

12/28/2023, 10:45 PM
the files get generated only when I have access to a k8s cluster
yeah they only get updated based on pulumi state
b

billions-river-87988

12/28/2023, 10:46 PM
ah interesting, let me ask around about that
(the cluster connection)
b

boundless-petabyte-41580

12/28/2023, 10:46 PM
to get them recreated you have to
pulumi destroy
the state
@quaint-spring-93350 any idea?
b

billions-river-87988

12/29/2023, 5:38 PM
I have looked into whether a connection to the cluster is required to generate the files- Here is an example repository where i generate successfully without any cluster, so I am not sure what is going on. https://github.com/automagic/pulumi_k8s_app
if you can share a longer code example including resources i can take a look
i am definitely able to render without a kube context set
just pushed a change to that repo that should work
my
.kube/config
Copy code
apiVersion: v1
clusters: []
contexts: []
kind: Config
preferences: {}
users: []
Validated if i disable the renderYamlToDirectory it will throw an error trying to create an acutal pod
q

quaint-spring-93350

12/29/2023, 6:24 PM
Sorry, my brain is flu'd and I barely connect any dots. We've had issues with serde layer for provider args in pulumi-scala but the end result was that it did render manifests on every run disregarding what was passed to the renderYaml... parameter. We fixed that (we needed to filter out all null values from proto structs) and I havent seen any issues around this later.
b

billions-river-87988

12/29/2023, 6:26 PM
not sure i totally follow- how is it rendering yaml reguardless of the
renderYamlToDirectory
flag?
b

boundless-petabyte-41580

12/29/2023, 6:43 PM
I was able to do the same now, both without any
.kube/config
and with the config you provided @billions-river-87988
it may be because of an old version of
@pulumi/kubernetes
I upgraded from
3.0.0
to
4.5.6
thanks for checking!
and feel better @quaint-spring-93350
q

quaint-spring-93350

12/29/2023, 6:47 PM
thx!
b

billions-river-87988

12/29/2023, 6:50 PM
ah, ok- there is still some weird behavior around the functionality i am aware of, even on the latest. For example: • YAML files are not recreated on
up
if they are removed (as mentioned)- but, they are always recreated on
pulumi preview
. The downside here is that if a resource changes, its doesn't update the file, it just creates a new file for the same resource (so now there are two). • It will create the files without a cluster as we have found, but if you do a
pulumi up --refresh
it will error out
b

boundless-petabyte-41580

12/29/2023, 7:03 PM
interesting: I restored my original
.kubeconfig
and now its trying to talk to the cluster again
b

billions-river-87988

12/29/2023, 7:06 PM
yeah not surprising- its definitely always looking at the kubeconfig, seems not to care if no clusters are defined but will use it if present.
b

boundless-petabyte-41580

12/29/2023, 7:21 PM
if I try to direct it to an empty
.kubeconfig
explicitly, it complains
Copy code
error: pulumi:providers:kubernetes resource 'k8s': property kubeconfig value {.kubeconfig} has a problem: "kubeconfig" arg is not compatible with "renderYamlToDirectory" arg