https://pulumi.com logo
Title
f

future-refrigerator-88869

03/31/2021, 10:39 AM
Hi everyone. I have made a k8s cluster with aws load balancer controller. I have managed to expose the right things and it generally works fine. However, every time i run
pulumi up
, it will update the secret token of the service account (that's what i believe it does at least). That will always show up as a pulumi update. My question now is: Is that intentional ? Is it supposed to show a pulumi update with every
pulumi up
? If not, can anyone point me in the right direction of what to check in order to fix this ?
👆 2
💯 1
b

billowy-army-68599

03/31/2021, 5:19 PM
hey, sorry for the delay replying here @future-refrigerator-88869! the reason this happens is because we use
helm template
to render the charts, so anything that generates a TLS certificates gets regenerated every time the chart is run. To get around this issue, I would recommend omitting those resources from the chart, and generated your own TLS certificates using the tls provider. You can see an example of me doing this here: https://github.com/jaxxstorm/pulumi-aws-loadbalancercontroller/blob/main/nodejs/src/index.ts#L276-L295 https://github.com/jaxxstorm/pulumi-aws-loadbalancercontroller/blob/main/nodejs/src/index.ts#L501-L544
f

future-refrigerator-88869

03/31/2021, 8:08 PM
Thanks for the answer @billowy-army-68599. Quick follow-up: I can see that in your examples you're building self signed certificates. Are those used by the load balancer internally in k8s ? The reason i`m asking is to figure out if i could use certificates from
aws certificate manager
(or if it is necessary even).
b

billowy-army-68599

03/31/2021, 8:19 PM
you totally can use certs from certificate manager! it's not strictly necessary because it's the k8s API server that needs to validate it, so in this case I just build a self signed cert, but any valid cert works
f

future-refrigerator-88869

04/05/2021, 11:40 PM
Hey @billowy-army-68599. I have been trying to use a custom certificate creating it locally like in your example. I have tried to use it with the helm chart but i can't seem to find a way to make it work. Are you aware if it is possible to overwrite the secret using the helm chart? checking this (https://github.com/aws/eks-charts/blob/master/stable/aws-load-balancer-controller/templates/deployment.yaml#L36) it seems like if you name it right, it should be working ? However, I still have another one created all the time.
b
f

future-refrigerator-88869

04/06/2021, 7:31 PM
@billowy-army-68599 thanks a lot for your help. I feel like I am very close to finishing this. I have checked the template a bit more and it seems that the certificate options
caCert
,
clientCert
and
clientKey
are loaded from the template and are automatically generated. They don't seem to have a
values
equivalent (https://github.com/aws/eks-charts/blob/master/stable/aws-load-balancer-controller/values.yaml). I have found an option for
cert-manager
but I am not sure if it can actually solve my issue (never used it before). I have found the following options for me: 1. edit the helm chart and allow values for cert 2. use pretty much your solution from your github with some modifications 3. use cert-manager (if it can be applied). What is your suggestion ? Please let me know if i'm missing something.
Just to be clear on this. Everything works fine. I managed to use ssl and dns for the services. The only issue is that every time i run
pulumi up
it will update the secret from the chart
b

billowy-army-68599

04/06/2021, 7:34 PM
@future-refrigerator-88869 you can use a transformation, so you'd intercept the helm template and inject the correct values, which languages SDK are you using, i can whip up example
can you share your diff too/
f

future-refrigerator-88869

04/06/2021, 7:35 PM
I am using typescript. What diff are you referring to ? I have shared what pulumi generates on every
pulumi up
(in the thread)
Or maybe you`re looking for the
details
option ?
b

billowy-army-68599

04/06/2021, 7:36 PM
yeah the details
f

future-refrigerator-88869

04/06/2021, 7:38 PM
give me a few minutes to recreate the env
just to make sure i have a clean output
@billowy-army-68599 can i send the diff in private channel ?
b

billowy-army-68599

04/06/2021, 7:57 PM
sure!
here we go, comments inline: https://gist.github.com/jaxxstorm/6317d030307c34783ba82afb2769fb3c let me know if there's any confusion
(transformations are really, really powerful)
f

future-refrigerator-88869

04/06/2021, 8:10 PM
Perfect. I get the idea of what it does. I'll give it a try and report back. Thanks a lot for the help
Hey @billowy-army-68599. I have managed to make it work ! I had to add the webhooks and the cert myself and after that, all good ! Now I have an issue with destroying the k8s cluster. It seems that some resources are unresponsive but I think i can figure it out. Thanks again for the help 🙂
b

billowy-army-68599

04/07/2021, 7:21 PM
i'm happy to hear it!