Having trouble with our current state paid customer we re us

Question

Having trouble with our current state paid customer we re using a Helm Chart to install Harbor in our EKS cluster This chart generates several `Secret` s used for encryption and mounts those as volumes At some point during an update the `Secret`s are no longer in the k8s cluster but Pulumi believes they are These missing `Secret`s are preventing pods from starting and the pods are stuck in the creation state `ContainerCreating` or `CreateContainerConfigError` I ve attempted refreshing multiple times and Pulumi still believes these `Secret`s are there and are never updated in the state I did find these entries in the job run logs from a job I ran to Preview stack changes a few hours ago we re using GH Actions ``` kubernetes core v1 Secret dev jobservice delete original + kubernetes core v1 Secret dev jobservice replace diff ~data ++ kubernetes core v1 Secret dev jobservice create replacement diff ~data kubernetes core v1 Secret dev trivy kubernetes core v1 Secret dev registry delete original + kubernetes core v1 Secret dev registry replace diff ~data ++ kubernetes core v1 Secret dev registry create replacement diff ~data kubernetes core v1 ConfigMap dev core kubernetes core v1 Secret dev core delete original + kubernetes core v1 Secret dev core replace diff ~data ++ kubernetes core v1 Secret dev core create replacement diff ~data ``` the ` dev trivy` is the only `Secret` that remains in the cluster

square-laptop-45713 · Answer

in a follow up job to perform the refresh I found these logs ``` ~ kubernetes core v1 Secret dev jobservice refreshing 0s kubernetes core v1 Secret dev jobservice 0 00s ~ kubernetes core v1 ConfigMap monitoring prometheus server refreshing 0s kubernetes core v1 ConfigMap monitoring prometheus server 0 00s ~ kubernetes core v1 Secret dev core refreshing 0s kubernetes core v1 Secret dev core 0 00s ~ kubernetes core v1 Secret dev registry refreshing 0s kubernetes core v1 Secret dev registry 0 00s ```

square-laptop-45713 · Answer

and these lower in the same logs ``` Refreshing kubernetes core v1 ConfigMap dev registry 8s kubernetes core v1 Secret dev trivy 9s kubernetes core v1 ConfigMap dev jobservice env 9s ```

billowy-army-68599 · Answer

if I m understanding correctly the secret is disappearing from the cluster

square-laptop-45713 · Answer

more like it disappeared I don t know exactly when and running pulumi doesn t bring it back up and pulumi itself gets stuck and fails b c the pods relying on the secret can never finish creating and timeout

billowy-army-68599 · Answer

okay so this is likely because when you re deploying a helm release The release itself hasn t changed so therefore Pulumi think it s fine The helm release itself manages the resource

square-laptop-45713 · Answer

does it I thought pulumi renders the templates using helm but then applies it using equivalent of `kubectl apply`

square-laptop-45713 · Answer

how do I resolve it

billowy-army-68599 · Answer

sorry are you using `helm Release` or `helm Chart`

billowy-army-68599 · Answer

if it s the latter you re right it templates and I m wrong

square-laptop-45713 · Answer

`helm Chart`

square-laptop-45713 · Answer

afaik they both do that

square-laptop-45713 · Answer

got it `helm Release` is delegating to the helm binary

square-laptop-45713 · Answer

< billowy army 68599> do you think if I convert the `helm Chart` to a `helm Release` in my IaC it will redeploy everything

square-laptop-45713 · Answer

the db is external as well as using s3 buckets so those can still be referenced and have no state changes