Hello k8s peeps I m not sure how this has happened but my pu

Question

Hello k8s peeps I m not sure how this has happened but my pulumi script for deploying apps into our cluster nginx ingress controller fluentd certmanager etc has gotten into a condition where the stack thinks there should be some resources in the cluster that are simply not there and `pulumi refresh` doesn t work to bring the stack into sync with the cluster The concerning thing is that these resources were removed by the pulumi script but the script failed in a latter part of the execution couldn t get ip address property from `ingressController` object and so the stack is now out of sync with what the script removed and I can t get the stack back in sync I cannot `pulumi destroy` the stack since there are things in place that would be very problematic to re create

gentle-diamond-70147 · Answer

When you say pulumi refresh doesn t work can you elaborate

ancient-megabyte-79588 · Answer

as I understand it `pulumi refresh` inspects the stack and the script and tries to bring them together So if the app doesn t have a resource described and the cluster doesn t have it but the stack does have that resource it would remove it from the stack

ancient-megabyte-79588 · Answer

As an example I have removed certmanager from the script and it was removed from the cluster but the stack seems to still think that all of the certmanager resources are still present

ancient-megabyte-79588 · Answer

Currently the script does not have a cert manager chart defined anywhere the cluster does not have any cert manager resources so this picture of the stack and resources is incorrect and I would expect `pulumi refresh` to fix that

ancient-megabyte-79588 · Answer

So we have exported the stack to a json file and we ve manually removed all of the dangling resources and re imported it and that seems to have worked

gentle-diamond-70147 · Answer

Yes I would have expected `pulumi refresh` to fix that too To expand a bit on the behavior you described if you have a resource in your Pulumi state but that resource does not exist in your provider `pulumi refresh` should fix that meaning recognize the resource no longer exists and remove from your state file

gentle-diamond-70147 · Answer

Having said that I believe the `v3 Chart` resource itself is a component resource in Pulumi s terms so it does not exist in that exact form in your k8s cluster So `pulumi refresh` should have removed any of the sub resources of that chart if they no longer existed but the chart component might not be removed I ll need to confirm the component refresh behavior with a Pulumi engineer

ancient-megabyte-79588 · Answer

Thanks < gentle diamond 70147> for looking into it I recently updated to the latest CLI so I was a bit concerned a regression had been introduced

ancient-megabyte-79588 · Answer

I ve never had to manually manipulate the export of a stack to get to clean