This may be a little of topic but I think the community here

Question

This may be a little of topic but I think the community here must have some insight on this I just updated a pulumi stack which deploys the <https github com prometheus community helm charts tree main charts kube prometheus stack> helm chart If you have a look at that page you ll see that major version upgrades require the update of the CRD as helm doesn t support updating CRD they provide yaml to do it I found out about this the hard way by having my helm chart deploy fail which amongst other things deleted the PV and PVC that grafana use and have all the dashboards I wrote on them So what do people do about such upgrades Do you pin all the helm chart versions and then work out a release process for each major version upgrade Do you backup all your PVs etc before every upgrade just in case I m trying to gain confidence in the Pulumi k8s helm ecosystem but it seems to be full of footguns confused

polite-napkin-90098 · Answer

My route forward is to create the PV and PVC before the helm release and then use a feature of this chart which allows me to specify an existing PVC for use for grafana s persistence I can then set `retainOnDelete` on it and it should be safe

polite-napkin-90098 · Answer

However this doesn t resolve the bigger issue with CRDs not being updated by helm and major version upgrades of helm charts which use them requiring special attention to run CRD updates which seem hard to automate and destined to break your deployed monitoring stack if you re not careful

steep-toddler-94095 · Answer

yes definitely pin all the chart versions once you have them pinned it sounds like it might be pretty simple to do these types of upgrades you could add the CRDs using <https www pulumi com registry packages kubernetes api docs yaml configfile >