https://pulumi.com logo
Join Slack
Powered by
i just can't seem to get the nginx ingress helm ch...
# general
b
i just can't seem to get the nginx ingress helm chart installed any more. i always end up with
Copy code
error: Plan apply failed: 2 errors occurred:
	* 'nginxingresscontroller-nginx-ingress-controller' timed out waiting to be Ready
	* Minimum number of live Pods was not attained


error: Plan apply failed: 2 errors occurred:
	* 'nginxingresscontroller-nginx-ingress-default-backend' timed out waiting to be Ready
	* Minimum number of live Pods was not attained
b
Has this just started happening or has it never worked for you?
b
i'm not sure updating the chart seamlessly has ever worked
b
Ok
Has this been for a long time or you’ve only started recently using it?
b
i've only used pulumi for the past month or so.
i first ran into the problem where the helm chart sucks so i had to add values for it to not break on upgrade. now it doesn't really break, but pulumi always thinks it doesn't succeed, but i'm not sure why. nothing seems to update besides the chart label
i should probably just port all the resources to pulumi because helm is terrible, but i'd like to avoid that because i do have other things to do. but given how often it's breaking deployments.... maybe i should just bite the bullet and do it 😕
or pin it, and just know that any time it upgrades it'll break deployments.
probably worth pointing out, if i wipe out the cluster and recreate everything, the new version deploys correctly
b
Ok - that’s interesting to know
I’m asking in the team atm
b
based on the diff output, the only field that's changing is [deployment].metadata.labels.chart. looks like what changed is using a value if given, but doesn't affect the rendering for what i'm doing.
c
We used to not use helm. Now we make heavy use of it. It’s a lot easier for third party applications. We only use native pulumi resources for anything not in the chart, or for our own applications. Having gone down the road of porting everything to pulumi previously, I warn you not to.
Have you looked at what the logs are showing for the pods? Or does it never update/create the Deployment?
b
@cool-egg-852 i appreciate the input. do you use nginx for your ingress controller or something else? it seems the nginx-ingress chart and pulumi don't get along.
i have. everything is a-ok.
my deployments deploy correctly
c
We do not use nginx, we use istio ourselves. But having used helm with pulumi many times, I can’t say I’ve experienced any issues like this. Issues yes, but due to the charts themselves (not terribly hard to fix)
Can you post the full output (not the diff) from your 
pulumi up
?
b
i mean, issues with charts are really issues with helm, wouldn't you say? the design issues of helm translates to issues with charts. poorly-made charts go hand-in-hand with poorly-designed helm.
i can
i run a 
pulumi up --diff -y
, do you want that full output?
or just the 
pulumi up --skip-preview
output?
c
I can’t speak to helm long ago, but these days, I would say it’s just on the charts themselves. The only real issues I have are either badly designed charts, or charts that are missing the 
namespace
because of the Tiller design. This will all be fixed though as Helm 3 doesn’t have tiller. Charts will have to add in 
namespace
Just preview actually, not up
b
yeah, i don't know how it took until version 3 to create a sane design. being open source, you'd think the k8s community would have realized how crazy it was and made a better alternative. i'm hoping the saner design of v3 will influence the use of helm, but i'm not gonna hold my breath. even the charts in stable aren't using explicit namespaces. anyway, i'll stop with the ranting.
c
Ok, and what is the diff in the metadata? I’m wondering if you are running into the same issue I have in the passed. It wouldn’t be helm related if it’s the case.
b
i think it's probably something pulumi-related, since the chart/resource does update:
Copy code
kubectl get deployment nginxingresscontroller-nginx-ingress-controller -o yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  ...
  labels:
    app: nginx-ingress
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: pulumi
    chart: nginx-ingress-1.20.0
but for whatever reason, pulumi thinks the deployment doesn't succeed (it does)
c
I’ve experienced something similar in the past, though I don’t recall it being helm related. Have you tried 
pulumi refresh
?
b
for past breakages from the same issue, yes, but let me try again and see what happens
well, even after the 
pulumi refresh
, the 
pulumi stack export
resource is still marked as unhealthy
now when i run 
pulumi preview
, it wants to update the two deployments, but there's no diff.
i guess i will just file an issue against pulumi-kubernetes
c
I had this before. I don’t recall how I fixed it.
I think I probably ended up just deleting the deployment and redeploying.
Do you mind sharing your pulumi code?
I actually remember there being 1 time I had this and the issue was caused by an integer overflow
b
c
Have you tried without the transformations?
b
i added them because of this error, but sure i can try that.
c
I’m not seeing anything that could possibly be wrong with it other than that, and I don’t see why that would cause it either, but worth a shot.
Other than that, I’d see if you can just remove the resource and recreate it if you need it fixed immediately. Otherwise file a pulumi issue. I’ve definitely run into this exact same issue previously.
Something causes pulumi to think it needs to be updated, but it doesn’t actually need an update, and thus it hangs.
And just to ask, what version of the chart are you using?
Given that you are not specifying a version, I’m not sure if pulumi updates the repo before executing or not.
Ah wait I see, 1.20.0
b
it's happened to me for both 1.17.x => 1.19.1 and from 1.19.1 => 1.20.0
last time i thought pulumi had just gotten into a bad state, so i blew away the cluster and recreated. now that it's happened again for the same reason, i know for sure it's a problem the update of the nginx-ingress chart.
also, it doesn't really matter if pulumi updates the repo, since i'm running in CI
the pulumi/pulumi image doesn't have helm, so i have to install it manually
so every time the CI runs, the repos should be up-to-date
c
Ah. We’re not using the docker image for pulumi quite yet.
We’ll be moving to that as we replace Jenkins.
g
Sorry about that, just responded on the issue. This turned out to be a bug with 
extensions/v1beta1/Deployment
logic that only applied on updates.
c
Fast as always.
😁 2
b
was that just released?
c
Yep, about 14 minutes ago
b
damn, fast indeed. @gorgeous-egg-16927 think you can kick off a docker image build? 👼
c
It was just a release of the provider, not sure a new docker image is needed is it?
b
oh, good point
g
Right, you just need to update the provider version
5 Views
Open in Slack
PreviousNext
Powered by