https://pulumi.com logo
Join Slack
Powered by
This message was deleted.
# kubernetes
s
This message was deleted.
b
how are you referring to the certificate in the ingress? Do you have a corresponding Kubernetes secret for it?
can you share some code?
@steep-portugal-37539 can you keep it in this thread please so we don't spam the channel
s
so sorry, yes
don’t think i can copy into this thread
b
no that's okay: based on your answers, it's not configured correctly. The AWS LB controller secrets are not TLS secrets for your host, they are TLS secrets for the internal webhook that the load balancers uses
you need to generate a new certificate, using something like ACM and then create a new secret with the correct certificate value
s
it’s weird bec it used to work…. What about tls cert auto discovery? I shouldn’t even need that section then. But currently, we create three acm certs for each hostname. we stick them all in the tls section together. Should they be separate?
b
that should work, but I don't see you specifying the certificate ARN in your ingress rule?
oh if you're using auto discovery and it's not working, you'll need to check the ingress controller logs to see what's going on
in any case, this feels more like an issue with the controller than a Pulumi problem 🙂
s
yes, shouldn’t need the cert arn anywhere or need to specify tls. I deleted the alb cntrl and redeployed. Ingress are deploying now. But getting now:
Copy code
error: 2 errors occurred:
    	* resource waterrecharge/waterrecharge-ingress was successfully created, but the Kubernetes API server reported that it failed to fully initialize or become live: 'waterrecharge-ingress' timed out waiting to be Ready
    	* Ingress has at least one rule that does not target any Service. Field '.spec.rules[].http.paths[].backend.serviceName' may not match any active Service
all of my svc’s in the namespace of my ing exist. could it be referring to 
"ssl-redirect"
service? I don’t know why pulumi isnt being more explicit there are no errors in the alb pod logs
the only think i’m curious about is this in the logs:
Copy code
"name": "waterrecharge-ingress",
  "namespace": "waterrecharge",
  "operation": "CREATE",
  "userInfo": {
    "username": "kubernetes-admin",
    "uid": "heptio-authenticator-aws:358459329979:AIDAVG5OXJW5UBJAFH7RX",
    "groups": [
      "system:masters",
      "system:authenticated"
    ],
    "extra": {
      "accessKeyId": [
        "AKIAVG5OXJW5YUHERVWE"
      ]
    }
  }
i’m wondering what 
username: kubernetes-admin
is referring to
b
that's the service account that's being used
'.spec.rules[].http.paths[].backend.serviceName' may not match any active Service
This indicates there's a configuration error, no?
s
would see like it although every service exists…. I’m going to try process of elimination on each ing rule. thank you so far
b
can you do do 
kubectl describe ing waterrecharge-ingress -n waterrecharge
s
Copy code
❯  k describe -n waterrecharge ingress waterrecharge-ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use <http://networking.k8s.io/v1|networking.k8s.io/v1> Ingress
Name:             waterrecharge-ingress
Namespace:        waterrecharge
Address:          <http://k8s-waterrec-waterrec-7dbf3cfe64-1532717845.us-west-2.elb.amazonaws.com|k8s-waterrec-waterrec-7dbf3cfe64-1532717845.us-west-2.elb.amazonaws.com>
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                                     Path  Backends
  ----                                     ----  --------
  <http://waterrecharge-pulumi-aryeh.tqhosted.com|waterrecharge-pulumi-aryeh.tqhosted.com>
                                           /*   ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
                                           /*   frontend:http (172.31.18.20:80)
Annotations:                               <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>:
                                             {"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}
                                           <http://alb.ingress.kubernetes.io/healthcheck-path|alb.ingress.kubernetes.io/healthcheck-path>: /chains/main/blocks/head/header
                                           <http://alb.ingress.kubernetes.io/healthcheck-port|alb.ingress.kubernetes.io/healthcheck-port>: 8732
                                           <http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: [{"HTTP": 80}, {"HTTPS":443}]
                                           <http://alb.ingress.kubernetes.io/load-balancer-attributes|alb.ingress.kubernetes.io/load-balancer-attributes>: idle_timeout.timeout_seconds=180
                                           <http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
                                           <http://ingress.kubernetes.io/force-ssl-redirect|ingress.kubernetes.io/force-ssl-redirect>: true
                                           <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
Events:
  Type    Reason                  Age               From     Message
  ----    ------                  ----              ----     -------
  Normal  SuccessfullyReconciled  3s (x4 over 71m)  ingress  Successfully reconciled
Copy code
/*   ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
i heard from co-worker that this error is ok
hmmm maybe not??
Copy code
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
maybe this one is ok? ☝️
even if i get rid of the ssl route, pulumi errors out still. the ing is still updated/created
hmmm https://pulumi-community.slack.com/archives/CRFURDVQB/p1623087255188800?thread_ts=1623084439.181200&amp;cid=CRFURDVQB i don’t see any svc account in my cluster with that name. Cntrlr stilll does not error
b
yeah it's really hard to know what's wrong here, there seems to be a lot of things that could be at fault
do you have a service in the same namespace like this?
Copy code
backend: {
                      serviceName: "frontend",
                      servicePort: "http",
                    },
s
Hey Lee, thank you again for the help! I’m still having an issue sadly…. My route53 function can’t get the hostname of the ingress… For one it isn’t awaited for anymore as we talked about.
route53 alias record needs the ingress and tries to resolve hostname of the ing
Copy code
ingress.status.loadBalancer.ingress[0].hostname.apply(
    (mylb) => mylb
  )
but it always returns undefined
b
you'll have to make your 
apply
a little more elegant, you can do pretty much anything you want inside the 
apply
- including retries. So you'll probably have to check the value is defined and then when it is, create a new resource
it's not ideal
I generally try use external-dns to solve this kinda problem
s
hmmm ok thanks. we were actually looking into external dns a little bit ago. How difficulut is it to set up
b
not too hard, just need to make sure you have the right IAM roles
s
Hey Lee 🙂 we’ll need cert-manager too if we go with external-dns, right? Not familiar with this stuff
b
not necessarily, external-dns just registers DNS records with your chosen DNS provider
you can still use ACM
s
so the tls section of the ingress can remain the same. It should reference the secret name and the hosts. (or maybe auto discovery would work now?)
b
yeah the TLS certificate just needs to be valid for the hostname you're specifying
s
thanks so much! Will give this a spin
5 Views
Open in Slack
PreviousNext
Powered by