https://pulumi.com logo
Powered by
Title
s

steep-portugal-37539

06/07/2021, 4:47 PM
To summarize this, using .networking ingress, we get this error: 
Failed build model due to ingress: waterrecharge/waterrecharge-ingress: none certificate found for host: <http://waterrecharge-pulumi-api-aryeh.tqhosted.com|waterrecharge-pulumi-api-aryeh.tqhosted.com>"
b

billowy-army-68599

06/07/2021, 4:48 PM
how are you referring to the certificate in the ingress? Do you have a corresponding Kubernetes secret for it?
can you share some code?
@steep-portugal-37539 can you keep it in this thread please so we don't spam the channel
s

steep-portugal-37539

06/07/2021, 4:53 PM
so sorry, yes
don’t think i can copy into this thread
i’ll just copy links: https://pulumi-community.slack.com/archives/CRFURDVQB/p1623084650182000
https://pulumi-community.slack.com/archives/CRFURDVQB/p1623084704182300
b

billowy-army-68599

06/07/2021, 4:54 PM
no that's okay: based on your answers, it's not configured correctly. The AWS LB controller secrets are not TLS secrets for your host, they are TLS secrets for the internal webhook that the load balancers uses
you need to generate a new certificate, using something like ACM and then create a new secret with the correct certificate value
s

steep-portugal-37539

06/07/2021, 4:56 PM
it’s weird bec it used to work…. What about tls cert auto discovery? I shouldn’t even need that section then. But currently, we create three acm certs for each hostname. we stick them all in the tls section together. Should they be separate?
https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/ingress/cert_discovery/#discover-via-ingress-rule-host
b

billowy-army-68599

06/07/2021, 4:58 PM
that should work, but I don't see you specifying the certificate ARN in your ingress rule?
oh if you're using auto discovery and it's not working, you'll need to check the ingress controller logs to see what's going on
in any case, this feels more like an issue with the controller than a Pulumi problem 🙂
s

steep-portugal-37539

06/07/2021, 5:30 PM
yes, shouldn’t need the cert arn anywhere or need to specify tls. I deleted the alb cntrl and redeployed. Ingress are deploying now. But getting now: 
error: 2 errors occurred:
    	* resource waterrecharge/waterrecharge-ingress was successfully created, but the Kubernetes API server reported that it failed to fully initialize or become live: 'waterrecharge-ingress' timed out waiting to be Ready
    	* Ingress has at least one rule that does not target any Service. Field '.spec.rules[].http.paths[].backend.serviceName' may not match any active Service
all of my svc’s in the namespace of my ing exist. could it be referring to 
"ssl-redirect"
service? I don’t know why pulumi isnt being more explicit there are no errors in the alb pod logs
the only think i’m curious about is this in the logs: 
"name": "waterrecharge-ingress",
  "namespace": "waterrecharge",
  "operation": "CREATE",
  "userInfo": {
    "username": "kubernetes-admin",
    "uid": "heptio-authenticator-aws:358459329979:AIDAVG5OXJW5UBJAFH7RX",
    "groups": [
      "system:masters",
      "system:authenticated"
    ],
    "extra": {
      "accessKeyId": [
        "AKIAVG5OXJW5YUHERVWE"
      ]
    }
  }
i’m wondering what 
username: kubernetes-admin
is referring to
b

billowy-army-68599

06/07/2021, 5:34 PM
that's the service account that's being used
'.spec.rules[].http.paths[].backend.serviceName' may not match any active Service
This indicates there's a configuration error, no?
s

steep-portugal-37539

06/07/2021, 5:37 PM
would see like it although every service exists…. I’m going to try process of elimination on each ing rule. thank you so far
b

billowy-army-68599

06/07/2021, 5:37 PM
can you do do 
kubectl describe ing waterrecharge-ingress -n waterrecharge
s

steep-portugal-37539

06/07/2021, 6:18 PM
❯  k describe -n waterrecharge ingress waterrecharge-ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use <http://networking.k8s.io/v1|networking.k8s.io/v1> Ingress
Name:             waterrecharge-ingress
Namespace:        waterrecharge
Address:          <http://k8s-waterrec-waterrec-7dbf3cfe64-1532717845.us-west-2.elb.amazonaws.com|k8s-waterrec-waterrec-7dbf3cfe64-1532717845.us-west-2.elb.amazonaws.com>
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                                     Path  Backends
  ----                                     ----  --------
  <http://waterrecharge-pulumi-aryeh.tqhosted.com|waterrecharge-pulumi-aryeh.tqhosted.com>
                                           /*   ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
                                           /*   frontend:http (172.31.18.20:80)
Annotations:                               <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>:
                                             {"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}
                                           <http://alb.ingress.kubernetes.io/healthcheck-path|alb.ingress.kubernetes.io/healthcheck-path>: /chains/main/blocks/head/header
                                           <http://alb.ingress.kubernetes.io/healthcheck-port|alb.ingress.kubernetes.io/healthcheck-port>: 8732
                                           <http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: [{"HTTP": 80}, {"HTTPS":443}]
                                           <http://alb.ingress.kubernetes.io/load-balancer-attributes|alb.ingress.kubernetes.io/load-balancer-attributes>: idle_timeout.timeout_seconds=180
                                           <http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
                                           <http://ingress.kubernetes.io/force-ssl-redirect|ingress.kubernetes.io/force-ssl-redirect>: true
                                           <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
Events:
  Type    Reason                  Age               From     Message
  ----    ------                  ----              ----     -------
  Normal  SuccessfullyReconciled  3s (x4 over 71m)  ingress  Successfully reconciled
/*   ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
i heard from co-worker that this error is ok
hmmm maybe not??
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
maybe this one is ok? ☝️
even if i get rid of the ssl route, pulumi errors out still. the ing is still updated/created
hmmm https://pulumi-community.slack.com/archives/CRFURDVQB/p1623087255188800?thread_ts=1623084439.181200&amp;cid=CRFURDVQB i don’t see any svc account in my cluster with that name. Cntrlr stilll does not error
b

billowy-army-68599

06/07/2021, 7:01 PM
yeah it's really hard to know what's wrong here, there seems to be a lot of things that could be at fault
do you have a service in the same namespace like this? 
backend: {
                      serviceName: "frontend",
                      servicePort: "http",
                    },
s

steep-portugal-37539

06/07/2021, 9:47 PM
Hey Lee, thank you again for the help! I’m still having an issue sadly…. My route53 function can’t get the hostname of the ingress… For one it isn’t awaited for anymore as we talked about.
route53 alias record needs the ingress and tries to resolve hostname of the ing
ingress.status.loadBalancer.ingress[0].hostname.apply(
    (mylb) => mylb
  )
but it always returns undefined
b

billowy-army-68599

06/07/2021, 9:57 PM
you'll have to make your 
apply
a little more elegant, you can do pretty much anything you want inside the 
apply
- including retries. So you'll probably have to check the value is defined and then when it is, create a new resource
it's not ideal
I generally try use external-dns to solve this kinda problem
s

steep-portugal-37539

06/07/2021, 10:12 PM
hmmm ok thanks. we were actually looking into external dns a little bit ago. How difficulut is it to set up
b

billowy-army-68599

06/07/2021, 10:14 PM
not too hard, just need to make sure you have the right IAM roles
s

steep-portugal-37539

06/08/2021, 5:54 PM
Hey Lee 🙂 we’ll need cert-manager too if we go with external-dns, right? Not familiar with this stuff
b

billowy-army-68599

06/08/2021, 5:56 PM
not necessarily, external-dns just registers DNS records with your chosen DNS provider
you can still use ACM
s

steep-portugal-37539

06/08/2021, 5:59 PM
so the tls section of the ingress can remain the same. It should reference the secret name and the hosts. (or maybe auto discovery would work now?)
b

billowy-army-68599

06/08/2021, 6:05 PM
yeah the TLS certificate just needs to be valid for the hostname you're specifying
s

steep-portugal-37539

06/08/2021, 6:11 PM
thanks so much! Will give this a spin
4 Views
#kubernetesJoin Slack
Powered by