This message was deleted.
# kubernetess
sparse-intern-71089
04/13/2020, 9:35 PMThis message was deleted.
b
billowy-army-68599
04/13/2020, 10:12 PM
do you have any error message? Generally, all outbound ports will work inf k8s in default setups, but it's very possible it's getting blocked by something externally
👍 1
a
ancient-megabyte-79588
04/14/2020, 2:16 PMThanks for your thoughts Lee. Here are what I've found so far.
This picture is what lead me to believe it was the SMTP provider. (ProofPoint)
ancient-megabyte-79588
04/14/2020, 2:17 PMI found these in the nginx ingress controller pod in the logs
Copy code
[Apr 13, 2020 11:06:49 AM GMT-6]
2020/04/13 17:06:49 [error] 35#35: *3067368 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.244.1.1, server: auth.<snipped>.com, request: "POST /Manage/SendVerificationEmail HTTP/2.0", upstream: "<http://10.244.0.45:80/Manage/SendVerificationEmail>", host: "auth.<snipped>.com"
[Apr 13, 2020 11:06:49 AM GMT-6]
10.244.1.1 - - [13/Apr/2020:17:06:49 +0000] "POST /Manage/SendVerificationEmail HTTP/2.0" 504 173 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0" 405 60.407 [default-<snipped>-identity-sts-svc-80] [] 10.244.0.45:80 0 60.004 504 57dd3ac17974fbce71f247782d6f52ceancient-megabyte-79588
04/14/2020, 2:18 PMMucking around with ports and config, I eventually get this from nginx as well.
ancient-megabyte-79588
04/14/2020, 2:20 PMI put a dnsutils sidecar in the pod and tried to ping proofpoint from within the pod.. this might be due to the k8s Services not being setup correctly as well. I'm flailing a little. This eventually did 30 rows of
* * *ancient-megabyte-79588
04/14/2020, 2:21 PMAnd my currnent egress network policy for k8s
Copy code
apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: NetworkPolicy
metadata:
name: backend-network-policy
namespace: default
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- {}ancient-megabyte-79588
04/14/2020, 2:21 PMThis may be a problem with proofpoint. We're going to try something else today.
ancient-megabyte-79588
04/14/2020, 2:45 PMBy trying a different SMTP (not ProofPoint we are getting failure to authenticate messages from that server, so this is progress. It isn't the k8s egress policy. Thanks for helping!
b
billowy-army-68599
04/14/2020, 4:03 PM
looks like you went through most of the troubleshooting steps I would have recommended, glad I could be of some help! I had a quick search, and it seems proofpoint needs you to add your outbound address to allowed IPs: https://help.proofpoint.com/Proofpoint_Essentials/Unable_to_send_emails_through_Proofpoint_Essentials_outbound_relay_feature.
Which would explain the connection timed out message
a
ancient-megabyte-79588
04/15/2020, 1:59 PM@billowy-army-68599 Thanks for your remote rubber duck debugging 😄 In the end, we sorted the authentication issues and created an account for the service to email from directly on O365. We do have a ticket in with ProofPoint to try and sort this out, but that may be a while, so we just forged ahead.
So in the end, it was a Proofpoint problem and not a k8s problem, especially with my very liberal egress policy, which I'll work to dial back over time.
5 Views