https://pulumi.com logo
Join Slack
Powered by
Does `pulumi_eks` support creating a fully private...
# kubernetes
n
Does 
pulumi_eks
support creating a fully private EKS cluster (no public api access) with storage classes? Running into a strange issue:
Copy code
eks:index:VpcCni (staging-eks-vpc-cni):
    error: Command failed: kubectl apply -f /tmp/tmp-37930G3QLEJ0E0vMu.tmp
    Unable to connect to the server: net/http: TLS handshake timeout

  kubernetes:<http://storage.k8s.io/v1:StorageClass|storage.k8s.io/v1:StorageClass> (staging-eks-gp2):
    error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: Get "<https://A5F09297FEECD3B9C32CA.gr7.ca-central-1.eks.amazonaws.com/openapi/v2?timeout=32s>": net/http: TLS handshake timeout

  kubernetes:core/v1:ConfigMap (staging-eks-nodeAccess):
    error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: Get "<https://A5F09297FEECD3B9C32CA.gr7.ca-central-1.eks.amazonaws.com/openapi/v2?timeout=32s>": net/http: TLS handshake timeout

Resources:
    21 unchanged

Duration: 7m42s
Here is the eks creation code:
Copy code
eks_cluster = eks.Cluster(
    resource_name=cluster_name,
    vpc_id=vpc_id,
    cluster_security_group=cluster_security_group,
    desired_capacity=desired_cluster_size,
    enabled_cluster_log_types=cluster_log_types,
    # Public subnets will be used for load balancers
    public_subnet_ids=public_subnet_ids,
    # Private subnets will be used for worker nodes
    private_subnet_ids=private_subnet_ids,
    # Do not give worker nodes a public IP address
    node_associate_public_ip_address=False,
    # Change configuration values to change any of the following settings
    name=cluster_name,
    skip_default_node_group=True,
    storage_classes={
        storage_class["name"]: storage_class["args"]
        for storage_class in storage_classes
    },
    tags={"Name": f"{name_tag}-eks"},
    version=eks_version,
    # Uncomment the next two lines for private cluster (VPN access required)
    endpoint_private_access=True,
    endpoint_public_access=False
)
b
yes, but you need to have the ability to contact the API for the kubernetes cluster from where your Pulumi program runs
ie a VPN
it’s really a network problem, rather than a pulumi program 🙂
n
confirmed, able to access from my laptop:
Copy code
$ telnet <http://a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com|a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com> 443
Trying 10.36.2.223...
Connected to <http://a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com|a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com>.
Escape character is '^]'.
b
can you 
kubectl
it?
n
unable to, same issue
Copy code
$ KUBECONFIG=~/.kube/eks-works.yaml kubectl get nodes
Unable to connect to the server: net/http: TLS handshake timeout
b
do you use a http proxy?
n
none, that i'm aware. I'm on a company vpn connection though
b
can you show me 
cluster_security_group
definition?
n
sure, from code or aws_console?
Copy code
def create_security_group(vpc_id: str, sg_name: str) -> aws.ec2.SecurityGroup:
    security_group = aws.ec2.SecurityGroup(
        f"{sg_name}-securityGroup",
        name_prefix=sg_name,
        description="EKS-Cluster-security-group",
        vpc_id=vpc_id,
    )
    security_group.id.apply(
                lambda sg_id: aws.ec2.SecurityGroupRule(
                    f"{sg_id}-https-443",
                    description="allow inbound 443 access from connected networks",
                    type="ingress",
                    from_port=443,
                    to_port=443,
                    protocol="tcp",
                    cidr_blocks=["0.0.0.0/0"],
                    security_group_id=sg_id,
                    opts=pulumi.ResourceOptions(depends_on=[security_group]),
                )
            )
    return security_group


cluster_security_group = create_security_group(vpc_id, f"{name_tag}-cluster-security-group")
b
hmm 🤔 well, the issue here is that the eks provider uses kubectl to make some changes, and ultimately there’s either a network error or a proxy intercepting. I imagine you’ll get the same response if you try to 
curl
the k8s api too. i’d double check your network connectivity (I know the telnet works, so something may be amiss elsewhere)
but it definitely is supported
n
yeah curl fails too
Copy code
$ curl -k -v <https://a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com>
*   Trying 10.36.2.223:443...
* Connected to <http://a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com|a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com> (10.36.2.223) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: Connection reset by peer in connection to <http://a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com:443|a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com:443> 
* Closing connection 0
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to <http://a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com:443|a5f09297feecd3b9c32ca.gr7.ca-central-1.eks.amazonaws.com:443>
b
can you check your security group in the console and make sure that rule applied?
n
looks like they were:
b
hmm 🤔
n
i even opened up ephemeral ports (0-65535), based on a colleagues suggestion, but still the issue persists. I'm using a AWS vpn connection, via a transit gateway.
c
Check your NACLs as well, especially if TGW is in play.
n
@cuddly-computer-18851 - thanks, I did, as I'm able to connect locally via telnet.
Just to close the loop on this issue. It turned out that the network where 
pulumi up
was run didn't have full access to the VPN connection.
Thanks jaxxstorm, your hunch was correct.
Open in Slack
PreviousNext
Powered by