https://pulumi.com logo
Powered by
#kubernetes
Title
# kubernetes
b

brash-mechanic-26159

05/06/2022, 4:36 PM
Is there any way I can easily customize the kubernetes Provider to open up an SSH tunnel to a bastion-host before performing any operations against the cluster?
For context: I’m deploying a private GKE cluster that I’m accessing through a private bastion using an SSH tunnel. Everything is implemented using pulumi: 1. Create the cluster 2. Create the bastion 3. Open the SSH tunnel (using 
bastion.id.apply(lambda x: open_ssh_tunnel_to_bastion(x))
) 4. Create the k8s provider that uses the bastion as a proxy 5. Create a bunch of resources on the cluster Everything kind of works, but step 3 is very awkward and doesn’t work automatically in every situation. I frequently find myself having to open up the SSH-tunnel manually before running 
pulumi up
when making changes to deployments and services (step 5). I think it would be better if I could just modify the lifecycle-hooks of the kuberneters Provider to ensure that the tunnel is always up. Any suggestions? Maybe there are other approaches that work better here?
b

brave-ambulance-98491

05/06/2022, 5:11 PM
Can you define "doesn't work"? And follow-up: Are you blocking your later operations on the 
apply
having completed? The obvious possible problem would be the k8s provider is trying to run commands before the tunnel is created, which would be solved by strictly ordering the operations.
b

brash-mechanic-26159

05/06/2022, 5:18 PM
I’m blocking all the other updates by returning the port-number from the 
apply
function, and then using that as an input to the k8s provider:
Copy code
ssh_tunnel_port = gke_bastion.name.apply(set_tunnel_target_and_open_ssh)

k8s_info = Output.all(
        cluster.name, cluster.endpoint, cluster.master_auth, ssh_tunnel_port
    )
k8s_config = k8s_info.apply(
    lambda info: _kubeconfig_template.format(
        ca_cert=info[2]["cluster_ca_certificate"],
        endpoint=info[1],
        context="gke_{0}_{1}_{2}".format(project, zone, info[0]),
        proxy_url=f"<http://127.0.0.1>:{info[3]}",
    )
)

provider = Provider(
    "gke_k8s",
    kubeconfig=k8s_config,
    suppress_helm_hook_warnings=True,
)
One thing that fails consistently is deleting deployments and services by commenting them out. The tunnel function gets called, but the delete operations are called before the tunnel is up (it takes a couple of seconds for it to start).
b

brave-ambulance-98491

05/06/2022, 5:24 PM
Hrm, interesting! I don't know how to block resource removal on other operations. That seems like something that would be a common problem, though ... I feel like the right solution would be to consider your 
Provider
dependent on the SSH tunnel - but it looks like you're already doing that!
7 Views
Powered by