https://pulumi.com logo
Join Slack
Powered by
Hey folks. I'm using pulumi to deploy helm charts ...
# getting-started
b
Hey folks. I'm using pulumi to deploy helm charts into my GKE cluster but for whatever reason, the admin kubeconfig isn't consistently used. I'll need to sign in with my own gcloud get-credentials, type my password, etc. before pulumi will work. This also means running pulumi in github actions is just downright broken. I'm using the gcp-py-gke example exactly but the workflow seems broken. Hopefully I'm just doing something a little bit wrong. Any advices? (cc @full-xylophone-8606)
👍 1
d
You should only need to login once. For within github actions, you'd use Workload Identity to authenticate your default Google credentials, which the kubeconfig will then use
b
Is the above pulumi example wrong, then? Shouldn't you be able to pull out the cluster master auth and endpoint and access the cluster without workload identity?
Thanks for helping, btw!
d
Workload Identity is just a way to auth with Google and get oauth credentials. The example follows the recommended way of authenticating with GKE, which is to use the 
gke-gcloud-auth-plugin
command
b
Yeah... that's what I'm doing and I can confirm the kubeconfig it's generating has the right creds but locally pulumi fails because it wants my local gcloud setup to authenticate again (why?) and on github it just doesn't work at all even though it too should be using gke-gcloud-auth-plugin
Couldn't find any docs other than that code example
d
You might need to run 
gcloud auth application-default login
for local use
How are you currently configuring github actions to authenticate to Google?
b
I'm not. I didn't think I'd need to since I'm using the google provider and it has a service account.
Everything else works, just not the GKE helm chart part
d
Yes, how is the service account configured in github action
b
Well it's a PR trigger that runs in pulumi cloud. Pulumi is configured with env vars. I have a google service account json and then a 
GOOGLE_APPLICATION_CREDENTIALS
path. Before I import pulumi and providers and read from env and write that to disk:
Copy code
# Load the GCP credentials from an env variable if available.
import configure_gcp_sdk # <--- here

import pulumi
import pulumi_gcp as gcp
Copy code
if os.getenv("GCP_CREDS_JSON_CREDS_BASE64"):
    AUTH_FILE = "./gcp_service_account.json"
    with open(AUTH_FILE, "w") as f:
        json.dump(
            json.loads(base64.b64decode(os.getenv("GCP_CREDS_JSON_CREDS_BASE64"))), f
        )
So all things google should be configured
d
And GOOGLE_APPLICATION_CREDENTIALS is set to an absolute path?
b
Relative path.
Copy code
GOOGLE_APPLICATION_CREDENTIALS=./gcp_service_account.json
But... it does work
(not sure if relative vs abs is going to be a problem though)
d
It needs to be absolute from what I recall. That's also set as an environment variable before you run pulumi too?
b
Yeah. It's set before I run pulumi and it's written to disk before the python providers are imported
I'll try an abs path, but the pulumi gcp provider is currently working. It's a bit of a mess because GOOGLE_APPLICATION_CREDENTIALS expects a file path and with pulumi cloud I can only set env variables
It's hard to know where to write that to because I don't know where pulumi execs my code, so relying on /tmp for example seems a little sketchy
Looks like relative paths are 👍 as of 2022: https://stackoverflow.com/a/71198300
d
It depends on where the gke-gcloud-auth-plugin gets run from
b
True
d
You could try running this command, however it's probably best to use the OIDC integration if you're using Pulumi Cloud Deployments
b
Also... big derp again from me. The GOOGLE_APPLICATION_CREDENTIAL path was for a custom python / gcp provider... but I am setting the credentials to the pulumi provider via 
gcp:credentials
Well.. not custom, but I had some customer mappings in bigquery and used those to load... long story but it works. Since 
gcp:credentials
is set, I'd expect it to work. Your point is valid about the GOOGLE_APPLICATION_CREDENTIALS env var might work if there was an abs path, but it's pretty vague about what Pulumi does to configure both the box with those creds and how gke-gcloud-auth-plugin is loaded and configured too.
d
There's an issue open around simplifying kubeconfig use with GKE, though I imagine the first implementation of that will still use gke-gcloud-auth-plugin (which I'm also not a fan of 🙂) https://github.com/pulumi/pulumi-gcp/issues/1064
The way to go really does seem to be to use oidc instead of SA keys though
75 Views
Open in Slack
PreviousNext
Powered by