https://pulumi.com logo
c

colossal-quill-8119

07/20/2023, 5:21 PM
@many-telephone-49025 As you asked me to tag you on twitter i’m tagging you here. I’ve been trying to use pulumi deployment to build a docker image, push it to GAR and then deploy a cloud run service. But this requires I run
gcloud auth configure-docker <http://asia-south2-docker.pkg.dev|asia-south2-docker.pkg.dev>
which does run successfully. But after that pulumi crashes with following error
You do not currently have an active account selected. See <https://cloud.google.com/sdk/docs/authorizing> for more information.
. I’m using Workload Identity Federation and the service acc has all the permissions it should.
m

many-telephone-49025

07/20/2023, 5:22 PM
hey @colossal-quill-8119, do you have a way to share your code as gtist? and can you print your
pulumi about
that would help to understand it better
c

colossal-quill-8119

07/20/2023, 5:23 PM
Give me a min i’ll share it
output from pulumi about:
Copy code
CLI
Version      3.75.0
Go Version   go1.20.6
Go Compiler  gc

Plugins
NAME    VERSION
docker  4.3.0
gcp     6.56.0
nodejs  unknown

Host
OS       darwin
Version  13.4.1
Arch     arm64

This project is written in nodejs: executable='/Users/debkanchan/.nvm/versions/node/v18.12.1/bin/node' version='v18.12.1'

Backend
Name           <http://pulumi.com|pulumi.com>
URL            <https://app.pulumi.com/DebkanchanSamadder>
User           DebkanchanSamadder
Organizations  DebkanchanSamadder, ride

Dependencies:
NAME            VERSION
@types/node     16.18.32
@pulumi/docker  4.3.0
@pulumi/gcp     6.56.0
@pulumi/pulumi  3.68.0

Pulumi locates its logs in /var/folders/7d/cln6swv17mq5fmjkpd6z6hnr0000gn/T/ by default
warning: Failed to get information about the current stack: No current stack
s

salmon-account-74572

07/20/2023, 5:37 PM
We recently had to make some changes to some of our GCP container templates to accommodate the switch from GCR to GAR. As part of that, I had to spend a fair amount of time in Docker authentication. If you are using the
docker-credential-gcloud
credential helper (the default), then you need to make sure you run
gcloud auth login
before running a Pulumi operation. If you are using the
docker-crdedential-gcr
credential helper, then you need to run
gcloud auth application-default login
before running a Pulumi operation. This isn’t something that we can work around in Pulumi code, AFAIK.
I’ll go back and double-check some of my notes and add more information here as needed. But I would suggest you make sure you’ve run both
gcloud auth login
and
gcloud auth application-default login
.
b

billowy-army-68599

07/20/2023, 5:40 PM
there is a previous convo here about this: https://pulumi-community.slack.com/archives/CRFUR2DGB/p1689342329458449 Looks like - using deployments with OIDC complicates this a little
c

colossal-quill-8119

07/20/2023, 5:43 PM
@salmon-account-74572 running both
gcloud auth login
and
gcloud auth application-default login
as a Pre-run command emits a browser link to login
s

salmon-account-74572

07/20/2023, 5:45 PM
Yes, it will do that. I don’t know of any other way around getting appropriate credentials for Docker to auth properly against GAR. However, I have a couple of contacts at Google, let me try contacting them to see if they can offer any insight.
c

colossal-quill-8119

07/20/2023, 5:46 PM
Thank you. Please keep me updated this is really important for us.
s

salmon-account-74572

07/20/2023, 5:48 PM
You got it---I just reached out to one of my contacts and will let you know as soon as I hear back.
c

colossal-quill-8119

07/20/2023, 5:49 PM
Thanks
s

salmon-account-74572

07/20/2023, 8:19 PM
This isn’t an answer to your question, but I wanted to point out that we have a #pulumi-deployments channel here that you might find helpful. Still waiting to hear back from my Google contact.
c

colossal-quill-8119

07/25/2023, 7:22 AM
@salmon-account-74572 any updates from google?
s

salmon-account-74572

07/25/2023, 3:20 PM
Unfortunately, my contact at Google wasn’t able to offer anything different than what we’ve already suggested. Are you a customer (i.e., do you pay for Pulumi Cloud)? If so, then I’d suggest opening a support ticket. If not, that’s OK; I’m going to keep digging on this.
c

colossal-quill-8119

07/25/2023, 4:21 PM
Unfortunately I'm not on an enterprise plan. I am on the team plan but was unable to file a ticket
s

salmon-account-74572

07/26/2023, 3:31 PM
@colossal-quill-8119 There is a thread going internally on your problem, and we’ve opened an issue in our Pulumi Cloud repo to look into how our OIDC integration works (so that we can better support
gcloud
being able to find credentials where it expects credentials). One question you had in the other thread was in regard to running
gcloud auth activate-service-account
. One of our engineers had this suggestion: 1. Set the contents of
key-file
as a secret environment variable in the Deployment configuration 2. Have a pre-run command that echos the environment variable into a file on disk 3. Run the
gcloud auth activate-service-account
command referencing the file that was written to disk Would you mind trying that and seeing if it helps the situation at all? Thanks!
c

colossal-quill-8119

07/26/2023, 3:36 PM
I can try that but I won't be able to revert back with the results before Friday. Is that ok?
s

salmon-account-74572

07/26/2023, 3:45 PM
No worries, I just want to try to get you unstuck. Give it a try and let me know (when you’re able) how it turns out.
b

bland-dog-47600

07/27/2023, 9:00 PM
Hi folks! I had the same issue and Scott’s advice is spot on. I added a pre-run command and successfully built an image. Thanks! 🙏🏻
s

salmon-account-74572

07/27/2023, 9:25 PM
Awesome, thanks for confirming it works and for sharing your experience @bland-dog-47600!
@colossal-quill-8119 Let me know (when you have time) if you see the same success.
c

colossal-quill-8119

07/29/2023, 10:36 AM
I tried it and it build and pushes successfully. But there’s now a new issue regarding using that image to deploy a cloud run revision. I’m getting the following error:
Copy code
gcp:cloudrun:Service (service): 
245
     error: 1 error occurred: 
246
     	* updating urn:pulumi:dev::api-gateway::gcp:cloudrun/service:Service::service: 1 error occurred: 
247
     	* Error updating Service "locations/asia-east1/namespaces/ride-app-dev-2/services/api-gateway": Put "<https://asia-east1-run.googleapis.com/apis/serving.knative.dev/v1/namespaces/ride-app-dev-2/services/api-gateway?alt=json>": oauth2/google: unable to generate access token: Post "<https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/cloud-run-service-manager@ride-app-dev-2.iam.gserviceaccount.com:generateAccessToken>": oauth2/google: status code 400: {"error":"invalid_target","error_description":"The target service indicated by the \"audience\" parameters is invalid. This might either be because the pool or provider is disabled or deleted or because it doesn't exist."}
b

bland-dog-47600

07/29/2023, 5:41 PM
@colossal-quill-8119 I had a similar problem. https://pulumi-community.slack.com/archives/C048NVDH6DV/p1690173567200419?thread_ts=1690170041.730299&amp;cid=C048NVDH6DV The audience must be your organisation name as in Pulumi.
c

colossal-quill-8119

07/29/2023, 7:08 PM
@bland-dog-47600 how do I do that? What’s the assertion I need to make? I already have my org name as allowed audiences
b

bland-dog-47600

07/29/2023, 7:31 PM
You don’t have to specify assertions.
c

colossal-quill-8119

07/29/2023, 7:35 PM
so I have my org already enabled as allowed audiences
@salmon-account-74572 It works perfectly now! Although it’s still a workaround. Looking forward to direct docker auth support through oidc
s

salmon-account-74572

07/31/2023, 2:27 PM
@colossal-quill-8119 Glad it’s working! As you mentioned, though, the ultimate “fix” belongs to Docker. Our OIDC support enables us to create resources, but it’s the Docker-Artifact Registry auth that was tripping things up.