Hello! :wave: I’m still trying to automate our inf...
# general
m
Hello! 👋 I’m still trying to automate our infrastructure using Github Actions. 😁 I’m trying to use Pulumi to create and push an image to ECR, but i can’t authenticate to ECR from a Github Action. Locally, i can inspect and push the image normally, so i’m not sure if there are any differences between how Pulumi handles the registry authentication locally vs using a CI provider.
Looking at the implementation, i wonder if the login result is being persisted to the machine. https://github.com/pulumi/pulumi-docker/blob/master/sdk/nodejs/docker.ts#L447-L450
w
I am surprised that that could happen.
docker
itself stores logins in the filesystem, but I would not expect that to be a problem. @most-pager-38056 Do you see any messages about logging in in the output previous to what's in the screenshot? cc @lemon-spoon-91807 any thoughts? Is there additional logging @most-pager-38056 could turn on to get more of the diagnostic events?
l
what happens if you just run that command manually?
i'm curious what error is actually being produced.
m
I don’t see any error messages in the action output. And i believe Pulumi would exit the process before building the image if the login couldn’t be completed, that happened to me sometimes when i was implementing the authentication. 😅 The error itself comes from
docker inspect
, i imagine this command runs after
docker login
.
l
Hi @most-pager-38056 can you clarify this
I don’t see any error messages in the action output.
is that if you run the command manually?
i wonder if the login result is being persisted to the machine.
m
My bad. I don’t see any login error messages in the action output.
l
'docker login' shoudl be doing that automatically. it is persisted into: .docker/config.json
unless you have a credential manager
What happens if you try to run the
docker image inspect ...
command manually? Thanks!
m
Yes, the problem is that i’m using Github Actions, so i can’t login into the machine and check these files. =/
l
ick.
m
I thought it could be something related about how credentials are being persisted because Pulumi should immediately stop the update if credentials are not valid. I’m pretty sure that’s not the case, a lot of resources are updated before the docker image itself and all these resources use the same AWS credentials.
l
If we got some error from docker we should always be logging it as per:
w
@most-pager-38056 Do you see
Executing 'docker login' ...
anywhere in your output?
l
Copy code
if (stderr.length > 0) {
                if (code && !reportErrorAsWarning) {
                    // Command returned non-zero code.  Treat these stderr messages as an error.
                    pulumi.log.error(stderr, logResource, streamID);
                }
                else {
                    // command succeeded.  These were just warning.
                    pulumi.log.warn(stderr, logResource, streamID);
                }
}
so you should at least see warnings or errors printed. so, it seems like the command is terminating in error, but without having printed any console output.
w
@lemon-spoon-91807 The error he is getting appears to be
Error response from daemon: Forbidden
. It is not clear to me whether the library every tried to do
docker login
.
m
@white-balloon-205 no, i don’t see these messages.
Before the docker build steps, i see all the resources preview.
w
Could you share the whole log somewhere? DM with @lemon-spoon-91807 and I, or email at luke@pulumi.com work if it's something you can share.
l
Ah.
We have this bug that we fixed:
There was a loic bug where if you had multiple images with the same repo, we could attempt to starting working on the second image prior to the first completing its login step, causing us to fail with authentication errors
m
@white-balloon-205 done! I just sent via DM to @lemon-spoon-91807.
I’m using the
dev
branch, so i imagine the fix is already there, right? (ah… i’m only building a single image right now)
l
can you take a look at the contents of the docker.ts or .js that you're pulling in?
i want to see if it has:
(or send to me in dm)
i need to see what loginToRegistry looks like in it
m
Yes! I’m going to send you right now.
l
actually... @white-balloon-205 i don't see how this could be a login issue... the pull completed. and to pull you have to login first.
instead of the logs @most-pager-38056 can you send me the actual pulumi code you've written?
also, it would likely be good to have you run that code locally versus on ci/cd, so we can look at the difference in logs
m
Yep! I just sent you via DM.
l
to see why it succeeds locally for you, but not in github
m
Hi! Just sending an update. Github Actions simply doesn’t allow the
docker inspect
command. 😑 I’ve tried to authenticate manually using the Docker Login action before Pulumi and i’ve gotten the following error:
This Docker operation is forbidden by GitHub Actions, you can find documentation at https://developer.github.com/actions/
🤨
I ended up moving to CircleCI. Everything runs perfectly now, thanks! 😅