https://pulumi.com logo
Title
f

fresh-spring-82225

05/02/2023, 6:01 PM
Thanks @dry-journalist-60579 for sharing an oidc role setup that works. • pulumi deployments assumes the role
arn:aws:sts::MGT_ACCT_ID:assumed-role/PulumiOIDC/pulumi
in my control tower management account • in
Pulumi.dev.yaml
I have
aws:assumeRole
set to
arn:aws:iam::APP_ACCT_ID:role/AWSControlTowerExecution
• when I log the result of
aws.getCallerIdentity
I get
arn:aws:sts::APP_ACCT_ID:assumed-role/AWSControlTowerExecution/aws-go-sdk-1683049420464527015
, which shows me that `aws:assumeRole`is working • … but I’m no longer able to push a docker image to ecr. It seems the code invoked by
docker.Image
doesn’t use the role given by `aws:assumeRole`:
docker:index:Image APP_IMAGE updating (1s) [diff: ~build];
warning: Failed to pull cached image <http://APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest|APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest>:
Error pulling cached image <http://APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest|APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest>:
Error response from daemon: pull access denied for <http://APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO|APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO>,
repository does not exist or may require 'docker login': denied:
User: arn:aws:sts::MGT_ACCT_ID:assumed-role/PulumiOIDC/pulumi is not authorized
to perform: ecr:BatchGetImage on resource: arn:aws:ecr:us-east-2:APP_ACCT_ID:repository/APP_REPO
because no resource-based policy allows the ecr:BatchGetImage action
Here’s a snippet showing how I create the docker image. It works on my laptop if I assume an admin role in the app account and comment out
aws:assumeRole
I wonder if the
aws ecr get-login-password
command is the problem …
d

dry-journalist-60579

05/02/2023, 6:19 PM
ahh interesting—haven’t run into this as we have a separate CI/CD process on buildkite that builds images and pushes them
f

fresh-spring-82225

05/02/2023, 6:23 PM
ahhh I like that
okay, yes, that is a good separation of build from deployment @dry-journalist-60579
sometimes when a problem seems unnecessarily hard it’s because I’m trying to solve the wrong problem lol
d

dry-journalist-60579

05/02/2023, 6:32 PM
❤️ Yeah we set up the Repo and RepoPolicy with Pulumi but use buildkite for the building/pushing… One thing to note is that there may be a bug where pulumi shows that there’s an update for the repo every pulumi run, but it doesn’t cause any issues… just a tad annoying: https://pulumi-community.slack.com/archives/CRH5ENVDX/p1679512594322899
f

fresh-spring-82225

05/02/2023, 6:36 PM
Okay, so I confirmed that if I do a
local.Command
(or
local.run
which I just discovered lol) of the aws cli, it runs as the original role and not the
aws:assumeRole
. So the result of executing this code is this:
current caller identity: arn:aws:sts::APP_ACCT_ID:assumed-role/AWSControlTowerExecution/aws-go-sdk-1683051997761320969
    {
        "UserId": "USER_ID:pulumi",
        "Account": "MGT_ACCT_ID",
        "Arn": "arn:aws:sts:MGT_ACCT_ID:assumed-role/PulumiOIDC/pulumi"
    }
Okay so it looks like if I use
aws.ecr.getAuthorizationTokenOutput
instead of running
local.Command
with
aws ecr get-login-password
then it works
d

dry-journalist-60579

05/02/2023, 9:23 PM
ahh interesting, good to know!
f

fresh-spring-82225

05/04/2023, 7:12 PM
Trying out building in CI (github actions in my case) as you suggested, @dry-journalist-60579 A couple of twists: • where previously I created the repo, built the image, and created the aws lambda which uses the image in the same pulumi program, now I rely on the image being built/pushed first. But to push the image I need the repo to exist. So I can either create the repo in github actions, or accept an initial run of the pulumi program that creates the repo but fails when it doesn't find the image • now that I rely on images being created external to the pulumi program, I have the ability to parametrize the pulumi deployment with an image tag ◦ this would let me rollback easily ◦ I could just use "latest," but I'm not sure if the lambda would be updated when a new image is updated, since its imageUri wouldn't change ◦ would the tag be an environment variable? a config setting?
d

dry-journalist-60579

05/04/2023, 8:12 PM
hmm so I don’t know how this relates to what you’re doing, but our infra is in a separate monorepo called
infrastructure
with all our Pulumi projects. Each Pulumi project uses Pulumi Deployments set up via GitHub integration to deploy the infrastructure that will act as a … “receptacle” for our application. Our application lives in another repo with its own CI/CD process on BuildKite (but could be GitHub Actions) where the images are built and pushed to the registry
I’m not sure I have a good understanding of your “application” here
f

fresh-spring-82225

05/04/2023, 9:36 PM
Well, currently I'm keeping shared infrastructure like vpcs in a separate repo. I'm keeping application-specific infrastructure in the same repo as the application code itself, although that detail isn't especially relevant here. The application in this case is basically just a containerized lambda. So the chicken and egg problem is that the lambda will be deployed by the app-specific pulumi stack, but the lambda depends on the image being deployed, which depends on the repo existing. But the repo feels like it should be deployed by the app-specific pulumi stack. Not a huge deal, really!