Thanks @dry-journalist-60579 for sharing an oidc role setup that works. • pulumi deployments assumes the role

arn:aws:sts::MGT_ACCT_ID:assumed-role/PulumiOIDC/pulumi

in my control tower management account • in

Pulumi.dev.yaml

I have

aws:assumeRole

set to

arn:aws:iam::APP_ACCT_ID:role/AWSControlTowerExecution

• when I log the result of

aws.getCallerIdentity

I get

arn:aws:sts::APP_ACCT_ID:assumed-role/AWSControlTowerExecution/aws-go-sdk-1683049420464527015

, which shows me that `aws:assumeRole`is working • … but I’m no longer able to push a docker image to ecr. It seems the code invoked by

docker.Image

doesn’t use the role given by `aws:assumeRole`:

docker:index:Image APP_IMAGE updating (1s) [diff: ~build];
warning: Failed to pull cached image <http://APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest|APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest>:
Error pulling cached image <http://APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest|APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO:latest>:
Error response from daemon: pull access denied for <http://APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO|APP_ACCT_ID.dkr.ecr.us-east-2.amazonaws.com/APP_REPO>,
repository does not exist or may require 'docker login': denied:
User: arn:aws:sts::MGT_ACCT_ID:assumed-role/PulumiOIDC/pulumi is not authorized
to perform: ecr:BatchGetImage on resource: arn:aws:ecr:us-east-2:APP_ACCT_ID:repository/APP_REPO
because no resource-based policy allows the ecr:BatchGetImage action

ahh interesting—haven’t run into this as we have a separate CI/CD process on buildkite that builds images and pushes them

ahhh I like that

❤️ Yeah we set up the Repo and RepoPolicy with Pulumi but use buildkite for the building/pushing… One thing to note is that there may be a bug where pulumi shows that there’s an update for the repo every pulumi run, but it doesn’t cause any issues… just a tad annoying: https://pulumi-community.slack.com/archives/CRH5ENVDX/p1679512594322899

Okay, so I confirmed that if I do a

local.Command

(or

local.run

which I just discovered lol) of the aws cli, it runs as the original role and not the

aws:assumeRole

. So the result of executing this code is this:

current caller identity: arn:aws:sts::APP_ACCT_ID:assumed-role/AWSControlTowerExecution/aws-go-sdk-1683051997761320969
    {
        "UserId": "USER_ID:pulumi",
        "Account": "MGT_ACCT_ID",
        "Arn": "arn:aws:sts:MGT_ACCT_ID:assumed-role/PulumiOIDC/pulumi"
    }

ahh interesting, good to know!

Trying out building in CI (github actions in my case) as you suggested, @dry-journalist-60579 A couple of twists: • where previously I created the repo, built the image, and created the aws lambda which uses the image in the same pulumi program, now I rely on the image being built/pushed first. But to push the image I need the repo to exist. So I can either create the repo in github actions, or accept an initial run of the pulumi program that creates the repo but fails when it doesn't find the image • now that I rely on images being created external to the pulumi program, I have the ability to parametrize the pulumi deployment with an image tag ◦ this would let me rollback easily ◦ I could just use "latest," but I'm not sure if the lambda would be updated when a new image is updated, since its imageUri wouldn't change ◦ would the tag be an environment variable? a config setting?

hmm so I don’t know how this relates to what you’re doing, but our infra is in a separate monorepo called

infrastructure

with all our Pulumi projects. Each Pulumi project uses Pulumi Deployments set up via GitHub integration to deploy the infrastructure that will act as a … “receptacle” for our application. Our application lives in another repo with its own CI/CD process on BuildKite (but could be GitHub Actions) where the images are built and pushed to the registry

Well, currently I'm keeping shared infrastructure like vpcs in a separate repo. I'm keeping application-specific infrastructure in the same repo as the application code itself, although that detail isn't especially relevant here. The application in this case is basically just a containerized lambda. So the chicken and egg problem is that the lambda will be deployed by the app-specific pulumi stack, but the lambda depends on the image being deployed, which depends on the repo existing. But the repo feels like it should be deployed by the app-specific pulumi stack. Not a huge deal, really!