So I’m setting up Pulumi Deployments and have set up an OIDC provider and role in an AWS account that Pulumi deployment executor can assume. Now… any recommendations for set up if different stacks are in different AWS accounts? E.g. we have

dev

,

staging

, and

prod

stacks each in a separate account. Do I need to set up the OIDC provider and role to assume in every account? Or can I have one and somehow use cross-account role assuming?

You can do either. I think which is better is dependent on how you access your accounts for other reasons. For example, if you prevent all direct access to sensitive accounts and require people to assume roles to get into those accounts, then you should probably stick with that approach for deployments. Having one role that allows deployments in your sensitive account is good, because you only have to make changes once in order to update deployments, whether it's from Pulumi Deployments (OIDC) or directly via a super-developer account of some sort. If you have no reason to use cross-account role assumption at the moment (e.g. SSO access to each account, managed by something like Okta), then an OIDC connection per account is a reasonable way to go. If neither of these conditions apply, then it probably comes down to personal preference. Do you prefer to have all your Pulumi-AWS auth configuration in a single role, with a single trust relationship? Easy to find all the info you ever need (on the topic) in one place, but also, easy to mess up typing all that JSON. For simpler resources spread out over more places (accounts), then have multiple OIDC connections.

So I’m using Pulumi to bootstrap the creation of the OIDC provider and the role that the Pulumi Deployment executor will be assuming during automatic deployments to other stacks. This bootstrap stack is obviously one I need to run manually from the CLI locally. We basically have the following projects/stacks: • app-project ◦ dev-stack ◦ staging-stack ◦ prod-stack • bootstrap-project ◦ prod-stack And the following AWS accounts (set up à la Control Tower): • management (id:

XXXXXX1

) • workloads-app-dev (id:

XXXXXX2

) • workloads-app-staging (id:

XXXXXX3

) • workloads-app-prod (id:

XXXXXX4

) • workloads-cicd (id:

XXXXXX5

) In the bootstrap-project I am now creating a single OIDC provider and a role that Pulumi can assume using

AssumeRoleWithWebIdentity

in the

workloads-cicd

account. Now, I’ve set up the Pulumi Deployments configuration in the UI (anxiously awaiting https://github.com/pulumi/service-requests/issues/169 to not have to use the UI) to use the Role ARN created from the bootstrap stack. The app/dev stack targets the workloads-app-dev account, but I can’t quite figure out the right way to do this. I guess I’ll need a role within that account that the aforementioned role can assume and I’ll need to add that to my

aws:assumeRole:

in app-project/Pulumi.dev.yaml? Right now I’m getting an error during the deployment/update of:

AWS account ID not allowed: XXXXXX5

because I have a

aws:allowedAccountIds: [XXXXXX2]

in the app/dev stack to ensure the correct placement… Sorry if that is a lot of context. Ideally I’d prefer to not create the OIDC provider in each account.

My suggestion would be to enhance the bootstrap-project code to cover the Control Tower work, including creating the roles in all the other accounts that trust the role in the workloads-cicd account. You might even create exports from that project for outputting Pulumi code that you can use as a template for creating AWS providers in projects that deploy to those accounts.

const stagingProvider = new aws.Provider("staging", { assumeRole: deploymentRoleInStagingAccount });

Ah thank you, makes sense! I was leaning that way… so yeah, loop over all the accounts in my bootstrap stack to create a PulumiDeploymentRole that trusts the one in the

workloads-cicd

account. I can then specify that the app/dev stack should use the PulumiDeploymentRole either by explicitly creating a provider in the code or by specifying it in the Pulumi.dev.yaml:

config:
  aws:region: us-east-1
  aws:allowedAccountIds: [XXXXXX2]
  aws:assumeRole:
    roleArn: arn:aws:iam::XXXXXX2:role/PulumiDeploymentRole

You can do that, though that relies on the default provider.. against which I have a personal gripe 🙂

ahh interesting—do you always create an explicit provider?

Always. Coding style guide / code reviews, Pulumi Policy, transformations and the Pulumi setting all prevent it.

So interestingly I’ve found I have to use the default provider for my bootstrap stack because it’s the only way I can assume a role to assume a role as the

provider

code does not accept another provider

I'd have to see that to understand...

When running locally, I’m using aws sso

Then use your SSO profile.

so my active profile is:

role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AWSAdministratorAccess_xxx

You can use

new aws.Provider("bootstrap", { profile: new pulumi.Config("aws").profile })

.

hmmmm 🧠 working….

The default AWS profile is just that, except with logic around AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY too.

arn:aws:iam::XXX:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AWSAdministratorAccess_xxx

=>

arn:aws:iam::XXX:role/service-role/AWSControlTowerAdmin

=>

arn:aws:iam::{0}:role/AWSControlTowerExecution

organization = aws.organizations.get_organization()

for account in organization.accounts:
    # Skip deactivated accounts and the management account
    if account.status != "ACTIVE":
        continue

    role_arn = pulumi.Output.format(
        "arn:aws:iam::{0}:role/AWSControlTowerExecution", account.id
    )

    # Create Provider to assume role
    provider = aws.Provider(
        f"Provider: {account.name}",
        assume_role={
            "roleArn": role_arn,
        },
    )

    password_policy = aws.iam.AccountPasswordPolicy(
        f"AccountPasswordPolicy: {account.name}",
        **PASSWORD_PARAMS,
        opts=pulumi.ResourceOptions(provider=provider),
    )

In the bootstrap project, that seems reasonable. I note that that account.id is an output but account.name isn't.. is that right? You might be missing some interpolation?

hmm it seems to just work like that

Good stuff. The workloads provider is created before the loop? That looks right.

Thank you for the help, @little-cartoon-10569!