https://pulumi.com logo
Powered by
#automation-api
Title
# automation-api
l

lemon-dog-29241

09/13/2021, 1:08 AM
Hey there everyone! I am trying to launch pulumi in my aws account and have it create resources in a different account. I already have a role set up that allows my lambda admin access (just for testing) to the other account. How do I pass in that role to pulumi? Currently I'm passing in the aws access key and secret, and I'm just not sure how I pass in a role instead...?
l

little-cartoon-10569

09/13/2021, 1:29 AM
This is an AWS problem rather than a Pulumi one. There's a few ways. The way I do it is to create profiles. One profile will have the access and secret keys, the other has a role and a source profile (which will be the first one). This is documented in a load of places.. for some reason the PHP devguide is the first on my search results... https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_profiles.html
l

lemon-dog-29241

09/13/2021, 1:43 AM
@little-cartoon-10569 Does this mean I would need to store the second (so not-my-main-account) profile locally? I'm trying to do everything progromatically, so that there's no interaction from getting the role arn to launching the resources in not-my-main-account. I see this, but I'm not sure that's exactly what I'm looking for.
l

little-cartoon-10569

09/13/2021, 1:44 AM
You can do it that way, and Pulumi assumes the role for you. I prefer not to do that, since then Pulumi has access to two sets of information: the original user, and the assumed role.
Note that everything we're talking about is within one account, and just one IAM user within that account.
l

lemon-dog-29241

09/13/2021, 1:45 AM
Ohhhh. Yeah I am looking for cross-account stuff. Any ideas for that?
l

little-cartoon-10569

09/13/2021, 1:46 AM
This all applies to cross-account stuff too. The ARN of the role you assume includes information about the account you're assuming into.
āœ… 1
l

lemon-dog-29241

09/13/2021, 1:46 AM
I'm running pulumi on a lambda, and was hoping I could have lambda assume the cross-account role (well, I know I can do this). But then I'm not sure how to have pulumi use that role too
l

little-cartoon-10569

09/13/2021, 1:47 AM
Don't lambdas run under a role, like EC2 instances run under an instance profile?
If that role has your permissions, that should be enough.
FWIW I don't think running Pulumi in a lambda is often recommended. Lambdas have memory and duration limitations that Pulumi might not conform to....
If you want to use the lambda's role to allow Pulumi to assume a 2nd role, that means that anything running in that lambda will be able to assume that 2nd role. Is that ok?
l

lemon-dog-29241

09/13/2021, 1:49 AM
But I am currently passing in access and secret keys to my pulumi stack:
Copy code
stack.set_config("aws:accessKey", auto.ConfigValue(value="accesskeyhere"))
stack.set_config("aws:secretKey", auto.ConfigValue(value="secretkeyhere"))
So I guess my wuestion is what do I pass in there? Because all I have from the not-my-account is the role
yeah it should be fine. i haven't had any issues with memory or duration limits yet šŸ¤ž
l

little-cartoon-10569

09/13/2021, 1:51 AM
This is the page you want, since you're trying to configure the Pulumi provider: https://www.pulumi.com/docs/reference/pkg/aws/provider/
You need to add the assumeRole config option.
āœ… 1
If you weren't in a lambda, I'd suggest also replacing the accessKey and secretKey with a profile.
l

lemon-dog-29241

09/13/2021, 1:53 AM
And do I need to add the resource options for every resource so that it uses this provider I set up? Or will it default to this provider?
can I set the provider as a part of my project settings?
l

little-cartoon-10569

09/13/2021, 2:20 AM
Settings the aws:accessKey etc. properties configures the default provider, so you shouldn't need anything else.
If you create a new provider with your settings, instead of using the default one, then you need to pass that provider into every resource.
The normal pattern for ComponentResources is to pass the same opt into all resources, so normally that's fine.
l

lemon-dog-29241

09/13/2021, 3:27 AM
@little-cartoon-10569 Thanks so much! I appreciate all your feedback here!
šŸ‘ 1
b

billowy-army-68599

09/13/2021, 4:53 AM
@lemon-dog-29241 you can assume the role from within the provider: https://www.pulumi.com/docs/reference/pkg/aws/provider/#providerassumerole as part of the project settings if neede
AWS profiles are indeed one way, but you can do it this way if necessary
šŸ™ 1
l

lemon-dog-29241

09/14/2021, 1:04 AM
thank you @billowy-army-68599!
@billowy-army-68599 How would I go about doing it the above way with the automation API? I don't currently have a designated provider block, I just set up my project, stack, and pass in my access key and secret as config values. I tried setting a config value for 
aws:assumeRole.roleArn
but that didn't work. Then I tried doing that and setting my aws secret and access key and that didn't work either. I'm getting the error: 
* Invalid or unknown key
. I also saw in some docs somewhere that when you set up a provider, you have to pass them into the 
ResourceOptions
for the resources you want to use that provider for ... what if I want all resources to be created with this role?
Copy code
project_settings = auto.ProjectSettings(
            name=project_name,
            runtime="python",
            main="/tmp",
            backend=auto.ProjectBackend(url=f"<s3://pulumi-state>-{org_id}/{state_path}"))
stack = auto.create_or_select_stack(stack_name=stack_name,
                                project_name=project_name,
                                program=program,
                                work_dir="/tmp",
                                opts=auto.LocalWorkspaceOptions(project_settings=project_settings))
stack.workspace.install_plugin("aws", "v4.15.0")
stack.set_config("aws:assumeRole.roleArn", auto.ConfigValue(value=EnVar))
stack.set_config("aws:accessKey", auto.ConfigValue(value="EnvVar"))
stack.set_config("aws:secretKey", auto.ConfigValue(value="EnVar"))
b

billowy-army-68599

09/16/2021, 11:22 AM
@lemon-dog-29241 see this issue reply from @red-match-15116 https://github.com/pulumi/pulumi/issues/6700#issuecomment-813119114
āœ… 1
l

lemon-dog-29241

01/19/2022, 4:46 PM
@red-match-15116 Hey there! šŸ™‚ I took a break from my project and am now back to working on this and found I am still having trouble actually getting this cross-account roll sutff to work. ā€¢ I am running pulumi in lambda. ā€¢ I want to have the lambda in my account launch resources into another user's account (currently just my own second test account). ā€¢ I have cross-account permissions, roles, trust-policies, etc all set up. ā€¢ In my lambda, I'm assuming the role I need in order to deploy resources into the test account. ā€¢ Then, I'm taking that output (an access key and secret) and trying to pass that into the pulumi project settings. Buuuut... it's not working šŸ˜• Hoping you might be able to help since you were so awesome last time!
b

billowy-army-68599

01/19/2022, 4:54 PM
just to be clear, your pulumi program RUNS inside a lambda function?
l

lemon-dog-29241

01/19/2022, 4:55 PM
yep
b

billowy-army-68599

01/19/2022, 4:58 PM
okay, the steps you need are: ā€¢ be extra sure ensure your lambda function has the right role. you''ll more than likely do an 
aws sts assume-role
call inside the lambda, and then you need to check you have the right role with 
aws sts get-caller-identity
- do all of this before you do anything with Pulumi Once you've done that, your Pulumi program can either: ā€¢ use ambient/env var credentials ā€¢ use configuration options One thing you haven't mentioned is setting a session token. the STS service with assume role also emits a session token, and you'll need to pass that to do cross account provisioning
šŸ™Œ 1
another, unrelated thing: lambda has an aggressive timeout, and you'll likely run into it when running Pulumi there, I did šŸ˜‰
if your pulumi program takes 15 mins to run, it's going to time out in lambda
l

lemon-dog-29241

01/19/2022, 5:01 PM
Awesome thanks! yeah my stacks are really small and take under a couple minutes to build. šŸ™‚ So I think it should work out haha. Am I using the correct configuration for the AWS access key and secret? Like, is it "aws:accessKey" and "aws:secretKey"? Same question them with the session token -- what is pulumi expecting? "aws:sessionToken"?
b

billowy-army-68599

01/19/2022, 5:04 PM
weirdly, session token is defined as "token" for some reason https://www.pulumi.com/registry/packages/aws/api-docs/provider/#token_nodejs
I'm very intrigued by what you're building too, once you get it working, would you be interested in blogging about it with us?
šŸ™ 1
l

lemon-dog-29241

01/19/2022, 5:07 PM
sure thing! šŸ™‚ I'm really excited about it too and would be happy to share more once it's in a demo-able state. I'm gonna try access_key, token, and secret_key, as well as verifying my role. If that fails.... I'll probably be back šŸ˜‚
r

red-match-15116

01/19/2022, 10:40 PM
ooh yeah this does sound super interesting! glad @billowy-army-68599 was able to help you out and looking forward to hearing more about it
f

fresh-minister-66960

11/24/2022, 7:48 AM
@billowy-army-68599 @lemon-dog-29241 I am in the process of building something very similar (running Pulumi code in a lambda function to provision resources on other accounts). Everything was going well but seems like now Provider requires a Profile and since I am sending AWS KEY ID and AWS SECRET KEY it stopped working and now I get the error that no AWS credentials are set. Do you have an alternative to this? It has been such a pain to implement this thing in a lambda that I am thinking about using a different AWS service.
l

lemon-dog-29241

12/02/2022, 7:56 PM
@fresh-minister-66960 I haven't run into that issue honestly. The way I do it is by assuming a role. hang on lemme find the relevant code. So I use those values but also a session token which might be why I'm not having issues. Although I probably haven't updated my pulumi version in a bit either.
Copy code
def main(org_id, env, state_path, program, role_arn):
    try:
        client = boto3.client('sts')
        ...
        response = client.assume_role(RoleArn=role_arn,
                                      RoleSessionName='GetBuildBot',
                                      ExternalId=org_id)

        os.environ['PULUMI_CONFIG_PASSPHRASE'] = f'{project_name}{stack_name}'
        os.environ['AWS_ACCESS_KEY_ID'] = response['Credentials']['AccessKeyId']
        os.environ['AWS_SECRET_ACCESS_KEY'] = response['Credentials']['SecretAccessKey']
        os.environ['AWS_SESSION_TOKEN'] = response['Credentials']['SessionToken']
Unless you mean something else? I might be able to help if you share a bit more.
l

little-cartoon-10569

12/03/2022, 1:07 AM
The AWS provider doesn't require a profile. Don't set the profile if you're not using it. The provider supports using explicit accessKey and secretKey if you want. If you're finding that it's using a profile when you don't want it to, check your environment variables. If AWS_PROFILE is set, it will use that.
Powered by