https://pulumi.com logo
#aws
Title
# aws
w

wooden-action-47328

03/06/2024, 5:21 PM
Can a single pulumi project manage stacks in different AWS accounts? Details in thread.
Right now for my POC projects, I am using an S3 backend state bucket which lives in the AWS account where the resources will go. That seems fine - the backend url config attribute in
Pulumi.yaml
references that bucket and aws profile. But if I want to put staging and production in two different AWS accounts, how do I arrange things to do that? Can the
Pulumi.<stack>.yaml
files have their own backend url config attributes?
Each stack will also have its own AWS role profile for accessing resources
l

little-cartoon-10569

03/06/2024, 7:51 PM
Yes. You configure your AWS provider(s) appropriately. A project can have any number of AWS providers in use, each stack can have all of the providers configured differently.
Can the Pulumi.<stack>.yaml files have their own backend url config attributes?
No. The backend for the project is configured in the project. All the stack metainfo is stored in the one place. All the resources in the stacks can be in any account. There is no requirement for the Pulumi information to be stored anywhere near the resources.
w

wooden-action-47328

03/06/2024, 7:54 PM
I’m not sure I follow what you mean by “each stack can have all of the providers configured differently”. Where does that configuration happen?
l

little-cartoon-10569

03/06/2024, 7:55 PM
In code. You don't have to (and I recommend not to) use the default AWS provider. Create as many instances of
aws.Provider
as you need, and pass them to the resources as you construct them. https://www.pulumi.com/registry/packages/aws/api-docs/provider/
This is also how you put resources in different regions.
w

wooden-action-47328

03/06/2024, 7:56 PM
Does this need to be explicitly passed as an argument to each resource?
l

little-cartoon-10569

03/06/2024, 7:56 PM
It really should be.
Allowing resources to be constructed in whatever account the current user has set as their default is dangerous. Don't do it.
If your code doesn't specify, check and double-check, then your infrastructure is at risk. In my opinion.
w

wooden-action-47328

03/06/2024, 7:57 PM
But the project specifies an account to connect to?
l

little-cartoon-10569

03/06/2024, 7:57 PM
Your code should do that. Not your Pulumi configuration, or your AWS configuration.
But by default, your AWS configuration does that. Which means if you run that code on a different machine, something different might happen.
Because the default configuration, which isn't checked, might be pointing somewhere else.
w

wooden-action-47328

03/06/2024, 8:00 PM
We’re using the
backend
attribute in Pulumi.yaml, which specifies a named aws role profile to use for the connection
l

little-cartoon-10569

03/06/2024, 8:00 PM
There are loads of exceptions to this idea/rule, but for any project that could be deployed in a prod or prod-like scenario, then IMO you must ensure that where you're deploying is where you ought to be deploying. Leaving it up to the default configuration is dangerous.
w

wooden-action-47328

03/06/2024, 8:00 PM
We are not using the [default] connection in ~/.aws/credentials
l

little-cartoon-10569

03/06/2024, 8:00 PM
We’re using the backend attribute in Pulumi.yaml, which specifies a named aws role profile to use for the connection
That's the configuration defining where to store the Pulumi state. It's not the configuration used to deploy the actual resources.
w

wooden-action-47328

03/06/2024, 8:00 PM
what determines that?
l

little-cartoon-10569

03/06/2024, 8:01 PM
Instances of
aws.Provider
, or the default AWS provider if you're not specifying that..
So: specify that.
w

wooden-action-47328

03/06/2024, 8:02 PM
does the default aws provider honor the configawsprofile attribute from Pulumi.stack.yaml?
l

little-cartoon-10569

03/06/2024, 8:03 PM
Yes. And if you check that in your index.ts , and throw exceptions when your default profile is configured inappropriately, then you'll be fine.
But who does that?
w

wooden-action-47328

03/06/2024, 8:04 PM
does it matter if the default profile isn’t the correct one if there’s a profile specified in Pulumi.<stack>.yaml? Won’t that override the default?
l

little-cartoon-10569

03/06/2024, 8:09 PM
The profile is specified by name. The creds in that profile could be different on different machines. "myapp-prod" could be a different account id, different region, different SSO information, etc.
w

wooden-action-47328

03/06/2024, 8:10 PM
yes, they absolutely should be though - everyone running this is using their own credentials
l

little-cartoon-10569

03/06/2024, 8:13 PM
Absolutely. And in many cases, people require that a specific project / stack must be deployed to exactly one well-known location, and all other deployment targets are illegal and must be prevented. In order to ensure this is done, you must configure the targets in code. You shouldn't rely on everyone doing it right. Eventually, someone will make a mistake, or maybe even deliberately deploy your valuable IP to the public domain and steal all your hard work.
So you specify the target in code. And the individual credentials are used (via SSO or whatever) and if they don't have permissions to deploy to the explicitly-defined target, then it fails. Correctly.
If you're creating a project, for example like the Pulumi example projects, where it can be deployed anywhere, wherever the user wants it to, in whatever account you like: then use the default AWS credentials. That's perfect! But many times, that's not appropriate.
w

wooden-action-47328

03/06/2024, 8:16 PM
is there documentation on how to configure and use a custom aws provider?
w

wooden-action-47328

03/06/2024, 8:18 PM
That’s kind of thin on details and doesn’t have any examples
l

little-cartoon-10569

03/06/2024, 8:18 PM
w

wooden-action-47328

03/06/2024, 8:19 PM
There’s no way to set a new provider as the default for inheritance without passing it to every resource call?
l

little-cartoon-10569

03/06/2024, 8:21 PM
I recommend against doing that. And setting the require-explicit-providers option in Pulumi (I'll look that up in a second). Not sure if there's a way to set an explicitly-configured provider as the default. That would be marginally better than allowing the unspecified provider to be the default, but it is still (IMO) bad.
w

wooden-action-47328

03/06/2024, 8:22 PM
doesn’t requiring the provider be added to each resource call put a bunch of burden on the people writing those resources to get it right?
l

little-cartoon-10569

03/06/2024, 8:22 PM
All our projects have lint rules to prevent building a project that has a resource that doesn't have an explicit provider. The risk of anything else is too high for us.
No. IDEs sort that.
It's like requiring types. No extra effort once you are expecting it.
w

wooden-action-47328

03/06/2024, 8:23 PM
well, we’re using python and not typescript
l

little-cartoon-10569

03/06/2024, 8:23 PM
The config for disabling default providers:
Copy code
pulumi:disable-default-providers:
    - aws
It goes in your Pulumi config, either Pulumi.yaml or your stack yaml file.
w

wooden-action-47328

03/06/2024, 8:24 PM
also I don’t see where that ai example you linked to passes the provider
l

little-cartoon-10569

03/06/2024, 8:25 PM
Yep I've checked, that AI example creates the providers then doesn't use them! AI loses, again.
w

wooden-action-47328

03/06/2024, 8:25 PM
sad trombone
l

little-cartoon-10569

03/06/2024, 8:26 PM
This is how it should be:
Copy code
function createS3Bucket(bucketName: string, provider: aws.Provider): aws.s3.Bucket {
    return new aws.s3.Bucket(bucketName, {
        bucket: bucketName,
        acl: "private",
    }, { provider: provider});
}
So this has been a long thread about a personal opinion based on experience. It may very well be that your case doesn't need an explicit provider. But in my experience, all projects written outside of the open source world do need explicit providers.
w

wooden-action-47328

03/06/2024, 8:30 PM
I echo a lot of the concerns found here, which has been open for four years: https://github.com/pulumi/pulumi/issues/2059
l

little-cartoon-10569

03/06/2024, 8:38 PM
If you think the changes are appropriate for you, you can absolutely make the changes incrementally. This is what we did. There's no issue configuring an explicit provider to be the same as the default provider, then start changing resources to use the explicit provider. Iirc you need an alias, but that's easy, and you can have a single well-known function set both the provider and the alias. You can update the state file manually, as implied in that archive. I did that in the first project I converted, and it does work. But I believe it's not necessary. And of course you can always make the changes on a branch (with no other changes),
pulumi destroy
on the old branch and
pulumi up
on the new branch, then merge. If that's appropriate for your use case.
Or just make the rule apply from now on, and leave the "legacy" resources as-is! Just because an explicit provider may be better for your use-case does not mean it's essential. For example, you can check the default provider in a business process, or in a wrapping script that calls Pulumi, or even in the Pulumi code itself.
I've been reading your linked issues, and one points to this. Great initial explanation of why explicit providers are good: https://github.com/pulumi/pulumi/issues/3383
w

wooden-action-47328

03/06/2024, 8:44 PM
There’s no reason the custom provider can’t draw config values from the stack configuration file, right?
l

little-cartoon-10569

03/06/2024, 8:44 PM
Absolutely. Strongly recommend doing that. We set the region from stack config (though not the credentials).
w

wooden-action-47328

03/06/2024, 8:45 PM
where do you set the credentials?
l

little-cartoon-10569

03/06/2024, 8:45 PM
We require SSO. Onesec I'll grab a code snippet.
It's a bit hairy, since the AWS CLI tool does not support output in JSON for this! Even though it's a documented option and works for almost everything else 😞
Untitled
So we have a well-known SSO profile name that must exist for the user or else you don't get in. And the region is configured per stack.
Then we check that the SSO profile is in use and that AWS says that it has been authenticated using SSO. We trust the output of the
aws
command, so there is a weakness: someone could replace that command with one that returns valid fake details, which could cause us to deploy to the wrong place.
But this is a risk that is small enough for us to accept.
Oh, the accessRoleArns variable is a hard-coded map of valid role ARNs as deployed by IAM Identity Center. This is a second line of defense: if someone destroys our identity center, then no more deployments will work until we fix the code. This prevents a bad sysadmin-type actor. We would need a colluding dev and operator in order to break this.
w

wooden-action-47328

03/06/2024, 9:01 PM
where do you set
ssoProfileName
?
I’m feeling like that should also be set in the stack config
l

little-cartoon-10569

03/06/2024, 9:57 PM
I'll check. I think it might be a well-known constant, but it could be config. So long as code review is involved, it should be fine either way.
It's a well-known constant for us. Set in code.
w

wooden-action-47328

04/08/2024, 5:39 PM
One of my developers discovered that if you set a value for aws:profile in the Pulumi.<stack>.yaml file, it gets picked up and used by the default provider, and doesn’t require using a custom provider in order to force a profile to use in code. Is this new? Is this documented anywhere?
s

salmon-account-74572

04/08/2024, 7:03 PM
I don't think this is new, but I also don't know if it is explicitly documented. The idea of the default AWS provider is that it will use "ambient" credentials & configuration. Setting "aws:profile" in the stack config file just tells it a specific local configuration to use. You could accomplish the same thing by setting the
AWS_PROFILE
environment variable (say, using something like
direnv
) in the Pulumi project directory.
w

wooden-action-47328

04/08/2024, 8:09 PM
I explicitly do not want to rely on having people set environment variables
Are the precedence rules for how the default provider gets credentials documented anywhere? Can aws:profile be overridden with environment variables?
l

little-cartoon-10569

04/08/2024, 8:13 PM
"Documented" may be a bit strong, but the rules can be inferred from the docs for properties like profile on this page: https://www.pulumi.com/registry/packages/aws/api-docs/provider/
If not set, the default profile created with aws configure will be used.
I read that as: the profile property trumps the non-Pulumi configuration. If profile isn't set, then use whatever's documented by AWS.
w

wooden-action-47328

04/08/2024, 8:18 PM
I’m more concerned about whether it can overridden if it is set
Is there any way to use a custom provider and have it be automatically used, or does it have to be the case that if I want to use a customer provider it has to be passed into the options for every declared resource?
s

salmon-account-74572

04/08/2024, 8:20 PM
The latter (explicit providers have to be explicitly specified) BTW, this page contains configuration details for the AWS provider: https://www.pulumi.com/registry/packages/aws/installation-configuration/
w

wooden-action-47328

04/08/2024, 8:23 PM
This doesn’t seem conclusive about whether aws:profile in the stack config will take precedence over an AWS_PROFILE environment variable
s

salmon-account-74572

04/08/2024, 8:27 PM
Unfortunately, you are correct. It's not clear. I believe (but have not definitively tested) that an environment variable will override the stack configuration.
w

wooden-action-47328

04/08/2024, 8:28 PM
That’s not great, we’ll have to test that
that shouldn’t be the case for an explicit provider though where the profile is specified, right?
Only for the default provider?
s

salmon-account-74572

04/08/2024, 8:47 PM
Correct. The explicit provider has its configuration explicitly defined in the code.
c

cuddly-journalist-77210

04/13/2024, 10:53 AM
w

wooden-action-47328

04/13/2024, 9:59 PM
But this code is impenetrable without some explanation of how it works, and the comments for the profile attribute just say “The profile for API operations. If not set, the default profile created with
aws configure
will be used.“, which is not terribly helpful.
c

cuddly-journalist-77210

04/14/2024, 8:07 AM
Hm, isnt this the standard for AWS SDK? Also, did you check the README.md of the repository. All the configs which will be read by pulumi aws provider are mentioned there.
for example: here is the one for golang. https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/
w

wooden-action-47328

04/14/2024, 3:57 PM
No, it isn’t the standard for the AWS SDK, the aws:profile configuration variable is a pulumi thing. This is the AWS SDK standard for credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html