https://pulumi.com logo
Join Slack
Powered by
:question: Can a single pulumi project manage stac...
# aws
w
❓ Can a single pulumi project manage stacks in different AWS accounts? Details in thread.
Right now for my POC projects, I am using an S3 backend state bucket which lives in the AWS account where the resources will go. That seems fine - the backend url config attribute in 
Pulumi.yaml
references that bucket and aws profile. But if I want to put staging and production in two different AWS accounts, how do I arrange things to do that? Can the 
Pulumi.<stack>.yaml
files have their own backend url config attributes?
Each stack will also have its own AWS role profile for accessing resources
l
Yes. You configure your AWS provider(s) appropriately. A project can have any number of AWS providers in use, each stack can have all of the providers configured differently.
Can the Pulumi.<stack>.yaml files have their own backend url config attributes?
No. The backend for the project is configured in the project. All the stack metainfo is stored in the one place. All the resources in the stacks can be in any account. There is no requirement for the Pulumi information to be stored anywhere near the resources.
w
I’m not sure I follow what you mean by “each stack can have all of the providers configured differently”. Where does that configuration happen?
l
In code. You don't have to (and I recommend not to) use the default AWS provider. Create as many instances of 
aws.Provider
as you need, and pass them to the resources as you construct them. https://www.pulumi.com/registry/packages/aws/api-docs/provider/
This is also how you put resources in different regions.
w
Does this need to be explicitly passed as an argument to each resource?
l
It really should be.
Allowing resources to be constructed in whatever account the current user has set as their default is dangerous. Don't do it.
If your code doesn't specify, check and double-check, then your infrastructure is at risk. In my opinion.
w
But the project specifies an account to connect to?
l
Your code should do that. Not your Pulumi configuration, or your AWS configuration.
But by default, your AWS configuration does that. Which means if you run that code on a different machine, something different might happen.
Because the default configuration, which isn't checked, might be pointing somewhere else.
w
We’re using the 
backend
attribute in Pulumi.yaml, which specifies a named aws role profile to use for the connection
l
There are loads of exceptions to this idea/rule, but for any project that could be deployed in a prod or prod-like scenario, then IMO you must ensure that where you're deploying is where you ought to be deploying. Leaving it up to the default configuration is dangerous.
w
We are not using the [default] connection in ~/.aws/credentials
l
We’re using the backend attribute in Pulumi.yaml, which specifies a named aws role profile to use for the connection
That's the configuration defining where to store the Pulumi state. It's not the configuration used to deploy the actual resources.
w
what determines that?
l
Instances of 
aws.Provider
, or the default AWS provider if you're not specifying that..
So: specify that.
w
does the default aws provider honor the configawsprofile attribute from Pulumi.stack.yaml?
l
Yes. And if you check that in your index.ts , and throw exceptions when your default profile is configured inappropriately, then you'll be fine.
But who does that?
w
does it matter if the default profile isn’t the correct one if there’s a profile specified in Pulumi.<stack>.yaml? Won’t that override the default?
l
The profile is specified by name. The creds in that profile could be different on different machines. "myapp-prod" could be a different account id, different region, different SSO information, etc.
w
yes, they absolutely should be though - everyone running this is using their own credentials
l
Absolutely. And in many cases, people require that a specific project / stack must be deployed to exactly one well-known location, and all other deployment targets are illegal and must be prevented. In order to ensure this is done, you must configure the targets in code. You shouldn't rely on everyone doing it right. Eventually, someone will make a mistake, or maybe even deliberately deploy your valuable IP to the public domain and steal all your hard work.
So you specify the target in code. And the individual credentials are used (via SSO or whatever) and if they don't have permissions to deploy to the explicitly-defined target, then it fails. Correctly.
If you're creating a project, for example like the Pulumi example projects, where it can be deployed anywhere, wherever the user wants it to, in whatever account you like: then use the default AWS credentials. That's perfect! But many times, that's not appropriate.
w
is there documentation on how to configure and use a custom aws provider?
l
w
That’s kind of thin on details and doesn’t have any examples
l
w
There’s no way to set a new provider as the default for inheritance without passing it to every resource call?
l
I recommend against doing that. And setting the require-explicit-providers option in Pulumi (I'll look that up in a second). Not sure if there's a way to set an explicitly-configured provider as the default. That would be marginally better than allowing the unspecified provider to be the default, but it is still (IMO) bad.
w
doesn’t requiring the provider be added to each resource call put a bunch of burden on the people writing those resources to get it right?
l
All our projects have lint rules to prevent building a project that has a resource that doesn't have an explicit provider. The risk of anything else is too high for us.
No. IDEs sort that.
It's like requiring types. No extra effort once you are expecting it.
w
well, we’re using python and not typescript
l
The config for disabling default providers:
Copy code
pulumi:disable-default-providers:
    - aws
It goes in your Pulumi config, either Pulumi.yaml or your stack yaml file.
w
also I don’t see where that ai example you linked to passes the provider
l
Yep I've checked, that AI example creates the providers then doesn't use them! AI loses, again.
w
sad trombone
l
This is how it should be:
Copy code
function createS3Bucket(bucketName: string, provider: aws.Provider): aws.s3.Bucket {
    return new aws.s3.Bucket(bucketName, {
        bucket: bucketName,
        acl: "private",
    }, { provider: provider});
}
So this has been a long thread about a personal opinion based on experience. It may very well be that your case doesn't need an explicit provider. But in my experience, all projects written outside of the open source world do need explicit providers.
w
I echo a lot of the concerns found here, which has been open for four years: https://github.com/pulumi/pulumi/issues/2059
l
If you think the changes are appropriate for you, you can absolutely make the changes incrementally. This is what we did. There's no issue configuring an explicit provider to be the same as the default provider, then start changing resources to use the explicit provider. Iirc you need an alias, but that's easy, and you can have a single well-known function set both the provider and the alias. You can update the state file manually, as implied in that archive. I did that in the first project I converted, and it does work. But I believe it's not necessary. And of course you can always make the changes on a branch (with no other changes), 
pulumi destroy
on the old branch and 
pulumi up
on the new branch, then merge. If that's appropriate for your use case.
Or just make the rule apply from now on, and leave the "legacy" resources as-is! Just because an explicit provider may be better for your use-case does not mean it's essential. For example, you can check the default provider in a business process, or in a wrapping script that calls Pulumi, or even in the Pulumi code itself.
I've been reading your linked issues, and one points to this. Great initial explanation of why explicit providers are good: https://github.com/pulumi/pulumi/issues/3383
w
There’s no reason the custom provider can’t draw config values from the stack configuration file, right?
l
Absolutely. Strongly recommend doing that. We set the region from stack config (though not the credentials).
w
where do you set the credentials?
l
We require SSO. Onesec I'll grab a code snippet.
It's a bit hairy, since the AWS CLI tool does not support output in JSON for this! Even though it's a documented option and works for almost everything else 😞
Untitled
Untitled
So we have a well-known SSO profile name that must exist for the user or else you don't get in. And the region is configured per stack.
Then we check that the SSO profile is in use and that AWS says that it has been authenticated using SSO. We trust the output of the 
aws
command, so there is a weakness: someone could replace that command with one that returns valid fake details, which could cause us to deploy to the wrong place.
But this is a risk that is small enough for us to accept.
Oh, the accessRoleArns variable is a hard-coded map of valid role ARNs as deployed by IAM Identity Center. This is a second line of defense: if someone destroys our identity center, then no more deployments will work until we fix the code. This prevents a bad sysadmin-type actor. We would need a colluding dev and operator in order to break this.
w
where do you set 
ssoProfileName
?
I’m feeling like that should also be set in the stack config
l
I'll check. I think it might be a well-known constant, but it could be config. So long as code review is involved, it should be fine either way.
It's a well-known constant for us. Set in code.
w
❓ One of my developers discovered that if you set a value for aws:profile in the Pulumi.<stack>.yaml file, it gets picked up and used by the default provider, and doesn’t require using a custom provider in order to force a profile to use in code. Is this new? Is this documented anywhere?
s
I don't think this is new, but I also don't know if it is explicitly documented. The idea of the default AWS provider is that it will use "ambient" credentials & configuration. Setting "aws:profile" in the stack config file just tells it a specific local configuration to use. You could accomplish the same thing by setting the 
AWS_PROFILE
environment variable (say, using something like 
direnv
) in the Pulumi project directory.
w
I explicitly do not want to rely on having people set environment variables
Are the precedence rules for how the default provider gets credentials documented anywhere? Can aws:profile be overridden with environment variables?
l
"Documented" may be a bit strong, but the rules can be inferred from the docs for properties like profile on this page: https://www.pulumi.com/registry/packages/aws/api-docs/provider/
If not set, the default profile created with aws configure will be used.
I read that as: the profile property trumps the non-Pulumi configuration. If profile isn't set, then use whatever's documented by AWS.
w
I’m more concerned about whether it can overridden if it is set
Is there any way to use a custom provider and have it be automatically used, or does it have to be the case that if I want to use a customer provider it has to be passed into the options for every declared resource?
s
The latter (explicit providers have to be explicitly specified) BTW, this page contains configuration details for the AWS provider: https://www.pulumi.com/registry/packages/aws/installation-configuration/
w
This doesn’t seem conclusive about whether aws:profile in the stack config will take precedence over an AWS_PROFILE environment variable
s
Unfortunately, you are correct. It's not clear. I believe (but have not definitively tested) that an environment variable will override the stack configuration.
w
That’s not great, we’ll have to test that
that shouldn’t be the case for an explicit provider though where the profile is specified, right?
Only for the default provider?
s
Correct. The explicit provider has its configuration explicitly defined in the code.
c
w
But this code is impenetrable without some explanation of how it works, and the comments for the profile attribute just say “The profile for API operations. If not set, the default profile created with 
aws configure
will be used.“, which is not terribly helpful.
c
Hm, isnt this the standard for AWS SDK? Also, did you check the README.md of the repository. All the configs which will be read by pulumi aws provider are mentioned there.
for example: here is the one for golang. https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/
w
No, it isn’t the standard for the AWS SDK, the aws:profile configuration variable is a pulumi thing. This is the AWS SDK standard for credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
Open in Slack
PreviousNext
Powered by