https://pulumi.com logo
Title
e

elegant-dog-76355

03/02/2023, 9:17 PM
Hi all, Looking at the comments to me it seems like the recommend way of implementing a backend is to configure it in
Pulumi.yaml
instead of using the environment variable
PULUMI_BACKEND_URL
. Is this correct ? This would have the limitation of having one backend
per project
and cannot have one
per stack
. If I only have a backend per project in the
Pulumi.yaml
file and that backend is stored in
S3
AWS Account
1111111
but each
stack
[*dev*|*stage*|*prod*] is per AWS Account `222222`|`3333333`|`4444444` how do I configure `Pulumi`/`Github Actions` to use AWS Account
1111111
for the backend operation but deploy the resources in the case of dev in AWS Account
222222
? At the moment i have the following in
GitHub actions
but this only works if the backend and the resources been deployed are in the same AWS Account:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1-node16
with:
aws-region: ${{ env.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_DEV_PULUMI_ROLE }}
role-session-name: OIDCSession
- name: Deploy changes to Infra
uses: pulumi/actions@v4
with:
command: up
stack-name: ${{ env.PULUMI_DEV_INFRA_STACK }}
work-dir: ${{ env.PULUMI_DEV_INFRA_CWD }}
env:
PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_INFRA_DEV_ACCESS_TOKEN }}
l

little-cartoon-10569

03/02/2023, 9:25 PM
I think the only way to have multiple backends per project is to omit it from Pulumi.yaml, and to use
pulumi login <backend>
before
pulumi stack select
.
This is not (afaik) possible with the Pulumi GitHub action; you would need to use a job with the Pulumi Docker image and the appropriate commands, or similar.
b

billowy-army-68599

03/02/2023, 9:27 PM
yes this is correct, you need a single state bucket for every stack in a project, we don’t yet support per stack backends
e

elegant-dog-76355

03/02/2023, 9:30 PM
okay thanks for that ... i was thinking that would be a possible solution but been fairly new to pulumi i was not sure if that considered best practise or not. again thank you 👍
e

echoing-dinner-19531

03/02/2023, 9:32 PM
we don’t yet support per stack backends
Just to set expectations, I don't think we'll ever support this. The backend tells us the stack, its very flipped logic to somehow get a stack to tell us what backend to use.
l

little-cartoon-10569

03/02/2023, 9:36 PM
You could automate it with automation-api. Setting ProjectBackend can be done before referring to any stacks.
e

elegant-dog-76355

03/02/2023, 9:44 PM
okay thanks for letting me know @echoing-dinner-19531 i guess it's a matter of perspective and logic i guess :-) what i mean is the pulumi doc i have read give example of having a stack for
dev
and another for
staging
etc. I have been deploying to aws for a while now with diffrent customers and it has always been the case that each of these environments (`dev`|`stage`|`prod`) are in diffrent aws accounts. Coming from a terraform/cloudformation background this pretty standard setup. Your comment "The backend tells us the stack" now makes sense why this would not be supported. I just need to get my head around a bit more on how pulumi works 🙂 very happy to go ahead and try out @little-cartoon-10569 suggestion
b

billowy-army-68599

03/02/2023, 9:46 PM
it’s been a few years since I worked with terraform, but for terraform’s implementation of stacks (workspaces) the same applies. You generally need a third party wrapper like terragrunt to handle a state per account model
e

elegant-dog-76355

03/02/2023, 9:59 PM
if this was terraform one way would be to simply have a backend file for each environment which just has the parameters for that environment. For example a file
dev_backend.config
which contains the following:
bucket         = "terraform-backend-eks-hello-dev"
region         = "eu-west-1"
key            = "terraform-backend-eks-hello-dev/eu-west-1/dev/terraform.tfstate"
dynamodb_table = "dynamo-terraform-state-lock"
encrypt        = true
then in github actions you pass this file into terraform to initialise the setup
- name: Terraform Init
run: |
terraform init -input=false -backend=true -backend-config="backend.config"
shell: bash
working-directory: ${{ env.TF_EKS_ENTRY_POINT }}
you can then have a backend end file for called
prod_backend.config
and so on. Terragrunt is a great tool for keeping your code DRY but you don't need it to support multiple backends. again thanks for all the advise on this. Only first week with Pulumi so I have allot to learn
b

billowy-army-68599

03/02/2023, 10:02 PM
right, but the
terraform init
phase is analogous to
pulumi login
l

little-cartoon-10569

03/02/2023, 10:06 PM
You are not required to use the same account for the backend as for everything else. If you have multiple customers that you deploy for, I recommend having an account / bucket that belongs to you that stores all backends.
The backends belong to you, not your customers
b

bitter-carpenter-93554

03/04/2023, 1:36 AM
In my opition, it would be good and flexible feature if we can specify env variable in Pulumi.yaml file like this:
backend:
    url: s3://${AWS_BUCKET}?region=us-east-1
@elegant-dog-76355, another option is to use PULUMI_BACKEND_URL env variable.
export PULUMI_BACKEND_URL="<s3://your-bucket?region=us-east-1>"
e

elegant-dog-76355

03/04/2023, 8:39 AM
thanks @bitter-carpenter-93554 ... this link was pointed out to me which has that env variable option you suggested. putting it here in case others find it usefull I think the safer option is to use the
pulumi login
cmd because if that fails you can catch and react to the error but thats just my option 1 week in using pulumi :-)