Hi everyone, Working in aws environment since a ye...
# general
a
Hi everyone, Working in aws environment since a year and still a noob on IaC but would like to change that. Does anyone have some guidance/references regarding layout /workflow/design-pattern for multi-account AWS deployment ? Layout on code (on gitlab), layout on state/stack (pulumi cloud) Goal : organize all the chain to be simple and clean
s
That's a pretty broad question, but the quick answer is that you probably want a Pulumi stack per AWS account. Tell me more about the infrastructure you're building and I can give you a more confident answer.
a
Hi, yes no easy answer for sure haha. Currently we have a single aws account for all ressources and for all env. We want to shift to a multi-account design/workflow, and embrace that through IaC (already using Pulumi for that single account but not full coverage on ressources). We already deployed an AWS Org and several accounts (1 aws account per env + monitoring, security, audit etc), but it's not used yet. We migrated our users to Identity Center in Management Account and hence, they assume a team-role into that 1 single workload account. Tomorrow I would like to have a dev being able to deploy his dev app to the dev account. And when done deploy to the prod account. My questions/concerns are : how can I structure my gitlab app repo, my pulumi state, and my CI pipeline in order to deploy smoothly in a multi-account setup ? We would like to have the devs having the ownership of what they deploy for specific app projects, and the infra team would handle the pulumi code for shared/infra ressources throughout the different accounts.
I don't know if it's more clear. I would highly appreciate any docs/tips you could share on that architecture topic.
s
It's clear! This is a pretty typical scenario. What you want is something like this: • 1 program for shared infra (VPCs, K8s clusters), 1 stack per account • 1 program for each workload running on the shared infra, 1 stack for each developer working on the project (if they each deploy their own version of the code to a personal or shared dev env), plus 1 stack for each shared environment (qa, staging, prod)
There's a couple of variations there - you might wanna put VPCs in 1 program and then K8s clusters in another program. Yours is the most typical scenario. I wasn't sure if you were attempting to deploy hot-hot DR or if you had e.g. multiple prod environments because you had a multi-tenant scenario where each tenant is in a different AWS account.
l
I think an important aspect is also blast radius - how much do you want to deploy together. We have individual programs for kubernetes clusters for instance, so that we can easily update one without affecting others
a
Thanks both for your insights. The layout that you suggest makes sense and is straight-forward. Does it matter to have everything inside a mono-repo ? Say I identify 3 type of infra use-cases : VPC, K8S, other-shared-services). Do you recommend a monorepo with 3 different subdirectory, each with their distinct Pulumi program ? (I like the idea of spliting programs to avoid blast radius) Or would you rather go with a dedicated repo for each use-case ? Also we tend to make the deployment happen only through the CI gitlab pipeline with a dedicated AWS CI role. That allows us to not give all the rights to developers considering we have a single account including prod workloads and don't want to give sensitive privileges. But for big workloads/apps, having many resources inside a single Pulumi program can make the pipeline deploymet very long, specially in dev stage where errors will happend and triggered only after several minutes sometimes. That's why we really need to move to multi-account soon. We think about splitting big workloads into smaller programs such as : infra-iam/DB/compute. Is it overkill, that would make lot of stacks given that we have different env.
s
I recommend separate repos so that you can give them separate permissions.
And so that each pipeline can be triggered only by changes to a single program.