I come from Terraform and am evaluating Pulumi vs Terraform's CDK (this is my first day trying it out) and I want to use Pulumi to provision resources in 3 separate AWS accounts that is managed under the same AWS Organization -

play

,

dev

, and

prod

.

play

can contain anything - it's a place for experimentation

dev

contains dev deployments of our API (perhaps different feature/hotfix branches), other internal dev tools

prod

contains the tested version of our API How should I be organizing my code? 1) Use a single project and create the following stacks:

<org>/<project>/prod/infra

<org>/<project>/prod/api

<org>/<project>/dev/infra

<org>/<project>/dev/api/featureX

<org>/<project>/dev/api/featureY

<org>/<project>/dev/api/bugA

<org>/<project>/dev/api/bugB

<org>/<project>/play

But then I would have to add a lot of conditional statements in my program so that they are only deployed into the right stack. 2) Use a separate project for each environment and concern. Define my API and infrastructure as separate Component Resources (https://www.pulumi.com/docs/intro/concepts/resources/#components) and 'import' them into each project.

| project | sub-project | stack    |
    |---------|-------------|----------|
    | prod    | infra       | infra    |
    | prod    | api         | api      |
    | dev     | infra       | infra    |
    | dev     | api         | featureX |
    | dev     | api         | featureY |
    | dev     | api         | bugA     |
    | play    | infra       | infra    |
    | play    | any         | any      |

I understand that projects and stacks are designed to be flexible, but I would appreciate a bit of guidance. I get the feeling that the examples in the documentation expects parity between dev and prod - you deploy this project in dev, and then deploy the same project in prod. How do you organize the code when dev and prod are somewhat different (e.g. internal tools only deployed in dev) but also somewhat similar (e.g. networking)? Thanks in advance for any help you can provide 🙇

Hi @billions-ghost-96421 I had exactly the same questions myself when I started with pulumi a ~month ago. It is still not clear to me either, but I create separate projects for each concern and create configurable components. There is an example of a big project: https://github.com/mitodl/ol-infrastructure/tree/main/src/ol_infrastructure Then put as much as you can in

Config

object. my current approach is

core-project # for shared infra, vpc network..
app-project/
  __main__.py
  Pulumi.dev.yaml
  Pulumi.staging.yaml

I don’t know if this helps, but here are some bits and thoughts. • Stacks are meant to be instantiations of a given project. A stack generally maps to an “environment.” Each stack may be somewhat different (say different numbers of resources or regions, etc) and those differences can be driven by for loops and conditionals, etc. And those differences would be driven by the stack config file (Pulumi.XXX.yaml file). • Given the description above, I would think that

play

is it’s own project since it seems it can be anything and is not all that aligned with the production environment. Maybe it’s a single stack or maybe different stacks for different users to test in? Not 100% sure. • It seems like the

dev

and

prod

stacks are based on the same project. And you may want to think of making

dev

to be multiple stacks - one for each branch/PR before being merged to main and that main drives the

prod

stack. • As far as the question of deploying internal tools in dev vs prod, then you probably want a different project for those non-prod tools. You can use stack references to allow the main application stack to find information it needs from the internal tools stack or perhaps more likely, vice versa. • So in the end, if the above is valid, you may have a project for the

play

environment(s). And then one or more stacks to instantiate that/those play environment(s). Then a project for the dev/branch-PR/prod environments that represent the system you are deploying for your business. And then a separate project to deploy the internal tooling needed for the non-prod environments with one or more stacks depending on what makes sense.

Thanks @great-sunset-355 and @witty-candle-66007 for the guidance. It helps a lot. I think the initial problem I had was I think of a 'project' in the Terraform way - which is a separate project for each environment (as Jan laid out in the second comment). Once I got over that it was not clear how I should organize my stacks. But with what you folks have said, I think I'll try something like this:

projects | stacks:
- infra
    - <org>/infra/prod
    - <org>/infra/dev
    - <org>/infra/play
- api
    - <org>/api/prod
    - <org>/api/dev/main
    - <org>/api/dev/featureX
    - <org>/api/dev/featureY
    - <org>/api/dev/bugA
- quota
    - <org>/quota/prod
    - <org>/quota/dev/main
    - <org>/quota/dev/featureZ
- play
    - <org>/play/default

(

play

is meant to be a free-for-all, anything-goes sandbox so it makes sense that that particular environment is its own project)

From what you described, that looks good. As as noted, Pulumi stack references will be useful if different stacks need to know about each other. https://www.pulumi.com/docs/intro/concepts/stack/#stackreferences

@billions-ghost-96421 I think that your layout will work well. It really depends on the difference between environments and your software architecture skills. If all that becomes too small for you you can always try to invent something using Automation API and control the whole pulumi programmatically

Hi all,