I’ve been thinking about the scopes of Pulumi projects 🤔 It’d be interesting if you could have multiple stacks that shared some state. For example, project A that was for a single microservice instance and a project B which was a monolithic project for an entire environment, that included multiple services including project A’s resources. If both shared the same state for this resource, you could apply changes independently. Obviously there’d be a challenge in reading multiple pieces of state across multiple storage backends (eg.GCS buckets); co-ordination/locking; and generally the pitfall of having 2 different (potentially conflicting) declarations of that infrastructure. I then got to thinking about what the scope of a Pulumi project should be. We generally do 1 project per service (whether that’s a business app or a piece of shared infrastructure like RabbitMQ), but I’ve also wanted to co-ordinate lots of these individual projects to spin up an entire environment. I’ve mostly done this via. the automation API but it turns lots of declarative processes into imperative ones, and you lose a lot of the magic of Pulumi. That got me to wondering - does anyone use Pulumi projects in a monolithic approach (ie. encompassing entire environments/clusters)? I can envision a project that deploys the entire production GKE cluster and all of the many apps and infrastructure components. Then you could have a single repo and for the CI/CD pipeline, you could use the

--target

flag to limit the scope of the changes depending upon the trigger (ie. triggered by another repository) . This would be especially easy if you used a

ComponentResource

to encapsulate each application’s resource boundarys. Plus you could still get re-use of these individual `ComponentResource`s by packaging them (npm) and importing them. I also think it’d be pretty cool if Pulumi had a first class notion of tags/labels that you could then limit the scope of changes to (eg.

--target-tag="<http://my.org/application=foo|my.org/application=foo>"

) (Sorry for the brain dump)

It’d be interesting if you could have multiple stacks that shared some state - ie. a project that was for a single microservice instance and a monolithic project that was for an entire environment that included the microservice. Obviously there’d be a challenge in reading multiple pieces of state across multiple storage areas (eg.GCS buckets) and co-ordination/locking.

The popular method for achieving this is with a "shared-" or "base-" project that has the resources which other stacks share. The "child" stacks get the values of the Outputs via StackReferences, and use the resource getters to get a copy of the resource in the current scope. https://www.pulumi.com/blog/iac-recommended-practices-using-stack-references/

interesting - that’s new to me

The more resources you manage, the longer deployments will take just to calculate the resource graph. When your plan fails, it will fail big. Operationally, micro stacks help you limit the damage in the case of an incident.

my gripe is that stacks feel more like different envs (staging/test/prod) instead of different vertical slices

That's where I think you're looking for the Stack References pattern

oh, it works across projects?

It's also worth noting there is "project level configuration" that all stacks can inherit, but not resources.

Is this relatively new?

Right. Nope.

do StackReferences work when not using Pulumi’s backend?

Yes, there is a note about a recent change regarding that though

One thing to remember about the stack references is that the parent stack can decide to remove the reference and there's no way to notify downstream about the change or removal. So I'd advise combining stack references with lookup aka

get_<resource>

functions.