Hi! I have a conceptual question I have been having trouble figuring out: My current setup, which I have been running for quite a while is: • a pulumi project that deploys the infrastructure necessary to run my app, including multiple services, databases, networking etc to two stacks (prod, staging). This is a single project that lives in its own git repo (

infra

). • a few application services that live each in their own git repo (e.g.

serv1

,

serv2

, etc). Each service has its own CI/CD pipeline for deploying a new version upon merging to the main branch. • The CI/CD pipeline of each service does not use pulumi to publish a new version, but rather some some bash scripts (due to legacy reasons) I would like to use pulumi to substitute said bash scripts, but I'm having trouble understanding how I could achieve this. I understand that, if I had one stack per service, I could easily build docker images as part of each stack and update the resources accordingly. However, as far as I understand, this would come at the cost of a complex setup (tradeoffs described here) using stack references. So I guess what I'm trying to figure out is: is there a way I can still have a centralized definition of resources but have each separate app be able to publish their new versions? Does this even make sense or is it an anti-pattern?

The infrastructure for the pipeline should not be related to the infrastructure for the application. Entirely separate projects. I cannot think of a reason for anything from one project to have to be a StackReference in the other. Consider the requirement to deploy the application from a developer machine: what information would it want to pull from the project that creates GitHub pipelines?

@little-cartoon-10569 in the setup I described I'm not using pulumi to create the pipelines themselves. In more practical terms, what I have today is the

infra

project creating a service. and the scripts update which version (e.g. image) that service runes over time. I could have separate projects both create their services and build/push an image that will be run by the service, but I'm wondering if I can keep things separate:

infra

project creates the service and individual applications build/push images and update the service to point to the new image (here's where I understand stack references would be used).

I don't follow, sorry. But in general: stack references are easy. More small projects tends to be easier to work with than fewer large projects, though figuring out where to break projects up can be tricky. Projects should generally group resources based on deployment / destruction cycles: if you want to destroy two things under different circumstances, then you should not put them in the name project.

Building / pushing an image is probably packaging, and comes at the end of a CI pipeline; that packaged image is used by a deployment pipeline, but only in the same way that a source file is used by a CI pipeline.

In this context, would you say that pulumi is appropriate for the deployment step? For example, if the output of the CI/CD pipeline is a deployable artifact (e.g. a docker image, a jar file, etc), would it make sense to use pulumi to patch the running services to use this new artifact?

There are times when there are better ways to do it. But often, Pulumi is appropriate. I suppose it depends on how it's being deployed. If the only thing you need to do to deploy an app upgrade is to push a new image to a repository somewhere (and everything else just works), then don't use Pulumi, it's just not necessary. But if you need upgrade lambdas, add a new target group to the LB, add a new folder to your logging bucket for the new logging agent, and so on and so on, then Pulumi becomes a very sensible option.