Hi, I'm migrating a manually maintained set of lambdas, databases, vpc's, security groups etc. to one which is instead fully managed by Pulumi. I've run into a slightly confusing problem regarding database migrations, and I'm struggling to find any examples documenting best practices. We use C# with Entity Framework, and the current build process is: Build projects -> Run DB Migrations -> Deploy code. We can do this because the manually created databases are known in advance, so the connection strings can be injected into the build. However with Pulumi the connection string is now calculated within the Deploy step, therefore I'm thinking the DB Migration step needs to be folded within the Pulumi Deploy step. The problem is that the only resource I seem to be able to find on the topic is this Medium post from 2019 which makes we worry that I'm going about this completely wrong. Is this solution of having a one-off task which is deployed, run, and monitored for completion before continuing to deploy code changes an expected normal solution, or is there some far easier way?

Hi @plain-area-81454 I may not be understanding the exact problem, but I think you could probably decouple the pulumi infrastructure deploy from db migrations. Just have the db migrations, or something, read the connection string from pulumi, with

pulumi stack output dbConnectionString

or whatever your equivalent is.

If you'll be deploying the app more frequently than the infra (which is a pretty common situation) then you'll have to do something like this. If you have a deployment management tool (e.g. Octopus), then the infra deployment step can push a new release to the app deployment tool, so that subsequent app deployments use the new configuration. Note that if you do something like this, it is important to properly version things like the DB connection string: we've made the mistake of having just "the" db connection string and updating it, but then wanting to deploy the app with the old values. You will almost certainly need to do something like this at some point...

Ah, thanks both! It looks like I got myself stuck thinking that pulumi up would be run from a single project. Taking a little away from what you've both said, it seems like I can split it into two projects - an infra project, and a code deploy project. Then I can continue with a similar process to what we already have (Build -> Deploy Infra -> DB Migrations -> Deploy Code), and don't need to worry about building a special DB migration one-shot server.

Splitting projects is often the right call. My rule of thumb is to think about the deployment lifecycle of each resource. If there's a reason to regularly deploy one group of things at a different time or a different rate than another group of things, then those groups of things should be in different projects.