Not a dumb question! You can set up <deployment w...
# general
h
Not a dumb question! You can set up deployment webhooks to cascade deployments. The auto-deploy package can help, but secretly its just setting up deployment webhook for you using the pulumi-service provider. For example: • A PR for stack A triggers a deployment on stack A • The deployment webhook on stack A fires afterwards and triggers a downstream deployment of stack B
Do you have PRs that touch both stack A and stack B in the same PR?
c
Yes, a single PR may touch the configs for both stacks A and B (they’re in the same repo, after all). I guess that raises an interesting possibility: what if we moved stack A into its own repo? That would guarantee separate PRs, but there would still be the practical need to require that A goes out before B. Thank you for the link @hallowed-baker-22997! Let me read through that and ponder possibilities. I might as well come clean and admit that our actual setup is more complex than my A + B example. In reality, we have a “pre” stack A, regional stacks B + C (…), and then a “post” stack D.
👍 1
l
Workflow steps work fine for this, they even allow parallel steps. I would solve the problem in GitHub rather than Pulumi.
👍 1
Workflow triggers are also helpful (where a step in one workflow triggers another workflow). Which solution is right depends on your context.
c
Good point… I’m already slicing things like this in Github workflows — that might be an easier lift than dealing with webhooks. 🤔 Thank you @little-cartoon-10569! I guess it’s worth asking: in either case, if there’s a problem (e.g. some error during deploy), is it common for the devops peeps to jump in and run manual commands when necessary?
b
At my workplace we have developed a set of bespoke orchestration scripts for this exact purpose - with some help from AI and the Automation API getting a PoC ready took less than an afternoon and we've been quite happy with the setup. One big advantage is that you can use the scripts locally as well as in CI. We have a quite a few projects/stacks in a single "infra" git repo - some global AWS infra (IAM, S3 buckets and whatnot), several ECS clusters with tens of microservices (each service being a separate project with a stack per cluster), as well as some auxiliary projects such as configuration for New Relic dashboards and alerting Some examples of what we can do with our setup: •
deploy-infra --stack=dev
- deploys dev cluster with all services in the correct order (cluster first, then services, then New Relic) •
bump-service --project=payment --stack=dev --latest
- fetches latest version tag of the payment service from ECR and updates the
dev
configuration with the version. Optionally we can also pass
--deploy
to immediately deploy There's more but you get the idea 🙂
l
Running commands for problem diagnosis and investigation is inevitable. Running commands because it's just the way to do this thing is (imo) something akin to code smell. It's a sign that you should be looking for a "better" way.