https://pulumi.com logo
Join Slack
Powered by
Hi! I need help. 1. I need guidance on how to red...
# general
t
Hi! I need help. 1. I need guidance on how to redo our current deploying process via git workflows. Currently whenever you push to a staging branch, github runner pushes that build to 1 single server that had a docker compose setup and hosted everything from db, frontend, backend, etc a. how would my team push new code now? 2. i have a new devops git repo, and then we have our main app monorepo. Is this a way to do infra now? I've seen a few ways but i am lost now.
l
Unless you're building and deploying on-demand environments, you're unlikely to want a push or PR trigger: a merge trigger is more likely, so that deployments happen when a PR is closed-merged. Pushing a readme file change to a branch should probably not run 
pulumi up
🙂
1.a. There are many ways to arrange pipelines, builds and deployments. My preference is to think firstly about the developer experience: don't make things harder for the people writing and pushing the code. Often this involves having separate pipelines for CI (going as far as packaging and deploying to the various registries) and CD. Build the pipelines to meet your devs' and your users' requirements; don't change their work experience based on your pipelines' requirements.
2. Yes, sure, this is a way to do infra. You can have multiple repos, mono repos, submodules, whatever works! There is no best way, do whatever works for you, your context and your developers.
t
1. ok well, idk what the new process will be so what do you suggest? my goal is adding scaling to our infra. i imported our staging server and added a aws rds node so that we can scale backend/frontend as multiple instances going forward.
dev team is just 3 including me, and me acting as the devops engineer
l
Two app devs and you, and you want to be able to 1. Make infra code changes and have them auto-deploy (to some but maybe not all environments?) once they're reviewed and merged; and 2. Make app code changes and have them auto-deploy to somewhere they can be tested once they're reviewed and merged?
If it was me, I'd go with two repos and a CD system: 1. the app repo where automatic pipelines (launched on PR merge) finish at package deployment (to NPM, Docker Hub, local registries, etc.). 2. the infra repo where automatic pipelines (launched on PR merge) finish at deploying infra to ephemeral environments where no app gets deployed (infra test only). 3. CD deployments that trigger on tagging either/both deployed packages and infra main branch, and deploy to named environments based on the tag. However, all my clients are old-school and don't like unguarded CD; everything has to be manually gated. There are smoother routes to prod than this 🙂
t
and we are a payments tech starttup to give you the context and scale we need
1 sec , reading your msgs
l
Payments = Fintech = no unguarded CD 🙂 So my pattern might be good to start with?
t
yes fintech
l
Since you're a start-up, be flexible. Try something that "makes sense", identify its weak points based on experience, and improve and iterate.
t
what is a unguarded CD?
l
"Real" continuous deployment: a change in code means a change in prod.
Guards = code reviews, change board approvals, etc.
t
ah yes, we do that on code push, pr merge = live
l
Right. Well that likely won't scale in fintech: when you have other people's money on the line, "LGTM" won't be enough to save the company.
It's great for agility though, so if agility is what you need right now, then stick with it.
t
what do you use for CD system?
i want to move awauy from slow GH runners lol
l
Different clients use different things. GH runners aren't the worst; the slow bit is usually the container prep and that's easy to fix by building your deployment image in advance, with all the necessary stuff installed. Octopus is popular. Pulumi's Deployments are good if you're already paying for Enterprise, though they're very similar to GH in a lot of ways. I see Spinnaker and Argo a fair bit. There's lots more: anything that you can use to map a tag or release of some kind, to a named environment, should be fine.
t
so you suggest that the app monorepo just builds/packages into docker img, then CD deploys that?
l
Or into whatever gets deployed, yes. Having a single pipeline that goes from commit to live is great in a special small number of cases. In particular, cases where there is no infrastructure change possible (e.g. you're deploying to Vercel or some other platform where you're not managing any of the infra). If you want to be able to deploy some changes (e.g. front end) and not other changes (e.g. firewall), then a single pipeline and single deployment schedule is not workable (imo).
I have had to help people who wondered why their prod database was being force-restored every time they changed a .tsx file... one pipeline. Oops.
t
lol
we use flyway for db migs
l
This particular problem wasn't a migration issue, it was a force-restore. There was an bug in a conditional in a pipeline, and it was doing the infra-rollback step every time a PR was merged. It never needed even to be a risk: it shouldn't have been in the same pipeline at all.
t
we had a bug where the db mounted as an annoy vol up container start, that was.........interesting...
where does db mig run in your example 2 repos + CD system?
l
That depends on the deployment cycle. Do you migrate every time you deploy the app? Less frequently? The deployment cycles are (imo) the key decision-drivers of all this.
If you need to (plucking random stuff from the sky here) rotate your TLS keys every time you update your WAF rules, then your WAF-rule-changing deployment and your TLS-key-rotation deployment should happen together. They might even be in the same repo and in the same Pulumi project.
t
we only db mig when we need a mig basically, usually during dev locally then staging/prod
l
Then it probably needs to happen in its own pipeline, manually triggered. It could be triggered by a special git tag or image tag push, I guess? Depends.
It can live in any repo, so long as its pipeline isn't triggered just because some other pipeline is triggered.
t
is that a new git repo
ah
i see
l
The repo layout isn't too important.. so long as the sequence of deployments is independent when it needs to be, or chained when it needs to be, or whatever.
Usually the hardest problem my clients have to solve is "should these 200 resources be in one project or in 10 projects or ...". In those cases, the deployment cycle question is really helpful.
4 Views
Open in Slack
PreviousNext
Powered by