https://pulumi.com logo
Title
b

bulky-area-51023

09/29/2021, 7:14 PM
Hi folks! I’m currently working on pulumi CI/CD pipeline. When using monolithic repository scheme (multiple projects in a repo), what would be the best way to preview & deploy when the MR changes multiple projects at a time? I think that pulumi stack well corresponds to git branch, but I’m having a trouble how to conduct gitops on multiple projects
s

steep-toddler-94095

09/29/2021, 7:21 PM
If you use Githhub Actions they have a matrix strategy that allows us to easily configure
pulumi preview/up
on all projects in parallel. Other CI tools might have something similar.
m

millions-furniture-75402

09/29/2021, 7:22 PM
How you solution this is dependent on what CI/CD tool you're using. In general you want to collect the projects, then operate across them, ideally in parallel where you can:
find . type -d -maxdepth 1 | xargs -I % sh -c 'pushd % && pulumi stack || echo "no pulumi stack"'
+1 for job matrix strategy
b

bulky-area-51023

09/29/2021, 7:33 PM
Wow thx for the fancy script, and matrix strategy! I’ve never heard of it. Actually I’m using gitlab CI/CD and it seems like there’s also tools for parallel executions. On top of that, I may do
git diff
and identify what projects have changed, and parallelize only those changed projects. Really grateful for your answers
m

millions-furniture-75402

09/29/2021, 7:42 PM
git diff --name-only HEAD HEAD~1 |grep "/" |awk -F "/" '{ print $1 }' |sort |uniq |grep -E "(project-1|project-2)"
"Get the difference between two git refs, grep for paths with directories, print the first directory, sort, unique, match a regex of project names"
So if you consider that's how you do your collection of projects, you can pipe that to xargs like above:
git diff --name-only HEAD HEAD~1 |grep "/" |awk -F "/" '{ print $1 }' |sort |uniq |grep -E "(project-1|project-2)" | xargs -I % sh -c 'pushd % && pulumi stack || echo "no pulumi stack"'
s

steep-toddler-94095

09/29/2021, 11:26 PM
gitlab has https://docs.gitlab.com/ee/ci/yaml/#ruleschanges which might allow you to trigger jobs only if certain files contain changes. Perhaps it also works for directories