https://pulumi.com logo
c

cuddly-smartphone-15267

09/09/2021, 1:50 AM
does anybody have any tips on setting up pulumi with CI/CD? we currently use appveyor for our code based CI/CD and i was hoping to use it for pulumi as well
for example, is it just a case of installing the pulumi CLI and running something like:
Copy code
pulumi login
pulumi stack select [environment]
pulumi up
we use typescript for our stacks
l

little-cartoon-10569

09/09/2021, 1:54 AM
In general: don't select a stack as a separate step in CI. It's too easy to make a copy-paste error, and it is not very self-documenting, especially in your build logs. Every pulumi command that operates on a stack takes the
--stack
option: use that.
c

cuddly-smartphone-15267

09/09/2021, 2:04 AM
ok, good to know thanks
l

little-cartoon-10569

09/09/2021, 2:05 AM
The Pulumi docs have recommendations for general CI and specific tools. See https://www.pulumi.com/docs/guides/continuous-delivery/other/ and https://www.pulumi.com/docs/guides/continuous-delivery/
c

cuddly-smartphone-15267

09/09/2021, 2:13 AM
thanks Paul. i'm reading through those articles now and I'm trying to wrap my head around the 'branch per stack' concept and how it applies to 'using branches for environments'. At the moment, we have a bunch of different environments set up and each has their own yaml 'settings file'. would that sort of thing remain and then somehow have the CI/CD determine which stack to select based on the branch?
l

little-cartoon-10569

09/09/2021, 2:15 AM
Could do, but I wouldn't. The idea that the files you can see in your git index or working directory differ based on which branch you're in doesn't sit well with me.
However, I don't deploy to multiple environments using a git-based flow. I use a separate CD tool. The only deployment in my system that has a git-based flow controlling it is the flow that tests deployments.
c

cuddly-smartphone-15267

09/09/2021, 2:18 AM
i see.. could you elaborate on how your workflow is set up?
l

little-cartoon-10569

09/09/2021, 2:20 AM
When we accept a PR, the build-test-deploy pipeline includes a `pulumi up --stack devtest`: the stack is hardcoded. We run post-deployment tests against that constant environment. If the deployment and tests pass, we push all the packages to OctopusDeploy, and that's the end of the git-based flow. Octopus is configured with all the environments that we can deploy to.
We're looking to change this in a few months though. Octopus is getting very expensive.
We may end up with a branch-per-stack, and only one Pulumi.<stack>.yaml file in each branch. Haven't begun to look into that yet.
c

cuddly-smartphone-15267

09/09/2021, 2:37 AM
ok yeah cool.. thanks for clarifying
f

famous-leather-94346

09/09/2021, 6:09 AM
@little-cartoon-10569 I can recommend not keeping Pulumi.<stack>.yaml in git. We struggled with that approach a few months ago. Rather use ad-hoc stacks in CI/CD (there is always some build id in pipelines to make them unique and it is possible to use branch name if needed - but beware of stack naming restrictions). These stacks (if not destroyed at the end of pipeline run) are always accessible with
pulumi stack ls
and also you can get actual config with
pulumi config refresh
(although there is an issue https://github.com/pulumi/pulumi/issues/7282 that makes it a bit harder than expected)
l

little-cartoon-10569

09/09/2021, 7:18 AM
We find it to be essential, because we deploy to each environment from many different machines, and this is the best way to ensure that the file is correct on each machine. If that situation doesn't apply for you, then you can avoid adding your stack yaml to git. However, it is important to back it up securely somewhere. Git might not be the best place for your particular circumstances.
f

famous-leather-94346

09/09/2021, 7:29 AM
@little-cartoon-10569 IMO There is no need to back up these configs for the most of cases. All the configuration is in the stack itself (and can be refreshed locally). There is only one case I can imagine - if you want to set configuration for the first use, but it doesn't apply for us.
b

bored-oyster-3147

09/09/2021, 12:16 PM
Our flow involves having the YAML in git and using the current branch to select the stack name. We like having the stack YAML in git alongside our Pulumi IAC because it means a PR that change’s infrastructure Config will also reflect those changes in that PR. Our CI/CD just spins up a Pulumi container and runs out of that infrastructure directory within our repo. We have had no issues with this that I can think of
f

full-artist-27215

09/09/2021, 1:49 PM
FWIW, we've also got our stack YAML stored in Git. Our stacks are also configured such that they have an "artifacts" object that contains a mapping of artifact name to artifact version (e.g., specific container tags for a service, or a particular AMI to use). In CI, after a PR has merged, we build the new artifacts, then push a new commit to an "rc" branch which has updated the stack files to contain the new artifacts. Downstream CI pipelines can then operate from this
rc
branch, and have a fully-locked down set of artifacts, thus keeping everything explicit and deterministic. For developers working on features, they can simply omit the artifact versions, and can deploy infrastructure based on artifacts built directly from the code they're working on.
l

little-cartoon-10569

09/09/2021, 8:02 PM
@famous-leather-94346 There is a need. Some of the configuration is in the yaml file. If the configuration is changed by one person, then we would like everyone else to review, accept and adopt those changes. This is what git is for. It is essential to our use case.
👍 1