https://pulumi.com logo
#general
Title
# general
f

freezing-umbrella-5483

05/11/2022, 3:48 PM
I'm looking through the Pulumi Registry and not finding any CI/CD providers. How do I go about provisioning CI/CD pipelines with Pulumi?
b

billowy-army-68599

05/11/2022, 5:05 PM
most ci/cd providers dont have apis for workflows, that's why
you can use aws codebuild i believe
f

freezing-umbrella-5483

05/11/2022, 5:38 PM
Oof, that kind of sucks. Is AWS really the only one?
b

billowy-army-68599

05/11/2022, 5:39 PM
most CI/CD providers expect you to include a configuration file in the pipeline, I don't know of any that has an API
f

freezing-umbrella-5483

05/11/2022, 5:40 PM
How is this usually solved? e.g. if you need to dynamically provision/tear down pipelines?
b

billowy-army-68599

05/11/2022, 5:41 PM
I have never seen that use case, why would you need dynamic pipelines?
f

freezing-umbrella-5483

05/11/2022, 5:44 PM
I'm currently working with CDK, and my app uses multiple container-based microservices. I can provision/tear down pipelines for each container by modifying my CDK code. I can also modify the pipelines, e.g. add/remove a stage, etc, for example if I need it to deploy to multiple environments based on my config.
b

billowy-army-68599

05/11/2022, 5:45 PM
is that provisioning cloudformation stacks or codebuild?
f

freezing-umbrella-5483

05/11/2022, 5:47 PM
The pipelines? It can be both, but I use CodeBuild to build and push container images to the repositories I need (they're dynamic, and a stage is created for each repository I need to push to, with the option to add a manual approval step to the stage if needed).
b

billowy-army-68599

05/11/2022, 5:48 PM
okay, you should be able to do that with Pulumi then. that feels very imperative though, and Pulumi is a desired state tool
f

freezing-umbrella-5483

05/11/2022, 5:48 PM
I can with AWS, yes, but I'm surprised it's the only provider that allows this.
I don't know how I would manage without being able to dynamically handle my pipelines the same way all my other infra is managed. A single line of code changed can lead to a pipeline being provisioned or torn down, or a stage being added/removed. That's the power of having IaC be defined with real programming languages like Python. I believe Pulumi embraces this in principle.
b

billowy-army-68599

05/11/2022, 5:52 PM
sure. if a provider supports it, Pulumi supports it
f

freezing-umbrella-5483

05/11/2022, 5:53 PM
Right. I guess I'm surprised that this isn't a common use case and that it's only supported by AWS's CodePipeline
This is not a jab at Pulumi, but rather at the status quo.
p

polite-napkin-90098

05/11/2022, 5:54 PM
GoCD is configurable from yaml/json. https://docs.gocd.org/current/advanced_usage/pipelines_as_code.html I used to use jsonnet to spit out json blobs to feed into it for Pipelines as Code. I imagine it wouldn't be impossible to get Pulumi to do this, but I'm pretty new to pulumi so I don't know how easy it would be.
👍 1
b

billowy-army-68599

05/11/2022, 5:55 PM
pulumi only talks to APIs, so unless gocd has an API its not possible to interact with it
p

polite-napkin-90098

05/11/2022, 5:57 PM
It has an api https://api.gocd.org/current/ but I guess it would need a provider, and I haven't checked out the api recently so I'm not sure as to its capabilities. But I may well revisit this when I get the round tuits to do PaC
g

great-sunset-355

05/12/2022, 7:20 AM
@freezing-umbrella-5483 if you can leave AWS CI ... imagine that almost all competitive CI solutions can dynamically create stages/jobs/pipelines from just 1 file ... eg.
buildspec.yaml
AWS makes this complicated because you need to deploy your pipeline to run
buildspec.yaml
... it's not needed anywhere else!
f

freezing-umbrella-5483

05/12/2022, 7:27 AM
It's not dynamic, though, you have to edit the file by hand. My point was to have it be dynamic via code. E.g. changing a single function parameter change leading to new container instances being spun up and a new pipeline to build the container being provisioned. This is very easy with AWS. I can definitely leave AWS CI if there's something better.
4 Views