Has anyone explored using Pulumi to template CI pi...
# generale
eager-wall-56838
12/13/2022, 5:54 PMHas anyone explored using Pulumi to template CI pipelines (GitHub Actions, GitLab CI, etc)? I think the existing functionality around Input and Output could be very useful for something like this. It would essentially be going from code to YAML, just like Pulumi can do for k8s manifests.
s
stocky-restaurant-98004
12/13/2022, 6:04 PMYou can definitely do this in AWS with CodePipeline/CodeBuild. There's also an Azure DevOps provider.
e
eager-wall-56838
12/13/2022, 6:06 PMMakes sense, but nothing for GitLab / GitHub?
eager-wall-56838
12/13/2022, 6:07 PMParticularly with YAML as the output instead of creating resources
eager-wall-56838
12/13/2022, 6:08 PMSome context: I built something without Pulumi, but realizing that it needs some sort of concept of deferred templating, similar to Pulumi’s input/output system, to unlock the full potential of the idea. https://github.com/adriangb/ciscript
eager-wall-56838
12/13/2022, 6:12 PMI’ve seen plenty of private orgs building very complex pipelines on GitHub/GitLab and even some open source projects have enough going on in their CI setups that they could benefit from this approach
e
echoing-dinner-19531
12/13/2022, 6:19 PMI have some ideas around extending pulumi to support this. I would love to write GHA yaml files with code rather than yaml, we'll probably play with this during a hackweek some time but it's more in the "fun experiments" rather than "business priority" camp.
e
eager-wall-56838
12/13/2022, 6:21 PMYeah makes sense. Seems kinda hard to sell since I imagine most users would template locally and not use a service. But it could be a cool way to introduce folks to the ecosystem! When’s your hack week btw?
e
echoing-dinner-19531
12/13/2022, 6:22 PMNext one is next week but I've already got something else lined up for that (although if that idea finishes early, this was going to be my back up hack project), but we have one every 3 months. So the next would probably be end of March.
e
eager-wall-56838
12/13/2022, 6:23 PMCool will keep an eye out, might experiment in the meantime
s
stocky-restaurant-98004
12/13/2022, 8:02 PM@eager-wall-56838 We have internal tooling to generate GHA for our providers, but it's not Pulumi per se. It is open source if you're interested.
e
eager-wall-56838
12/13/2022, 8:03 PMSure shoot me a link!
s
stocky-restaurant-98004
12/15/2022, 2:51 PMstocky-restaurant-98004
12/15/2022, 2:52 PMSpecifically, the
provider-cie
echoing-dinner-19531
12/24/2022, 9:18 AMSo I did end up ending my first hackweek project early (pluginification of secret providers, proved out the interfaces, need to do a load of engine clean up to really enable it, https://github.com/pulumi/pulumi/issues/11493) so I had a look at the idea of turning any JSON schema into a pulumi object model, the results of that are at https://github.com/Frassle/pulumi-jsonschema/
It needs a load more work, especially for the more complicated combinators like allOf/anyOf/oneOf but it seems a promising start. I'm going to see if I can get it to give a good object model based on the GHA schema from "https://raw.githubusercontent.com/SchemaStore/schemastore/master/src/schemas/json/github-workflow.json".