Hi all, I’m considering a potential use-case for embedding Pulumi automation API within a custom Kubernetes controller’s reconciliation loop to read custom resources (inputs) and create remote infrastructure. Does anyone have any experience, knowledge, or thoughts on doing this? Reasons to avoid or potential benefits / shortcomings?

Have you considered using Pulumi's own Kubernetes Operator? https://www.pulumi.com/docs/iac/using-pulumi/continuous-delivery/pulumi-kubernetes-operator/ or are you looking to build something a bit more bespoke?

Regrettably, something a bit more bespoke. We’re in a situation where many teams are already creating (fake) custom resources that are currently being read and parsed by Terraform as inputs.. The high level thought was to formalize these CRs to be deployed alongside CRDs in Kubernetes and have a proper custom operator read and manage the lifecycle of this infrastructure.

Understood. It's also worth noting the operator does support defining the pulumi resources directly in one of it's custom resources here - https://www.pulumi.com/docs/iac/using-pulumi/continuous-delivery/pulumi-kubernetes-operator/#using-a-program-object But that being said if you did want to go down the route of building something yourself - the operator is open source so you could have a peek at how they do similarly to what you're describing - https://github.com/pulumi/pulumi-kubernetes-operator/blob/master/operator/internal/controller/auto/update_controller.go I don't have any strong suggestions on this other than these being a bit of a beast to scaffold/maintain. You'll also need to consider a few things i.e stack locking, status reporting and concurrency/scaling

Thank you for the information and links! Yes, our use case has a bunch of already created YAML files with Kind’s already set that (currently) do not map to any CRDs.. the spec attributes are effectively being used as inputs.. and our goal would be to keep that abstraction in place and not have to ask the users to change anything on their end. I’ll digest the operator code, which does appear to be the best place the reference.. also useful as I plan to write a bespoke operator in go. I was definitely thinking about locking, as we’d be using a DIY backend and concurrency and scaling up possible deployments of 100s of stacks concurrency. More to research there for sure