No matter how you like to participate in developer communities, Pulumi wants to meet you there. If you want to meet other Pulumi users to share use-cases and best practices, contribute code or documentation, see us at an event, or just tell a story about something cool you did with Pulumi, you are part of our community.

Pulumi Community

hi <@U0493730JP5> - this looks like a good pattern. I wrote something up last week that may help: <https://leebriggs.co.uk/blog/2023/08/17/structuring-iac>

to answer your questions:

&gt; How does CI/CD work where we might want to only deploy only one or some of the stacks? Do we write a script that simply traverses the subdirectories and runs pulumi up on each, knowing that Pulumi should be smart enough to not cycle resources? A major disaster point would be cycling the experiment tracker and thereby losing the results of previous experiments.
There’s 2 options here - have a stack per environment in CI/CD and make sure that environment only runs on PR/merge or tags. The other is to use automation API to wrap the invocation and run it that way

&gt; How do we share similar but subtly different infrastructures between dev and prod? The main one here is workbench_* which can either be a dev workbench with one EC2 instance per data scientist to be used for development or a prod workbench which requires only a single EC2 instance for training runs. Likewise, different compute types might be required for better latency in prod, etc. We’ve settled on inheriting from classes in workbench_common and extending or overriding them in order to do this.
You’d use a stack and set configuration on the stack. You can be veru flexible here if you write a component, which has inputs which get taken from config

&gt; How do we handle IAM users? Ideally, we’d want a single config file where we can list users by email and which groups they should be in (data-science, ml-engineering, superuser) which can then be shared via stack references.

Personally I’d avoid IAM if you can, I wrote something about the right way to auth to AWS here: <https://leebriggs.co.uk/blog/2022/09/05/authenticating-to-aws-the-right-way>

thats really awesome thanks for your help

do you have any guidance on using ComponentResources in python for building up "modules" of resources for reuse and encapsulation?

no official guidance written except this <https://www.pulumi.com/docs/concepts/resources/components/>

Paging <@U029HQAKWQ0> who’s been working on some AI/ML-specific stuff and may have some additional input

I'm def into the microstack idea but have not practiced it myself before, though I am breaking up my pulumi python into smaller modules so may have more to say/share soon.