This message was deleted.

hi @ambitious-computer-3093 - this looks like a good pattern. I wrote something up last week that may help: https://leebriggs.co.uk/blog/2023/08/17/structuring-iac to answer your questions:

How does CI/CD work where we might want to only deploy only one or some of the stacks? Do we write a script that simply traverses the subdirectories and runs pulumi up on each, knowing that Pulumi should be smart enough to not cycle resources? A major disaster point would be cycling the experiment tracker and thereby losing the results of previous experiments.

There’s 2 options here - have a stack per environment in CI/CD and make sure that environment only runs on PR/merge or tags. The other is to use automation API to wrap the invocation and run it that way

awesome thanks and i will pass this on

How do we handle IAM users? Ideally, we’d want a single config file where we can list users by email and which groups they should be in (data-science, ml-engineering, superuser) which can then be shared via stack references.

Personally I’d avoid IAM if you can, I wrote something about the right way to auth to AWS here: https://leebriggs.co.uk/blog/2022/09/05/authenticating-to-aws-the-right-way

thats really awesome thanks for your help

no official guidance written except this https://www.pulumi.com/docs/concepts/resources/components/

perfect thanks!

Paging @busy-journalist-6936 who’s been working on some AI/ML-specific stuff and may have some additional input

I'm def into the microstack idea but have not practiced it myself before, though I am breaking up my pulumi python into smaller modules so may have more to say/share soon.