Hi all, I’m new to pulumi and have a question about micro-stacks with monorepo setup (typescript, aws), how I manage correctly the files structure to work with single root package.json and root node-modules. In the docs they not dive in to actual use case, have some one repo examples with micro stacks. My use case in general is a deploying stepFunctions each one is isolated stack that can use common lambdas that should be in different stacks each lambda sets base on the logical domains.

hey, I’ve faced the same problem, so I’ve created an example repository with my own implementation of this approach. I have sent it for review to a pulumi team member, but there is no answer yet, so maybe there are better options. but for now, you can take a look: https://github.com/germankrause/pulumi-ts-substacks-example if you have any questions about it or improvement considerations, feel free to contact me here in slack or create an issue in the repository

This article solved my question https://blog.bitsrc.io/managing-micro-stacks-using-pulumi-87053eeb8678 and i implemented it in nx-monorepo and it work good

Cool, I have seen the article too, before building my own wheel. But I still think that would be useful to have the solution in the pulumi itself, something like

pulumi up --step=build

, as it would simplify a project overall by removing the need to configure a monorepo

There's so many solutions to this problem that a single skeleton provided by Pulumi would probably miss more times than it hits. I use Yarn2+ workspaces to achieve the monorepo goal; the two solutions mentioned above are great too; I have a legacy project that hardwires a load of paths in the various Pulumi projects to point to top-level dirs; and I'm sure there's plenty more valid solutions too.

@little-cartoon-10569 Yes, in my actual project I had to use the path aliases too. And I agree, there are too many other solutions to choose from to best fit requirements of each project. But I don't propose any skeletons or another monorepo solution - this is definitely out of pulumi's scope. The growing amount of diverse solutions only proves that there is a need of a new concept: another level of resource scoping that would compose a stack from multiple parts, just like stacks compose a project. So what I do propose instead of opinionated templates is to standardize the concept. This would eliminate the need of any skeletons, additional research and setup (which is different for each language). So that all the supported languages would benefit from the standard approach, and there would be no need to mix in other tools anymore to achieve more flexibility.

Ah, that's a different topic I think? Composing a stack from multiple parts is already fully supported by your preferred language. Import files, defined classes, whatever works. There's no requirement for a stack to be built entirely in one file.

That's the same topic - "micro-stacks", which described by pulumi's team in the blog. Which you haven't dealt with obviously if you think that "import files" is a solution.

Sorry, I may be misunderstanding something. You said "compose a stack from multiple parts": isn't that just defining resources in multiple files? A stack does not have stacks within it: if you want a stack with resources defined in multiple files, you use normal programming techniques like classes and imports to compose those files into a single Pulumi program.

No, "just multiple files" has nothing in common with microstacks. The problem microstacks solve is splitting the program into multiple steps. For example in one step you provision infrastructure (like create a k8s cluster, which is usually done once) and in another step you configure your application inside the previously created cluster (which will be run on every change). The thing is those are parts of the same environment, but none of those should not be necessary to run every time.

Ah, yes the micro-stacks described in that blog article. Those are just normal Pulumi projects, with the additional constraint that they be small, and the convention imposed that all the projects use the same stack names. Which is great, I think that'd be suitable for lots of development. Do you think any new wiring is required to implement it? It seems to be fully supported by Pulumi right now?

Yes, exactly - just normal Pulumi projects. But why would I create another project if need the separation inside the project? Another project would require to deal with monorepos. And simply specifying a different entrypoint file to Pulumi program based on some CLI argument would achieve the same, but with less effort than multiple projects

But how would state be managed? If you enter the program from a different place than when it was originally deployed, the resource tree that Pulumi builds would look nothing like the resource tree in state. You'd need separate projects in the same code to separate the state. Which means you want.. automation-api. Would automation-api solve the same problem?

And yes, it is already supported to pick the entrypoint, as I did in my example repo. But it required to refactor everything from just exporting the outputs to wrapping all the resources into functions and returning the outputs from the functions.