Hey folks, what would you say is the most appropriate way to share a resource between different stacks of the same project? Let's say I have stacks

dev

and

prod

, but they both want to use the same IAM Role. Should I create a separate stack just for the role, and then use stackReference in the other projects?

I'm interested in the answer to this as well.

Do you want to share the IAM Role between stacks in the same project, or between stacks in different projects? If the latter (stacks in different projects), then a stack reference is one way to do it. I haven’t verified, but there’s probably a “getter” function to look up an IAM Role that you could use, too. If you have multiple stacks that need to reference the same IAM Role, then you could consider breaking the IAM Role into its own stack. If the former (stacks in the same project)…well, stacks in the same project run the same code (perhaps with different parameters), so I’m honestly not really sure how you would go about doing that. There’s no way of which I know to have one stack in the project create the role but another stack in the same project consume a role (I mean, I guess you could build some logic into the code based on the stack name or something).

Same project, different stacks. My point is as such. The IAM Role has no difference between the stacks. Same name, same policy, same everything. So when I switch between

dev

and

prod

and run

pulumi up

in either of them, I assume that pulumi would either get confused about what to do with the role, OR try to recreate it / update it. Which would be pointless since the role is already there. I have come up with an idea which I'm gonna test soon, but I'm not sure if that's a proper approach. Give me about 30 minutes

OK, so what are you using different stacks for? Different regions, different AWS accounts, different parameters for resources…?

Different parameters for resources essentially. I'm building some data pipelines using AWS Glue jobs. I'm gonna have a staging stack that runs the pipelines on fake staging data in a

staging

bucket. Once I know everything works, I will deploy the prod stack which is going to create a separate job that parses prod data in a

production

bucket. That's more or less the gist of it

Gotcha. In that case, I’d probably split the IAM Role into a separate project and use stack references (or a “getter” function) in the data pipeline project and stacks.

@salmon-account-74572 here's what I ended up doing before you provided your latest suggestion. My repo has the following structure:

project
├── job1/
│   └── __init__.py
├── job2/
│   └── __init__.py
├── __main__.py
├── iam.py
├── Pulumi.yaml
├── Pulumi.iam.yaml
└── Pulumi.dev.yaml

In

__main__.py

, I have the following code:

import pulumi

current_stack = pulumi.get_stack()
if current_stack == "iam":
    import iam
else:
    import job1, job2

This way, if the stack is

iam

, only the IAM role and policies are gonna get updated. Else, the actual glue jobs. In the glue jobs definitions I use stack reference with

require

to the ARN exported by

iam.py

, so the program crashes if no export is found. What do you think ?

Yep---this is what I alluded to when I said you could build some logic into the code based on the stack name (what you built is exactly what I was thinking).

Neat! Does it seem reasonable? I'm quite new with Pulumi and I feel like I'm stretching its capabilities in some cases

Yeah, it’s a reasonable approach. I would caution you to make sure you document this arrangement really well so that your successor (or you in 12 months) can understand/remember exactly what’s going on with this project and its stacks. 🙂

document this arrangement really well

good idea. I'm definitely gonna forget in 12 months ahahaha 😄 Since you're here, I have one last question. I noticed that the newly created stack (in this case,

iam

), doesn't have the

vcs:

tags set yet. I know these tags are set automatically by pulumi but I'm not sure when. I'm asking cause it's convenient when browsing the UI to see the different stacks grouped together. Right now it shows like this:

Are they in a version control repository? Pulumi will set the

vcs

tags automatically if their in a version control repository, but if they aren’t you can manually set the

vcs

tags to group them together: https://www.pulumi.com/docs/reference/cli/pulumi_stack_tag/

Yeah all this project is in a github repo. Admittedly, the stack

iam

and related code was created later. Nonetheless, the

vcs

tags are currently missing. I did not set them manually. See:

Yeah, that’s odd. I’d open an issue in https://github.com/pulumi/pulumi (the team will sort it into the correct repository from there).

will do, thanks!