Hello, can anyone point me towards guidance on how stackreference works between projects when using an azure storage backend? When I start a new project I typically run "pulumi login azblob://pulumi/<projectname>" e.g. "pulumi login azblob://pulumi/project1". Does this mean that I will only be able to reference other stacks created while logged in to "azblob://pulumi/project1" or can I also reference stacks in other projects sharing the same storage account e.g. project 2 created when logged in to "pulumi login azblob://pulumi/project2"? Thanks , Alan

A user wrote a great blog post on ways he's been doing this: https://martinjt.me/2021/03/04/pulumi-multiple-projects-with-custom-backends/

Thanks @brave-planet-10645 👍 That pretty much matches my experience i.e. I need to manually add a project name to the stack and I can't reference stacks in different containers within the same storage account.

Correct. This is one of those situations where if you use the Pulumi console as a backend it's a lot easier because we do a lot of that work for you

@brave-planet-10645 Is there a nice summary somewhere that details the advantages of moving to a Pulumi hosted backend?

We have docs available here: https://www.pulumi.com/docs/intro/concepts/state/#deciding-on-a-backend and also some of the features available here: https://www.pulumi.com/docs/intro/console/

Hello @brave-planet-10645, would it be technically possible to map a project to a folder at the root of an Azure blob container instead of having a blob container per project? Also, would it be possible to map organizations to Azure blob containers? And if it’s technically possible, could such a change be accepted?

An organisation is a feature within the Pulumi console. If you're self-managing your state by using Azure blob storage you don't get the same organisation management. With regards to the folders of Azure blob storage, I honestly don't know but there's nothing wrong with you trying it is there?

Sure we can’t get the same organization functionalities with an Azure backend than with the Pulumi backend, but it would allow stack references to reference stack in another blob container (kind of organization). We can effectively use folders for projects, but stack references don’t support them. It could be cool if a stack reference to "my-org/my-proj/my-stack" was able to reference the stack my-stack in the folder my-proj of the blob container my-org. It would allow for a much cleaner stack organization and avoid the downsides one get when prefixing its stack name by its project name like explained in the (great) blog post.

So there's a great post here about how to do this kind of thing with self hosted backends. Definitely worth a read

Yes, that’s the post I was referring to. The problem is that one can’t organize stacks of different projects separately without loosing stack reference support between projects.

If that's a requirement, then what's stopping you moving over to the Pulumi console?

My client wants to keep stacks state on Azure. Should I understand that this limitation of the Azure backend is deliberate so that people are encouraged to switch to Pulumi console?

I don't think it's deliberate, but it is one of the limitations. Pulumi is open source and there's nothing to stop you from raising a PR to fix this.

Pulumi is open source and there's nothing to stop you from raising a PR to fix this.

Sure! My initial question was: is there some technical explication to these limitations and, if no, would a PR lifting these be accepted (considering that would be a breaking change)?

I'm not the one to ask about technical limitations. I'll share this thread with our engineers and see what they say. All PRs are welcome