https://pulumi.com logo
Title
i

icy-dress-83371

03/24/2021, 3:31 PM
Hi all, I have a question, and I think it is an easy one, and that I am just being very dumb. I have tried a few different little projects. I am just using
pulumi login --local
. These two projects are in different folders, with different yaml files. Each had their own stack created, but when I say
pulumi stack ls
in one of the projects folders, I seem to get both stacks accross multiple projects. I noticed this only because I tried to name the stack in both projects "dev". Am I doing something wrong here, or is this a byproduct of using local login or something?
m

millions-furniture-75402

03/24/2021, 3:37 PM
Does your
Pulumi.yaml
look the same in both projects? This file contains the project name.
b

bored-oyster-3147

03/24/2021, 3:56 PM
It's because file state backends don't separate state by project. You'll see this issue if, for instance, you try to give two stacks the same name in 2 different projects. The names will collide. That's why, when using a file state backend, you want to either: • give each project it's own state directory - but this means you will only be able to
StackReference
stacks that are in the same project • OR, prefix your stack names with the project name. I.E. project
foo
may have stacks
foo.beta
and
foo.prod
i

icy-dress-83371

03/24/2021, 4:25 PM
@millions-furniture-75402 no they are different, with different names. @bored-oyster-3147 Thanks for the heads up. So because I use local login, they are all writing their stacks to the same place
~/.pulumi/stacks/
pulumi cannot differentiate them on a project level? These are mostly just local testing at this point anyways, my plan is to use blob storage for state as I used to with terraform, so I guess with that, this problem would solve itself?
b

bored-oyster-3147

03/24/2021, 4:26 PM
blob storage is the same problem unfortunately - functionally it is just a file state backend hosted in the cloud.
In our shop we don't need or use
StackReferences
so we just place all projects in their own directory in our S3 bucket and that works for us.
i

icy-dress-83371

03/24/2021, 4:27 PM
I meant, if each project, were to have it's own Container, they would have the logical separation
Yeah, I would be the same, each project would be independent and shouldn't need to use StackReferences of any kind either.
b

bored-oyster-3147

03/24/2021, 4:28 PM
yea - if each project has it's own "directory" than you won't have any issues. Unless you need cross-project stack references
i

icy-dress-83371

03/24/2021, 4:28 PM
Perfect, thanks for your help Joshua, I'll go look through the docs, but for local testing, is there a way to do that with local state?
b

bored-oyster-3147

03/24/2021, 4:29 PM
we use a single S3 bucket but when we provide the project a backend url we suffix the project name to achieve that separation. So like
s3://{bucketName}/{projectName}/
I think you can do it the same way as I just mentioned - except locally you may need to actually create those project name directories. You don't need to do that in blob storage because it just treats path segments as virtual directories.
Another thing to keep in mind when using file state backends is you lose the locking functionality on stacks that prevent concurrent updates. There is a new environment variable you can use to enable it but currently by default it is disabled
I will say that it is a lot easier to test this or even use file state backends in practice, if you completely drop your
pulumi login
usage. Start specifying your backend in the
project.yaml
. Because as you discovered when you use
pulumi login
it sets that file state backend as your current backend in the
credentials.json
file in your user directory and then it is very easy to forget that it will preserve that login across directories and pollute your backend
i

icy-dress-83371

03/24/2021, 4:38 PM
fair enough, I used to do it that way with terraform, but for local testing there, it stored my state directly in the projects directory, I never ran into issues like this. Oh well, now I know... Thanks again
šŸ‘Œ 1
one last thing, I guess, where/how does the project.yaml support the s3/azblob details instead of using a credentials file?
I guess to ask this more clearly, all examples show I NEED to set the ENv variables for the storage account name and key. Are those not something I can add under the backend section of the config as well? Having trouble finding that documentation
b

bored-oyster-3147

03/24/2021, 5:16 PM
The pulumi project configuration file has a
backend/url
parameter that you can set.
If you need environment variables though, could you not just place those things in the stack config file and then set the environment variables before instantiating your provider?
I'm sorry my experience is with AWS - so for Azure requirements I might be shooting in the dark
i

icy-dress-83371

03/24/2021, 5:19 PM
yeah, I have set that (although only url, not sure how I would set the account name and key), but you mentioned to drop pulumi login usage, but that is required to do anything, if I logout, and try to do any cli command, it will attempt to log me into pulumi web... Maybe I misunderstand the whole "drop pulumi login usage"
b

bored-oyster-3147

03/24/2021, 5:44 PM
this must be a difference between the azure blob backend implementation and the aws blob backend implementation. You're saying in order to use the azure blob backend you must have those environment variables set? And even with those environment variables set, it is trying to use the pulumi backend when you are logged out?
I don't think AWS needs required env vars for the s3 backend usage. Or at least without them it defaults to inferred auth. So that's where our disconnect is. My "drop pulumi login usage" recommendation was from personal experience when developing locally - because I will
pulumi login ...
in one pulumi project directory, and then
cd
to another project directory and forget that file state backends don't differentiate and that I need to
pulumi login ..
again. Me setting the
backend/url
in the
project.yaml
is my way of not causing issues when I'm developing locally. In a pipeline, this is a non-issue because you can just have it
pulumi login
everytime - or in automation API you can put the
backend/url
in the
project.yaml
and just set your ENV vars on the workspace. But locally, if those environment vars are required for the azure blob backend, than my recommendation to ditch the pulumi login usage wouldn't work for you - because you also need to remember to change the values of those ENV vars and those can't be set in the YAML files. Unless of course all of your azure blob backends use the same ENV var values, in which case just setting the
backend/url
with the project name suffix in each project YAML will be fine because you don't ever need to change the ENV vars.
i

icy-dress-83371

03/24/2021, 6:44 PM
Yeah, so I have it working... Although I do NEED the variables as far as I can see, those are the storage account name and key. At least the url can be dynamic and point to a different container per pulumi project, which should allow me to avoid the clashing of projects
šŸ™Œ 1