https://pulumi.com logo
Join Slack
Powered by
Is there any advice around using a single configur...
# general
a
Is there any advice around using a single configuration structure and file across multiple stacks? Does pulumi offer any way to pick this up by say... having a stack config file in the parent folder of all my stacks?
g
It's possible by passing the location of the configuration file to Pulumi during 
up/preview
. The 
--config-file {location}
flag is what you're looking for. Just curious, why would you want to use the same configuration file for multiple stacks?
a
I've got a nice data structure that represents some common intentions/conventions in a multi-region setup.
I have it as a DTO in a common library that all my stacks refer to.
Which is really helpful, 'cause I can then do: 
configuration.GetObject<Project>("project");
And that value is strongly typed with information about all my regions.
What I was hoping to do was avoid repeating that structure once per 
Pulumi.stack.yaml
file. I guess what would be really cool is if I could import another 
.yaml
file. Or alternatively, provide multiple that then get merged together prior to being passed to Pulumi.
Copy code
myorg-core:project:
    Debug: true
    Name: "myorg-prototype-3"
    OrganizationName: "myorg"
    OrganizationNameShort: "mp"
    EnvironmentNameShort: "mpp3"
    DefaultLocation: "us-central1"
    Regions:
      ca:
        Code: "ca"
        Global: true
        Location: "northamerica-northeast1"
        Cidr: "10.8.0.0/28"
      us:
        Code: "us"
        Location: "us-central1"
        Global: false
        Cidr: "10.9.0.0/28"
  gcp:project: myorg-prototype-3
  google-native:project: myorg-prototype-3
☝🏻 This is repeated in all of my stacks.
g
Pulumi.stack.yaml
files are meant to be per stack, so the configuration isn't really designed to do what you'd like, although it will support it. You're using .net, right? What's stopping you from just using a common appsettings.json file and using .net apis, such as 
ConfigurationBuilder
to use your shared config file?
a
Probably nothing, but my default is to try and work within the system before I start doing my own thing 😅
g
I hear you, but it's just code, so .net configuration is working within the system
a
Alright, so I guess outside of parsing it, pulumi really has no preferences/involvement with the 
yaml
files?
Like it's not expecting them to be there?
Or will I end up with a vestigial 
yaml
AND the config.
g
That's true. One sec let me grab another option for you.
👍🏻 1
Disregard, what I was thinking isn't going to support the usecase you want. I think the easiest option for you is to pass the location of the stack configuration file when exeucting an 
up/prevew
. 
pulumi up --config-file ../my-shared-stack-config.yaml
a
That's not too too bad.
g
that was my original suggestion 🙂
a
Hmm....
What about the namespace though?
Copy code
config:
  my-org-core:project:
g
project namespace, is project level not stack
a
my-org-core:project:
seems to influence where 
configuration.GetObject<Project>("project");
begins looking.
g
Will all your stacks.reside in the same project? Or different projects?
a
Different projects.
So each stack is its own 
.csproj
with their own 
Pulumi.yaml
, in sibling folders.
And I guess - at least for the moment - I have a 1:1 correlation between projects and stacks as a result?
g
Stacks are meant to be members inside a single project. Think of a project as an application and stacks are the different versions of that application. For instance, dev, test, stage, prod...
a
But the stack code can be reused between multiple stacks, right?
(I think of 
public class MyStack : Stack
as more like a "stack template", is that right?)
g
But the stack code can be reused between multiple stacks, right?
There is no such thing as stack code. Projects contain code (program.cs, .csproj, etc). Stacks are instances of a project.
a
It's not like I have to make: • 
public class MyDevStack : Stack
• 
public class MyProdStack : Stack
• 
public class MyStageStack : Stack
• 
public class MyTestStack : Stack
...right?
g
Correct, instead of that, you'd have one single 
MyStack
class with four stacks. 
Pulumi.Dev.yaml
Pulumi.Test.yaml
Pulumi.Stage.yaml
Pulumi.Prod.yaml
a
Okay, so I'm on the right track there still.
I only have the one 
public class MyStack : Stack
.
g
Yes.
a
(FWIW, I think the name 
Stack
in code risks confusion, might be good to call that base class something like 
StackTemplate
, I might open a suggestion there 😉 )
g
The .net template is misleading because it autonames it "MyStack"
yeah, exactly.
a
Regardless, so coming back to the yaml files though... That 
Pulumi.yaml
does kind of result in an inference of the entrypoint.
g
Please, do open an issue to rename the main class in the .net template. I think that would be helpful
💓 1
a
I think because that 
Pulumi.yaml
dictates where things begin, I felt like I should have multiple 
.csproj
and multiple accompanying 
Pulumi.yaml
.
g
To you 
Pulumi.yaml
file question, that represents the file Pulumi uses to understand what language SDK you're using, what the project name is, and other optional things. It has nothing to do with individual stacks
a
Hmmm... Alright, so if I did this:
Copy code
my-git-repo-root/
        my-stack-project-one/
            MyStackProjectOne.csproj
            Pulumi.dev.yaml
        my-stack-project-two/
            MyStackProjectTwo.csproj
            Pulumi.dev.yaml
        my-stack-project-three/
            MyStackProjectThree.csproj
            Pulumi.dev.yaml
        Pulumi.yaml
Could I get away with that?
(Of note here: Only one 
Pulumi.yaml
, above all project dirs.)
Or do the environment-specific stack 
yaml
files have to be a sibling to 
Pulumi.yaml
?
g
I don't believe this will work. I think the 
Pulumi.yaml
file and the .csproj file must be in the same folder
a
Okay, so it does really govern the entrypoint.
So I think that brings me back to my one issue then...
I think this behaviour prevents the 
--config-file
option from working here.
g
I guess I don't understand the connection. Those values would reside in the stack configuration file aka 
pulumi.dev.yaml
which has nothing to do with a 
Pulumi.yaml
file.
a
Right, but I'd still be on the hook for repeating them for each 
Pulumi.yaml
I have.
So I'd have to do:
g
Correct, if you want all stacks to use a single configuration file, you'd need to make use of the 
--config-file
flag.
a
Copy code
config:
  my-org-core:project:
  my-org-other-stack-1:project:
  my-org-other-stack-2:project:
g
But that kind of defeats the purpose of a a configuration file?
why even break them up into different projects?
a
Because they have a data stucture in common.
All my projects have some config that I'll need to leverage.
g
If you're going to repeat the same data structure, 3 different times in the same file, why not just use 3 different stack configuration files? You're already repeating yourself 3 times.
a
And they're separate projects because I'd like to be able to deploy them separately.
Yeah, I don't think --config-file is really what I want here.
What I suspect I'm after is the ability to define a snippet/base 
.yaml
and then have the stack-specific 
.yaml
files import the common values.
But then be allowed to add their own values for any specifics beyond that.
Again, they may all have extra values above and beyond this.
But each one will start out with this data structure which defines a lot of values that each will use to establish conventions.
g
I guess that just doesn't make sense to me. I think you're going to paint yourself into a corner. While, stack configurations, can require some duplication, I think it makes more sense to keep things siloed
You want a common config file and a stack specific file and that is not supported.
a
Yarrr
It's just a pain because if I add a new region, I'm visiting one 
Pulumi.stack.yaml
for each project.
Is it the end of the world? No. I was just hoping for a nicer way to manage it.
😉
g
Of course, there is nothing stopping you from writing your own. I have a python example of exactly what you're looking for: https://github.com/phillipedwards/pulumi-custom-config
a
I wonder... Has anyone ever made a pulumi .net core config bridge?
g
The answer is yes, but only using automation api
a
Automation api?
g
a
Ah, I think I'd sooner take duplication than try and extend a tool too much.
g
How many stacks are we talking about?
5 or 50?
a
6ish?
Not 50 in order of magnitude, no. More like 10 in terms of top-magnitude here 😉
g
Then just duplicate them
a
It's going to shake out to one project-per-git-repo which is roughly one-git-repo-per-microservice.
Each project will build on top of conventions from a single core that defines some common infrastructure. And then I'll likely also have another project for ingress/application gateway-ing/load-balancing, just to prevent any circular referencing 😉
g
Are you using ComponentResources for your common infrastructure?
a
Within the common infrastructure project, yeah. I have three so far. One for persistence that makes a DB and storage bucket. One for secrets. Another for some preliminary compute resources, VPC for serverless config and IAM permissions surrounding that.
I'm not sure I'll need to use component resources in my individual microservice projects though.
Probably overkill in there 😉 (but who knows, it's still early days)
g
You can always refactor as you go.
Don't need to solve all the problems day one.
a
Yeah, man. Pulumi is brilliant for that.
g
Agreed. Its a good model to follow, IMO. It sounds like you're on the right track.
a
That has to be like... The biggest thing right now is I have obliterated the pain I experienced with terraform refactoring.
Right click in Rider, click "refactor" and sing while I'm winning.
g
😆 I hear you
a
😄
Thanks for the guidance BTW 🙂
g
anytime!
❤️ 1
a
I'm sure you'll see me lurking around with deep and complex ornate questions like this for the next week or two. 😅
Just trying to avoid being an astronaut overall.
g
The community is pretty rich in terms of Pulumi knowledge, so ask away. I'm out of here for the rest of the weekend. Enjoy the rest of your Saturday!
a
You as well!
4 Views
Open in Slack
PreviousNext
Powered by