https://pulumi.com logo
Join Slack
Powered by
This message was deleted.
# python
s
This message was deleted.
p
Not sure if that’s gonna help you but I personally rely heavily on structured configuration (https://www.pulumi.com/docs/intro/concepts/config/#structured-configuration) and load the whole nested object into pydantic model for further validation (thanks to that I can easily check if required fields are present and even check the type correctness). Could you describe a little bit more what you wanted to achieve?
Regarding your second question, I’m sorry but I don’t really get it 😞. Do you want to have multiple config files and “merge” them while doing 
pulumi up
?
a
Thank you for the quick response! 🙂 Regarding the structured configuration, exactly what I mean but I need this to be a bit more comprehensive. Let's take the example there for instance, what happens if the user hasn't defined 
nums
? When you perform this line:
Copy code
config = pulumi.Config()
data = config.require_object("data")
print("Nums:", data.get("nums"))
data.get("nums")
returns 
None
, obviously I can use 
data.get("nums", [])
to specify this manually, but that means I am aware that nums is a list, and therefore I know the falsy default value is 
[]
. But what happens when the type is not known? Or we decide to change nums into a dict?
Regarding the second question, yes, I want to be able to merge config files, and to have the ability to overwrite or append fields accordingly.
p
what happens when the type is not known
I’d joke that you’re already f**ed 😄 . Even in python (that is a language without static typing) you should at least have some assumptions regarding data types in order to work on them. If 
nums
was a list and you decided to change to into a dict, you’re most probably break your code. Check my suggestion regarding pydantic models usage:
Copy code
from typing import Dict, List, Mapping, Optional
from pydantic import BaseModel

class MyConfig(BaseModel):
    nums_with_default_empty: List[int] = []
    optional_nums: Optional[List[int]]
    required_nums: List[int]

...

pulumi_conf = pulumi.Config().get_object("config") or {}
config = MyConfig.parse_obj(pulumi_conf)
Once again, not sure if that’s what you’re looking for but I’m trying to partially guess what problem you want to resolve.
btw, pydantic also lets you create more complex validators and things like min_length of the list is already a built-in feature
I want to be able to merge config files
As far as I know, each stack can only have one, single config file (loaded with 
pulumi.Config()
). You can overcome that limitation but implementing some logic outside of this config (pulumi does not prevent you from using your custom files or even getting your config from the internet - not that I’d advice doing that in IaC project). If you want to make some modular projects and divide things into layers, I’d look into creating separate projects and bind them using Stack References (https://www.pulumi.com/docs/intro/concepts/stack/#stackreferences). One of my setup is divided into 3 different pulumi projects: • gcp-project-bootstrap (creates a GCP project, VPC inside it and some high-level IAM rules) • gcp-project (creates GKE, CloudSQL, MemoryStore, etc.) • k8s-addons (manages helm charts within GKE and secrets/configmaps)
a
Pydantic sounds exactly what I'm looking for. I do have my configuration separated to MyClass and MyClassArgs, so I might as well do it like that. I'll play around a little and get back to you, but so far it seems as though this has been extremely helpful.
👍 1
I'm trying to hone IaC principles and be able to control my infra better. Let's say I want to expose a new port to all my services, then I'd have to edit every single stack file, whereas with configuration layers I'd simply edit the base layer which they all effectively inherit from
I'll wrap my head better around the matter and I'll get back to you but I feel like you've given me a push in the right direction
Thank you for your time and patience! 🙂
8 Views
Open in Slack
PreviousNext
Powered by