https://pulumi.com logo
#general
Title
# general
l

limited-farmer-68874

06/15/2023, 6:23 PM
I am really having trouble wrapping my head around what makes sense to have in a
Pulumi.yaml
file VS what should just be defined in code. It seems like the docs recommend that you'd want to have things defined in your
Pulumi.yaml
config the stuff that is likely to vary between versions of your stack (e.g. development VS production). Okay, reasonable enough. However, I'm struggling with the proper way to structure this. I was hoping to have something like this in my Project-level config:
Copy code
project:mysql-instance:
    description: MySQL instance
    value:
      name: my-db
      version: MYSQL_8_0
      disk-size: 10
      availability: REGIONAL
      tier: db-custom-1-3840
and then something like this in my Stack-level config:
Copy code
project:mysql-instance:
      availability: ZONAL
To have a lower availability level in my dev environment. Unfortunately, it seems that this wipes out the entire
mysql-instance
config rather than just overwriting a single value. Are there any recommendations on how to achieve something like this? The only way I can think of is to ditch structured configuration values and have every single configuration option that would vary between stack as a top-level attribute in your config -- but that doesn't scale at all, you'll soon have hundreds of top-level properties in your config and it will be extremely challenging to figure out what corresponds to what.
f

fast-vr-6049

06/15/2023, 6:48 PM
I wasn't aware of config objects at the stack level replacing config objects at the project level; I'd have thought they'd do a merge instead. That sounds like an oversight. In the meantime, though, I think you could do a logical split between values you expect to be consistent and what you expect to always differ. E.g.
Copy code
# Pulumi.yaml
project:mysql-instance-consts:
  description: MySQL instance consts
  value:
    version: MYSQL_8_0

# Pulumi.<stack>.yaml
project:mysql-instance-vars:
  name: my-db
  disk-size: 10
  tier: db-custom-1-3840
  availability: ZONAL
Then access both as necessary like so:
Copy code
# C#
var configConsts = new Config("mysql-instance-consts");
var configVars = new Config("mysql-instance-vars");
s

salmon-account-74572

06/15/2023, 6:51 PM
AIUI, what’s supposed to happen is that a stack-level configuration value will overwrite a project-level configuration value with the same key. That doesn’t seem to be the behavior that’s happening here, though. @limited-farmer-68874, would you mind opening an issue on GitHub so that we can debug this behavior?
f

fast-vr-6049

06/15/2023, 6:53 PM
Scott, if that's the case, perhaps the key being referenced for this check is the top-level one,
mysql-instance
in this case, rather than expanding down to the child keys if the config being compared is an object
s

salmon-account-74572

06/15/2023, 6:55 PM
Yeah, that’s possible; it would be helpful to see the actual
Pulumi.yaml
and
Pulumi.<stack>.yaml
files being used.
l

limited-farmer-68874

06/15/2023, 6:55 PM
happy to share ..
s

salmon-account-74572

06/15/2023, 6:57 PM
You’re welcome to post them here, or we can open an issue on GH and collaborate there. Whichever is easiest for you!
l

limited-farmer-68874

06/15/2023, 6:58 PM
Pulumi.yaml
Copy code
name: project
runtime:
  name: python
  options:
    virtualenv: venv
description: GCP universe definition
config:
  # Defaults for GCP-wide configuration
  gcp:region:
    description: The GCP region
    value: us-central1
  gcp:zone:
    description: The GCP zone
    value: us-central1-a

  # project-db-lite MySQL instance
  project:project-db-lite-mysql-instance:
    description: MySQL instance
    value:
      name: project-db-lite
      version: MYSQL_8_0
      disk-size: 10
      availability: REGIONAL
      tier: db-custom-1-3840
      databases:
        - name: project-db
      users:
        - name: project-developer
Pulumi.dev.yaml
Copy code
#TEST-specific overrides
config:
  gcp:project: project-test-01
  project:project-db-lite-mysql-instance:
      availability: ZONAL
To sanity check...
print(config.require_object('project-db-lite-mysql-instance'))
with the
project:project-db-lite-mysql-instance
config present in Pulumi.dev.yaml:
Copy code
{'availability': 'ZONAL'}
without:
Copy code
{'availability': 'REGIONAL', 'databases': [{'name': 'project-db'}], 'disk-size': 10, 'name': 'project-db-lite', 'tier': 'db-custom-1-3840', 'users': [{'name': 'project-developer'}], 'version': 'MYSQL_8_0'}
happy to open an issue as well
just figured we could confirm this isn't a PEBKAC situation first
I am using Pulumi python just to be clear 🙂
s

salmon-account-74572

06/15/2023, 7:04 PM
Is your
Pulumi.dev.yaml
file hand-crafted? I’d be interested to see if using
pulumi config set project-db-lite-mysql-instance:availability ZONAL
writes it differently (going back to @fast-vr-6049’s thought on how the data is being structured as an object in the file).
f

fast-vr-6049

06/15/2023, 7:11 PM
My suspicion was actually about the behaviour of the Pulumi.Config package and how it evaluates the YAML files, though your suggestion definitely warrants testing
l

limited-farmer-68874

06/15/2023, 7:15 PM
Trying..
Copy code
pulumi config set --path project-db-lite-mysql-instance.availability ZONAL
sets it the same way I have it now
Copy code
pulumi config set project-db-lite-mysql-instance:availability ZONAL
sets it to
Copy code
project-db-lite-mysql-instance:availability: ZONAL
Which does not get picked up at all as overwriting any values in the object, which is what I expected, since that's the syntax for making
project-db-lite-mysql-instance
a namespace. Adding an additional "project:" to the front does not change anything.
s

salmon-account-74572

06/15/2023, 7:26 PM
OK, let’s create an issue for this on GitHub, if you don’t mind.
l

limited-farmer-68874

06/15/2023, 7:33 PM
s

salmon-account-74572

06/15/2023, 7:48 PM
🙏