I am really having trouble wrapping my head around what makes sense to have in a

Pulumi.yaml

file VS what should just be defined in code. It seems like the docs recommend that you'd want to have things defined in your

Pulumi.yaml

config the stuff that is likely to vary between versions of your stack (e.g. development VS production). Okay, reasonable enough. However, I'm struggling with the proper way to structure this. I was hoping to have something like this in my Project-level config:

project:mysql-instance:
    description: MySQL instance
    value:
      name: my-db
      version: MYSQL_8_0
      disk-size: 10
      availability: REGIONAL
      tier: db-custom-1-3840

and then something like this in my Stack-level config:

project:mysql-instance:
      availability: ZONAL

To have a lower availability level in my dev environment. Unfortunately, it seems that this wipes out the entire

mysql-instance

config rather than just overwriting a single value. Are there any recommendations on how to achieve something like this? The only way I can think of is to ditch structured configuration values and have every single configuration option that would vary between stack as a top-level attribute in your config -- but that doesn't scale at all, you'll soon have hundreds of top-level properties in your config and it will be extremely challenging to figure out what corresponds to what.

I wasn't aware of config objects at the stack level replacing config objects at the project level; I'd have thought they'd do a merge instead. That sounds like an oversight. In the meantime, though, I think you could do a logical split between values you expect to be consistent and what you expect to always differ. E.g.

# Pulumi.yaml
project:mysql-instance-consts:
  description: MySQL instance consts
  value:
    version: MYSQL_8_0

# Pulumi.<stack>.yaml
project:mysql-instance-vars:
  name: my-db
  disk-size: 10
  tier: db-custom-1-3840
  availability: ZONAL

Then access both as necessary like so:

# C#
var configConsts = new Config("mysql-instance-consts");
var configVars = new Config("mysql-instance-vars");

AIUI, what’s supposed to happen is that a stack-level configuration value will overwrite a project-level configuration value with the same key. That doesn’t seem to be the behavior that’s happening here, though. @limited-farmer-68874, would you mind opening an issue on GitHub so that we can debug this behavior?

Scott, if that's the case, perhaps the key being referenced for this check is the top-level one,

mysql-instance

in this case, rather than expanding down to the child keys if the config being compared is an object

Yeah, that’s possible; it would be helpful to see the actual

Pulumi.yaml

and

Pulumi.<stack>.yaml

files being used.

happy to share ..

You’re welcome to post them here, or we can open an issue on GH and collaborate there. Whichever is easiest for you!

Pulumi.yaml

name: project
runtime:
  name: python
  options:
    virtualenv: venv
description: GCP universe definition
config:
  # Defaults for GCP-wide configuration
  gcp:region:
    description: The GCP region
    value: us-central1
  gcp:zone:
    description: The GCP zone
    value: us-central1-a

  # project-db-lite MySQL instance
  project:project-db-lite-mysql-instance:
    description: MySQL instance
    value:
      name: project-db-lite
      version: MYSQL_8_0
      disk-size: 10
      availability: REGIONAL
      tier: db-custom-1-3840
      databases:
        - name: project-db
      users:
        - name: project-developer

Pulumi.dev.yaml

#TEST-specific overrides
config:
  gcp:project: project-test-01
  project:project-db-lite-mysql-instance:
      availability: ZONAL

To sanity check...

print(config.require_object('project-db-lite-mysql-instance'))

with the

project:project-db-lite-mysql-instance

config present in Pulumi.dev.yaml:

{'availability': 'ZONAL'}

without:

{'availability': 'REGIONAL', 'databases': [{'name': 'project-db'}], 'disk-size': 10, 'name': 'project-db-lite', 'tier': 'db-custom-1-3840', 'users': [{'name': 'project-developer'}], 'version': 'MYSQL_8_0'}

Is your

Pulumi.dev.yaml

file hand-crafted? I’d be interested to see if using

pulumi config set project-db-lite-mysql-instance:availability ZONAL

writes it differently (going back to @fast-vr-6049’s thought on how the data is being structured as an object in the file).

My suspicion was actually about the behaviour of the Pulumi.Config package and how it evaluates the YAML files, though your suggestion definitely warrants testing

Trying..

OK, let’s create an issue for this on GitHub, if you don’t mind.

🙏