Just wondering, why is there a concept of a namesp...
# general
l
Just wondering, why is there a concept of a namespace in the config files. Why isn't the key the namespace Current
Copy code
aws:someProperty: 'a'
aws:somethingElse: 'b'
vs.
Copy code
aws:
  someProperty: 'a'
  somethingElse: 'b'
I feel this helps clear the confusion, at least mine, around reading nested configs and allows you to fold/unfold related options Something also related to this, why isn't the configuration key types inferred from the config file rather than specified explicitly. As in why
requireString()
and
requireObject()
and so on instead of just
require()
and
get
. I feel the config file already all has the type and secret semantics that you need for that?
1
b
if it was just
require()
and
get()
, how would that work in strongly typed languages? what would those methods return?
would you rather have a method signature that was like
getBoolean()
and know that a
boolean
is returned? or would you rather have a generic result returned from a
get()
method and then have to do your own type checking to ensure that the result is the
boolean
you expect?
having
getX
and
requireX
methods results in a much cleaner API + much more intuitive usage for the consumer
as for your original question, I'm pretty sure that in YAML this:
Copy code
aws:someProperty: 'a'
is equivalent to this:
Copy code
aws:
  someProperty: 'a'
So you very well could write your file like that if you wanted, that just happens to not be the way that Pulumi writes the config when it writes to that file, so it would probably get overwritten the next time Pulumi serialized your config. I'm not 100% certain on those being equivalent though, YAML is weird
👀 1
h
I'd be really surprised if these are equivalent. Can somebody please confirm? I don't have access to my dev machine for a few days.
g
@bored-oyster-3147 Like @handsome-state-59775 suggested, they are not equivalent. I was also caught by surprise about the "namespaces" and the structure of the yaml file. essentially you can get the value only from the top-level item the namespace, everything else that is nested is an object. assume project name is
project
eg:
Copy code
config:
  project:key: value
  project:another:
    nested: value
  project:different:
    nested_different:
      - 1
      - 2
      - 3
Copy code
# returns value
value = cfg.require('key')

# returns {"nested": value"}
another = cfg.require_object('another')

# fails to find `project:another.nested: value
another_value = cfg.require('another.nested')
I'd recommend playing around with config but in general, if you want nested config you'll have to do with
get_object
l
Nested configs also don't allow nested secrets because you would have to treat the whole object as a secret which forces you to use namespaces which are essentially just different keys.
It's no deal breaker but unless there is a good reason for it I don't see why it is done that way
AFAIK YAML has types so they could still give you type safety in typed languages by inferring the type from YAML.
g
@silly-forest-38049 what do you mean by the nested secret?
l
Like if you have
Copy code
namespace:some-service-config:
  normalProperty: a
  anotherNormalProperty: b
  secretProperty:
   AdminEmail:
     secure: some-encrypted-value
   AdminPassword:
     secure: another-encrypted-value
There is no way to read the secretProperty without reading the entire
service-config
as secret So you would be forced to do this:
Copy code
namespace:some-service-config:
  normalProperty: a
  anotherNormalProperty: b
namespace:someSecretProperty:
 AdminEmail:
   secure: some-encrypted-value
 AdminPassword:
   secure: another-encrypted-value
b
Ah so they are not equivalent in Pulumi because Pulumi treats the top as the dictionary key. In YAML, though..
@little-market-63455 I think you are severely underestimating the complexity of inferring the type from the YAML and how the API would function in code.
Honestly I save myself this headache by treating Pulumi Config like a string:string dictionary. Structured objects in Pulumi Config is wholly unnecessary IMO, and comes with more trouble than it's worth. Their API has to support both YAML & JSON so they have to make concessions to not split the API. Alternatively if you want complex Config you could do the deserialization yourself, there’s nothing stopping you from doing that in your code.
l
Could be. This is definitely opinionated. Could be very complex but given the complexity of what actually Pulumi does I would imagine they are totally capable of handling it.
b
I will give you an example in the dotnet SDK. You have this shape:
Copy code
aws:
  someProperty: 'a'
  somethingElse: 'b'
I have called
config.Get("aws")
What is the method signature of
config.Get(...)
? You can't have a dynamic return type in .NET. So it must be some generic type
ConfigResult
. This object would need to look something like:
Copy code
class ConfigResult
{
   ConfigType Type,
   object? Value
}
Where
ConfigType
is an enum that looks something like:
Copy code
enum ConfigType
{
   String,
   Int16,
   Int32,
   Int64,
   Boolean,
   Object,
   etc..
}
Ok so let's say we did our
config.Get("aws")
call. This is an object. You're suggesting that they return a strong type.
AFAIK YAML has types so they could still give you type safety in typed languages by inferring the type from YAML
In .NET that means they would have to use reflection to search all the available types that: • Matches the properties • Is accessible • Is constructible That is a really big ask and reflection has very poor performance, but let's say they accomplish it. Now your code looks like:
Copy code
var config = new Pulumi.Config();
var awsConfigResult = config.Get("aws");
if (awsConfigResult.Type != ConfigType.Object)
    throw SomeException();

if (awsConfigResult.Value is not AwsConfig awsConfig)
    throw SomeException();

// now do work with awsConfig instance that is proper type
When if you had just provided the type to begin with, your code would remain:
Copy code
var config = new Pulumi.Config();
var awsConfig = config.GetObject<AwsConfig>("aws");
No type searching required, no inference required, and cleaner consumer code.
The whole point of the type specified overloads is so that Pulumi can handle the defensive coding for you.
l
To be honest the various
requireString()
and family of functions aren't really annoying me. It's mainly the namespace thing that does My config file is just growing larger and larger and being able to fold/unfold and read the configuration for a specific component in one go is what makes it attractive for me. Mainly the secrets part It's been a while that I have done some .NET but cannot you use generics to solve the problem. I am not saying it's easy but again I don't work for them and I don't know if this is the biggest deal. So a normal
get()
or
require()
could return indeed a generic
object
type by default and if you are interested in a typed option then you can provide the type as a generic parameter and Pulumi would just cast it as it is doing now with
requireObject<T>
So basically: • If you are interested in types then it's your responsibility as a consumer to specify that type and Pulumi does is cast • If you are not interested in the type then you get a generic
object
or
any
in TypeScript
b
Right but if you get an
object
you're still doing the defensive checking in your infra code when it could've been done in Pulumi. Meaning you would only use that overload for config that can be more than one type.
And if you have config that could be more than 1 type, asking Pulumi to allow one of those types to be a strongly typed class is a very big ask. To the point that it is not feasible.
It's more likely and doable that you would get a
Dictionary<string, string>
that has the keys
someProperty
and
somethingElse
g
I did this:
Copy code
secretProperty:
   AdminEmail:
     secure: some-encrypted-value
   AdminPassword:
     secure: another-encrypted-value
but you must use
get_secret_object
and then deal with the Output type or it is possible to get plain text values by
get_object
. IMO easiest way is to pass this responsibility from pulumi to pydantic and instantiate pydantic object instead In another language I'd use the same approach and created my own config object instead of relying on pulumi.
1