https://pulumi.com logo
Join Slack
Powered by
:thread: Is Config able to load a nested struct?
# kubernetes
c
šŸ§µ Is Config able to load a nested struct?
I'm pulling up
Copy code
Onlyyaml          bool   `yaml:"onlyyaml"`
	ConfigMountPath   string `yaml:"configmountpath"`
	Configdatamapname string `yaml:"configdatamapname"`
	Dapr struct {
		Enabled              bool   `yaml:"enabled"`
		AppID                string `yaml:"appid"`
		AppPort              int    `yaml:"appport"`
		EnableMetrics        bool   `yaml:"enablemetrics"`
		MetricsPort          int    `yaml:"metricsport"`
		SidecarCPULimit      string `yaml:"sidecarcpulimit"`
		SidecarCPURequest    string `yaml:"sidecarcpurequest"`
		SidecarMemoryLimit   string `yaml:"sidecarmemorylimit"`
		SidecarMemoryRequest string `yaml:"sidecarmemory-request"`
	} `yaml:"dapr"`
	Datadog struct {
		Enabled bool `yaml:"enabled"`
	} `yaml:"datadog"`
	Deployment  string `yaml:"deployment"`
	Environment string `yaml:"environment"`
	Image       struct {
		Pullpolicy string `yaml:"pullpolicy"`
		Repository string `yaml:"repository"`
		Tag        string `yaml:"tag"`
		Secretname string `yaml:"secretname"`
	} `yaml:"image"`
}
and i'm so confused šŸ˜­ after hours of debugging. In the example above say 
Dapr
is loaded fine by 
cfg.RequireObject("data", &configData)
However, despite Dapr struct loaded the nested values from pulumi config and the cli returning the 
pulumi get config --path 'data.problemstruct
when loaded by pulumi it only pulls in 3 values and all the rest are empty. I have tried 
inline
modifier and moved to it's own config path at the same level as 
data
, and nothing changes. I never see the values populated at any stage despite the cli being able to load and my yaml mapping being correct (i believe). Any hints?
b
Use the json struct tag instead of yaml
What does your config look like? Also, can you share a slightly more complete code snippet?
c
I'm thinking it has to do with
Copy code
RedisHost               string `yaml:"redis_host,omitempty"`
I want the yaml tag to be 
redis_host
when I unmarshal and also later when I marshal back to a string. However, it seems to be unable to match that until I changed to 
redisHost
.I have no idea why as I thought the tag was independent of the name.
Ok, I can do that.
b
Sorry I'm not at a computer so I can't paste in a code example.
c
I just tested and once I matched say
Copy code
APIHost `yaml:"APIHost"`
it found it
but not with 
api_host
being the matching word
So I should use json tag even though I'm remarshalling later back into a yaml file configmap string?
I thought I didn't need caps on anything other than the property in the struct for it to be exported.
b
You can have both json and yaml tags on the same property
It'll be a bit easier to comment with a more complete example
c
yes ok, let me trim it down then, just a sec
Copy code
type AppConfig struct {
	Env                     string `yaml:"env,omitempty"`
	Jsonlogs                bool   `yaml:"jsonlogs,omitempty"`
	Debug                   bool   `yaml:"debug,omitempty"`
	APIHost                 string `yaml:"APIHost,omitempty"`
	// Not working APIHost                 string `yaml:"api_host,omitempty"`
	// Not working APIHost                 string `yaml:"apihost,omitempty"`

}
var appConfig AppConfig
cfg := config.New(ctx, "")
cfg.RequireObject("appconfig", &appConfig)

// appConfig == env, jsonlogs, debug show up, but APIHost
That's the first example. All my other structs seem fine on config, but this one doesn't have exact 1-1 matching names and things broke i think due to that.
b
Cool. What does the config itself look like?
c
I then take that later and create a yaml string as I mount it in the pod with a configmap
Copy code
marshalledConfigVals, err := yaml.Marshal(appConfig)


	cfgMap, err = corev1.NewConfigMap(ctx, configData.ServiceConfigMapName(), &corev1.ConfigMapArgs{
		Metadata: &metav1.ObjectMetaArgs{
			Labels: configData.AppPulumiStringMap(),
		},
		Data:      pulumi.StringMap{configData.Configdatamapname: pulumi.String(string(marshalledConfigVals))},
		Immutable: pulumi.Bool(true),
	}, pulumi.Provider(prov))
The config originally was all under data. I moved it up to this like this:
Copy code
portal:appconfig:
    api_host: my_api_host
    apihost: my_apihost
    APIHost: my_APIHost
    api_port: 80
    db_connection: ifipostthisimadevopsN00b
    debug: true
    env: dev
    jsonlogs: true
    redis_host: redis:6379
pulumi and Go are longwinded so I hope this contains the core gist of what you need to have a better idea šŸ™‚ I'm marshalling to the string with 
yaml "<http://github.com/goccy/go-yaml|github.com/goccy/go-yaml>"
However, the core issue is even though it says the appdata was a required object, it wasn't able to map any of the properties apparently till I did that 
APIHost: val
which makes no sense to me at this time at least
brb for any further answers
b
So two things: 1. The portal prefix makes me think you should be config.New(ctx, ā€œportalā€) 2. I would try and change it from yaml: api_host to json: api_host in your struct tag. This is what has worked for me.
I'd first solve the part about Pulumi reading it, then figure out how to get it marshaled back to yaml šŸ˜€
c
cool. i'll try. I couldn't find anything mentioning the namespace. The docs show load the data with "" so I'll try it.
json tag worked fine.
OMG. Should have asked first. Lol. That was 3-4 hours of wrestling with stuff.
You rock šŸŽ‰ šŸ’Æ
b
Yeah this one was very surprising - I had the exact same issue as you. I found some example code that did this, so just laying it forward.
Paying it forward šŸ˜€
c
NICE. Thank you for this
b
I think internally Pulumi treats the config as json so this is why it's necessary.
šŸ‘ 1
c
I think a section on creating a configMap yaml file for mounting and mentioning the proper tag reading would be really important. I lost 3 days of app working, and a full day of trying to figure out what happened due to this. Be good to have a section in the Config page cover with admontion to ensure json tag is used even if yaml + when marshalling back. If there is a built in marshaller that would be good to mention too or provide in the pulumi examples repo. I basically had to create a configmap (you have nginx example for go and that's it in pulumi examples) and then generate a string yaml output so it created a mounted file rather than name value pairs.
b
Iā€™m not quite sure what you mean about the ConfigMap
you should not need to do your own marshalling there
c
I need to generate a yaml file to mount NOT name/value pairs in config map.
b
Are you trying to generate a configmap or is the value for the configmap itā€™s own YAML blob?
c
So basically the normal things might be this with name value pairs
Copy code
cfgMap, err = corev1.NewConfigMap(ctx, configData.ServiceConfigMapName(), &corev1.ConfigMapArgs{
		Metadata: &metav1.ObjectMetaArgs{
			Labels: configData.AppPulumiStringMap(),
		},
		Data:      pulumi.StringMap(appconfig), // <<<< Example. Whatever the type, I just pass in the stringmap.
		Immutable: pulumi.Bool(true),
	}, pulumi.Provider(prov))
instead I was asked to generate a yaml config that gets mounted into the container at a path, not env variables.
Copy code
cfgMap, err = corev1.NewConfigMap(ctx, configData.ServiceConfigMapName(), &corev1.ConfigMapArgs{
		Metadata: &metav1.ObjectMetaArgs{
			Labels: configData.AppPulumiStringMap(),
		},
		Data:      pulumi.StringMap{"/config/config.yml": pulumi.String(string(marshalledConfigVals))}, <<< example string map, but the key = file path and the value = the yaml marshalled string
		Immutable: pulumi.Bool(true),
	}, pulumi.Provider(prov))
I am using the 
type AppConfig struct
from the pulumi config and passing to the ConfigMap to create. This way my config stays in Pulumi, but generates the yaml in the plan.
Hope that makes sense?
b
OK, I see.
Yeah, itā€™s unfortunate about the 
json
piece being surprising. I think the rest is mostly just normal Go code.
FWIW, I think it would be general practice to not reuse the same struct for multiple targets, and to have distinct models for it.
c
I have AppConfig = unmarshal as is from pulumi config. Use this as is for the configmap generation. I want to be able to add a appconfig value to the json and it get to the configmap without effort, which is what I was doing that. Does that make sense in this use case?
b
It does.
Thereā€™s nothing wrong with what youā€™re trying to do, of course. The note was just that generally in Go, the preference is to use dedicated types/structs for each system (so youā€™d have one struct for what you want to read in from config, and another struct that you would populate from the first one with what you want to write into the ConfigMap. In your case, itā€™d be problematic as you want it to be ā€œautomaticā€, but just wanted to note the best practice.
c
Yes. Agree. I do try to avoid modification of a struct, but this is one of the cases of using pulumi config to set the values so exception. Makes sense though to keep things clean. Right now my pulumi code is a mess (first round with K8, still learning k8 too). Next step is cleanup of code and figuring if I can build some unit or integration tests possibly. Haven't seen a lot on that, so will be looking soon if I can make time!
b
On testing I have much less info šŸ™‚
55 Views
Open in Slack
PreviousNext
Powered by