https://pulumi.com logo
Powered by
#general
Title
# general
k

kind-fireman-33438

10/30/2023, 6:24 PM
Is there a best practice (specifically in Go) for splitting your resources / layers up into multiple files or libraries? I’d like to have a structure something like the following:
Copy code
├── infrastructure
│   ├── cloud
│   │   ├── aws
│   │   ├── azure
│   │   └── gcp
│   └── local
├── dependencies
│   ├── postgres-operator
│   ├── postgres-operator-ui
│   ├── monitoring (stuff)
├── applications
│   ├── dimo-identity
├── scripts
├── keys
├── README.md
├── Makefile
└── .gitignore
I started to create multiple pulumis (recipes? scripts?) then pull them together in the main.go file at the top level. Ran into issues with passing results from one to the other. Tried with context but realize I should probably use stack references instead.
Either that or just have a top level pulumi.Run and then pass the context from that to methods imported from other files.
s

salmon-account-74572

11/01/2023, 8:36 PM
The second approach is what I’ve used when breaking a Pulumi Go program into multiple files, but I’m not a Go expert.
I wouldn’t necessarily recommend breaking things apart into separate projects/stacks unless there are other considerations present as well. This blog post provides some guidelines you can use: https://www.pulumi.com/blog/iac-recommended-practices-structuring-pulumi-projects/
k

kind-fireman-33438

11/01/2023, 8:43 PM
Breaking into multiple files/directories is plenty for my current use case. I’ve reviewed that link and don’t need separate github repos so might not be my best option to go that route.
s

salmon-account-74572

11/01/2023, 8:47 PM
Based on your message I would agree that breaking into separate Pulumi projects (regardless of whether you use separate GitHub repos or not) is probably overkill. 🙂
k

kind-fireman-33438

11/01/2023, 8:48 PM
You happen to know of any examples in go where there are multiple pulumi files in a single project that pass ctx around?
s

salmon-account-74572

11/01/2023, 8:51 PM
I don’t. I have one I’m working on, but I haven’t had time to publish it yet. 😕
k

kind-fireman-33438

11/01/2023, 8:53 PM
🙂
Is it shareable by chance? 🙂
s

salmon-account-74572

11/01/2023, 9:07 PM
A colleague shared this with me, it might be helpful: https://github.com/pulumi/pulumi-self-hosted-installers/tree/master/ecs-hosted/go The stuff I’ve been working on isn’t shareable yet, but let me see if I can make it available in its current form later tonight.
k

kind-fireman-33438

11/01/2023, 11:03 PM
That would be awesome, thanks!
s

salmon-account-74572

11/02/2023, 3:57 PM
I haven’t gotten to this yet, but going to work on it today as I have time.
@kind-fireman-33438 I can’t promise this is the best/most elegant code, but you’re welcome to have a look and see if it’s helpful to you: https://github.com/scottslowe/learning-tools/tree/main/pulumi/eks-from-scratch
k

kind-fireman-33438

11/02/2023, 8:00 PM
This is great @careful-vase-44898, thanks!!!
@salmon-account-74572, I’ve restructured my app to call functions and pass in context. I’m stuck trying to use a variable that I exported to the context from another go module. Any guidance on how to do this? I’ve tried a number of ways and nothing seems to work…
Copy code
func InstallDependencies(ctx *pulumi.Context) (err error) {
	stackName := ctx.Stack()
	stackRef, err := pulumi.NewStackReference(ctx, stackName, nil)
	if err != nil {
		return err
	}

	// Get kubeconfig from exported to context
	kubeConfig := stackRef.GetOutput(pulumi.String("kubeConfig")) <---- is this the right way to get the variable?
	//var getKubeConfig *remote.Command = stackRef.GetOutput(pulumi.String("getKubeConfig"))

	kubeProvider, err := kubernetes.NewProvider(ctx, "k3s", &kubernetes.ProviderArgs{
		Kubeconfig: kubeConfig.ApplyT(func(s string) string {   <---- This fails!
			return s
		}).(pulumi.StringOutput),
	}) // May want to make this do better checking to ensure that the node is all the way up
	if err != nil {
		return err
	}
s

salmon-account-74572

11/09/2023, 5:20 AM
You don't want to use a stack reference; that's for passing values between separate stacks/projects. Without looking at your code, it's probably a variable scoping issue. Define the variable outside the other function so that it will be visible to the rest of your code.
k

kind-fireman-33438

11/09/2023, 4:24 PM
Yeah, I figured but couldn’t find a standard way to do that. If I have one main file then two packages within that file, if I call the first library and want to use its outputs in the second, I believe I’ll need to pass them from the main file into the second library. If I import the first library from the second, won’t I be getting a new instance of that and not have access to the values tied to it?
s

salmon-account-74572

11/09/2023, 4:58 PM
OK, so using my “eks-from-scratch” code as a reference, I knew I needed 
vpcId
, 
publicSubnets
, and 
privateSubnets
outside of the 
vpc.go
file. So I declare them (see lines 12-14) outside of the 
buildInfrastructure
function (starting on line 17) that is called in 
main.go
. As a result, they are globally scoped, and can be referenced anywhere in the 
main
package (which all files are part of). Does that help, or am I not understanding your question/issue correctly?
k

kind-fireman-33438

11/09/2023, 5:10 PM
Yes, that does definitely help. I just passed the kubeconfig into the second function but it seems to be an empty string so I’m wrestling with that now.
Wondering if it might be better to just pass the KubeProvider instance to the second funciton instead of creating a new provider in the second instance.
s

salmon-account-74572

11/09/2023, 5:15 PM
Yeah, I might go the provider route instead of the Kubeconfig itself. I haven’t tried that myself yet, so I don’t know exactly what would be involved. If you run into issues, let me know and I’ll pester some of the engineers. 🙂
k

kind-fireman-33438

11/09/2023, 5:16 PM
Awesome, thanks. Trying that now. Will let ya know. I appreciate the help!
Actually it looks like the KubeProvider in infrastructure.go isn’t working now either. 😕
I bet it has to do with the getKubeConfig command running on the second time
Jeez, well that was it. 😄
Had to ada an “Update” portion to the getKubeConfig command.
Powered by