https://pulumi.com logo
#golang
Title
# golang
m

miniature-library-50162

01/10/2024, 1:59 PM
I was hoping to build something in Go which could run anywhere... Now I'm asking myself ig it makes more sense to just continue using Terraform or maybe even binding everything hard to Kubernetes based on Crossplane operators as everything seem to be pretty hard to archive with Pulumi.
s

stale-france-31173

01/10/2024, 2:00 PM
I have the exact same feeling 😞
m

miniature-library-50162

01/10/2024, 2:01 PM
Everything looked really cool and interesting, but if you get your hands dirty there seem to be some challenging hurdles.
At least I'm not alone with that feeling...
s

salmon-winter-68864

01/10/2024, 8:56 PM
I really wanted Pulumi to work. I've spent a few weeks trying to give it a decent effort to use as a ubiquitous build tool. It does everything, but devex is horrible IMO. The magic is available, but it's not, again IMO, implied, easy to grok. You can get all the values you need from the ApplyT, so you have to put your JS callback brain in gear because everything is async. That helps with problems when you're in that context. I don't get how Pulumi creates it's build order dependency graph, it's based on outputs it seems. So I had a tough time spinning up an instance and then waiting for the running state so I could run some cli tasks via ssh. I got there in the end...but...now I'm doing the same again with Terraform and it's far more enjoyable. They seem to have the balance right, where magic happens etc. Pulumi is powerful, but there in lies it's fault too. To me it's a Go Vs Rust comparison if that makes sense?
m

miniature-library-50162

01/10/2024, 9:04 PM
I'm coming from Terraform and wanted to build some kind of micro service architecture based on event bus to deploy different applications... The documentation seems more or less outdated and further information like retrieving outputs in string format for things like template files is entirely lacking... Maybe it's even easier to use crossplane and use Kubernetes as "database" 😂
s

salmon-winter-68864

01/10/2024, 9:09 PM
I feel like you can have your build tool act on events and use/apply your TF plans. That way the plan is a stand alone thing so anyone can stand up "that" piece of arch (proving it all works) and you micro service could call the same plans on event.
So then, Pulumi is probably not required, you could use the tf SDK
s

salmon-account-74572

01/10/2024, 10:32 PM
Hey all, thanks for this honest feedback regarding Pulumi and your experiences with Pulumi. While it's hard to hear sometimes, honest feedback is the only way to improve and get better.
The documentation seems more or less outdated
Would it be possible to provide an example? We're constantly working to improve our docs, and I'm keen to see if there's something we've missed.
s

salmon-winter-68864

01/11/2024, 12:58 AM
I'm not ignoring @salmon-account-74572, I haven't had an issue with docs in general. Tho examples are missing for Golang code samples for sure.
m

miniature-library-50162

01/11/2024, 6:19 AM
I can talk about go examples only as I never tried any other language for Pulumi, but as an example this simply doesn't work, that's why I think it's just outdated https://www.pulumi.com/registry/packages/hcloud/api-docs/primaryip/#example-usage
@salmon-account-74572 don't get me wrong, there isn't everything bad... It's just hard to get used to some scenarios and to really get the concept of some parts. Like how can I access outputs as strings... Or embedding pulumi into a go program. For the latter there had been good examples within the auto example repo, this works without an issue.
s

salmon-account-74572

01/11/2024, 9:51 AM
Thank you for responding, and for providing some additional information; I appreciate it. @miniature-library-50162, thanks for the specific example that doesn’t work; we’ll look into it. If you find others, you are always welcome to open an issue on our docs repo: https://github.com/pulumi/docs @salmon-winter-68864 If you think of any specific instances where Golang code samples are missing, I’d love to know. (You can also create an issue if that’s easier/faster/better for you.)
s

salmon-winter-68864

01/16/2024, 6:23 PM
Hi @salmon-account-74572 I'm back a keyboard - but to help. From the default "Using Pulumi" intro docs, there are no Go examples to follow https://www.pulumi.com/docs/using-pulumi/define-and-provision-resources/ TypeScript, Python, YAML provided. Again, lack of docs hasn't been as much of an issue for me in using Pulumi, but I did notice places where Go examples were missing.
s

salmon-account-74572

01/16/2024, 6:46 PM
@salmon-winter-68864 Thanks for this---I’ll raise an issue with the team that manages this content and let them know!
f

freezing-vase-18205

01/17/2024, 8:53 AM
@salmon-account-74572 Here is my experience with a few weeks of pulumi under my belt 1. Mostly very good docs, tbh. Kudos to the team developing the infra to support multi lang nature of it. As a doc person myself I feel it in my bones this was not easy and fun ride. 2. Anyways, I found the learn-pulumi track insanely good, it must be linked somewhere in getting started (if it is not already), as it gives a reader a walkthrough experience. Links to blog posts from there would be nice as well. Some blog posts are really nice (like deploying wordpress with pulumi and ansible integration via local/remote command. 3. Not everything is rosy though. The outputs problem that seems to be a problem for most pulumi users must be clarified more extensively. Clearly people have hard time understanding the concept, some real examples are needed to cross the t’s. @miniature-library-50162 brought a good example. Deploy some stuff with pulumi and generate a yaml (ansible inventory) with the outputs. Show users how it is done and reason why this makes sense. This is what I think the key I found missing. 4. And then project structure. I expanded on this here https://pulumi-community.slack.com/archives/C01PF3E1B8V/p1705480431178569?thread_ts=1705424367.004349&cid=C01PF3E1B8V
you can use this for the blog post that covers the apply-all
s

salmon-winter-68864

01/17/2024, 9:04 AM
Just thinking about this... Because it makes sense that we don't have values until a point in time, a promise basically. And we have a callback/then scenario....so... I guess as a Go dev, you'd likely want a channel and a common response struct to populate and "act" upon. Thinking out loud. I personally fired up an instance on my cloud provider and needed to "wait" for a particular state to then make my next call. I ended up using a ctx deadline inside an ApplyT to send the val down the Chan, otherwise timeout/cancel timeout err returned. All worked as expected. Thinking about how to handle ApplyT in a app wide way.
f

freezing-vase-18205

01/17/2024, 9:11 AM
Had the same idea about channels but I think this is a burden from an auto generated code for multiple languages. Makes it hard to use the language-native tools “right”. Maybe not impossible, but certainly way harder Great use cases @salmon-winter-68864 on the
waitForResource
I have the same problem and I haven’t gotten to it just yet. Would love to see your code piece on that.
the “promise” maps nicely to a channel and a goroutine that for-selects on that chan and does the work when the value is received. Haven’t looked at ApplyT, maybe that’s how it works? But then why not letting the actual value to be returned back (as
any
)
and it feels like
waitForResource
is a common thing to have under
pulumi
itself
s

salmon-winter-68864

01/17/2024, 9:15 AM
This is what I've used this far
Copy code
package instance
import (
    "context"
    "time"
    "<http://github.com/pulumi/pulumi-linode/sdk/v4/go/linode|github.com/pulumi/pulumi-linode/sdk/v4/go/linode>"
    "<http://github.com/pulumi/pulumi/sdk/v3/go/pulumi|github.com/pulumi/pulumi/sdk/v3/go/pulumi>"
    manu "<http://gitlab.com/kylehqcom/manu/providers/linode/pulumi|gitlab.com/kylehqcom/manu/providers/linode/pulumi>"
)
// CreateInstance will create a linode resource
func CreateInstance(ctx *pulumi.Context, name string, config *manu.LinodeConfig) (*linode.Instance, error) {
    if config.SSH.Key.Private == "" {
        return nil, ErrMissingConfigSshPrivate
    }
    if config.SSH.Key.Public == "" {
        return nil, ErrMissingConfigSshPublic
    }
    return linode.NewInstance(ctx, name, &linode.InstanceArgs{
        Type:   pulumi.String(config.Instance.Type),
        Region: pulumi.String(config.Instance.Region),
        Image:  pulumi.String(config.Instance.Image),
        AuthorizedKeys: pulumi.StringArray{
            pulumi.String(config.SSH.Key.Public),
        },
    })
}
func AwaitInstanceStatus(targetStatus string, instance *linode.Instance, opts ...manu.AwaitOption) error {
    awaitOptions := manu.AwaitOptions{}.New()
    for _, opt := range opts {
        opt(awaitOptions)
    }
    // Add any delay return duration to the overall deadline duration. So that
    // when delaying, a false positive deadline error isn't returned
    if awaitOptions.DelayReturn > time.Nanosecond {
        awaitOptions.Deadline = awaitOptions.Deadline + awaitOptions.DelayReturn
    }
    // Pass a context with a timeout to tell a blocking function that it
    // should abandon its work after the timeout elapses.
    ctx, cancel := context.WithTimeout(context.Background(), awaitOptions.Deadline)
    defer cancel()
    done := make(chan (struct{}))
    instance.Status.ApplyT(func(status string) string {
        if status == targetStatus {
            close(done)
        }
        return status
    })
    for {
        select {
        case <-done:
            if awaitOptions.DelayReturn > time.Nanosecond {
                time.Sleep(awaitOptions.DelayReturn)
            }
            return nil
        case <-ctx.Done():
            return ctx.Err()
        }
    }
}
I am trying to build the abstraction on top of Pulumi. A project called "Manu". I believe Pulumi to be very powerful, the building blocks to make the generic builder project "Manu", but atpit, I feel that Terraform has a slightly higher abstraction which makes it a little easier to use and therefore enjoyable. I'm not at the fundamental building block level yet personally, so I see it's potentially and power, but I'm not there yet and can achieve my needs with Terraform in a way that suits me better at this time.
Copy code
AwaitOptions struct {
		Deadline    time.Duration
		DelayReturn time.Duration
	}
	AwaitOption func(*AwaitOptions)
f

freezing-vase-18205

01/17/2024, 9:23 AM
I went to pulumi because I couldn’t find a juice in me to re-learn terraform DSL once again. Love the idea of using a proper programming lang constructs instead of a flat tf files structure. I think with the right docs and examples even the outputs challenge can be masked away
s

salmon-winter-68864

01/17/2024, 9:26 AM
Yeah I'm with you, I like Pulumi, I do. And I really wanted it to work, and that's not to say I won't continue down this path, but I need to confirm first that my 100% effort could be better used per time via tf right now.
Hopefully the AwaitStatus code makes sense/helps. I have AwaitRunningStatus proxy funcs that pass the constant strings for me etc
f

freezing-vase-18205

01/17/2024, 9:41 AM
Yes, this makes sense. Although my case is a little bit different. I presume in your project you’re creating the instances, thus you can await on the desired status. In my case a 3rd party (cloud controller manager) creates a LB instance based on the services created in a cluster. This “external” actor that I do not control creates a resource that my pulumi stack depends on to continue its operations. Therefore, I need to use one of the get/lookup functions to query for a resource, and if it doesn’t yet exist — retry fetching it. Go’s chans and for/select make it so easy to work in that mindset, but now since I moved my stack from Go to Py I will have to chatgpt my way into how to do that in idiomatic python
@salmon-winter-68864 one other thing. In your code you’re awaiting on instance status to reach some
targetStatus
but then I guess it is interesting to know how you make the rest of your pulumi code to “block” until the status is reached? since code exec is async, creation of resources happens concurrently, so I guess your
AwaitInstanceStatus
func itself should return a pulumi output that would be an input to the rest of your stack code to make sure that resources dependable on Instance.targetStatus are not attempted to be created first
s

salmon-winter-68864

01/17/2024, 11:09 AM
That's where Pulumi "magic" happens tho...and something I don't particularly enjoy thus far. Pulumi determines when an output is required and builds the dependency on that. So I was getting really frustrated at first because I couldn't understand why the order of my go code wasn't being respected. In my main block, I call that status check and then run a bash script on the resource as expected. But it's still magic how this happens. Not a fan of this kind of magic.
At first I tried the "depends on" option, but that didn't work. Using the ApplyT in a previous call, and my code execution worked. I read somewhere in docs that output determines execution order and that's something Pulumi handles under the hood. They must have to do some serious introspection to figure that out.
f

freezing-vase-18205

01/17/2024, 1:34 PM
@salmon-winter-68864 but this is another type of magic what you described is a natural dependency graph build out I am talking about something different (at least that is what I think is happening) Let’s check with the community if the below pseudo code is executed how I describe:
Copy code
r1 = newResource1()

awaitR1StatusRunning()

r2 = newAnotherResource(input=r1.Name)
My line of thinking is that even though
awaitR1StatusRunning
is waiting till
r1.Status=="Running"
it WILL NOT prevent
r2
creation API to be triggered. And this is because r2 depends on
r1.Name
to be ready, but not necessarily to r1.Status to be Running. It can be
r1.Status==Init
and r2 will still be attempted to be created
s

salmon-account-74572

01/17/2024, 5:23 PM
@freezing-vase-18205 & @salmon-winter-68864: Thanks for the honest feedback, I really appreciate it. We recognize that Outputs are a sore spot, and we have a workstream specifically dedicated to improving that documentation. (Slight aside: when dealing with Outputs across stacks via stack references, you might want to look into OutputDetails: https://www.pulumi.com/blog/stack-reference-output-details/) Code organization (within a project---one file or multiple files, etc.) is another topic that has come up before. We’ve generally punted that to the language communities (i.e., do whatever the Python/TypeScript/Go community recommends), but it might be time to address it more directly. Finally, thanks for the positive feedback on Pulumi Learn. I know the team producing that content will appreciate the feedback! Keep the feedback coming…I’m taking notes, opening issues, and forwarding stuff to our internal teams!
f

freezing-vase-18205

01/17/2024, 5:55 PM
crazy idea @salmon-account-74572: imagine if pulumi had a “linear”
up
mode where main function behaves linearly and gives the programmer full control over the execution concurrency and dependency graph Like really standing on the promise of using a real programming language to define the infra. If you want to create 100 containers - use a goroutine/asyncio/await - you are the programmer, you decide. If you are a devops person, here is the original (default) flow that pulumi does with the hidden concurrency and code path management
That way you have outputs in native types (and not
pulumi.Output
) that you don’t have to jump hoops through to work with.
s

salmon-account-74572

01/17/2024, 6:11 PM
Interesting idea…feel free to open an issue on
pulumi/pulumi
to generate some discussion around the idea and whether folks feel like it’s something that could/should be implemented.
s

salmon-winter-68864

01/18/2024, 12:31 AM
@freezing-vase-18205 https://pulumi-community.slack.com/archives/CCWP5TJ5U/p1705498487205629?thread_ts=1704895154.944869&amp;cid=CCWP5TJ5U That's exactly what the Pulumi behaviour magic is though, it changes the execution order so in your example, due to the output of one is a dependency of two, it "should" wait. You can pass an explicit "depends_on" option to your calls but I found that didn't change my program flow. I can tell you my bash script did run after my wait for status Go/Chan code example however. That's as far as I have got atpit soz.
I really like your linear mode though. Although, I also need to remind myself to turn on async js brain with Pulumi too.
f

freezing-vase-18205

01/18/2024, 9:31 AM
@salmon-winter-68864 well, I don’t think so. What you’re saying is aboslutely true IF your goal is for r2 to wait for ANY r1.State BUT you wait for a specific status,
r1.State==Running
Let me reiterate 1. r1 is created 2. its r1.Status after creation is
r1.Status==Init
3. your await func starts to wait for
r1.Status==Running
4. BUT your next resource creation step depends just on
r1.Status
which is already present, it just happens to be
Init
which is NOT what you want, you want it
Running
but that won’t be passed to r2 creation loop 5. unless your await func returns another pulumi output and you pass this output as an input to your r2 creation logic
apparently the outputs are key force in the preview https://github.com/pulumi/pulumi/discussions/15172
s

salmon-winter-68864

01/19/2024, 7:28 AM
My code example works for me waiting for a running state - using the chan ctx deadline/cancel setup I provided. When my linode is created, it's initial state is "booting", not the state I want. I wait until the state is "running". So I can only assume that when the state is changed, which to your point, could be "any state", then this ApplyT call must be being invoked with every state change. Because I have an explicit check on the state value inside a for/loop select, each state change gets sent down the chan but is skipped if its not the "running" I want. So I think you are correct, that "any" state is fired, but I handle that in my for/loop with a ctx deadline. I'm a noob at the pulumi process too, so I can only share what I have working thus far. ✌️
I feel like your use case is more in depth that my own at this time. I think you're dealing with multiple resources, where my example was waiting for the same resource to be in a certain state, before making more calls on the 1 resource (instance) I have in scope.
My next steps would be to setup the network resources, which require the instance to be in a running state - so if I continue down this road, and have similar issues, I will reconnect to findings.
I can probably confirm easily that every instance state change invokes the AppyT func by simply Printf the value. I'm pretty sure this is how I concluded to design the code the way I did. The Pulumi linode sdk handles calling whenever the state changes.
f

freezing-vase-18205

01/19/2024, 9:37 AM
then I still don’t understand how the r2 resource won’t get triggered immediately after r1 resource is created ADD1: or maybe I do now.. Need to think about it more looking at your code.
s

salmon-winter-68864

01/19/2024, 9:42 PM
Because outputs determine dependency order, Pulumi handles this. It's what messed with my head initially because I was looking at my Go code and couldn't understand the execution order. Then I realised my Go code flow be damned. Instead I had to turn on my js promise/then brain (this is how I understand it anyway) because this is where the Pulumi magic happens, it determines what/when to fire. And I read in the docs that the output helps determine that. Take the above with a grain of salt. I don't fully understand the magic and stepping through the source code atpit didn't clarify things for me. That's when I took a step back from the keys. If my Spidey senses are telling me to hold here, then that's a good personal check point for me to confirm my requirements and stack choices. IAC is hard. I was immediately drawn to Pulumi because it's USP of your code, of choice. I thought I would be able to use my Go to stand up my ubiquitous build repo. But when the Go code in front of you doesn't behave like the Go code you write, then what am I really writing? If I continue down this path, I will keep you in the loop. I don't mind reviewing/helping where able. Best k
3 Views