https://pulumi.com logo
#golang
Title
# golang
p

prehistoric-kite-30979

01/15/2021, 11:55 AM
Hi all, I need something similar to a dynamic provider in Go. As this isnt supported, how have other people been doing this? My initial use case is creating a MySQL schema after creating an rds instance.
1
l

lemon-agent-27707

01/15/2021, 6:16 PM
Not a replacement for dynamic providers, but automation api can help you manage related workflows like database migraitons: https://github.com/pulumi/automation-api-examples/tree/main/go/database_migration
p

prehistoric-kite-30979

01/15/2021, 6:29 PM
thanks, from the looks of it this is basically creating a "dynamic stack" and then using up/destroy
how would you handle the update equivalent?
I guess basically by ensuring an idempotent creation/update?
l

lemon-agent-27707

01/15/2021, 7:32 PM
thanks, from the looks of it this is basically creating a "dynamic stack" and then using up/destroy
Not sure what you mean by this. This program invokes the pulumi engine with a pulumi program and generates a state file.
p

prehistoric-kite-30979

01/15/2021, 7:33 PM
yes, thats what I meant 🙂
l

lemon-agent-27707

01/15/2021, 7:34 PM
Have you had the chance to read about the automation api? https://pkg.go.dev/github.com/pulumi/pulumi/sdk/v2/go/x/auto https://www.pulumi.com/blog/automation-api/ There is some conceptual information that might be helpful.
I guess basically by ensuring an idempotent creation/update?
Calling
stack.up()
will be idempotent (running the pulumi program). It would be your job to make sure that your database migrations are invoked in the program in an idempotent fashion.
🙌 1
r

rhythmic-nail-73192

03/17/2021, 10:03 PM
We also really want to use Go for Pulumi if possible (90% of our codebase is already Go), but the lack of dynamic providers is challenging - we need to hit our own application APIs at various points in the deploy process for each customer. In the GitHub issue and this thread the most common workaround suggested is the automation API - this allows a single Go program to chain multiple different stacks together and pass outputs between them, for example. It's a great feature, but then we can't get the optimal parallelism for resource graph resolution - I have to add sequential barriers to break the deploy process at one or more points, which is what we already do with CloudFormation nested stacks and what I was hoping to avoid. I got excited by lambda.GetInvocation and resource transformations as places to insert arbitrary code, but these run during resource graph construction (which is already arbitrary code) and not during the update phase. Is there any way to inject arbitrary Go code during the resource creation process, particularly for AWS? I don't even need a full CRUD model, if I could just invoke a Lambda function as resources are updated, we'd be set.
l

lemon-agent-27707

03/18/2021, 12:17 AM
@rhythmic-nail-73192 rather than implementing a dynamic resource, you could implement a full custom resource or component resource in a provider (make your own provider). https://github.com/justinvp/vpc/blob/main/provider/cmd/pulumi-resource-vpc/vpc.go I have done similar things to model local processes as a part of pulumi workflows (run arbitrary commands, shell scripts, binaries, etc, as a part of workflow).
r

rhythmic-nail-73192

03/18/2021, 12:25 AM
Neat, thank you @lemon-agent-27707! And then how do I register the custom provider? Or would I re-compile pulumi from source? Is this a valid (albeit hacky) workaround?
output.Apply
blocks until the input is ready, so I could trigger actions as a side-effect of the apply e.g.
Copy code
bucket = s3.Bucket(...)

def side_effect(x: str) -> str:
    lambda._get_invocation(...)
    return x

# Force this resource to wait until bucket is finished, and our custom code has run
other_bucket = s3.Bucket(
    logging_prefix=bucket.id.apply(
        side_effect
    )
)
I see the warning in the docs, don't worry 😅
you should avoid side-effects within the callbacks. For this reason, you should not allocate new resources inside of your callbacks either, as it could lead to 
pulumi preview
 being wrong.
l

lemon-agent-27707

03/18/2021, 1:39 AM
If implemented in go, providers are binaries that live on your path. Not sure what platform you are on, but on mac you can see this in ~/.pulumi/plugins
r

rhythmic-nail-73192

03/18/2021, 2:02 AM
Awesome! Good to know, thanks
3 Views