Working on my first attempt at a dynamic provider ...
# general
d
Working on my first attempt at a dynamic provider / resource. The immediate problem I’ve run in to: It’s very very slow! Just planning an empty dynamic resource (written in typescript) takes about 1min 30sec. That can’t be right… Any ideas?
w
Hmm - that doesn't sound right at all. Any details on exactly what you are trying to do?
d
pulumi preview
this file:
it completes after about 1min 30 sec
which seems suspiciously like a timeout, but unclear why
if i comment out the
new ApiGateway
, it is snappy
oh and there is a main file that basically just imports that WebSocketGateway and calls
const gateway = new WebSocketGateway('gateway')
w
Huh - very odd - I can reproduce this as well. Will take a look to see what's causing this.
d
Thanks!
My life as an unintentional human fuzz tester continues.
w
So - it turns out you can fix this by moving this line:
Copy code
const client = new apigatewayv2();
From being outside of the
create
callback to being inside the
create
callback. It's a little involved why this makes such a big difference, but in short - dynamic providers take advantage of the same "function serialization" logic we use for turning callbacks into Lambdas (documented https://pulumi.io/reference/serializing-functions.html). To serialize the reference to the
client
defined outside of the callback, it looks like we effectively have to serialize a large part of the AWS SDK, and this takes a long time (as it's not really optimized for this). The experience here isn't ideal - it would be preferable to error instead of hang for so long for example. But the workaround should be simple.
d
Whoa! Crazy, but thanks for debugging it for me.
Let me give it a shot
yup - sure enough, that’s much faster!
Thanks.
One other quick bit of feedback on dynamic providers: I initially omitted a delete method and expected removing the resource to fail on delete, but it silently passed!
that’s a a tad scary of a default
w
Yes - dynamic provider in current form is very low level - and missing many convenience safe gauards. That's part of why we haven't formally documented it yet. But it's super useful as a low level escape hatch. Lot's of work we want to do to make this simpler/safer for "normal" usage!
d
Yeah, I’m pretty interested in it, since a couple of services i’ve inherited lately have a sort of “plugin” concept that i’d like to make dynamic/stateful resources so that 5 teams aren’t all editing/deploying the same service that really shouldn’t have to care about the plugin implementations - it would be nice to expose a pulumi lib to consumers and they could deploy their plugins & dynamic resources seems like the perfect mechanism for that.
Got this working: https://gist.github.com/brandonbloom/b0e7a77c5cab6247320e290402e87a52 - pretty happy with it, thanks!