Working on my first attempt at a dynamic provider ...
# general
Working on my first attempt at a dynamic provider / resource. The immediate problem I’ve run in to: It’s very very slow! Just planning an empty dynamic resource (written in typescript) takes about 1min 30sec. That can’t be right… Any ideas?
Hmm - that doesn't sound right at all. Any details on exactly what you are trying to do?
pulumi preview
this file:
it completes after about 1min 30 sec
which seems suspiciously like a timeout, but unclear why
if i comment out the
new ApiGateway
, it is snappy
oh and there is a main file that basically just imports that WebSocketGateway and calls
const gateway = new WebSocketGateway('gateway')
Huh - very odd - I can reproduce this as well. Will take a look to see what's causing this.
My life as an unintentional human fuzz tester continues.
So - it turns out you can fix this by moving this line:
Copy code
const client = new apigatewayv2();
From being outside of the
callback to being inside the
callback. It's a little involved why this makes such a big difference, but in short - dynamic providers take advantage of the same "function serialization" logic we use for turning callbacks into Lambdas (documented To serialize the reference to the
defined outside of the callback, it looks like we effectively have to serialize a large part of the AWS SDK, and this takes a long time (as it's not really optimized for this). The experience here isn't ideal - it would be preferable to error instead of hang for so long for example. But the workaround should be simple.
Whoa! Crazy, but thanks for debugging it for me.
Let me give it a shot
yup - sure enough, that’s much faster!
One other quick bit of feedback on dynamic providers: I initially omitted a delete method and expected removing the resource to fail on delete, but it silently passed!
that’s a a tad scary of a default
Yes - dynamic provider in current form is very low level - and missing many convenience safe gauards. That's part of why we haven't formally documented it yet. But it's super useful as a low level escape hatch. Lot's of work we want to do to make this simpler/safer for "normal" usage!
Yeah, I’m pretty interested in it, since a couple of services i’ve inherited lately have a sort of “plugin” concept that i’d like to make dynamic/stateful resources so that 5 teams aren’t all editing/deploying the same service that really shouldn’t have to care about the plugin implementations - it would be nice to expose a pulumi lib to consumers and they could deploy their plugins & dynamic resources seems like the perfect mechanism for that.
Got this working: - pretty happy with it, thanks!