https://pulumi.com logo
#general
Title
# general
m

microscopic-park-81177

07/28/2023, 3:16 PM
have you tried putting a mutex in your dynamic provider's create / update functions?
e

echoing-dinner-19531

07/28/2023, 3:25 PM
I don't think dynamic providers are currently stateful, so I don't think a mutex would help here.
Maybe an operating system named mutex, but now you've got machine global state to worry about
m

microscopic-park-81177

07/28/2023, 3:25 PM
so this is turning into 'you need your own full fledged provider"
e

echoing-dinner-19531

07/28/2023, 3:25 PM
Probably
It's possible, but totally undocumented and tricky how to do that for Python
m

microscopic-park-81177

07/28/2023, 3:38 PM
most full fleged providers are golang, right?
e

echoing-dinner-19531

07/29/2023, 8:04 PM
Yes most, but we do have awsx which is a TypeScript "component package" but that's technically just a type of provider, and the C# dynamic provider. Other languages are possible
r

rhythmic-sunset-53997

07/31/2023, 10:01 AM
Thanks for the reply. Indeed I am using a dynamic provider so I am fully (?) in control of what is actually done to implement that resource graph I initially create.
have you tried putting a mutex in your dynamic provider’s create / update functions?
Yes, I tried these:
I tried using the standard threading Python library, but this doesn’t seem to work, the requests are made concurrently. Also no luck with fasteners.
To my surprise, neither worked - I added some
sleep()
calls and from the logs it was clear that all of the resource creation “threads” obtain the mutex at approx. the same time. Which is odd, because the latter library is supposed to support inter-process file-based locking. So either the library is bugging out or something really odd is happening. At that point I decided to ask here if there is a better, recommended approach for this. I find that the official/supported providers are quite nice to work with in Pulumi, but adding anything custom (which I understand Dynamic providers are for) is clunky. I wonder if I am missing something, or is the “custom extensibility” more of a nice-to-have-sidethought that doesn’t have much use? (No offense here, it’s perfectly okay to not support that - I just want to decide whether I should stop trying 😁 )
e

echoing-dinner-19531

07/31/2023, 3:32 PM
more of a nice-to-have-sidethought that doesn’t have much use
It's more that it's a lot of work to support this well and we haven't had time. It would be lovely for dynamic providers to just work and for it to be easier to write native providers, but we just haven't had the capacity to work on them. In this case what I think is happening is that the dynamic provider "re-hydrates" your provider code for every resource method call, so if you make a mutex object it's only actually shared for that one resource call. That's why I suggested using a named mutex, because then the state is stored in the operating system itself.
3 Views