Hello AWS Pulumi people. Is there a way to rate-l...
# awsc
curved-jordan-5346
10/28/2025, 5:58 AMHello AWS Pulumi people. Is there a way to rate-limit the AWS api calls made by the Pulumi provider?
We are randomly hit with "Throttling Exception: Rate exceeded" errors when updating AWS SES receipt rules. Digging into the documentation it appears that this API is limited to 1 request per second.
This is an intermittent problem; AWS does not always enforce this limit. They do, however, seem to enforce this limit whenever we have a critical, time-sensitive operation to complete. 😉
Anyone run into this problem and might have a solution?
https://docs.aws.amazon.com/ses/latest/dg/quotas.html#quotas-email-general
curved-jordan-5346
10/28/2025, 5:59 AMimage.png
e
echoing-dinner-19531
10/28/2025, 9:08 AMWe're looking at adding configurable retry options soon, which will allow you to catch throttling exceptions and retry them. Might also be worth asking in the aws provider repo if it makes sense for the provider to always auto-retry throttle errors.
👌 1
l
lively-crayon-44649
10/28/2025, 9:44 AMIn the interim I could imagine a hack where you use before hooks to implement a queue in your Pulumi program, assuming that your Pulumi programs are the only source of requests...
👋 1
lively-crayon-44649
10/28/2025, 9:44 AMBut this is a hack
c
curved-jordan-5346
10/28/2025, 9:46 AMThanks @echoing-dinner-19531 - I will do that. I have a feeling this is not the only AWS API that has a low throttle rate.
curved-jordan-5346
10/28/2025, 9:49 AM@lively-crayon-44649 Eeeeeeeenteresting!
I imagine I would set up a ~1 second "sleep" timer in the beforeCreate, beforeUpdate and beforeDelete hooks.
The next question is would I need to disable pulumi parallelism to ensure the API calls are executed serially?
l
lively-crayon-44649
10/28/2025, 9:50 AMI don't think so -- there will only be one instance of your program running at once, so all hooks will go back into the same language host
lively-crayon-44649
10/28/2025, 9:50 AMSo (and again, this is horrible, and probably not a good idea) you could imagine something like
c
curved-jordan-5346
10/28/2025, 9:53 AMThanks @lively-crayon-44649 - we are going to try that route
l
lively-crayon-44649
10/28/2025, 9:54 AM Copy code
// Making this up, should probably think about whether this works/is data-race-free etc.
let locked = false
const x = new Resource("x", { ... }, {
hooks: {
beforeCreate: [() => {
while (locked) { await new Promise(resolve => setTimeout(resolve, 1_000)) }
locked = true
}],
afterCreate: [() => { locked = false }],
},
})lively-crayon-44649
10/28/2025, 9:54 AM^ This is probably not going to work ever but maybe some better version of the idea might
lively-crayon-44649
10/28/2025, 9:54 AMI'm more of a specialist in introducing race conditions rather than eliminating them unfortunately
😅 1
lively-crayon-44649
10/28/2025, 9:54 AMAnd yea, sleeps in the before are probably simpler/good enough
c
curved-jordan-5346
10/28/2025, 9:58 AMI will implement this later this week and report back... mid-week holidaytomorrow :D
curved-jordan-5346
10/31/2025, 2:30 AMThanks @lively-crayon-44649 - adding a sleep timer to the resource hooks worked like a charm:
Copy code
...
from time import sleep
...
def sleep_hook(args: pulumiResourceHookArgs):
sleep(1)
...
_ = aws.ses.ReceiptRule(
...
opts=pulumi.ResourceOptions(
hooks=pulumi.ResourceHookBinding(
before_create=[sleep_hook],
before_update=[sleep_hook]
)curved-jordan-5346
10/31/2025, 8:38 AM@echoing-dinner-19531 Added a feature req to
pulumi-awsl
lively-crayon-44649
10/31/2025, 9:31 AMNice! I'm torn between "don't let perfect be the enemy of good" and "the thing that should not be", but glad it's working 😄
c
curved-jordan-5346
10/31/2025, 9:48 AMhah - it's not an ugly solution. Thanks sir
2 Views