Hi... We've got a Pulumi stack that manages a few ...
# general
g
Hi... We've got a Pulumi stack that manages a few thousand PagerDuty resources (teams, schedules, etc). The code works - but intermittently (~30% of the time) our refresh process fails. This failure happens in the refresh process, so as far as I know the
customTimeouts
configuration is not relevant here. We've already tried dropping our parallism down to
4
... Anyone else have ideas on how we can handle this? I really hate that it fails entirely rather than having some ability to just retry...
Copy code
2025-08-12T19:07:45.4386997Z  ~  pulumi:pulumi:Stack production refreshing (79s) error: update failed
2025-08-12T19:07:45.4387240Z  ~  pulumi:pulumi:Stack production **refreshing failed (0.00s)** 1 error
2025-08-12T19:07:45.4387318Z Diagnostics:
2025-08-12T19:07:45.4387799Z   pagerduty:index:EventOrchestrationService (...):
2025-08-12T19:07:45.4388495Z     error:   sdk-v2/provider2.go:572: sdk.helper_schema: Get "<https://api.pagerduty.com/event_orchestrations/services/XXX/active>": net/http: request canceled (Client.Timeout exceeded while awaiting headers): provider=pagerduty@4.27.1
2025-08-12T19:07:45.4389448Z     error: refreshing urn:pulumi:production::...Service$pagerduty:index/eventOrchestrationService:EventOrchestrationService::...: 1 error occurred:
2025-08-12T19:07:45.4389908Z     	* Get "<https://api.pagerduty.com/event_orchestrations/services/XXX/active>": net/http: request canceled (Client.Timeout exceeded while awaiting headers)
m
@echoing-dinner-19531 could be useful to have a look into this https://github.com/pulumi/pulumi/issues/7932 as @great-sundown-78827 could benefit from this too?
@great-sundown-78827 not sure if the Resource hooks could help here, especiall
beforeUpdate
? https://www.pulumi.com/blog/resource-hooks/
e
We're looking into support retry hooks this year. Hope to have a public RFC for the design by end of quarter, and then implementation before end of year.