Does using a self-hosted backend affect the concurrency of deploying resources? The stacks I have on app.pulumi.com seem to update all resources in parallel that can be, but my stacks with an S3 backend seem to only update one resource at a time. Is that expected?

it largely depends on the backend object store, and is down to the go-cloud underlying library we use to talk to object store buckets

Got it, it looks like https://github.com/pulumi/pulumi/issues/4605 has a reference from someone who noticed similar things to me on the S3 backend. If there's any room for me to make a PR to help here, if you could point me to where the S3 communication happens I'll gladly take a look. I know Terraform talks to the S3 backend considerably quicker, and I'm pretty familiar with how they do it in their codebase

it's all in here https://github.com/pulumi/pulumi/tree/master/pkg/backend/filestate

It looks like the blob library exposes

MaxConcurrency

field: https://github.com/google/go-cloud/blob/60d79299da9fb31d6bd022ae834b6766f45b55b1/blob/driver/driver.go#L65 that could be passed through in the

*blob.WriterOptions

of the

WriteAll

method: https://github.com/pulumi/pulumi/blob/ca3b9a57276751c71a4ca7fd10710edcdd72cfd3/pkg/backend/filestate/bucket.go#L22 Would you be open to a change where

pulumi up

could have a new flag named

--max-state-concurrency

where if set, that value would be passed through to the blob for whatever state is used? Looking through the code, it looks like Azure and S3 are the only blob backends that support it right now

I'm open to it, but I don't speak for the engineers who manage that code. I'd recommend opening an issue first and discussing there

Sounds good. Created an issue: https://github.com/pulumi/pulumi/issues/10057

Once the thread appears in archive.pulumi.com we can link to it on your issue

Oh cool, I'll update the link when I can