Hello, Everyone can anyone please confirm whether pulumi work on concurrent provisioning of infrastructure. I am using golang sdk for pulumi, when I used to provision/modify/destroy resources with single thread everything works fine. But whenever I am using multiple go routine to provision resources it’s givving error some time giving fatal error.

or visit <https://pulumi.com/docs/reference/install/> for manual instructions and release notes.
error: read ".pulumi/Pulumi.yaml": blob (key ".pulumi/Pulumi.yaml") (code=Unknown): RequestError: send request failed
caused by: Get "https://<s3_bucket>/.pulumi/Pulumi.yaml": read tcp 192.168.1.45:56411->3.5.213.145:443: read: connection reset by peer
{"level":"error","@timestamp":"2023-04-13T19:52:16.998165+05:30","caller":"command/command.go:94","msg":"failed to apply this stack","error":"failed to run update: exit status 255\ncode: 255\nstdout: \nstderr: warning: A new version of Pulumi is available. To upgrade from version '3.59.1' to '3.63.0', run \n   $ brew update && brew upgrade pulumi\nor visit <https://pulumi.com/docs/reference/install/> for manual instructions and release notes.\nerror: read \".pulumi/Pulumi.yaml\": blob (key \".pulumi/Pulumi.yaml\") (code=Unknown): RequestError: send request failed\ncaused by: Get \"https://<s3_bucket>/.pulumi/Pulumi.yaml\": read tcp 192.168.1.45:56411->3.5.213.145:443: read: connection reset by peer\n\n"}
{"level":"error","@timestamp":"2023-04-13T19:52:16.998262+05:30","caller":"processor/infra_processor.go:221","msg":"failed to run apply"}
{"level":"error","@timestamp":"2023-04-13T19:52:16.998302+05:30","caller":"processor/infra_processor.go:199","msg":"error while modifying members in this slack channel: <slack_channel_name>, with err: failed to run update: exit status 255\ncode: 255\nstdout: \nstderr: warning: A new version of Pulumi is available. To upgrade from version '3.59.1' to '3.63.0', run \n   $ brew update && brew upgrade pulumi\nor visit <https://pulumi.com/docs/reference/install/> for manual instructions and release notes.\nerror: read \".pulumi/Pulumi.yaml\": blob (key \".pulumi/Pulumi.yaml\") (code=Unknown): RequestError: send request failed\ncaused by: Get \"https://<s3_bucket>/.pulumi/Pulumi.yaml\": read tcp 192.168.1.45:56411->3.5.213.145:443: read: connection reset by peer\n\n"}

though for each resources pulumi s3 bucket is different.

Is this using automation api, or just trying to call the CLI from multiple go routines?

No I am running everuthing inside go code base, not using cli

So with automation api? It should work in parallel, I know of customers using the typescript and python apis for that, I'd think Go would work as well.

func  provisionResource() error {
	var wg sync.WaitGroup
	responseChan := make(chan error, len(allSlackChannels))
	defer close(responseChan)

	for _, rsc := range listOfResource {
		wg.Add(1)
		go func(req type, ch chan error) {
			defer wg.Done()
			// pulumi go sdk logic to create resource
			}
			ch <- err
		}(slChannel, responseChan)
	}
	wg.Wait()
	for id := 0; id < len(listOfResource); id++ {
		err := <-responseChan
		if err != nil {
			return err
		}
	}
	
	return nil
}

This is kind of pseudo code. If I removed go routines and just run normal serial execution, it works fine.

What does "// pulumi go sdk logic to create resource" look like? Cause you don't have a pulumi context in that code shown

yeah actually inside the pulumi resource creation, I setup pulumi stack code, which uses pulumi context, then execute this code. https://www.pulumi.com/registry/packages/slack/

Ah right. So making multiple calls to pulumi.Run in parallel. That is not supported, you should have one pulumi.Run call per-process.

Ok, cool got it. I was also thinking so, like in my local setup I can see one file like this is automatically created, now whenever multiple go routine are fired in this block

backend.url

is getting changed everytime Thanks for the confirmation.