Is something like this possible? Essentially it is python threads running check_output() calls to invoke the cli multiple times on multiple stacks in the same folder.

I'm using typescript on the pulumi side, python for my orchestration cli

The python is installed at runtime on the container build

Since you're using Docker, you could orchestrate outside Docker, and use a different container (same image) for each deploy?

And if your outDir points to a non-bind-mounted location, you should be able to avoid that source of errors.

Not that conflicting output files is definitely the problem.. just a guess.

so I'd have multiple threads in python CD to the same dir, then run

pulumi stack select A

,

pulumi update -y

via subprocess

That's what tsconfig.json's outDir property is for. If every process is compiling the same files to the same directory, the you may get race conditions. If they're all compiling to different directories, you may avoid that.

is that a runtime flag option for pulumi to set outDir?

Good luck!

just FYI, tried this today and no dice, I used a quick and dirty way to do this, I used jinja CLI to update the tsconfig.json on each thread run right before running an update: this is a bit of the mixed-thread log outputs, this type of random error is what occurs when I run them too close together. This time I added a couple of commands to create a local tmp dir for each deploy, and then update the tsconfig to point to that new folder just before running pulumi up, hoping that would get each compiliation to use its own dir - didn't work 😕 if I space the threads out 30 sec apart, it is ok, but then it sorta nullifies the benefit of the threading with 15 services going up simultaneously.

[DEBUG] MYCOMPANY SERVICE DEPLOY PULUMI CMD LIST: [
  'pulumi stack select mycompany-ecs-services.us-east-1.stage.userservice.blue', 
  'pulumi refresh --yes', 
  'pulumi config set aws:region us-east-1', 
  'mkdir tmp/HXKdIUPPXo', 
  'OUTDIR=HXKdIUPPXo jinja2 tsconfig.json.j2 > tsconfig.json', 
  'TASK_REVISION=169 pulumi update --yes'
]                   
 +  pulumi:pulumi:Stack mycompany-ecs-services-mycompany-ecs-services.us-east-1.stage.apigateway.blue create error: an unhandled error occurred: Program exited with non-zero exit code: -1
 +  pulumi:pulumi:Stack mycompany-ecs-services-mycompany-ecs-services.us-east-1.stage.apigateway.blue create 1 error

Diagnostics:
  pulumi:pulumi:Stack (mycompany-ecs-services-mycompany-ecs-services.us-east-1.stage.apigateway.blue):
    error: an unhandled error occurred: Program exited with non-zero exit code: -1

Go inline programs can run concurrently. Other languages can achieve the same results by opting for local programs instead of inline programs.

how is the Automation API maturity with Go ?

it's possible to write a CLI with Go and use the Automation API in Go to trigger/run stacks written in Typescript?

Yes. See this example: https://github.com/pulumi/automation-api-examples/tree/main/go/local_program The local program is written in Go here (https://github.com/pulumi/automation-api-examples/tree/main/go/local_program/fargate), but it could be any other pulumi language.

hi @lemon-agent-27707 - so I've been working on this, I hope you don't mind me asking a couple more questions. I prefer to write micro-stacks. I'm thinking that I would like to have a folder of micro-stacks, and then above that I'd like to have a folder of automation scripts. The automation scripts would basically orchestrate the deployment of an "area" which may be like ALB + ECS cluster + IAM roles + Autoscaling + Logging etc. Most of the micro-stacks are already written in Typescript. I'd like to write a CLI tool in GO + cobra to help simply the CLI aspects. My question is about the automation scripts. I'd like the CLI to be able to simply load an automation script via CLI command, something like

ops -script services-refresh -env prod -region us-west-2

so the GO would (I assume) run an inline automation program, which will then run updates on several micro stacks, probably setting some config values to them as passed in by the CLI arguments. I'd like your advice on what language you think those should be, and if they are GO, I'm assuming the GO cli can basically sub-execute the automation scripts as an inline GO program. Does that sound reasonable? I think that seems more reasonable than using a GO cli to execute inline node automation programs (or maybe it doesn't really matter). But I'd like to avoid having to bake in the automation scripts to the GO cli. I'd like the CLI to be able to be ran from inside of different repos containing this type of configuration so it is portable to many projects. Any advice is appreciated 🙏