https://pulumi.com logo
Title
s

some-nail-32772

01/19/2023, 3:23 PM
Hi, When using automation API in NodeJS via Typescript, Is it possible to run 2 stacks in parallel?
b

billowy-army-68599

01/19/2023, 3:24 PM
if you run multiple
stack.up
they’ll run in parallel, this is up to your automation API wrapper to handle
s

some-nail-32772

01/19/2023, 3:36 PM
My issue here is that it’s not that simple since we are using NodeJS which is single-threaded by definition. Is there any chance that you have encountered such an example of running multiple stacks using NodeJS? Maybe you can share some sample code? Thanks a lot!
b

billowy-army-68599

01/19/2023, 3:41 PM
I don’t have sample code, but you can definitely achieve this with node using worker pools
s

some-nail-32772

01/19/2023, 3:50 PM
Thanks a lot!! Seems to be exactly what I looked for!!! I’ll explore it and update you if it fits our implementation. Seems that this code is without workers, right?
l

lemon-agent-27707

01/19/2023, 11:06 PM
The node process will spawn multiple pulumi processes under the hood, important to remember.
r

rapid-receptionist-28528

01/20/2023, 9:05 AM
Hi @some-nail-32772 @billowy-army-68599 sorry to jump in your thread - But I think it’s really interesting topic. from checking the provided source code - looks like it does not use Workers. NodeJS by his nature does not work in Parallel - It’s asynchronous single threaded language which mean it will run concurrently and not parallel at any single point of time only one execution task is running. I also run the code and from viewing the logs it looks like stack2 run only after stack1 is done. Btw, The following line of code (from the provided example) suggest the each stack run one by one in sequential manner since the
await
statement before the function call is blocking the rest of the code until it finished.
await Promise.all(stackNames.map(async stackName => await testStackLifetime(stackName)));
Please let me know if my understanding is not correct, and if possible please elaborate on the last sentence you mention:
The node process will spawn multiple pulumi processes under the hood
How did you come to that conclusion? I mean where in the code ? Thanks!!
l

lemon-agent-27707

01/20/2023, 4:10 PM
Automation API spawns a new process for each pulumi command you run as a part of your program: https://github.com/pulumi/pulumi/blob/5cfad4a909eb6162b57c770a9174340ec823bf97/sdk/nodejs/automation/cmd.ts#L64