https://pulumi.com logo
#general
Title
# general
b

bright-motherboard-93404

10/12/2018, 1:08 PM
Is it normal for pulumi-aws ec2 resources to get spun up/down serially, even though `pulumi up`/`down` is being called with
--parallel 100
. I simply have a list of ec2 instances that I'm spinning up and down, and I've notices that the bigger the list, the longer these operations take. Have I accidentally introduced some kind of data dependency between my machines that forces Pulumi to process them one-by-one?
b

big-piano-35669

10/12/2018, 3:14 PM
Unfortunately, until https://github.com/pulumi/pulumi/issues/1063 is implemented, Python programs cannot be parallelized; so they always run serially. I think we have a reasonable idea of the design there -- it's "just" a matter of work (which we can prioritize higher to unblock you for sure). Are all of your programs in Python, or do you also use Java/TypeScript?
b

bright-motherboard-93404

10/12/2018, 3:21 PM
All in python, but that's only because we run Python apps with wallaroo. For consistency. I guess there's not reason not to do it in typescript if that means true parallelism
Let me rewrite the stack definition in typescript and see the difference
👍 1
b

big-piano-35669

10/12/2018, 3:23 PM
In general, TypeScript is a bit further along than Python -- in this regard and in the richness of higher level libraries like https://github.com/pulumi/pulumi-aws-infra, but we are committed to supporting them both at the same level long-term, it's just a moment in time thing.
Definitely eager to hear what you think after rewriting.
b

bright-motherboard-93404

10/12/2018, 3:24 PM
Hmm, there's one thing that I forgot about. I generally wanted to keep the number of dependencies low -- if we keep the stack definition in Python, that means the entire wallaroo deployment system is just a
pip install -r requirements.txt
away
b

brave-angle-33257

10/12/2018, 4:54 PM
extra vote for python3x support as well!
i

incalculable-sundown-82514

10/12/2018, 4:57 PM
@brave-angle-33257 come join #python! I’
’d love to hear about people’s python 2x vs 3x requirements
b

bright-motherboard-93404

10/12/2018, 5:27 PM
at Wallaroo, we've currently got python3 support on our plate. python2 will retire soon: https://pythonclock.org/
Well, the rewrite in JS resulted in lighting-fast spin-up times regardless of cluster size 🎉
but cluster shutdown is still serial
i

incalculable-sundown-82514

10/15/2018, 8:36 PM
Starting with the 0.16 SDK (which should be released soon!) we will delete clusters in parallel too!
b

bright-motherboard-93404

10/15/2018, 8:38 PM
That's awesome news!