https://pulumi.com logo
Title
m

mammoth-caravan-51104

03/12/2021, 11:07 AM
Hello, I'm experiencing very slow
pulumi up
execution times. Preview phase and update phase are excruciately slow. stack has ~200 resources. is there a way to speed it up?
b

broad-dog-22463

03/12/2021, 11:13 AM
Hey @mammoth-caravan-51104 - we have a guide here on how you can help us understand these problems - https://www.pulumi.com/docs/troubleshooting/#diagnosing-issues
can you give that a try and see what you find?
m

mammoth-caravan-51104

03/12/2021, 11:23 AM
thanks Paul, will do!
opening app.pulumi.com is slow as well..
update page load time is ~ 3 minutes
b

broad-dog-22463

03/12/2021, 11:57 AM
@brave-planet-10645 Do you know of any issues here?
b

brave-planet-10645

03/12/2021, 11:58 AM
No, all looks fine from my end
@mammoth-caravan-51104 can you run a traceroute to app.pulumi.com and see what you get?
@mammoth-caravan-51104 outside of app.pulumi.com, we have been seeing issues recently with stacks that have a lot of secrets. How many do you have in the stack you're trying to preview/update?
m

mammoth-caravan-51104

03/12/2021, 12:17 PM
@brave-planet-10645 thank you for checking that out. I'm having 29 secrets in a stack, and traceroute
traceroute: Warning: <http://app.pulumi.com|app.pulumi.com> has multiple addresses; using 34.216.230.31
traceroute to <http://app.pulumi.com|app.pulumi.com> (34.216.230.31), 64 hops max, 52 byte packets
 1  [my local] ([my local])  1.901 ms  1.004 ms  0.987 ms
 2  [my local] ([my local])  2.887 ms  2.833 ms  1.979 ms
 3  [my ip]-<http://isp.superkabel.de|isp.superkabel.de> ([my ip])  57.343 ms  54.761 ms  21.933 ms
 4  [my provider].<http://static.kabel-deutschland.de|static.kabel-deutschland.de> ([my provider])  60.172 ms
    [my provider].<http://static.kabel-deutschland.de|static.kabel-deutschland.de> ([my provider])  30.443 ms  16.873 ms
 5  145.254.3.92 (145.254.3.92)  26.281 ms
    145.254.3.66 (145.254.3.66)  17.645 ms
    145.254.3.92 (145.254.3.92)  11.107 ms
 6  145.254.2.217 (145.254.2.217)  27.511 ms
    145.254.2.215 (145.254.2.215)  19.030 ms  18.807 ms
 7  145.254.2.217 (145.254.2.217)  19.039 ms  31.977 ms
    145.254.2.215 (145.254.2.215)  17.880 ms
 8  <http://ae9-100-xcr1.hac.cw.net|ae9-100-xcr1.hac.cw.net> (195.89.99.1)  19.355 ms  21.289 ms  17.757 ms
 9  <http://hbg-b2-link.ip.twelve99.net|hbg-b2-link.ip.twelve99.net> (80.239.193.100)  17.717 ms  23.430 ms  17.916 ms
10  <http://hbg-bb4-link.ip.twelve99.net|hbg-bb4-link.ip.twelve99.net> (62.115.119.86)  163.768 ms * *
11  <http://ffm-bb2-link.ip.twelve99.net|ffm-bb2-link.ip.twelve99.net> (62.115.138.172)  164.398 ms
    <http://ffm-bb1-link.ip.twelve99.net|ffm-bb1-link.ip.twelve99.net> (62.115.123.76)  168.169 ms  166.890 ms
12  <http://prs-bb2-link.ip.twelve99.net|prs-bb2-link.ip.twelve99.net> (62.115.122.138)  163.632 ms
    <http://prs-bb2-link.ip.twelve99.net|prs-bb2-link.ip.twelve99.net> (62.115.114.98)  167.576 ms *
13  <http://ash-bb2-link.ip.twelve99.net|ash-bb2-link.ip.twelve99.net> (62.115.112.242)  113.285 ms  124.722 ms  119.950 ms
14  <http://las-b23-link.ip.twelve99.net|las-b23-link.ip.twelve99.net> (62.115.137.37)  165.923 ms * *
15  <http://vadata-svc071503-lag003400.ip.twelve99-cust.net|vadata-svc071503-lag003400.ip.twelve99-cust.net> (213.248.100.93)  167.024 ms  184.681 ms  169.374 ms
16  150.222.234.49 (150.222.234.49)  169.991 ms
    150.222.234.53 (150.222.234.53)  167.595 ms
    <http://vadata-svc071502-lag003385.ip.twelve99-cust.net|vadata-svc071502-lag003385.ip.twelve99-cust.net> (213.248.100.83)  168.588 ms
17  54.239.102.59 (54.239.102.59)  166.483 ms
    52.93.34.12 (52.93.34.12)  177.142 ms
    54.239.102.25 (54.239.102.25)  169.519 ms
18  * 150.222.234.65 (150.222.234.65)  177.589 ms *
19  * 52.93.34.18 (52.93.34.18)  169.998 ms *
20  * * *
21  * * *
22  * * *
23  150.222.249.165 (150.222.249.165)  321.026 ms
    150.222.249.175 (150.222.249.175)  283.789 ms
    150.222.249.173 (150.222.249.173)  309.209 ms
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
31  * * *
32  * * *
33  *
b

brave-planet-10645

03/12/2021, 12:45 PM
Ok can you also follow the performance instructions on the link Paul posted to grab a trace file please
m

mammoth-caravan-51104

03/12/2021, 1:38 PM
i've got the file, a lot of
api/decryptValue
calls with ~300ms each
where can i send the trace file?
b

brave-planet-10645

03/12/2021, 2:00 PM
@mammoth-caravan-51104 you can email me piers@pulumi.com
But that's what I'd expect.
Where are you located by the way?
m

mammoth-caravan-51104

03/12/2021, 2:01 PM
berlin, germany
b

brave-planet-10645

03/12/2021, 2:03 PM
Yeah if you can send that trace file to me then I'll talk to people internally
m

mammoth-caravan-51104

03/12/2021, 2:03 PM
done
b

brave-planet-10645

03/12/2021, 2:04 PM
thank you
m

mammoth-caravan-51104

03/12/2021, 2:04 PM
thank you, hope that helps!