Hello wave I m wondering if people are using `mypy` or any o Pulumi Community #python

Hello :wave: I’m wondering if people are using `my...

enough-garden-22763

04/21/2021, 9:58 PM

Hello 👋 I’m wondering if people are using

mypy

or any other static checkers with Pulumi, and how’s the experience been?

enough-leather-70274

04/21/2021, 10:00 PM

I have both mypy and pylance enabled in vscode, and they've worked fine for me so far.

enough-garden-22763

04/21/2021, 10:01 PM

For some context I’ve been looking at some rather slow (400s) mypy check times on some programs and considering introducing lazy module imports to speed up the situation. There is however a cost, there is a minor breakage of mypy for programs with a certain style of imports: https://github.com/pulumi/pulumi/issues/6837

enough-garden-22763

04/21/2021, 10:02 PM

That is, there is no effect on runtime behavior of

pulumi preview, up, etc

enough-garden-22763

04/21/2021, 10:02 PM

Feedback is very welcome. 🙏

enough-garden-22763

04/21/2021, 10:02 PM

I should check out pylance, thanks for that pointer.

enough-leather-70274

04/21/2021, 10:07 PM

OK - I'll admit type checking can be a bit slow at times, but that's not isolated to pulumi. And I've seen nothing like the 400s you're reporting (we're aws)

enough-garden-22763

04/21/2021, 10:08 PM

Azure provider is probably an outlier here. I guess I should measure the impact on AWS provider as a useful datapoint.

handsome-state-59775

04/22/2021, 3:52 AM

Copy code

# mypy .
helper_k8s.py:193: error: Argument "string_data" to "Secret" has incompatible type "Output[Mapping[str, str]]"; expected "Union[Mapping[str, Union[str, Awaitable[str], Output[str]]], Awaitable[Mapping[str, Union[str, Awaitable[str], Output[str]]]], Output[Mapping[str, Union[str, Awaitable[str], Output[str]]]], None]"
Found 1 error in 1 file (checked 7 source files)                                                                                                                                                              
~4m-23.0s

provided type fits into expected type set as far as i can see. not sure how to fix this - any insights? mypy 0.812

handsome-state-59775

04/22/2021, 3:53 AM

^ expanded expected type, highlighted provided type

enough-garden-22763

04/22/2021, 2:22 PM

It’s a bit unrelated but an interesting error. I filed your issue as https://github.com/pulumi/pulumi/issues/6843 suggestion. It might be difficult to “fix” since marking Output covariant in T can have far-reaching implications. So perhaps you can consider a workaround here. The workaround is using

string_data.apply(lambda v: v)

👍 1

enough-garden-22763

04/22/2021, 3:28 PM

Coming back with a bit more data. https://github.com/pulumi/pulumi/issues/6837 I am observing a large mypy speedup on AWS example as well, 53s -> 0.74s.

enough-garden-22763

04/22/2021, 3:28 PM

53s is a lot, I am beginning to wonder if something is abnormally slow with my setup, as I’d expect people be complaining more about it. I will try to get some more independent measurements.

handsome-state-59775

04/22/2021, 5:16 PM

the azure-native code i posted above took 4m23s to run, so i'm sure it's not an issue with your setup 🙂 i'm running it on an 8 core i9 2019 macbook pro. mypy for it exhausts all of my 16G of RAM and proceeds to devour swap at rate that puts chrome to shame

handsome-state-59775

04/22/2021, 5:43 PM

Copy code

# mypy --show-error-codes helper_k8s.py                      
Success: no issues found in 1 source file                                                                                                                                                                      
~14m7.3s

Good news: workaround satisfies mypy Bad news: the time it took for checking 1 file

handsome-state-59775

04/22/2021, 5:44 PM

^ imports in the file:

Copy code

import base64
import itertools
import json
import pulumi as p
import pulumi_azure_native as az
import pulumi_kubernetes as k8s
import shlex
import subprocess
from typing import Iterable, Mapping, MutableMapping

enough-garden-22763

04/22/2021, 5:50 PM

Yes, yes. Thanks for this.

enough-garden-22763

04/22/2021, 5:51 PM

I know Pulumi staff is still quite busy with 3.0 but I will wait for the feedback on my proposal above to speed up mypy at the cost of breakage. We’ll see.. It would be awesome if there was a non-breaking way but I do not see it yet.

👍 1

enough-garden-22763

04/22/2021, 5:52 PM

Perhaps an upstream fix if mypy could be made to chase import refs lazily would be ideal, but that sounds a lot harder.

handsome-state-59775

04/27/2021, 7:29 AM

i renamed my project, and mypy checks broke completely 😞

Copy code

resource_group.py:3: error: Cannot find implementation or library stub for module named 'pulumi'  [import]
resource_group.py:4: error: Cannot find implementation or library stub for module named 'pulumi_azure_native'  [import]
aks.py:7: error: Cannot find implementation or library stub for module named 'pulumi_kubernetes'  [import]
aks.py:8: error: Cannot find implementation or library stub for module named 'pulumi_tls'  [import]

already tried clearing .mypy_cache, reinstalling dependencies, and recreating venv entirely.

pulumi up

, etc. works without issue. any tips regarding this please?

enough-garden-22763

04/28/2021, 2:21 PM

mypy

installed into the same venv as Pulumi (just guessing maybe it’s system-level mypy not finding pulumi imports)? Do you think you can jot down some repro steps for me?

👍 1

handsome-state-59775

05/01/2021, 7:53 AM

this is exactly the case - thanks! after install mypy to the venv, had to reactivate the venv

👍 1

144 Views

Open in Slack

Previous Next