https://pulumi.com logo
Title
c

clever-painter-96148

10/09/2022, 8:20 PM
I wrote a lot of Pulumi code in TypeScript and got used to its extensive type definition, which is very convenient when working with VSCode. I recently started a Python-based Pulumi project. I am surprised that it looks like typing support is inferior in Python. Am I wrong? Am I missing something?
e

echoing-dinner-19531

10/09/2022, 8:57 PM
We do have typing annotations, but we don't run a type checker by default (unlike for TypeScript). You can use mypy or similar to type check your pulumi python programs, we've discussed internally about how we could make this better. Maybe running mypy automatically by default, or adding an option to Pulumi.yaml to run it?
c

clever-painter-96148

10/09/2022, 9:02 PM
I use mypy. But the way types are defined doesn't feel as natural as I am used to with TypeScript. For example, I am trying to do a simple thing: an
iam.Role
subclass with default tags. Since `___init___`is overloaded with multiple definitions, it's not clear what is the clean way to achieve this while keeping typing benefits in my subclass? For reference, I see:
def __init__(__self__,
                 resource_name: str,
                 opts: Optional[pulumi.ResourceOptions] = None,
                 assume_role_policy: Optional[pulumi.Input[str]] = None,
                 description: Optional[pulumi.Input[str]] = None,
                 force_detach_policies: Optional[pulumi.Input[bool]] = None,
                 inline_policies: Optional[pulumi.Input[Sequence[pulumi.Input[pulumi.InputType['RoleInlinePolicyArgs']]]]] = None,
                 managed_policy_arns: Optional[pulumi.Input[Sequence[pulumi.Input[str]]]] = None,
                 max_session_duration: Optional[pulumi.Input[int]] = None,
                 name: Optional[pulumi.Input[str]] = None,
                 name_prefix: Optional[pulumi.Input[str]] = None,
                 path: Optional[pulumi.Input[str]] = None,
                 permissions_boundary: Optional[pulumi.Input[str]] = None,
                 tags: Optional[pulumi.Input[Mapping[str, pulumi.Input[str]]]] = None,
                 __props__=None):
    def __init__(__self__,
                 resource_name: str,
                 args: RoleArgs,
                 opts: Optional[pulumi.ResourceOptions] = None):
    def __init__(__self__, resource_name: str, *args, **kwargs):
Duplicating #1 doesn't seem wise but this API is the most user-friendly. Duplicating #2 looks easier to maintain, but is less Pythonic when using it. #3 is the actual implementation, easy to override, but not typing-friendly.
ideally, I'd like to use
CustomRole
the same way I use the original
Role
(aka keep all options open) - but that looks hard to achieve?
Not pretty IMO šŸ˜•
In TypeScript, we can do nice type composition like
Omit<RoleArgs, 'assumeRolePolicy'>
. In Python,
RoleArgs
is a class, so I don't see any easy way to do that kind of things. And I'd like to avoid writing a ton of boilerplate/redundant code. Any suggestion?
e

echoing-dinner-19531

10/09/2022, 9:38 PM
#2 is definitely the cleanest. #1 has issues when resources have properties also called "name" or "opts". It is hard trying to balance well typed code with "pythonic" code.
c

clever-painter-96148

10/09/2022, 9:41 PM
Is it possible to keep typing over Input/Output wrapping/unwrapping like in TS?
e

echoing-dinner-19531

10/09/2022, 9:43 PM
I don't think so. TypeScript has a really good type system which lets us express adding/removing Input recursively to the fields of a structure. Python just has simple generics.
c

clever-painter-96148

10/09/2022, 9:43 PM
That's what I was afraid of šŸ˜ž
Thanks @echoing-dinner-19531
e

echoing-dinner-19531

10/09/2022, 9:44 PM
dotnet and java are similar to Python in this regard. Like I don't think I can emphasise just how good and advanced the TypeScript type system is. So moving from that to something else does always feel like you've lost something.
c

clever-painter-96148

10/09/2022, 9:45 PM
Haha. You are making me wonder how dumb it is to write my Pulumi code in Python.
e

echoing-dinner-19531

10/09/2022, 9:45 PM
I mean there are good reasons to write python code, that's why we support it šŸ™‚
c

clever-painter-96148

10/09/2022, 9:45 PM
It's a Python project, so I felt like it's wiser to write everything in Python... šŸ˜…
e

echoing-dinner-19531

10/09/2022, 9:46 PM
I mean that might be the right call, but if you know the team will also have good TS experience it might be better to have Python appcode, TS infra code.
c

clever-painter-96148

10/09/2022, 9:47 PM
It's a personal project, so I am the team šŸ˜›
I just wanted to avoid context switching due to multiple languages / more complexity due to 2 very different stacks
e

echoing-dinner-19531

10/09/2022, 9:48 PM
Yeh that's reasonable, there's a million and one tradeoffs with choices like this it's hard to say what's right.
But if you do stick with Python feel free to raise issues / comment here about any rough spots.
c

clever-painter-96148

10/09/2022, 9:49 PM
I'm too old to look for perfection, so I'm looking for simplicity, which is already very hard to achieve šŸ˜›
Thanks for your support šŸ™‚
One more question, which is not Pulumi-specific, but related to Python typing: is it just me or mypy is very slow in your experience too?
e

echoing-dinner-19531

10/09/2022, 9:58 PM
Yeh it's not fast
Had similar slowness with a non-pulumi code base at my previous job
c

clever-painter-96148

10/09/2022, 10:00 PM
Okay, thanks!
f

fierce-ability-58936

10/10/2022, 9:28 PM
pyright can check types (or pylance) and do it much faster than mypy!
c

clever-painter-96148

10/10/2022, 9:40 PM
That's good to know, thanks @fierce-ability-58936 šŸ™‚
g

great-sunset-355

10/11/2022, 1:19 PM
First word of warning: I'd stay away from pulumi + python ... TypeScript is much better. (I used pulumi with multiple languages) You will soon start missing interfaces from TS. ā€¢ MyPy is extremely slow ā€¢ Types are lacking and sometimes code completion does not work - it is especially awkward when working with outputs ā€¢ Writing classes in python is somewhat clunky - I started with
pydantic
but it turned out pretty bad because again
Outputs
messed it up. So now I'm writing classes with
Attrs
.
c

clever-painter-96148

10/11/2022, 1:22 PM
Haha. Funny to read that as I'm completing my migration away from Python. It has been a pleasure to rewrite it in TypeScript. My tiny experience with Pulumi+Python makes me completely agree with you @great-sunset-355 šŸ™‚
f

fierce-ability-58936

10/11/2022, 10:38 PM
I tend to disagree - my experience with Pulumi+Python has been amazing!
pyright -w
in one console, the same thing as a language server, pytest wth watchexec and everything is nice and tidy. Pulumi with Python feels much less clunky than Go, for example. On the other hand, yes, you don't get these features straight from the box, and it involves installing lots of different tools and utilities. I'd personally prefer Go with generics, really looking forward to it!
c

clever-painter-96148

10/12/2022, 7:40 AM
@fierce-ability-58936 I've no doubt that pyright is a good way to ease the pain of mypy's poor perfomance. However, typing coverage in TypeScript is awesome compared to Python. And the way Python works, there things you can't do.
For example, in TS, you can easily build new types based on existing ones:
type ServiceRoleArgs = Omit<iam.RoleArgs, "assumeRolePolicy"> & {
  service: string;
};
Maybe you can achieve the same by metaprogramming Python classes? Don't know. But even if it's possible, it's very complicated.
TypeScript is awesome
e.g. type inference works perfectly across value wrapping/unwrapping. For example,
id
is inferred as a string in TS:
const bucket = new Bucket(...);
bucket.id.apply(id => ...);
But in Python it is `Any`:
bucket = Bucket(...)
bucket.id.apply(lambda id: ...)
e

echoing-dinner-19531

10/12/2022, 8:43 AM
But in Python it is `Any`:
Oooh we should look into that! Python has enough support for generics that I'd expect
id
to be correct there.
c

clever-painter-96148

10/12/2022, 8:44 AM
Now you make me doubt. I wrote that based on memories. šŸ˜… Do you want me to reproduce it?
e

echoing-dinner-19531

10/12/2022, 8:51 AM
If you can do raise a bug about it
c

clever-painter-96148

10/12/2022, 8:58 AM
OK, my bad,
bucket.id.apply(lambda id: id.split())
works. I must've been doing something more complex. I'll play a bit with it and raise an issue if I find something weird.
@echoing-dinner-19531 I played with type inference a bit and I did not find any issue. Sorry for my misleading statement!
Only issue I actually had was slow feedback from mypy. šŸ˜‰
e

echoing-dinner-19531

10/12/2022, 9:58 AM
No worries šŸ™‚