https://pulumi.com logo
Title
m

millions-pharmacist-626

04/05/2023, 10:57 AM
Hey everyone, I was wondering, what is the purpose of using classes such as
glue.JobArgs()
to define args of a
glue.Job()
resource, instead of just passing them directly to
glue.Job
? Is there an advantage with this approach over the other? Reference docs: https://www.pulumi.com/registry/packages/aws/api-docs/glue/job/#inputs
g

great-sunset-355

04/05/2023, 11:12 AM
JobArgs is not a class it is an interface describing the type of
args
passed to
glue.Job(args)
This is a common pattern which helps with testing, etc...
m

millions-pharmacist-626

04/05/2023, 11:14 AM
Thanks for the reply. Can you expand a bit more on what you mean with the last part? I'm not fully understanding the benefit here
g

great-sunset-355

04/05/2023, 11:21 AM
IMO there isn't a simple explanation, this highly depends on your programming experience. You should have decent software development experience when using pulumi otherwise there's a high chance you'll shoot yourself in a foot.
m

millions-pharmacist-626

04/05/2023, 11:26 AM
I'd say I'm about intermediate, been working as a data eng for roughly 3 years, but I'm quite new with pulumi and built relatively straightforward stuff only.
g

great-sunset-355

04/05/2023, 11:40 AM
I assumed so 🙂 (please do not feel offended by the following text). From my experience data engineers typically are not very experienced software developers and lack the understanding of basics. IMO this is mainly caused by the broad scope of data engineering itself. Here are some articles about interfaces and their use-cases https://www.digitalocean.com/community/tutorials/how-to-use-interfaces-in-typescript https://www.softwaretestinghelp.com/typescript-interface/ And since you are working with Glue, you may find my article series helpful. It may be a bit outdated but I tried to put some developer practices into glue development. https://dev.to/1oglop1/aws-glue-first-experience-part-4-deployment-packaging-5708 PS. You can write pulumi in Python as well. In that case, you can think of an
interface
as a
typed Dict
or a
dataclass
. Instantiating a class in python is not as elegant as creating an object in JS/TS. Object in JS/TS is more close to Dict in python however, pulumi does not support type discs (yet, there's an issue about it's support). Also note, pulumi has to bend some good practices to make things work, so it may not be obvious why they do things the way they do. I'd suggest reading a smaller project like this https://github.com/pulumi/pulumi-synced-folder/blob/main/provider/cmd/pulumi-resource-synced-folder/s3-bucket-folder.ts#L29 to learn more about how to write pulumi.
m

millions-pharmacist-626

04/05/2023, 11:47 AM
Thanks a lot for the detailed explanation. Yeah there's a lot to know about in DE, it can be daunting at times. No offense taken, I know what my limits are which is why I try to broaden my knowledge as much as possible by not sticking to a single niche of work. Regarding interfaces, I do use them profusely when I am working with TypeScript, as I also work with the company's frontend team, but Python is my language of choice so I'm using that to write pulumi stuff. I guess this is probably the source of confusion, as you said it's not as elegant as when using TS. I will read your articles, thanks again
Yeah to be fair after checking the TypeScript tab in the pulumi docs I understand exactly where my confusion came from. This is much more clear than its python counterpart.
l

limited-rainbow-51650

04/05/2023, 11:55 AM
One of the reasons to have an object with it's own properties as
args
, followed by
opts
, is that we can make this 3 piece setup (
name
,
args
,
opts
) to create a resource consistent over the different programming languages. You will see this pattern consistently applied over our supported programming languages and providers.
m

millions-pharmacist-626

04/05/2023, 11:56 AM
Yup, I noticed right after checking the other languages tabs
g

great-sunset-355

04/05/2023, 11:56 AM
Python contain loads of overloads https://github.com/pulumi/pulumi-synced-folder/blob/main/sdk/python/pulumi_synced_folder/azure_blob_folder.py#L131 So you can do
Job("name", prop=val, prop2=val)
or
Job("name", args=JobArgs(prop=val, prop2=val), opts=ResourceOptions())
- In the beginning I noticed that IDEs did not really work well with the former
l

limited-rainbow-51650

04/05/2023, 11:57 AM
In Python, each resource has 2 constructors. You have the 3-piece constructor, which allows for type validation, but we also generate a (more Pythonic)
kwargs
version of the constructor. Given we flatten this via
kwargs
, it is no longer possible to perform a certain level of type checking, delaying some validation until runtime. It is up to you what you use, but we propose to use the less Pythonic 3-piece constructor if you want a faster development roundtrip.
m

millions-pharmacist-626

04/05/2023, 12:01 PM
Thanks for the explanation @limited-rainbow-51650. Seems to me that even the kwargs approach is benefitting from type checking tho? See screen:
l

limited-rainbow-51650

04/05/2023, 12:04 PM
OK, I said
a certain level of type checking
, while I should have said:
a certain level of input validation
. The
kwargs
version of the constructor doesn't support validating if all required properties are passed, let alone nested properties.
m

millions-pharmacist-626

04/05/2023, 12:07 PM
Got it, thanks for the clarification 🙂