https://pulumi.com logo
Join Slack
Powered by
I'm trying to piece together the steps needed to a...
# general
l
I'm trying to piece together the steps needed to author a component resource in typescript, so that its usable by other langages. I have figured out to do it locally, but I keep coming back to this: "Build the resource provider plugin: the binary file that contains all of the components or resources you defined in your source code". I cannot find any description about how to actually do this. Anyone had any luck? The pulumi docs are woefully spotty in this area. I'm going in circles 😐
There is a https://github.com/pulumi/pulumi-component-provider-ts-boilerplate but theres very little documentation, and it refers to a section in docs (https://www.pulumi.com/docs/iac/guides/building-extending/components/build-a-component/) which is very light on detail, its more like a concept guide. So if anyone have found a proper step by step guide I'd greatly appreciate it!
Hm I can see that the ts-boilerplate repo was recently slimmed down a lot, and the PR for that commit says: "Our current guidance to users who want to create reusable components in python is that they should use source distribution and local packages (by using 
pulumi package add
), rather than trying to build a binary plugin and publishing SDKs to NPM.". I have not seen anything of that guidance. In the guide linked by the README at the bottom it actually says:
Publishing a component as a Pulumi package makes it easier to distribute and integrate into Pulumi workflows. This method enables community contributions and ensures that infrastructure components remain modular and maintainable.
That sounds like the opposite of recommending users not to do it, so its mega-confusing
s
You don't need the boilerplate repos anymore. We should look at deprecating them.
l
yeah, that's the problem I think there so much content that I get the feeling might be out of date, it feels very conflicting. Thanks for the url, tho it doesnt mention how to package a a component resource as a package, it just links to Publishing packages
s
I'm working on an update to the Packages page in the docs that should explain this much better, but if you have 
PulumiPlugin.yaml
, that's basically the magic that lets us know it's a component, and we can internally auto-generate the schema and the consumer's SDK.
l
my goal is to host the built binary (?) on a company internal http server, and generate .net + typescript sdks for it. Having dug into this now for 3-4 hours I'm pretty sure it's not documented anywhere how to do that.
s
You don't have to do that anymore. All you have to do is commit it to git, optionally tag the repo with a semantic version e.g. 
v1.2.3
, optionally run 
pulumi package publish
if you are a Pulumi Cloud customer and want the component to appear in Pulumi IDP.
You reference the component with 
pulumi package add
and then the git source. When the component is pulled down, we auto-generate the schema and SDK so the component can be consumed in any language.
(The generated SDK is in the language of the consuming program.)
If you want to, you can do a more heavyweight provider, with a schema file and pre-generated, hosted SDKs, but that's overkill for most use cases.
It's not you, btw - the docs need updating on this subject to ensure we clearly explain the options. We're actively working on it.
l
thanks for the info, much appreciated. There are a few "problems" with using the git approach, especially around auto-picking versions. By using the language's native package provider (npm, poetry, nuget) there's at least some logic for ensuring references stay up to date. If we just tell users to point at a certain version its a lot more manual. So I prefer the approach of publishing language-native sdks and then the actual "binary" (elusive since I don't yet know how it's generated) to hold the actual logic
package managers are able to determine prerelease from release versioning (assuming semver) etc, all things that are important for an enterprise. So I think the guidance of publishing via git fails to take into consideration the nuances around versioning of these components
You can also target a standard internally hosted git service, just by providing the repo URL without the 
<release-version>
portion.
there's very little info here about how version updates would be handled in real-life. Our devs don't have time to manually go thru their referenced packages and check if there might be an update available
s
pulumi package add
automatically selects the most recent semver tag.
It's based off of Go modules, which use the same mechanism for versioning. Obviously, Go has pretty widespread adoption, so it's at least a viable system (if not ideal).
If you want pre-packaged SDKs, then I recommend you write your component in Go and use https://github.com/pulumi/pulumi-go-provider as it has the best and best-supported tooling.
Go also, of course, is distributed as a proper binary, which means your consumers do not need the language runtime of the upstream component.
So it's easier from an operational perspective.
l
which means your consumers do not need the language runtime of the upstream component
yeah, this is what I was looking for more information on. How does the "consumer" actually execute the a component resource written in another language - tied to "how to package it". Again, references to this elusive "binary" 🙂. Very much looking forward to reading a revamped Packages docs page.
> It's based off of Go modules, which use the same mechanism for versioning. this is also very valuable info and not stated anywhere I've seen in the docs. I think a walk-thru of something as simple as bumping the version of an internal component resource would be extremely helpful
btw we're a polyglot shop with python, c#, typescript used internally, so adding Go to the mix is something we do not take lightly. It's tricky enough to manage 3 completely different languages
s
Very much looking forward to reading a revamped Packages docs page.
Working on it today. The Packages page revision will mostly explain it conceptually: how the provider "binary", schema, and SDK work, and where you use locally generated vs pre-built SDKs.
How does the "consumer" actually execute the a component resource written in another language - tied to "how to package it".
The short answer is, whether the SDK is pre-generated or locally generated, you must have the "runtime" (Go has none, which is why it's the recommended choice, absent any governance questions for pre-built SDKs) of the upstream component on your system.
Again, references to this elusive "binary" 🙂. Very much looking forward to reading a revamped Packages docs page.
The "binary" is basically the guts of the package where you define the component.
The "binary" part should be clearly defined. For pre-generated SDKs, there's an init script that installs the binary. You can see this when you do 
npm i @pulumi/aws
for a new version of the AWS provider - it downloads the provider binary (in this case, an actual binary file, which is cached in 
~/.pulumi/something-or-other
The init script (pretty sure) is part of the 
pulumi package gen-sdk
command's output.
l
thanks! I've dug into the pulumi codebase to see how it behaves when doing pulumi package add <git source> in the meantime. As far as I understand it has some similarities to golang resolves packages, but the behavior is different. For instance, golang calls out to "git" which means that any git config get taken into account, while pulumi does not - it uses a go-based git library which doesn't support the full git config. This caused problems for me, and the error messages it produces are less than ideal. I'd be tempted to create a PR to allow setting a "PULUMI_USE_NATIVE_GIT" to make it fall back to regular git behavior, that would help us a lot if we decide to use this
The init script (pretty sure) is part of the 
pulumi package gen-sdk
command's output.
I only get:
Copy code
❯ pulumi package gen-sdk .
SDKs have been written to ./sdk
The "binary" part should be clearly defined
I'm not sure if you mean "defined in docs" or how but it would be fantastic if someone at pulumi had time to step thru this from the perspective of a user trying to create and package a component resource library in a language besides golang from start to finish - I'm sure a lot of rough edges would be encountered that could result in more complete docs coverage
💯 1
from what I can see right now, the "install from git" with "pulumi package add" is not mature enough for us to be able to use it
btw one thing that struct me is that is not just "package add", its also " how do I stay up to date with packes". If pulumi's recommendation is to install from git, then a pseudo-package manager is needed to manage package versions is needed too. It would be good to see some real-life guidance around this.
1062 Views
Open in Slack
PreviousNext
Powered by