https://pulumi.com logo
Powered by
#package-authoring
Title
# package-authoring
g

great-night-26645

11/07/2023, 12:05 PM
Hi, I'm trying to wrap a Terraform provider (https://github.com/poseidon/terraform-provider-ct) and use the resulting Python SDK to try out if it worked. The 
make build_sdks
succeeds (except for C# which I don't care about at the moment). Now I tried to reference the generated Python SDK from another repository based on its path in my filesystem. With Poetry:
Copy code
[tool.poetry.dependencies]
...
pulumi-butane = {path = "/home/dev/DEVOPS/pulumi-butane/sdk/python"}
...
With this configuration PyLance in VS code is able to resolve the Python code. When I run 
pulumi up
it fails with something like:
Copy code
error: Could not automatically download and install resource plugin 'pulumi-resource-butane' at version v0.0.0, install the plugin using `pulumi plugin install resource butane v0.0.0 ...
I followed the instructions and installed the plugin:
Copy code
❯ poetry run pulumi plugin install resource butane v0.0.0 --file /home/dev/DEVOPS/pulumi-butane/sdk/python
[resource plugin butane-0.0.0] installing
❯
But it still doesn't work:
Copy code
error: failed to load plugin /home/dev/.pulumi/plugins/resource-butane-v0.0.0/pulumi-resource-butane: loading PulumiPlugin.yaml: open /home/dev/.pulumi/plugins/resource-butane-v0.0.0/PulumiPlugin.yaml: no such file or directory
What am I doing wrong?
l

limited-rainbow-51650

11/07/2023, 3:04 PM
@great-night-26645 the install command is not needed. In development mode, it suffices that you have 
pulumi-resource-butane
somewhere in your 
PATH
.
b

big-architect-71258

11/07/2023, 3:08 PM
@limited-rainbow-51650 I wanted to write the same thing, but it does not explain the error. It should no longer occur after an install. Thinking about that the version doesn't match, because when you develop locally you get something like this 
0.0.1-alpha.1694635523+8f2e8c13.dirty
and not a clean 
v0.0.0
or something. However, it is of course sufficient to adjust the PATH variable.
Especially for Python the version gets written to 
bin/setup.py
which is used to setup the Python SDK.
So I suspect, when using the, not needed, approach via 
pulumi install
, the correct version that gets displayed during 
make build_python
or 
make provider
must be used to install the provider.
l

limited-rainbow-51650

11/07/2023, 3:18 PM
the 
install
copies the built provider binary. If you are iterating fast on the pulumi provider, using 
PATH
is what I rely on myself.
b

big-architect-71258

11/07/2023, 3:19 PM
I know, I know 🙂 You would never do that. But I'm wondering what the root cause of the error described really is.
g

great-night-26645

11/07/2023, 4:25 PM
Thanks for the quick answers! I will try my luck tomorrow and let you know how it goes!
I removed all installed plugins from the home folder and added the folder to my path like this:
Copy code
export PATH=$PATH:/home/dev/DEVOPS/pulumi-butane/provider/cmd/pulumi-resource-butane
It still looks for the v0.0.0 version though I have no clue where that is coming from. The failure is as follows:
Copy code
Diagnostics:
  pulumi:providers:butane (default_github_/api.github.com/.../pulumi-butane):
    error: Could not automatically download and install resource plugin 'pulumi-resource-butane' at version v0.0.0, install the plugin using `pulumi plugin install resource butane v0.0.0 --server <github://api.github.com/.../pulumi-butane>`: error downloading provider butane to file: failed to download plugin: butane-0.0.0: 404 HTTP error fetching plugin from <https://api.github.com/repos/.../pulumi-butane/releases/tags/v0.0.0>. If this is a private GitHub repository, try providing a token via the GITHUB_TOKEN environment variable. See: <https://github.com/settings/tokens>
b

big-architect-71258

11/08/2023, 3:20 PM
@great-night-26645 Is this an isolated error with Python i.e. do the other SDKs show the same error?
g

great-night-26645

11/09/2023, 8:33 AM
Good question. I'll try it out with TypeScript
I just saw that you (@limited-rainbow-51650) implemented the pulumi-matchbox provider that serves as an example for me. Too bad you didn't also implement a provider for the terraform-provider-ct that is from the same context. Its one of the last pieces that keeps me from switching from Terraform as there is no way to easily convert Butane to Ignition in Pulumi
l

limited-rainbow-51650

11/09/2023, 8:44 AM
@great-night-26645 indeed, but that is part of my personal homelab effort, not part of my job unfortunately. Now, looking at your 
PATH
export above, I think this isn't correct. If you started from the terraform-bridge boilerplate, the 
Makefile
should build your provider locally in the 
bin
folder. If you then use this:
Copy code
export PATH=$PATH:/home/dev/DEVOPS/pulumi-butane/bin
Pulumi should then pick up 
pulumi-resource-butane
from your 
bin
folder.
g

great-night-26645

11/09/2023, 8:48 AM
That's too bad but totally understandable
I was just surprised to find the Matchbox provider to be brigeded over but not the butane one
The updated export seems to have worked:
Copy code
warning: using pulumi-resource-butane from $PATH at /home/dev/DEVOPS/pulumi-butane/bin/pulumi-resource-butane
    warning: using pulumi-resource-butane from $PATH at /home/dev/DEVOPS/pulumi-butane/bin/pulumi-resource-butane

    /home/dev/DEVOPS/pulumi-butane/bin/pulumi-resource-butane: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /home/dev/DEVOPS/pulumi-butane/bin/pulumi-resource-butane)
    /home/dev/DEVOPS/pulumi-butane/bin/pulumi-resource-butane: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /home/dev/DEVOPS/pulumi-butane/bin/pulumi-resource-butane)
l

limited-rainbow-51650

11/09/2023, 8:49 AM
I did the Talos Linux one too, if that might interest you.
g

great-night-26645

11/09/2023, 8:50 AM
I haven't used that one before
We usually use butane configs to deploy Flatcar VMs to vSphere
That is our main usecase next to K8s deployments
l

limited-rainbow-51650

11/09/2023, 8:50 AM
g

great-night-26645

11/09/2023, 8:51 AM
But I don't see a way to do it in Pulumi without briding over the butane provider
b

big-architect-71258

11/09/2023, 8:51 AM
@great-night-26645 Is the issue with Python fixed as well? Or only for Typescript?
l

limited-rainbow-51650

11/09/2023, 8:52 AM
I was looking into Flatcar as well, before a colleague pointed me to Talos.
g

great-night-26645

11/09/2023, 8:52 AM
@big-architect-71258 I got further with Python though now it is complaining about glibc being outdated ... Typescript I didn't get enough along to tell.
@limited-rainbow-51650 For deploying VMs without having to resort to additional configuration steps with something like Ansible Flatcar is great. You can configure everything at runtime through Ignition configs
l

limited-rainbow-51650

11/09/2023, 8:54 AM
But I don't see a way to do it in Pulumi without bridging over the butane provider
For building/verifying proper Butane config, you don't necessarily need a Pulumi provider. Don't forget you use a normal programming language. E.g. if you use Python, you can mix in this library: https://pypi.org/project/bupy/
g

great-night-26645

11/09/2023, 8:55 AM
I had a look at that route first but I can't use 
bupy
because it does not support Butane merging (snippets), which is key for writing modular resources
The terraform provider I'm trying to bridge is quite small and does everything we need. Though I really struggle with the briding part of it
I guess you deploy Talos via Matchbox?
l

limited-rainbow-51650

11/09/2023, 8:57 AM
That is the idea, yes. But I only got my local netboot setup for my RPi CM4s last Sunday. Haven't been able to progress since then as I wasn't home for a few days.
@great-night-26645 coming back to the bridging part, given your plugin binary is now picked up from the path, does it unblock you on the bridging aspect?
g

great-night-26645

11/09/2023, 8:58 AM
We tried to use Matchbox for deploying K8s (via Typhoon) but found that it wasn't very stable and took quite a long time compared to spinning up Flatcar VMs from Terraform. That was straigtforward and took only a fraction of the time.
I first have to see if I can resolve the glibc thing before I really see if it works. I'll let you know if I make progress
Powered by