Hey guys - I am having trouble using Pulumi Packages with a private repository.. Can anyone please help? I am building the package successfully and I have the binary + package in Github (using Github releases), and I am able to do npm install locally and fetch the package as well. The problem is when I run ‘pulumi up’ - I get the following error:

warning: resource plugin xyz is expected to have version >=0.0.23, but has 0.0.1; the wrong version may be on your path, or this may be a bug in the plugin

My suspect is that it is unable to install the pulumi plugin, and that my

pluginDownloadURL

inside my schema.yaml is wrong - the issue is that I can’t find any examples online on how to do it properly... my schema.yaml file is as follows:

# yaml-language-server: $schema=<https://raw.githubusercontent.com/pulumi/pulumi/master/pkg/codegen/schema/pulumi.json>
---
name: xyz
pluginDownloadURL: <github://api.github.com/amitOrganization/pulumi-amit-packages>
resources:
  xyz:index:StaticPage:
    isComponent: true
    inputProperties:
      indexContent:
        type: string
        description: The HTML content for index.html.
    requiredInputs:
      - indexContent
    properties:
      bucket:
        "$ref": "/aws/v4.0.0/schema.json#/resources/aws:s3%2Fbucket:Bucket"
        description: The bucket resource.
      websiteUrl:
        type: string
        description: The website URL.
    required:
      - bucket
      - websiteUrl
language:
  go:
    generateResourceContainerTypes: true
    importBasePath: <http://github.com/pulumi/pulumi-xyz/sdk/go/xyz|github.com/pulumi/pulumi-xyz/sdk/go/xyz>
  nodejs:
    packageName: "@amitOrganization/xyz"
    dependencies:
      "@pulumi/aws": "^4.0.0"
    devDependencies:
      typescript: "^3.7.0"

You probably have a copy of the provider binary on PATH (likely in go/bin), the engine always prefers to use the binary on PATH vs what's in the plugin cache. You can turn this off with the envvar PULUMI_IGNORE_AMBIENT_PLUGINS.

@echoing-dinner-19531 Thanks for the quick answer! How can I set this env variable? Can’t find in the documentations anything about it..

PULUMI_IGNORE_AMBIENT_PLUGINS=true

?

Yeh it's not documented anywhere, but "true" or "1" will enable it.

@echoing-dinner-19531 So looks like I get a different error message which is good, but still it fails:

error: Could not automatically download and install resource plugin 'pulumi-resource-xyz' at version v0.0.23, install the plugin using `pulumi plugin install resource xyz v0.0.23`.
Underlying error: error downloading plugin xyz to file: failed to download plugin: xyz-0.0.23: 403 HTTP error fetching plugin from <https://get.pulumi.com/releases/plugins/pulumi-resource-xyz-v0.0.23-darwin-arm64.tar.gz>

Maybe my

pluginDownloadURL

is still wrong? its value is:

<github://api.github.com/amitOrganization/pulumi-rise-packages>

Have you got an envvar GITHUB_TOKEN set with a github pac token set? Plugin downloads always fall back to get.pulumi.com (I should probably fix that), but it probably did the fallback because the github download failed. If you turn on verbose logs (-v11 --logtostderr) I think it should print the github requests

@echoing-dinner-19531 Yup I got

GITHUB_TOKEN=ghp_...

set (I set it in my terminal before running pulumi up) Looking at the logs I found kind of the same error message:

I0206 20:37:41.095459   48664 plugins.go:788] full plugin download url: <https://api.github.com/repos/pulumi/pulumi-xyz/releases/tags/v0.0.23>
Underlying error: error downloading plugin xyz to file: failed to download plugin: xyz-0.0.23: 403 HTTP error fetching plugin from <https://get.pulumi.com/releases/plugins/pulumi-resource-xyz-v0.0.23-darwin-arm64.tar.gz>\u003c{%reset%}\u003e\n","color":"raw","severity":"error"}},{"sequence":4,"timestamp":1675708662,"summaryEvent":{"maybeCorrupt":false,"durationSeconds":2,"resourceChanges":{"create":1},"PolicyPacks":{}}}]}

The part

pulumi-resource-xyz-v0.0.23-darwin-arm64.tar.gz

looks correct, even though entering: https://api.github.com/repos/pulumi/pulumi-xyz/releases/tags/v0.0.23 In my browser or doing cURL - I get not found error

Have you got a github release for that version? e.g. if you look at one of our providers we have a GH release with all the .tar.gzs for the different platforms in a release notice: https://github.com/pulumi/pulumi-aws/releases/tag/v5.29.1 That's what the downloader is looking for

@echoing-dinner-19531 Yes - thats the release, maybe I did it incorrectly?

No that looks ok. I think the download url is wrong, because it hit:

<https://api.github.com/repos/pulumi/pulumi-xyz/releases/tags/v0.0.23>

That's our org, it should of hit your org...

@echoing-dinner-19531 Nope there is only one log containing full plugin download url:

I0206 20:53:03.655370   49041 plugins.go:788] full plugin download url: <https://api.github.com/repos/pulumi/pulumi-xyz/releases/tags/v0.0.23>

So in that case, where do I configure it? in the schema.yaml file inside my package? which field should I use?

The schema looks right, if you look at the SDKs generated they should have some utility files with the url in it as well, can you check those are pointing at your org?

@echoing-dinner-19531 I see a

utilities.ts

file with this part:

/** @internal */
export function resourceOptsDefaults(): any {
    return { version: getVersion(), pluginDownloadURL: "<github://api.github.com/@amitOrganization/pulumi-rise-packages>" };
}

@amitOrganization

looks odd? where did the

@

come from?

@echoing-dinner-19531 I’ve got it in the shcema.yaml file:

packageName: "@amitOrganization/xyz"

and in scripts/install-pulumi-plugin.js:

var res = childProcess.spawnSync("pulumi", ["plugin", "install", "--server", "<github://api.github.com/@amitOrganization[/pulumi-rise-packages]>"].concat(args), {
    stdio: ["ignore", "inherit", "inherit"]
});

Is your github org actually named

@amitOrganization

I've never seen a github org with an at symbol in the name?

@echoing-dinner-19531 It has a different name but don’t know if I can share it, doesn’t start with ‘@’ though

Ah there's your error!

@echoing-dinner-19531 I’ve added it in multiple places inside my action \ makefile, still the same error... I see that it is run from the yarn install command

"install": "node scripts/install-pulumi-plugin.js resource xyz "

and inside install-pulumi-plugin there is this part:

var res = childProcess.spawnSync("pulumi", ["plugin", "install", "--server", "<github://api.github.com/amitOrganization[/pulumi-rise-packages]>"].concat(args), {
    stdio: ["ignore", "inherit", "inherit"]
});
.
.
} else if (res.error || res.status !== 0) {
    console.error("\nThere was an error installing the resource provider plugin. " +
            "You may try to manually installing the plugin by running " +
            "`pulumi plugin install " + args.join(" ") + "`");
}

which the error comes from maybe the url is still incorrect? should it be with the square brackets? (e.g

[/pulumi-rise-packages]

)

It shouldn't have the square brackets...

@echoing-dinner-19531 I see - its just that the documentation is kinda confusing regarding the square brackets😅 my

pluginDownloadUrl

is:

pluginDownloadURL: <github://api.github.com/amitOrganization/pulumi-rise-packages>

So I removed the square brackets from the nstall-pulumi-plugin and it still didn’t help, same error from the CICD ..

Ah that's meant to be that the repository name is optional!

Interesting, I will try without!

If you just have "github://api.github.com/amitOrganization" the engine will look at your package name for the repo name and if your package is "rise-packages" then it will result in the same URL of "github://api.github.com/amitOrganization/pulumi-rise-packages"

@echoing-dinner-19531 Sorry for the long response didn’t have time to keep working on it just re-ran everything, and stumbled upon a new error on the Build sdk step for nodejs 😞 So I started the repo again from scratch, changed the bare minimum and it looks like the error is happening again - which makes me think this has something to do with the pulumi converter \ codegen? two lines for copy-paste purposes:

Error: node_modules/@types/node/ts4.8/util.d.ts(1485,42): error TS1005: ',' expected.
Error: node_modules/@types/node/ts4.8/util.d.ts(1485,44): error TS1068: Unexpected token. A constructor, method, accessor, or property was expected.

Full error:

Ah

@echoing-dinner-19531 Thank you so much!! Fixed it

Still around the pluginDownloadURL?

@echoing-dinner-19531 yeah basically.. even though I am trying different things, still trying to handle this error from the build sdk stage:

error: [resource plugin xyz-0.0.6] downloading from : failed to download plugin: xyz-0.0.6: 403 HTTP error fetching plugin from <https://get.pulumi.com/releases/plugins/pulumi-resource-xyz-v0.0.6-linux-amd64.tar.gz>
There was an error installing the resource provider plugin. You may try to manually installing the plugin by running `pulumi plugin install resource xyz v0.0.6`

Right, same as before but to summerize things to check: pluginDownloadURL in the schema that GITHUB_TOKEN is set to access the repo that there's a github release in the repo for 0.0.6 with the expected .tar.gz files in it

So I added these two fields:

publisher: amitOrganization
repository: my-repo

Which now gave me this error which is better, it’s looking at the right spot now:

error: [resource plugin xyz-0.0.7] downloading from <github://api.github.com/amitOrganization>: failed to download plugin: xyz-0.0.7: 404 HTTP error fetching plugin from <https://api.github.com/repos/amitOrganization/pulumi-xyz/releases/tags/v0.0.7>. If this is a private GitHub repository, try providing a token via the GITHUB_TOKEN environment variable. See: <https://github.com/settings/tokens>

Which makes me believe I need to pass GITHUB_TOKEN some place else (I am confident it has the right permissions) p.s -regarding the things you mentioned:

pluginDownloadURL: <github://api.github.com/amitOrganization>

plus the release does exists.

Not sure secrets.access_token_github is right? https://docs.github.com/en/actions/security-guides/automatic-token-authentication#using-the-github_token-in-a-workflow

@echoing-dinner-19531 from my understanding it’s the name of your github secret , can be anything, for example I have an organization secret named

USERS_S3_FILE_PATH

which is totally custom and works great

yeh I'm not a github actions expert, just the docs style is what I'm used to seeing so something to double check