When running pulumi in a dockerized CI environment...
# general
When running pulumi in a dockerized CI environment like CodeFresh, we’re getting:
Copy code
Previewing update (LinioIT/support):                                                                                              
[resource plugin aws-1.2.1] installing                                                                                            
Downloading plugin: 60.87 MiB / 60.87 MiB  100.00% 4sm1s                                                                          
Moving plugin... done.                                                                                                            
    pulumi:pulumi:Stack rsin-provider-support running                                                                             
    pulumi:providers:aws default_1_4_0  error: no resource plugin 'aws-v1.4.0' found in the workspace or on your $PATH, install th
e plugin using `pulumi plugin install resource aws v1.4.0`                                                                        
    pulumi:pulumi:Stack rsin-provider-support                                                                                     
    pulumi:providers:aws default_1_4_0  1 error                                                                                   
  pulumi:providers:aws (default_1_4_0):                                                                                           
    error: no resource plugin 'aws-v1.4.0' found in the workspace or on your $PATH, install the plugin using `pulumi plugin instal
l resource aws v1.4.0`
Any idea? It seems like pulumi is installing the wrong version, but it is detecting it needs to install the plugin at minimum.
My guess it is sees the provider used was 1.2.1, but 1.4.0 is what is in yarn.lock, and it isn’t installing that plugin version
@gentle-diamond-70147 ping on this if you have any ideas.
Can you elaborate on the 1.2.1 <-> 1.4.0 point... You have 1.4.0 in your yarn.lock? Where is the 1.2.1 coming from?
I’m guessing 1.2.1 is coming from the stack
We updated to 1.4.0 in package.json
I’m confirming 1.4.0 is in yarn.lock right now
And yes, 1.4.0 is in yarn.lock
If you export your stack - is
referenced? If so where?
Getting that for you now
Copy code
                "urn": "urn:pulumi:development::rsin-provider::pulumi:providers:aws::default_1_2_1",
                "custom": true,
                "id": "869d7928-4dc2-486e-a517-6641d3dcfece",
                "type": "pulumi:providers:aws",
                "inputs": {
                    "region": "us-east-1",
                    "version": "1.2.1"
                "outputs": {
                    "region": "us-east-1",
                    "version": "1.2.1"
All the resources are referencing that provider
My guess is this: Pulumi detects it needs to reinstall the plugin. It checks the stack for the version. It installs that version. Preview/Up try to use the updated version installed from the
. Pulumi does not try to reinstall the plugin for this version.
Ahh - I see what's going on here. I assume you are not running
npm install
in your CI? The plugins needed by your NPM modules are installed at
npm install
time. You need to do at least one of: 1. Run
npm install
2. Run
pulumi plugin install
for the same version manually 3. Copy the plugins in question manually to the path (possibly from a previous run of one of the above) Plugins needed by your state file will be downloaded on demand during
if needed.
I am, but it’s in a previous step.
Therefore ~/.pulumi is not copied to the next step
You do need to ensure your
is maintained - or else re-do the plugin installations.
If you see the issue mentioned, you said that shouldn’t be the case anymore.
The reinstallation happens, but it’s the wrong versions.
What really needs to happen is pulumi needs that env var as discussed there.
I know Fernando made the decision to simply remove the node_modules and reinstall in each step, but this is an issue for anyone who uses a CI system like CodeFresh where the entire filesystem is not kept the same between steps.
I might have been mistaken in the issue - in the case where there are different versions in the checkpoint than in package.json - I think it's expected that you would get only the version in the state file. As you say - the "fix" for this is to make it possible to redirect where
is stored. But until then - you do need to make sure plugins are installed across
npm install
pulumi up
but this is an issue for anyone who uses a CI system like CodeFresh where the entire filesystem is not kept the same between steps.
Yep - definitely understood.
What’s the thought on timeframe for this fix? I really would rather not have to reinstall the node_modules multiple times. Adding 30 seconds or so per process sucks.
PRs definitely welcome! (adding an env var to manage this shouldn't be too deep a change I expect) But also hope we can have someone from the team look at this this milestone.
I’ll stick to the workaround for the immediate time because I dislike working with Go. Thanks for adding it to the milestone.
@cool-egg-852 We have a PR out for an env var to control where
is stored. Feedback on the PR welcome to ensure it would work for your environment. https://github.com/pulumi/pulumi/pull/3300