https://pulumi.com logo
Join Slack
Powered by
When running pulumi in a dockerized CI environment...
# general
c
When running pulumi in a dockerized CI environment like CodeFresh, we’re getting:
Copy code
Previewing update (LinioIT/support):                                                                                              
[resource plugin aws-1.2.1] installing                                                                                            
Downloading plugin: 60.87 MiB / 60.87 MiB  100.00% 4sm1s                                                                          
Moving plugin... done.                                                                                                            
                                                                                                                                  
    pulumi:pulumi:Stack rsin-provider-support running                                                                             
    pulumi:providers:aws default_1_4_0  error: no resource plugin 'aws-v1.4.0' found in the workspace or on your $PATH, install th
e plugin using `pulumi plugin install resource aws v1.4.0`                                                                        
    pulumi:pulumi:Stack rsin-provider-support                                                                                     
    pulumi:providers:aws default_1_4_0  1 error                                                                                   
                                                                                                                                  
Diagnostics:                                                                                                                      
  pulumi:providers:aws (default_1_4_0):                                                                                           
    error: no resource plugin 'aws-v1.4.0' found in the workspace or on your $PATH, install the plugin using `pulumi plugin instal
l resource aws v1.4.0`
Any idea? It seems like pulumi is installing the wrong version, but it is detecting it needs to install the plugin at minimum.
My guess it is sees the provider used was 1.2.1, but 1.4.0 is what is in yarn.lock, and it isn’t installing that plugin version
@gentle-diamond-70147 ping on this if you have any ideas.
g
Can you elaborate on the 1.2.1 <-> 1.4.0 point... You have 1.4.0 in your yarn.lock? Where is the 1.2.1 coming from?
c
I’m guessing 1.2.1 is coming from the stack
We updated to 1.4.0 in package.json
I’m confirming 1.4.0 is in yarn.lock right now
And yes, 1.4.0 is in yarn.lock
w
If you export your stack - is 
1.2.1
referenced? If so where?
c
Getting that for you now
Copy code
{
                "urn": "urn:pulumi:development::rsin-provider::pulumi:providers:aws::default_1_2_1",
                "custom": true,
                "id": "869d7928-4dc2-486e-a517-6641d3dcfece",
                "type": "pulumi:providers:aws",
                "inputs": {
                    "region": "us-east-1",
                    "version": "1.2.1"
                },
                "outputs": {
                    "region": "us-east-1",
                    "version": "1.2.1"
                }
            },
All the resources are referencing that provider
My guess is this: Pulumi detects it needs to reinstall the plugin. It checks the stack for the version. It installs that version. Preview/Up try to use the updated version installed from the 
node_modules
. Pulumi does not try to reinstall the plugin for this version.
w
Ahh - I see what's going on here. I assume you are not running 
npm install
in your CI? The plugins needed by your NPM modules are installed at 
npm install
time. You need to do at least one of: 1. Run 
npm install
2. Run 
pulumi plugin install
for the same version manually 3. Copy the plugins in question manually to the path (possibly from a previous run of one of the above) Plugins needed by your state file will be downloaded on demand during 
up
if needed.
c
I am, but it’s in a previous step.
Therefore ~/.pulumi is not copied to the next step
w
You do need to ensure your 
~/.pulumi
is maintained - or else re-do the plugin installations.
c
If you see the issue mentioned, you said that shouldn’t be the case anymore.
The reinstallation happens, but it’s the wrong versions.
What really needs to happen is pulumi needs that env var as discussed there.
I know Fernando made the decision to simply remove the node_modules and reinstall in each step, but this is an issue for anyone who uses a CI system like CodeFresh where the entire filesystem is not kept the same between steps.
w
I might have been mistaken in the issue - in the case where there are different versions in the checkpoint than in package.json - I think it's expected that you would get only the version in the state file. As you say - the "fix" for this is to make it possible to redirect where 
.pulumi
is stored. But until then - you do need to make sure plugins are installed across 
npm install
and 
pulumi up
steps.
but this is an issue for anyone who uses a CI system like CodeFresh where the entire filesystem is not kept the same between steps.
Yep - definitely understood.
c
What’s the thought on timeframe for this fix? I really would rather not have to reinstall the node_modules multiple times. Adding 30 seconds or so per process sucks.
w
PRs definitely welcome! (adding an env var to manage this shouldn't be too deep a change I expect) But also hope we can have someone from the team look at this this milestone.
c
I’ll stick to the workaround for the immediate time because I dislike working with Go. Thanks for adding it to the milestone.
w
@cool-egg-852 We have a PR out for an env var to control where 
.pulumi
is stored. Feedback on the PR welcome to ensure it would work for your environment. https://github.com/pulumi/pulumi/pull/3300
Open in Slack
PreviousNext
Powered by