https://pulumi.com logo
Powered by
#general
Title
# general
m

magnificent-lifeguard-15082

01/16/2022, 8:40 PM
It appears that if an absolute path is passed to 
FileArchive
then pulumi will have a hard time on another machine and will fail to update 😬 Is this expected? If so might be worth adding a warning or explicit mention of relative path requirement on the docs. I do wonder how we should handle non-deterministic paths though between builds? I have a "file not found" error where a 
BucketObject
is referencing a 
.zip
with an absolute path which was created running 
up
on another machine.
Kind of feels like the local path is insignificant to the actual resource representation. It's the resolved file (and it's data) that is important.
l

little-cartoon-10569

01/16/2022, 8:42 PM
That sounds like a normal programming issue. Exception handling and testing are the way to go, here.
m

magnificent-lifeguard-15082

01/16/2022, 8:47 PM
Yes.
l

little-cartoon-10569

01/16/2022, 8:48 PM
There isn't a relative path requirement, so it can't be added to the docs. The relative path thing is a requirement only when running on multiple machines with different directory hierarchies. Frequently, this isn't the case.
Absolute paths would be fine if deploying only from CI, or only within a containerized environment, or similar.
m

magnificent-lifeguard-15082

01/16/2022, 8:49 PM
It's not my code throwing the exception though. Pulumi is storing in state (validated via 
pulumi stack export
) an absolute path that is not valid across environments.
Your last statement is more true but not entirely. If I change the working directory configuration of my CI container tomorrow paths will be invalidated.
l

little-cartoon-10569

01/16/2022, 8:50 PM
So the exception handling would have to wrap the execution of the Pulumi engine, which is inconvenient but not difficult.
If you change the working directory configuration of your CI container, that's equivalent to changing the code. You need to test that.
m

magnificent-lifeguard-15082

01/16/2022, 8:55 PM
Err, not really agreeing with your sentiment. Of course we should "handle errors that could occur in your code". However, I believe this error should be handled by pulumi. In my configuration it is correctly pointing to the right file path that exists on the machine pulumi is running on. Pulumi is trying to do something with the old state and failing - which I'm pretty sure is the biggest jobs of pulumi (ie. to detect configuration drift so that it can perform the right CRUD actions). It's seeing "oh this FileArchive doesn't exist anymore" but I've got a new one in it's place and do not have control over what pulumi is doing in the background when it realises an old FileArchive resource is orphaned.
l

little-cartoon-10569

01/16/2022, 8:57 PM
If Pulumi did that, how would you handle the case where you require the file to be at a specific, absolute location? And where if it path changes, then deployment should correctly fail?
m

magnificent-lifeguard-15082

01/16/2022, 8:57 PM
The problem is apparent when you consider the 
Asset
resources are the only resources that do not have a stable name you can provide. It is expecting to use the file path as a stable reference for the "virtual resource" AND as a pointer to "physical resource".
l

little-cartoon-10569

01/16/2022, 9:04 PM
That does complicate matters...
In cases where this is likely to become a problem, it might be worth wrapping the Asset in a ComponentResource with the same name as the Asset's constructor. You'd achieve the same protection, but with annoying boilerplate code, and extra resources which you might have to pay for if you're using a paid plan.
Having separate name and path properties would be better. It's probably worth opening an issue explaining this.
m

magnificent-lifeguard-15082

01/16/2022, 9:08 PM
Yeah defo will do. Thanks for helping me validate the issue
for reference and anyone else exploring this issue: this is the offending BucketObject state that exists in the stack which it's trying to do something with even though it sees the path has changed. It may actually be isolated to just s3.BucketObject so will explore a bit deeper. (my config is simply {source: FileArchive(
/abs/path.zip
)
Copy code
"source": {
                        "4dabf18193072939515e22adb298388d": "0def7320c3a5731c473e5ecbe6d01bc7",
                        "hash": "63d94894644f928fdc44fc353c161ca544f477d28880d07c2f8634322ce3b1d6",
                        "path": "/Users/path/to/repo/modules.zip"
                    }
e

echoing-dinner-19531

01/17/2022, 9:24 AM
I think I understand your problem now. You had old state that refered to one file, and now you've moved to a new machine and that file doesn't exist anymore and you've changed your program to use a new file path. But when pulumi runs it's still trying to actually read the file at the old path and causing a file not found error? Could you ping us the output from up so we could see where exactly this error is showing up, a quick look at the engine code suggests we should be handling the case of the old file going missing without issue.
I assume this is https://github.com/pulumi/pulumi-aws/issues/1767, thanks for the issue.
✅ 1
m

magnificent-lifeguard-15082

01/18/2022, 10:40 AM
That's the one. The more I'm thinking about it the more likely it might be tied in with my other linked issue related to 
source: FileArchive
and 
sourceHash
Let me know if the output would still be useful (I imagine you'd want the result of -v ? with --logtostderr)
e

echoing-dinner-19531

01/18/2022, 2:44 PM
I don't think its needed but we'll ping on the ticket if that changes.
3 Views
Powered by