I ran into an issue after upgrading Pulumi to the latest version. The problem occurred during a CI build: after a refresh and an update, a resource was incorrectly updated. The smallest stack that I have replicated the issue in consists of a single gcp.storage.BucketObject resource using a FileArchive asset. The end result is that the BucketObject is replaced with an empty zip file instead of what the FileArchive asset should be.
I wanted to share my findings and the resolution I came up with in case someone else encounters this issue.
I discovered that the gcp.storage.BucketObject resource is handled using the Pulumi Terraform bridge. An important detail is that this bridge internally materializes archives as files and keeps them around at /tmp/pulumi-asset-<hash>.
Here's what I found: when a pulumi refresh is done (with no /tmp/pulumi-asset-<hash> files present), this results in the asset files being generated as an empty zip file. Subsequently, when the pulumi update is done, it doesn't rewrite the asset file. As a result, it detects changes and overwrites the bucket object with the empty zip file asset.
To work around this issue, after the pulumi refresh, I remove all /tmp/pulumi* files.
I've found that I can reliably recreate these empty zip assets by ensuring there are no /tmp/pulumi-asset-* files and performing a pulumi refresh on a stack that has been updated previously.
I'm not sure when the behavior changed that would cause an empty zip file to be generated during a refresh, and I'm also unsure whether it's an issue with Pulumi or the Pulumi Terraform bridge.
11/22/2023, 10:55 AM
Can you confirm this happens with the latest version of the gcp provider please (v7.2.0).
Reporting an issue here is best, and should be triaged fairly quickly as it can result in production data vanishing: https://github.com/pulumi/pulumi-gcp/issues
A code example will massively help