Unfortunately, I did not found a fix yet. It seems to be related to the provider making the push before the image is completed building locally, but the staff could not reproduce the issue yet.
For now, I am surviving on v3. I don't even mind the slower building time, but having the image being recreated on every pulumi preview is a productivity killer.
If you can, I suggest you to comment on the issue on github with information related to your scenario, so maybe with more information someone can pinpoint where the issue lies and fix it.