This is Docker responsibility I think unless Pulumi is forcing a rebuild, perhaps you do a COPY ./ in the same directory you are make changes to your Pulumi. If docker build takes time then Pulumi will take time. If Docker build is smart enough to detect nothing has changed then I presume Pulumi will gain this benefit too