No matter how you like to participate in developer communities, Pulumi wants to meet you there. If you want to meet other Pulumi users to share use-cases and best practices, contribute code or documentation, see us at an event, or just tell a story about something cool you did with Pulumi, you are part of our community.

Pulumi Community

In most cases, the answer is: you want different projects.

That said, if the constituents of the image haven't changed, the build shouldn't change so the task shouldn't get deployed. If the constituents haven't changed but you're still seeing unnecessary task deployments, you should review that part of the process.

yea different projects is where my mind was headed but I was hoping there would be a way to avoid that

No, and I think that's intentional. If two chunks of IaS can be deployed separately, then two projects is the intended way to represent that.