Currently I'm doing this, and feeling silly:

# Trigger the pulumi up => change the world.
      pulumi up --yes --message "$AUGMENTED_MESSAGE" ||
        # retry-as-a-service :P 
        pulumi up --yes --message "(1st retry) - $AUGMENTED_MESSAGE" ||
          # retry-as-a-service :P 
          pulumi up --yes --message "(2nd retry) - $AUGMENTED_MESSAGE" ||
            # retry-as-a-service :P 
            pulumi up --yes --message "(3rd and last retry) - $AUGMENTED_MESSAGE"

Wouldn't it be easier to handle the error and retry in code rather than retry on the command level?

There isn't a good way to retry pulumi operations in code, something we want to fix but it's tricky. Generally we say providers should have retry logic where it makes sense. @brash-alligator-49865 if there's one resource that's just being flaky worth pinging the repo for that resource provider about it with an issue, might be it ought to just retry in general.