If I stuff a random guid in there, the issue goes away. Obviously a random guid won't work but for debugging purposes. It's like it doesn't see that virtualmachinescaleset.identity is a Output.

If virtualmachinescaleset.identity is an Output<VirtualMachineScaleSetIdentityResponse> do I have to do something special to get at principalId? I'm thinking that I shouldn't.

Using a dummy value worked 😮

Which has the same workaround as you found.