In Github Actions, when a Preview job passes, it leaves this type of comment on the pull request (first screenshot). But in the docs here https://www.pulumi.com/docs/using-pulumi/continuous-delivery/github-actions/ I can see a better-formatted comment (see my second screenshot). Is this only available with Pulumi Cloud account?

Yes for GHA without pulumi cloud you just get the non-interactive CLI output, but pulumi cloud integration gives you much richer output