Or something else? Apologies for the dumb question.

Unfortunately APIGateway Websocket APIs are not supported yet. Pulumi exposes the upstream Terraform Resource Provider resources, and this has been pending in the Terraform AWS resource provider for a little over a year 😞. https://github.com/terraform-providers/terraform-provider-aws/issues/7004

Oh, bum. At least I feel a little less stupid. What would be the quickest way for me, in the future, to determine whether a service I want to use is or isn't supported by Pulumi?

In general -the answer is to look at the docs: https://www.pulumi.com/docs/reference/pkg/nodejs/pulumi/aws/ In this particular case, the confusing design of API Gateway (v1 vs. v2 instead of actual named things) in AWS makes it a little less clear what you should be looking for unfortunately.