Ok, now I'm trying to figure out how to configure the autoscale settings/rules for an AppServicePlan. Can't find any docs or samples on how to do that,

First, do you know if you're using the native provider (https://www.pulumi.com/registry/packages/azure-native/api-docs/web/appserviceplan/) or the classic provider (https://www.pulumi.com/registry/packages/azure/api-docs/appservice/appservice/) --- I'm guessing you're on native, but I want to be sure before I point you down the wrong path. Just in case it's helpful, the native provider is built directly from Azure's resource model, so I generally check Azure's specs, like this one, when I'm stuck.