Is there any documentation for configuring autoscaling for fargate services?

Appautoscaling is the one you want. There's an example of autoscaling an ecs service here: https://www.pulumi.com/docs/reference/pkg/aws/appautoscaling/target/#ecs-service-autoscaling

Thanks that looks like what I need 🙂