You definitely need an S3 VPC Endpoint if you want communications to go from the VPC directly to S3. And yes it is much faster than going out to the internet.
As for pulumi automatically making it...I would assume no.
Basically if you connect lambda to your VPC and don't have the S3 endpoint, it will just route out through the default gateway to the internet.