for instance using a mongodb instance to store pulumi.yaml and stacks - would my program need to pull these and push them to disk first before the automation would do its thing?

Yes they would need to be in a working directory for the automation api to operate. I’m using an S3 backend for state and migrating all my existing pulumi to inline functions so that I can ditch the project and stack YAML files. The automation api will still create them in the temp directory that it creates but my application will no longer depend on them.