So what I ended up doing:
Manually fork the repo (could have used a template instead)
Create a Private Key with the TLS module
Create a deploy key from the public key
Write the private key locally
Sync everything up and checkout the repo with ansible