:nerd_face: In the last 5 years there has widespre...
# general
a
🤓 In the last 5 years there has widespread shift in data and analytics platforms towards user independence, away from a centralized data team that delivers analytics to order. Data is heading towards a real platform where teams exchange data with each other without going through a middleman. Probably sounds familiar to folks 'round here 🤗 Exchanging data requires deploying a lot of infrastructure: IAM, network connectivity, storage, compute, streams, metastore resources, ... = a lot of repeating infrastructure, from a lot of different contributors. Abstraction of repeating parts and declarative provisioning fits like a glove. I've had a lot of success with self service data platform configuration on git, and Terraform deployment, but have struggled to reduce the turnaround time for user errors, and to avoid a beast of an integration pipeline for the platform team to manage. pulumipus chill hoodieFeeling like there has to be a better way, I found my way to Pulumi and Resource Components this year (thanks to @calm-horse-33012). I'm trying to design and implement a minimal data platform with Pulumi (attached image), and was wondering if anyone has any experience or interest. You can read a bit more about and follow my adventures 🍞 on my blog. Let's connect if you would like to compare notes!