Hey guys. So one of the challenges of centralised reuse is the issue of changes breaking downstream consumers. People solved this by adding versioning which allows consumers to pin to a version, and update in their own time instead of breaking instantly if the upstream changes something.
However a challenge that still remains unsolved as far as I can see, is tracking consumers. In a more anonymous decoupled ecosystem like npm packages, the package maintainer doesn't need to know what or who uses their package.
Pulumi has introduced outputs, and allows a pulumi project to utilise the outputs of another project - which is very useful, and addresses a big problem well.
However in the context of a single company or team's infrastructure, I do kind of need to know which parts of my infra consume an output I maintain.
Short of manually maintaining a list of everywhere that I use a certain output, which suffers from all the problems of any manually updated list, or using some kind of automated scanning tool to search across multiple repos or project folders, there is no good automated way I know of.
However I see an opportunity here - it would only work within the Pulumi ecosystem, but I can live with that. My idea is to add some kind of 'phone-home' hook or functionality into Pulumi such that when I consume an output from another project, Pulumi either detects this and updates a central storage somewhere, or I can add a tag or line in the code saying "update the source" and Pulumi will see this and do so.
For example
Project A creates a service which has an endpoint, OutputA
Project B uses the endpoint. So does Project C and let's just say Project D.
I could go to Project A and add comments above OutputA every time I utilise it, mentioning "Hey watch out, if you change this, it affects Projects A, B, C"
This is prone to human error even if it's just me maintaining all of them, let alone if we have a wider team making updates to all the projects.
If I could add a line in Project B in the resource which uses OutputA saying "updateSource: True" and Pulumi sees this and updates my resource graph centrally in my cloud account somehow (sorry, I'll leave the magic to you guys) this would be really super useful.
If I could get a warning on Pulumi Preview saying 'you've changed outputA which projects A,B,C rely on" that would be incredibly useful.
Don't know how feasible this is but just throwing it out there.
Thread in Slack Conversation