Pulumi team is working on supporting versioning wi...
# esc
p
Pulumi team is working on supporting versioning within ESC. At a high level, this is how it'd work 1. Every time you make a change and 'save' an environment, a new revision is created. Revisions are immutable 2. On the Version history page, you can add, edit, and remove tags to the environment revisions 3. You can import specific versions of the environment into other environments or Pulumi stacks by using tags or revision #. Here is the early wireframe of how we are thinking about seeing the history of revisions made to your environments and managing tags. Your feedback is much appreciated.
👍 1
🙌 3
m
Hey Arun! Eagerly looking forward to this feature in beta/alpha. Revision based UIs are really tricky... I think GitHub actually has a lot of amazing cues to take from. Suggestion 1: Revisions might be best served in the upper left, where they are intuitive to users (attached). For the page you are showing and requesting feedback on, I think something like the commits chart would be a great place to take cues from as well. The layout is already very similar (attached as 2nd image). Tags view being separate (attached 3) might also help. From a usability perspective, I think a diff would be by far one of your most requested features - just like they show on github for diff between two commits. In general, I highly recommend ya'll just use git under the hood for revision tracking. At least - the same mechanisms as git. Let me know if any of this feedback has been helpful 😄 I can't tell what FE UI framework pulumi.com uses but there are a lot of great pre-built open source components/tools for, for example React, that show diffs and such. Also, another very highly requested feature will probably be automatic upstream config change diffs. Ie., a user might want to see the history of the evaluated configuration (including automatic revision changes generated from upstream config changes). These would, for example, only be tracked when the upstream configs are opened and evaluated as changed (would be pretty hard to track at random). E.x.: config ABC is on Revision 5. No changes have been made to it, but a change to an upstream config imported in it has been made. This would get recorded as Revision 6, somewhat of a "commit by a service account", in config ABC.
g
Hey @plain-diamond-92898 Excited to see this. Bare minimum requirements for us are satisfied by the wireframe you posted (seeing when a change was made and by who) ~~~Version Pinning~~~ I'd like to know how you are imagining pinning syntax to work? E.g. in my stack file currently I import an env like this
Copy code
environment:
  - shopify-prod
I find the docker style of image tags intuitive and easy so
Copy code
environment:
  - shopify-prod:latest
or
Copy code
environment:
  - shopify-prod:rev2 # or whatever
~~~Diffs~~~ I agree with @magnificent-soccer-44287 diffs will be super useful. Seeing this in the UI will be good, being able to see on the cli would also be very handy. I imagine this to work git style i.e.
Copy code
pulumi esc diff shopify-prod:r2 # diffs the specified version against whatever the current pinned version is in the stack file
or
Copy code
pulumi esc diff shopify-prod:r2 shopify-prod:latest # diffs tag1 against tag2
Seeing the effects of 'upstream' changes would also be incredibly useful. I'd be fine with either UI or CLI. ~~~Dependency Visualisation~~~ I'd love to see a tree or some other graph view (like the pulumi resources graph) showing the imports e.g.
Copy code
shopify-global
|
+---shopify-prod
p
Thank you both for the detailed feedback! Apologies for the late response (just returned from my vacation) @magnificent-soccer-44287 - Yes, we've taken a lot of cues from Github. I haven't shown the new 'Editor' view - which we aim to have a similar capability where users can select a version by tags or by revision #. We thought about the commits chart - in the initial version we won't be supporting comments just yet - the commits chart looked bare without them! Diff is something we definitely want to do. It's sequenced in the next iteration of versioning. #1 - I'm curious about your suggestion for a separate tags view. In the wireframe that we shared, we get to the same tags view by clicking on the toggle (it is a representation of a filter for revisions with tags only vs. showing all revisions)
E.x.: config ABC is on Revision 5. No changes have been made to it, but a change to an upstream config imported in it has been made. This would get recorded as Revision 6, somewhat of a "commit by a service account", in config ABC.
#2 - In your example above, I'd imagine the environment ABC to not have any change in revision # because the definition of ABC itself hasn't changed. It's only when you 'open' the environment, that you import the upstream environment that has been changed but technically you haven't made any actual changes to ABC's config definition.
🙌 1
@gifted-gigabyte-53859 - Versioning pinning will be similar to Docker. We found that to be most intuitive as well. And yes definitely we want to support diffs in the next iterations! Seeing the effects of 'upstream' changes is indeed interesting. Yevgen also pointed to something similar. We hadn't thought of that deeply yet. Dependency visualization - definitely! This isn't part of the versioning work but we have planned for some visual representation of dependencies.
@gifted-gigabyte-53859 - Apart from the CLI commands to compare environments on the fly, is there anything that you'd expect out of the box to see the 'effects of upstream changes'? If say you are a consumer of an upstream environment, how would you like the upstream changes information presented to you?
g
Hi @plain-diamond-92898 - nice. Regarding viewing upstream changes, to be honest I haven't thought a lot about this yet. I guess what I was thinking when I wrote that was an easy way to review changes before accepting - I could do this using diffs on the console, but something like the

side by side diff view in some IDE

s comes to mind. Like a side by side 'compare' view is probably the best way to start off with - it's a battle tested industry standard. I am vaguely imagining something like this 1. I observe or come to know somehow (or even get notified in some way? a little overlay or label somewhere?) that an upstream I consume, has a new version) 2. I click a button somewhere to say "compare changes" 3. I am shown a side by side diff/compare view 4. Maybe I have a button to 'accept' or maybe I just manually update the tag (upstream:v2) if I want those changes, then commit it. These are just rough thoughts Some kind of 'git blame' type feature would also be magical - for example if I'm viewing a final merged/result view, would be nice to see which lines come from which imports. I can't recall tool other than git that does this, but it's something I've often wanted in many places. I can't make any good suggestions on how to display this visually, but here's one way which popped up on google (highlight lines with colour depending on which parent they came from).
p
Thank you, James! We plan to have webhooks that'll notify of updates but I hear you on a nice overlay or label somewhere that shows that upstream has a new version available. I'll add it to our backlog. Just for my curiosity in your organization who would produce the upstream environment that you'll consume? Interesting idea on showing from which environment a specific output path-value comes from. I'll add it to the backlog as well.
g
For our org we'd have for example a general parent environment which is not account specific, which most other environments would include/import. There would also be probably per account/environment configs, which projects deploying to those specific environments would import. So as for who would produce these, it would likely be engineering management/myself.
p
Got it. Thank you! I created these two Github issues to track the request, feel free to upvote! https://github.com/pulumi/esc/issues/272 https://github.com/pulumi/esc/issues/273
1