I started working on a Rust SDK but only got as far as generating code from the language and resource protos using tonic-build. I wanted to do this to learn Rust but realised that this is a massive undertaking that is currently beyond my skill level, as it requires knowledge of async, tokio, and a deep understanding of how pulumi works. Also there are a number of architectural decisions that must be made, such as deciding whether to use WASM for providers or using extism (as per github issue #3622). There are also decisions on how we want the API to look, whether to use macros etc etc. There seems to be a decent amount of interest in a rust SDK, I'd be interested in working on it further but I couldn't be the one to lead such an endeavour; it would require some leadership and guidance from pulumi devs and more experienced rust programmers. Let me know if there's any interest in working on it. The repo with the tonic-build codegen is here https://github.com/chambers/pulumi-rs/tree/dev
10/03/2023, 1:59 PM
You are completely right in your appraisal that this is a rather large endeavour. The glue code for protobuf is actually a very small piece of the SDK for a language. What's quite involved is the machinery around encoding/decoding values from JSON-like protobuf representation on the wire to idiomatic data structures for given language, asynchronous runtime that does an interesting fire-but-not-completely-forget forking of requests and subsequent promise fulfilling (you basically return to the user a struct/object with Outputs backed with Promises that forked request fulfils once you get a grpc reply) and the codegen to generate provider packages out of schemas. I'm planning to write a blog post about our journey through all of that but that's probably coming around nov/dec.
Source: we've recently released Scala SDK for Pulumi - https://virtuslab.github.io/besom/, I'm the lead of the project.
10/03/2023, 3:22 PM
@modern-cpu-56503 I’m also looking into a rust sdk ❤️ 🦀. I also agree that this would require some collaboration with those more familiar with pulumi under the hood.
https://github.com/pulumi/pulumi/issues/11882 also has some great insights!
10/03/2023, 6:56 PM
@mysterious-activity-97749 cool, how far have you gotten? pm if you want to chat more about it. @quaint-spring-93350 very cool to see an sdk in a functional style and interested to see whether FP is a good fit for IaC. I don't know anything about Scala but will defo try it out. For IaC (when what we're really doing most of the time is abstracting away YAML), verbosity can be a killer and FP could help there. As for deserialising JSON-like PB, rust is actually pretty great at that with crates like tonic, prost and serde. Will defo read your blog post, but perhaps you could give us a preview and share some of the key difficulties/pitfalls of making the sdk and how you solved them? :)
10/04/2023, 5:20 PM
@modern-cpu-56503, I haven’t gotten very far. Right now, I’m focusing on building more context and knowledge of how Pulumi works and what’s required for creating an SDK.