Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding a new system for bindings generation, modeled after a compiler pipeline with a lot of small steps. This is WIP a continuation of #2333.
Planning out the work
This time I only implemented the general parts of the IR, not the Python pipeline. My next step is to try to get the Gecko JS bindings to use this. I'm pretty confident that I can get Python working again so I'd rather work on a new language. Also, I think the Gecko JS bindings could really benefit from this system.
In the meantime, I expect this to be a long-lived PR/branch. Feel free to comment on it whenever you get some time to take a look. I don't expect many merge errors, since this is basically just adding new code.
I'm thinking that we should give external bindings authors plenty of time to see this before we force them to switch over. If we decided to go with this and we migrate all the builtin bindings generators to using the new system, I'd still like to see at least 2 UniFFI versions that support the current
ComponentInterface
-based generation before we force users to move to the new one. If it's not much work to keep supporting the legacy system, then I'd like it to be more like 4 versions.The IRs
The IR types are mostly similar to before, with one big difference: IRs contain items from every crate involved. My feeling is that this will set us up to finally tackle issues that require cross-module coordination like #1896 and #2430.
The passes
I tried sooooo many ways to express the logic of each pass in a nice way. In the end I was inspired by the nanopass framework, where you have a bunch of little passes. That keeps the logic modular and readable. When you read one of the
ir::general::pass
modules, it's doing a single thing. Before, we had one big pass that converted everything, which was harder to understand.The downside of a nanopass framework is that it's pretty easy to make
rustc
explode. My initial try created a separate set of IR node type for each pass, but once I got to a dozen or so the build time foruniffi_bindgen
went to about 30s. Then it started getting killed by the OOM killer and I decided it was time for a new direction.To avoid this, the new system creates a single IR for a pass, but then splits the pass up into multiple steps. Even though each step works on the same IR, it still kind of feel like they're transforming the types. Instead of adding a new field, they now set a value for a field that was previously empty.
This achieves more-or-less the same effect as the nanopass system, but with much faster compile times. My not-very-scientific measurement is that it went from 12s before to 17s now. That's not great, but if this is successful we can hopefully delete a bunch of the old code and get times down to something reasonable again.
The CLI
I replaced the
peek
anddiff
commands with thepipeline
command. If you have time, read the internals doc about it and take it for a test drive. IMO it's pretty fun walking through each pipeline step.