-
Notifications
You must be signed in to change notification settings - Fork 769
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[READY] Add support for the Rust programming language #268
Conversation
I made some random comments ;) Reviewed 4 of 6 files at r1. build.py, line 172 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 67 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 68 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 90 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 155 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 179 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 198 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 242 [r1] (raw file): Comments from the review on Reviewable.io |
I took a swift look too. Review status: 4 of 6 files reviewed at latest revision, 20 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 2 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 6 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 8 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 57 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 60 [r1] (raw file): I keep wondering if i should ycmd/completers/rust/rust_completer.py, line 67 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 68 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 90 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 106 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 107 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 140 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 196 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 241 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 242 [r1] (raw file): Comments from the review on Reviewable.io |
Review status: 4 of 6 files reviewed at latest revision, 20 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 60 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 90 [r1] (raw file): Comments from the review on Reviewable.io |
Reviewed 2 of 6 files at r1. ycmd/completers/rust/rust_completer.py, line 60 [r1] (raw file): Comments from the review on Reviewable.io |
Is there a way to get flake8 to validate the spaces for parens and other delimiters? |
@jwilm Awesome work, thanks for doing this! I gave this a light review since you're still working on it. I see both @puremourning and @vheon have provided excellent comments as usual, so you have feedback to go on. :) Review status: all files reviewed at latest revision, 21 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file):
What you have is 👍 for now. ycmd/completers/rust/rust_completer.py, line 2 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 6 [r1] (raw file): Make sure you go through your files before you submit and check that the copyright headers are correct. It's one of those annoying bureaucratic hassles that sadly need to be done right. :( ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): ycm_extra_conf should probably be used only for plugging in user-provided logic (like computing include paths for C++ etc). I'm not sure does a normal rust installation include the full source for the std library, but if it does, we might want to figure this out automatically by probing the system. Maybe not for a v1 though. Comments from the review on Reviewable.io |
☔ The latest upstream changes (presumably #272) made this pull request unmergeable. Please resolve the merge conflicts. |
Review status: 4 of 6 files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 6 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): This is definitely something we want configurable per project. I've got projects on my system that must be built with beta/nightly, and most of them are on stable. I would want completions/definitions to be found from the correct standard library. That said, one option would be configuring source trees for the various rust releases. The rust completer plugin could query rustc to see which version to use, but that incurs some runtime cost. Options summary:
Option 1 seems to be the most versatile, but it does come with extra challenges. ycmd/completers/rust/rust_completer.py, line 68 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 196 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 242 [r1] (raw file): Comments from the review on Reviewable.io |
Whew.. reviewable has a bit of a learning curve! |
Reviewed 1 of 6 files at r1. ycmd/completers/rust/rust_completer.py, line 242 [r1] (raw file): Comments from the review on Reviewable.io |
Review status: 4 of 6 files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 242 [r1] (raw file): Comments from the review on Reviewable.io |
Review status: 4 of 6 files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 242 [r1] (raw file): Comments from the review on Reviewable.io |
Reviewed 1 of 2 files at r3. ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file):
You have to consider that we're in the ycmd/completers/rust/rust_completer.py, line 60 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 68 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 196 [r1] (raw file): Comments from the review on Reviewable.io |
Review status: 5 of 6 files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): Comments from the review on Reviewable.io |
Added some waffly thoughts - more discussion points than anything else :) Review status: 5 of 6 files reviewed at latest revision, 13 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file): There's an existing rust plugin for Vim which provides an omnifunc. With tern completer I deliberately made sure that if the ycmd tern submodule had bot been "installed", that ycmd didn't create a Completer instance (in hook.py) and thus user's existing installations of the racer plugin continued to work as-is. IMO the omnifunc-based one is inferior due to not supporting LCS matching and a couple of other bits, but we don't want people to ycmd/completers/rust/hook.py, line 1 [r4] (raw file): ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): Here's my thinking:
I think we're setting a precedent here, so it probably bears the discussion :) ycmd/completers/rust/rust_completer.py, line 68 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 196 [r1] (raw file): At least, that's what i surmised. I've said before that i'm no expert in this newfangled python whatsit. ycmd/completers/rust/rust_completer.py, line 182 [r4] (raw file): ycmd/completers/rust/rust_completer.py, line 194 [r4] (raw file): I just realise now, however, that that change was wholly unnecessary because I removed the 1 command which used it. Please don't hate me. Comments from the review on Reviewable.io |
Reviewed 1 of 1 files at r4. Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 194 [r4] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 194 [r4] (raw file): I originally had I also intended to add In theory, the API supports it so it's not such a big deal... maybe... hopefully... I feel bad about this. I'm already punishing myself for it and grasping at justifications... Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 194 [r4] (raw file):
Don't be. As you said it will be useful in the near future 👍 Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file): That seems possibly controversial at first glance. But YCM provides a superior completion experience than using vim's omnifunc and the user already clearly agrees otherwise they wouldn't have installed YCM. We also provide fuzzy matching, better perf, minimal Vim GUI thread blocking etc. So if we add semantic completion for a new language, we should override specific plugins that used to provide such completion. Note that we have plenty of experience doing this; YCM launched with only C-family completion. Everything else was slowly added over time, and we've always been overriding. And we've never had a complaint about it; you've seen our issue tracker and know that our users have absolutely no qualms about complaining. :) We've also done this for diagnostics. At first, C-family diagnostics would come from Syntastic which would call compilers directly. Then we plugged ourselves into the Syntastic API and started returning the diagnostics we got from libclang. Eventually we replaced the Syntastic layer and just started showing diagnostics ourselves. In fact, we literally turn off Syntastic for c-family filetypes (see (And that reminds me, we should turn it off for other filetypes we provide diagnostics for.) Again, we've never had a complaint about it so I'm pretty sure we're making users happy with this. ycmd/completers/rust/rust_completer.py, line 6 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 55 [r1] (raw file): I think we should go with vimrc setting + fallback to env var. We can add support for a function in the extra conf file later if necessary, but hopefully not. I see the extra conf file as a measure of last resort, not something we should be picking by default. It's executable code which means security issues (don't forget, we've had a terrible RCE vulnerability with it once), not everyone knows Python nor wants to learn it etc. Plenty of reasons why not to pick it. ycmd/completers/rust/rust_completer.py, line 68 [r1] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file): That sounds like a separate issue in ycmd though, and one that we'd solve across the codebase if we decided to tackle it. For now, you might want to go with Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 6 [r1] (raw file): ycmd/completers/rust/rust_completer.py, line 164 [r1] (raw file):
The RustCompleter currently reads this line from racerd's stdout and uses it when making http requests. Unfortunately, it blocks while waiting for that output. In practice for racerd, it's a short amount of time. I haven't personally noticed any delay in the GUI, but that's obviously not representative of users as a whole. It seems like the consensus is that we just want to specify the port ahead-of-time. I'll update this accordingly. ycmd/completers/rust/rust_completer.py, line 194 [r4] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file): For now, don't feel the need to focus on diagnostics, but we'll probably add them for Rust at some point. Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. build.py, line 262 [r1] (raw file): Comments from the review on Reviewable.io |
Added HMAC support :) Review status: 3 of 6 files reviewed at latest revision, 10 unresolved discussions, some commit checks failed. Comments from the review on Reviewable.io |
👍 Reviewed 1 of 5 files at r5. ycmd/completers/rust/rust_completer.py, line 99 [r5] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 2 unresolved discussions. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 2 unresolved discussions. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 2 unresolved discussions. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file): Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 2 unresolved discussions. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file):
Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 2 unresolved discussions. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file): Comments from the review on Reviewable.io |
Awesomesauce. :) @micbou Could you create an issue on the tracker so we don't forget the necessary work when Rust stable 1.7 goes out? You seem to have a solid understanding of the intricacies. Reviewed 5 of 6 files at r22, 1 of 3 files at r23, 2 of 2 files at r24. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file): I'd lean towards us implementing Don't forget that we also document that ycmd/tests/rust/subcommands_test.py, line 27 [r24] (raw file): Comments from the review on Reviewable.io |
☔ The latest upstream changes (presumably #246) made this pull request unmergeable. Please resolve the merge conflicts. |
Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed. ycmd/tests/rust/subcommands_test.py, line 24 [r24] (raw file): ycmd/tests/rust/subcommands_test.py, line 27 [r24] (raw file): Comments from the review on Reviewable.io |
I believe that the tests wont start until you rebase against master. Reviewed 1 of 1 files at r25. ycmd/tests/rust/subcommands_test.py, line 57 [r25] (raw file): for command in [ 'GoTo', 'GoToDefinition', 'GoToDeclaration' ]:
yield test, command Comments from the review on Reviewable.io |
Review status: all files reviewed at latest revision, 3 unresolved discussions, some commit checks failed. ycmd/tests/rust/subcommands_test.py, line 57 [r25] (raw file): Comments from the review on Reviewable.io |
LGTM Reviewed 1 of 1 files at r25, 3 of 3 files at r26. Comments from the review on Reviewable.io |
@jwilm Tell us when you want this merged. Review status: all files reviewed at latest revision, 3 unresolved discussions. Comments from the review on Reviewable.io |
@jwilm Could you squash your commits? I see 62 commits here, and we'd rather not have that many in the history for one PR. :) Review status: all files reviewed at latest revision, 1 unresolved discussion. Comments from the review on Reviewable.io |
@Valloric I would love to clean up the commit log. Squashing it to a single commit would lose some potentially useful history (and my git activity!). However, there is a lot of incremental/fixup commits that don't add anything, and we can squash some history to tell the story more concisely. Review status: all files reviewed at latest revision, 1 unresolved discussion. Comments from the review on Reviewable.io |
Feel free to preserve as much history as you feel is useful to have, just not 62 commits worth. :D If you could keep it under 10, that would be splendid. Review status: all files reviewed at latest revision, 1 unresolved discussion. Comments from the review on Reviewable.io |
I added minor comments. Reviewed 1 of 1 files at r25, 3 of 3 files at r26. ycmd/completers/rust/rust_completer.py, line 173 [r23] (raw file): ycmd/completers/rust/rust_completer.py, line 198 [r23] (raw file): return { 'location': location } instead of these 3 lines. ycmd/completers/rust/rust_completer.py, line 200 [r23] (raw file): Comments from the review on Reviewable.io |
A Rust completer is added with support for semantic completions, GoTo, GoToDefinition, and GoToDeclaration. Since rust declarations are also definitions, GoToDeclaration and GoTo simply map to GoToDefinition. The Rust completer is powered by racerd, an HTTP/JSON wrapper around @phildawes' racer library. In addition to providing an HTTP API, racerd offers improvements including caching between requests and support for dirty buffers. Building ycmd with rust support requires adding the `--racer-completer` flag when running build.py. The build will fail if `cargo` (Rust's build tool) is not available on the system. Two configuration parameters are introduced for the Rust completer. 1. `rust_src_path` - tells racerd where to look for the rust standard library. 2. `racerd_binary_path` - override which `racerd` binary to use. This is primarily helpful for debugging and development of racerd.
After reading through the history, it was almost entirely incremental/fixup work on the feature without any real value. I squashed it into a single commit. Review status: 13 of 15 files reviewed at latest revision, all discussions resolved. Comments from the review on Reviewable.io |
Reviewed 2 of 2 files at r27. Comments from the review on Reviewable.io |
Hmm, maybe we should go ahead and merge this so the tests for my YCM PR will pass :). |
@homu r+ Review status: Comments from the review on Reviewable.io |
📌 Commit 4e11be5 has been approved by |
⚡ Test exempted - status |
[READY] Add support for the Rust programming language [racerd][] and the ycmd completer wrapper for racerd are finally ready for some peer review and testing. There are some blocking issues that need to get resolved before we can launch; these are enumerated later. ### Demonstration This gif shows semantic completion support (module paths and object members) and GoTo jumping (starting with BinaryHeap and further into the standard library). GoTo, GoToDefinition, and GoToDeclaration are all equivalent at this time.  ### Building ycmd with rust support 1. Install [multirust][] if you don't already have it. 2. Add this branch to your local ycmd installation. For example, ``` cd $YCMD_PATH git remote add jwilm https://github.com/jwilm/ycmd.git git fetch jwilm git checkout rust-support ``` 3. Run the ycmd build.py script with an additional argument. If you only care about rust support, the build command might look like this. ``` ./build.py --racerd-completer ``` ### Outstanding Issues - [x] Racerd `::racer::core::Session` usage leaks memory. Need to patch racer + racerd to support a mutable, long-lived file cache. Until then, the FileCache's Arena grows indefinitely (can be witnessed by watching racerd's memory usage). - [x] Add tests for the rust completer in ycmd - [x] racerd HMAC support - [x] RustCompleter utilizes racerd hmac - [x] Support specifying rust source path as a ycm option (eg. via .vimrc for vim) - [x] Remove `multirust` dependency from _build.py_ - [x] Tests pass on appveyor - [x] Add user option for overriding `racerd` path - [x] Add logging for racerd output - [x] Add synchronization for `self._racerd_phandle` (and friends) in RustCompleter - [x] Update racerd to handle `PoisonError` (jwilm/racerd#9) - [x] Eliminate OpenSSL dependencies (windows support) - [x] Rebase and include latest racerd [racerd]: https://github.com/jwilm/racerd [multirust]: https://github.com/brson/multirust <!-- Reviewable:start --> [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/ycmd/268) <!-- Reviewable:end -->
Add support for the Rust programming language Support for the Rust programming language is added using ycmd's racerd completer. This PR contains documentation for using the Rust completer and an update to the ycmd submodule. ## TODO - [x] Update ycmd submodule once ycm-core/ycmd#268 lands <!-- Reviewable:start --> [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/1888) <!-- Reviewable:end -->
racerd and the ycmd completer wrapper for racerd are finally ready for some peer review and testing. There are some blocking issues that need to get resolved before we can launch; these are enumerated later.
Demonstration
This gif shows semantic completion support (module paths and object members) and GoTo jumping (starting with BinaryHeap and further into the standard library). GoTo, GoToDefinition, and GoToDeclaration are all equivalent at this time.
Building ycmd with rust support
Install multirust if you don't already have it.
Add this branch to your local ycmd installation. For example,
Run the ycmd build.py script with an additional argument. If you only care about rust support, the build command might look like this.
Outstanding Issues
::racer::core::Session
usage leaks memory. Need to patch racer + racerd to support a mutable, long-lived file cache. Until then, the FileCache's Arena grows indefinitely (can be witnessed by watching racerd's memory usage).multirust
dependency from build.pyracerd
pathself._racerd_phandle
(and friends) in RustCompleterPoisonError
(Handle PoisonError when locking mutex jwilm/racerd#9)