-
Notifications
You must be signed in to change notification settings - Fork 13.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
self-contained linker: conservatively default to -znostart-stop-gc
on x64 linux
#137685
Conversation
Sanity check perf run: @bors try @rust-timer queue |
This comment has been minimized.
This comment has been minimized.
self-contained linker: conservatively default to `-znostart-stop-gc` To help stabilization, this PR disables an LLD optimization with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. This optimization has [no visible impact](rust-lang#112049 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe `@Kobzol` will look into it if they have time. r? `@petrochenkov`
☀️ Try build successful - checks-actions |
This comment has been minimized.
This comment has been minimized.
Finished benchmarking commit (ced69ff): comparison URL. Overall result: no relevant changes - no action neededBenchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf. @bors rollup=never Instruction countThis benchmark run did not return any relevant results for this metric. Max RSS (memory usage)Results (primary 3.4%, secondary -2.2%)This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
CyclesResults (secondary 1.7%)This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
Binary sizeThis benchmark run did not return any relevant results for this metric. Bootstrap: 771.284s -> 771.237s (-0.01%) |
Even better than emitting FCWs would be fixing LLD. Wild linker had the same issue, but the fix was quite easy: davidlattimore/wild#460 |
LLD has said on several occasions that they won't fix this, as they don't consider it to be an issue (https://lld.llvm.org/ELF/start-stop-gc.html). We're kind of being overly conservative here. |
To be clear, there is nothing to fix in either linkers per se: they both implement the two behaviors of conservative and precise GC. The default value is just different, GNU ld defaults to It's more of a difference of opinion, which is why I described this as disabling a size optimization. It just helps stabilization, to remove a concern of these different defaults, and delay possible FCWs and wait periods to where it's needed. We could equally do nothing and take the breakage. |
@bors r+ |
🌲 The tree is currently closed for pull requests below priority 100. This pull request will be tested once the tree is reopened. |
@bors rollup- |
self-contained linker: conservatively default to `-znostart-stop-gc` To help stabilization, this PR disables an LLD optimization with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe `@Kobzol` will look into it if they have time. r? `@petrochenkov`
self-contained linker: conservatively default to `-znostart-stop-gc` To help stabilization, this PR disables an LLD optimization with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ``@Kobzol`` will look into it if they have time. r? ``@petrochenkov``
self-contained linker: conservatively default to `-znostart-stop-gc` To help stabilization, this PR disables an LLD optimization with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ```@Kobzol``` will look into it if they have time. r? ```@petrochenkov```
…iaskrgr Rollup of 10 pull requests Successful merges: - rust-lang#132388 (Implement `#[cfg]` in `where` clauses) - rust-lang#134900 (Fix parsing of ranges after unary operators) - rust-lang#136938 (Remove `:` from `stack-protector-heuristics-effect.rs` Filecheck Pattern) - rust-lang#137054 (Make phantom variance markers transparent) - rust-lang#137525 (Simplify parallelization in test-float-parse) - rust-lang#137618 (Skip `tidy` in pre-push hook if the user is deleting a remote branch) - rust-lang#137685 (self-contained linker: conservatively default to `-znostart-stop-gc`) - rust-lang#137741 (Stop using `hash_raw_entry` in `CodegenCx::const_str`) - rust-lang#137849 (Revert "Remove Win SDK 10.0.26100.0 from CI") - rust-lang#137862 (ensure we always print all --print options in help) r? `@ghost` `@rustbot` modify labels: rollup
…iaskrgr Rollup of 10 pull requests Successful merges: - rust-lang#132388 (Implement `#[cfg]` in `where` clauses) - rust-lang#134900 (Fix parsing of ranges after unary operators) - rust-lang#136938 (Remove `:` from `stack-protector-heuristics-effect.rs` Filecheck Pattern) - rust-lang#137054 (Make phantom variance markers transparent) - rust-lang#137525 (Simplify parallelization in test-float-parse) - rust-lang#137618 (Skip `tidy` in pre-push hook if the user is deleting a remote branch) - rust-lang#137685 (self-contained linker: conservatively default to `-znostart-stop-gc`) - rust-lang#137741 (Stop using `hash_raw_entry` in `CodegenCx::const_str`) - rust-lang#137849 (Revert "Remove Win SDK 10.0.26100.0 from CI") - rust-lang#137862 (ensure we always print all --print options in help) r? `@ghost` `@rustbot` modify labels: rollup
This will help stabilization of lld.
So it's not the serialized vs rollup state, it's some combination of lld + unknown flag + cc + apple causing an error.
The flag is for ELF, but it should be fine as lld accepts unknown z flag and emits warnings for them. % ~/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin/gcc-ld/ld.lld -zbla
rust-lld: warning: unknown -z value: bla I suspect some issue on osx in passing the flag all the way down to the linker, as the error shows an unknown argument instead of an unknown -z value. In any case, we don't need to care: to avoid this entirely, and since the flag is only for ELF, I've limited this change to x64 linux only (and if others want to expand this to other ELF targets needing it it'll be easy to add). Now that CI has passed, we can re-test on the apple test builders, but nothing changes there. |
self-contained linker: conservatively default to `-znostart-stop-gc` To help stabilization, this PR disables an LLD optimization with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe `@Kobzol` will look into it if they have time. r? `@petrochenkov` try-job: x86_64-apple-1 try-job: x86_64-apple-2 try-job: aarch64-apple
☀️ Try build successful - checks-actions |
-znostart-stop-gc
-znostart-stop-gc
on x64 linux
CI passed on linux, and on the apple builders. I've updated the name and description to mention this is now for x64 linux only, as explained above. Let's have a final look over. |
@bors r+ |
self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux To help stabilization, this PR disables an LLD optimization on x64 linux with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe `@Kobzol` will look into it if they have time. r? `@petrochenkov`
Rollup of 29 pull requests Successful merges: - rust-lang#135733 (Implement `&pin const self` and `&pin mut self` sugars) - rust-lang#137107 (Override default `Write` methods for cursor-like types) - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion) - rust-lang#137327 (Undeprecate env::home_dir) - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book) - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden)) - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var) - rust-lang#137612 (Update bootstrap to edition 2024) - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck) - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums) - rust-lang#137685 (self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux) - rust-lang#137744 (Re-add `Clone`-derive on `Thir`) - rust-lang#137758 (fix usage of ty decl macro fragments in attributes) - rust-lang#137764 (Ensure that negative auto impls are always applicable) - rust-lang#137772 (Fix char count in `Display` for `ByteStr`) - rust-lang#137798 (ci: use ubuntu 24 on arm large runner) - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee) - rust-lang#137805 (adjust Layout debug printing to match the internal field name) - rust-lang#137808 (Do not require that unsafe fields lack drop glue) - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists) - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN) - rust-lang#137827 (Add timestamp to unstable feature usage metrics) - rust-lang#137832 (Fix crash in BufReader::peek()) - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT) - rust-lang#137920 (interpret/provenance_map: consistently use range_is_empty) - rust-lang#138038 (Update `compiler-builtins` to 0.1.151) - rust-lang#138046 (trim channel value in `get_closest_merge_commit`) - rust-lang#138052 (strip `-Wlinker-messages` wrappers from `rust-lld` rmake test) - rust-lang#138053 (Increase the max. custom try jobs requested to `20`) r? `@ghost` `@rustbot` modify labels: rollup
self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux To help stabilization, this PR disables an LLD optimization on x64 linux with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ``@Kobzol`` will look into it if they have time. r? ``@petrochenkov``
self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux To help stabilization, this PR disables an LLD optimization on x64 linux with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ```@Kobzol``` will look into it if they have time. r? ```@petrochenkov```
…mpiler-errors Rollup of 8 pull requests Successful merges: - rust-lang#136642 (Put the alloc unit tests in a separate alloctests package) - rust-lang#137337 (Add verbatim linker to AIXLinker) - rust-lang#137363 (compiler: factor Windows x86-32 ABI impl into its own file) - rust-lang#137685 (self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux) - rust-lang#138000 (atomic: clarify that failing conditional RMW operations are not 'writes') - rust-lang#138063 (Improve `-Zunpretty=hir` for parsed attrs) - rust-lang#138137 (setTargetTriple now accepts Triple rather than string) - rust-lang#138173 (Delay bug for negative auto trait rather than ICEing) r? `@ghost` `@rustbot` modify labels: rollup
Rollup of 5 pull requests Successful merges: - rust-lang#136642 (Put the alloc unit tests in a separate alloctests package) - rust-lang#137528 (Windows: Fix error in `fs::rename` on Windows 1607) - rust-lang#137685 (self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux) - rust-lang#137757 (On long spans, trim the middle of them to make them fit in the terminal width) - rust-lang#138189 (Mention `env` and `option_env` macros in `std::env::var` docs) r? `@ghost` `@rustbot` modify labels: rollup
Rollup merge of rust-lang#137685 - lqd:nostart-stop-gc, r=petrochenkov self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux To help stabilization, this PR disables an LLD optimization on x64 linux with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](dtolnay/linkme#63) or on nightly, need to disable lld. More information about all this, and the historical differences, can be found in: - https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order - https://lld.llvm.org/ELF/start-stop-gc This optimization has [no visible impact](rust-lang#137685 (comment)) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable. I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ````@Kobzol```` will look into it if they have time. r? ````@petrochenkov````
To help stabilization, this PR disables an LLD optimization on x64 linux with respect to
--gc-sections
and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using linkme, which doesn't work with lld or on nightly, need to disable lld.More information about all this, and the historical differences, can be found in:
This optimization has no visible impact on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without
-znostart-stop-gc
, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable.I've tested that this correctly links on
linkme
examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for-znostart-stop-gc
to only impact this... It should of course be doable though, maybe @Kobzol will look into it if they have time.r? @petrochenkov