Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

std::ptr::swap_nonoverlapping is not always untyped #134713

Open
zachs18 opened this issue Dec 23, 2024 · 5 comments · May be fixed by #137412
Open

std::ptr::swap_nonoverlapping is not always untyped #134713

zachs18 opened this issue Dec 23, 2024 · 5 comments · May be fixed by #137412
Assignees
Labels
C-bug Category: This is a bug. P-high High priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Comments

@zachs18
Copy link
Contributor

zachs18 commented Dec 23, 2024

Code

I tried this code:

#![allow(unused)]
use std::mem::{size_of, align_of};

#[repr(C)]
struct Foo(usize, u8);

fn main() {
    let buf1: [usize; 2] = [1000, 2000];
    let buf2: [usize; 2] = [3000, 4000];
    
    // Foo and [usize; 2] have the same size and alignment,
    // so swap_nonoverlapping should treat them the same
    assert_eq!(size_of::<Foo>(), size_of::<[usize; 2]>());
    assert_eq!(align_of::<Foo>(), align_of::<[usize; 2]>());
    
    let mut b1 = buf1;
    let mut b2 = buf2;
    // Safety: b1 and b2 are distinct local variables,
    // with the same size and alignment as Foo.
    unsafe {
        std::ptr::swap_nonoverlapping(
            b1.as_mut_ptr().cast::<Foo>(),
            b2.as_mut_ptr().cast::<Foo>(),
            1,
        );
    }
    assert_eq!(b1, buf2); // Fails: [3000, 160] != [3000, 4000] or [3000, 1952] != [3000, 4000]
    assert_eq!(b2, buf1); // Fails: [1000, 208] != [1000, 2000] or [1000, 4048] != [1000, 2000]
}

godbolt link

I expected to see this happen: The two assert_eq!s should succeed; b1 and b2 should be completely swapped, since std::ptr::swap_nonoverlapping::<T> claims to swap bytes, not Ts.

Instead, this happened: They are not entirely swapped. swap_nonoverlapping appears to be doing a typed swap at Foo, skipping/zeroing/not-correctly-swapping the 7 padding bytes at the end of Foo.

I think this only happens with types where size_of::<T>() > size_of::<usize>() from looking at the implementation, but I'm not sure.

In debug mode:
Rust 1.61-1.70 appear to consistently give [3000, 160].
Rust 1.71-nightly appear to consistently give [3000, 1952].
4000 is 0x0fa0
160 is 0x00a0
1952 is 0x07a0
2000 is 0x07d0
so it looks like Rust 1.61-1.70 are zeroing the padding bytes, and Rust 1.71-nightly are ignoring them.

Version it worked on

It most recently worked on: Rust 1.60

Version with regression

rustc --version --verbose:

rustc 1.61.0 (fe5b13d68 2022-05-18)
binary: rustc
commit-hash: fe5b13d681f25ee6474be29d748c65adcd91f69e
commit-date: 2022-05-18
host: x86_64-unknown-linux-gnu
release: 1.61.0
LLVM version: 14.0.0

@rustbot modify labels: +regression-from-stable-to-stable -regression-untriaged

@zachs18 zachs18 added C-bug Category: This is a bug. regression-untriaged Untriaged performance or correctness regression. labels Dec 23, 2024
@rustbot rustbot added I-prioritize Issue: Indicates that prioritization has been requested for this issue. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. and removed regression-untriaged Untriaged performance or correctness regression. labels Dec 23, 2024
@scottmcm
Copy link
Member

This is the classic problem that MaybeUninit promised two things and it can't actually promise both:

  • That it holds any byte pattern
  • That it has the same ABI

So copying as MaybeUninit<T> doesn't do what it says it would :(

@GKFX
Copy link
Contributor

GKFX commented Dec 27, 2024

I don't see any part of the MaybeUninit<T> documentation that would make the padding bytes of T not still be padding bytes of MaybeUninit<T>. So it doesn't look like swap_nonoverlapping_simple_untyped can be expected to copy padding bytes. Possibly the answer is that all of the copying should be done as MaybeUninit integers, never MaybeUninit<T>, or copy_nonoverlapping<T> should be used.

bors added a commit to rust-lang-ci/rust that referenced this issue Dec 31, 2024
Redo the swap code for better tail & padding handling

A couple of parts here

## Fixes rust-lang#134713

Actually using the type passed to `ptr::swap_nonoverlapping` for anything other than its size + align turns out to not work, so this redo goes back to *always* swapping via not-`!noundef` integers.

(Except in `const`, which keeps doing the same thing as before to preserve `@RalfJung's` fix from rust-lang#134689)

## Fixes rust-lang#134946

I'd previously moved the swapping to use auto-vectorization, but someone pointed out on Discord that the tail loop handling from that left a whole bunch of byte-by-byte swapping around.  This PR goes back to a manual chunk, with at most logarithmic more instructions for the tail.

(There are other ways that could potentially handle the tail even better, but this seems to be pretty good, since it's how LLVM ends up lowering operations on types like `i56`.)

## Polymorphization

Since it's silly to have separate copies of swapping -- especially *untyped* swapping! -- for `u32`, `i32`, `f32`, `[u16; 2]`, etc, this sends everything to byte versions, but still mono'd by alignment.  That should make it more ok that the code is somewhat more complex, since we only get 7 monomorphizations of the complicated bit.

(One day we'll be able to remove some of the hacks by being able to just call `foo::<{align_of::<T>()}>`, but since alignments are only powers of two, the manual dispatch out isn't a big deal.)
@scottmcm scottmcm self-assigned this Dec 31, 2024
@jieyouxu jieyouxu added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Jan 8, 2025
@Amanieu Amanieu added P-high High priority and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Jan 15, 2025
@Manishearth
Copy link
Member

Manishearth commented Jan 28, 2025

Uninitialized and padding bytes are different. I believe this code is UB: reading from padding bytes.

It would probably be good to document this behavior on the copy functions, though. "Copies bytes" is potentially misleading

@GKFX
Copy link
Contributor

GKFX commented Jan 28, 2025

The code snippet above never reads padding bytes; it should read fully initialized arrays of usize. Copying padding bytes around has the effect of making the target uninitialized (https://github.com/rust-lang/unsafe-code-guidelines/blob/master/reference/src/glossary.md#padding), but swap_nonoverlapping is documented specifically not to do that.

@RalfJung
Copy link
Member

Cc @rust-lang/opsem MaybeUninit ABI guarantee striking again...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. P-high High priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
8 participants