Set IPv6 hop limit for multicast traffic in sendmsg

[sgasse]

I tried a long time setting the hop limit for IPv6 multicast traffic sent with sendmsg until I read in a StackOverflow post that for IPv6 multicast and sendmsg, the hop limit has to be set with a control message.

For this, I created #2074 , which works on Linux. However, the is one slight difference to what is written in the man pages: When we pass the value -1, the hop limit is not set to the route default (64 on most machines including mine, see ``) but set to 1. Any idea what could cause this in `nix`?

Here is an example binary to test the hop limit:

use std::{
    io::IoSlice,
    net::{Ipv6Addr, SocketAddrV6},
};

use libc::{in6_addr, in6_pktinfo};
use nix::sys::socket::{
    sendmsg, socket, ControlMessage, MsgFlags, SockFlag, SockType, SockaddrIn6,
};

fn main() {
    let socket = socket(
        nix::sys::socket::AddressFamily::Inet6,
        SockType::Datagram,
        SockFlag::SOCK_NONBLOCK,
        None,
    )
    .unwrap();

    // Set source IP address and interface.
    let interface_id = 2;
    let ipv6_info = in6_pktinfo {
        ipi6_addr: in6_addr {
            s6_addr: Ipv6Addr::UNSPECIFIED.octets(),
        },
        ipi6_ifindex: interface_id,
    };

    let dst_addr: SockaddrIn6 =
        SocketAddrV6::new("ff02::11".parse::<Ipv6Addr>().unwrap(), 12345, 0, 0).into();

    let cmsgs = &[
        ControlMessage::Ipv6PacketInfo(&ipv6_info),
        ControlMessage::Ipv6MulticastHopLimit(&7),
    ];

    let iovs: Vec<_> = ["hello".as_bytes(), "world".as_bytes()]
        .iter()
        .map(|b| IoSlice::new(b))
        .collect();

    sendmsg(
        socket,
        iovs.as_slice(),
        cmsgs,
        MsgFlags::empty(),
        Some(&dst_addr),
    )
    .unwrap();
}

We can observe the hop limit as hlim in tcpdump:

sudo tcpdump -v port 12345

...
10:27:07.166865 IP6 (flowlabel 0x15ce0, hlim 7, next-header UDP (17) payload length: 18) ...

Would this be worth merging even if -1 does not lead to the route default? Or any suggestions what could be different? With Python, -1 as hop limit leads to the route default, however the cmsg info in strace already looks the same, below the example from Rust:

strace -s 256 --trace=sendmsg cargo run --bin main

sendmsg(3, {msg_name={sa_family=AF_INET6, sin6_port=htons(12345), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "ff02::11", &sin6_addr), sin6_scope_id=0}, msg_namelen=28, msg_iov=[{iov_base="hello", iov_len=5}, {iov_base="world", iov_len=5}], msg_iovlen=2, msg_control=[{cmsg_len=36, cmsg_level=SOL_IPV6, cmsg_type=0x32}, {cmsg_len=20, cmsg_level=SOL_IPV6, cmsg_type=0x34}], msg_controllen=64, msg_flags=0}, 0) = 10

[asomers]

Are you saying that sending this control message seems to have different effects in Python vs in Rust with Nix? We need to figure out why before we can merge this. Have you tried running both the Python and Rust programs under gdb and examining the cmsg ? That may reveal more detail than strace.

[sgasse]

Alright thanks for the suggestion with gdb! 🙏 I will take a look and try to understand why setting -1 has different behavior when using Python vs Rust.

[sgasse]

Hey @asomers , I took another look at the issue. Short story: The behavior is the same between Rust and Python. When I tested before, I just tested on different interfaces with Rust vs Python and they behaved differently. On one of my interfaces, setting the hop limit parameter to -1 yields 1 in the packet being sent while on another interface, -1 yields a packet with the route default, in my case 64. This is the same for Rust and Python.

Nevertheless, since you suggested it and to be sure, I looked at what is passed to sendmsg in gdb. Below is a screenshot with the test code and gdb sessions in which I break on __libc_sendmsg, get the address and length of the control messages and then dump the memory of it for both Rust and Python. The data is identical.

[asomers]

So now that the Rust-vs-Python issue is resolved, is there anything else to this issue that isn't resolved by #2074?

[sgasse]

No, everything should be fine now 👍 I updated #2074 with the changes you requested.