New ask Hacker News story: Ask HN: Multicast over UDP vs. Multicast Mellanox Infiniband
Ask HN: Multicast over UDP vs. Multicast Mellanox Infiniband
3 by scrubs | 0 comments on Hacker News.
I'm working on a pet project involving multi-cast. Unfortunately I don't know enough about how this scales. Consider a network of 50 Linux machines connected to 1 TOR switch. 10 processes in this network are multi-cast transmitting long lived streams of data. Other processes in the network may or may not elect to listen to these mcast packets forwarded by the switch. Is the following true? * If IPV4 UDP multi-cast is used, the switch is obliged to forward all mcast packets to all 50 machines. * If Mellanox Infinband mcast is used (and the TOR switch is infiniband compatible) the switch is not obliged to forward the mcast packets to all 50 machines. In fact, Mellanox Infiniband does mcast with subsciption. Receivers subscribe to a mcast channel (address). As such the switch only forwards mcast packets to subscribed receivers. All-in-all, the Mellanox mcast approach is net more efficient. The switch forwards fewer packets to targeted receivers meaning all the work to get the packets into the NIC, then into the kernel, and into the receiving processes is also net less work. In contrast the UDP mcast is net less efficient. For example, if there's a machine with no listening receivers, that box would still get all mcast packets from all 10 producers only to be dropped somewhere in the kernel.
3 by scrubs | 0 comments on Hacker News.
I'm working on a pet project involving multi-cast. Unfortunately I don't know enough about how this scales. Consider a network of 50 Linux machines connected to 1 TOR switch. 10 processes in this network are multi-cast transmitting long lived streams of data. Other processes in the network may or may not elect to listen to these mcast packets forwarded by the switch. Is the following true? * If IPV4 UDP multi-cast is used, the switch is obliged to forward all mcast packets to all 50 machines. * If Mellanox Infinband mcast is used (and the TOR switch is infiniband compatible) the switch is not obliged to forward the mcast packets to all 50 machines. In fact, Mellanox Infiniband does mcast with subsciption. Receivers subscribe to a mcast channel (address). As such the switch only forwards mcast packets to subscribed receivers. All-in-all, the Mellanox mcast approach is net more efficient. The switch forwards fewer packets to targeted receivers meaning all the work to get the packets into the NIC, then into the kernel, and into the receiving processes is also net less work. In contrast the UDP mcast is net less efficient. For example, if there's a machine with no listening receivers, that box would still get all mcast packets from all 10 producers only to be dropped somewhere in the kernel.
Comments
Post a Comment