From owner-freebsd-net@freebsd.org Sun Jan 3 18:42:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22BADA5F127 for ; Sun, 3 Jan 2016 18:42:55 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [IPv6:2a00:1450:4010:c04::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A45131BE0; Sun, 3 Jan 2016 18:42:54 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-lb0-x22e.google.com with SMTP id pv2so164579326lbb.1; Sun, 03 Jan 2016 10:42:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GEof0u9QZXzpazSUvDokdToRla7SoV7NrF+HgUQX2PE=; b=ym513fbVsXkCOjQNPbrD0w36uOzOLK8lQo2V+i2rLepLkgt3/eoJe0ZyfxivCK03IR I0tF1KV4d1tGc4ilWZe/3JDcW0qtfHE3OYp2nc5yrnnxSytWvr1NkXxp1ypA3/dA1nbb t7UQB7pxtnfQvGMLM2PF1F9b3J8OSbgch6MKPiZoixMN6XQ3v2WEUtBnOlpFuZ0p7pl9 i03cORT5wZQ+f4rFudCVL1L7AkPIwb3Up4ZKvDbD/OX34EMs2C6BoUZhESnYW/JRir65 gSL9bKbwyK5V2VjJN7Vgg6nt2s5i3mslJXH/WqFIR+97LxIieWOGJ/BJSimLvOkG73jS TtWg== MIME-Version: 1.0 X-Received: by 10.112.211.136 with SMTP id nc8mr29751507lbc.54.1451846572693; Sun, 03 Jan 2016 10:42:52 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.13.33 with HTTP; Sun, 3 Jan 2016 10:42:52 -0800 (PST) In-Reply-To: References: <1451841004.10139.34.camel@me.com> Date: Sun, 3 Jan 2016 19:42:52 +0100 X-Google-Sender-Auth: V3jYoCmzx4GR-T_qS8IpzBsNfDE Message-ID: Subject: Re: Does FreeBSD have sendmmsg or recvmmsg system calls? From: Luigi Rizzo To: Robert Watson Cc: Rui Paulo , HuanHuan , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 18:42:55 -0000 On Sun, Jan 3, 2016 at 6:46 PM, Robert Watson wrote: > On Sun, 3 Jan 2016, Rui Paulo wrote: > >>> NetBSD 7.0 has just introduced these two syscalls. And Linux also has >>> them. >>> >>> Does FreeBSD have them? Or plan to support them in the future? >> >> >> FreeBSD does not have them. It doesn't seem especially hard to implement, >> though. Do you know any major application already using them? > > > I see no harm in having the system calls. When I chatted with the authors > of nsd a couple of years ago (they had originally promoted the approach), > they told me they felt it only offered incremental benefit and didn't > particularly recommend that FreeBSD adopt it. However, it would undoubtably > see use and does offer some opportunities for future batching-related > performance operations, so if someone wants to pursue introducing it, do go > ahead. Or just introduce a libc implementation that can be converted to a > kernel implementation later if that is determined to be beneficial. But the > nsd observations don't (currently) lead me to believe that it is critical to > do so. Do not expect great speedups unless you are willing to put a ton of work in the network stack. When I looked at it (about 2012), the linux implementation for sendmmsg() just looped inside the kernel on the body of sendmsg(), so the advantage was just in the syscall enter/exit call, which on linux again was relatively cheap anyways. So think of some 10% gain for unconnected sockets on every msg after the first one, and maybe slightly larger on connected sockets. It is not easy to do better, especially for unconnected sockets, because you have to lookup addresses for each message, possibly going to different interfaces, so unless you are willing to implement batching throughout the entire stack you won't see much gain. On the receive side, the syscall is probably another story (one can grab a full batch of packets in one shot), though the expensive part of the work -- dispatching packets to the socket -- was already done on the receive interrupt handler. Same problems as in the tx path to be solved. cheers luigi