From owner-freebsd-net@freebsd.org Fri Mar 30 19:23:26 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9BB5BF5AC5E for ; Fri, 30 Mar 2018 19:23:26 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-qk0-x22b.google.com (mail-qk0-x22b.google.com [IPv6:2607:f8b0:400d:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3CA9A7A645 for ; Fri, 30 Mar 2018 19:23:26 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-qk0-x22b.google.com with SMTP id c188so9936842qkg.2 for ; Fri, 30 Mar 2018 12:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=X9zSYqjam8RVI6Am09DPui0fzdWbaJDNlRdm4L/A/1Q=; b=XINTda93MYOaLnPEGplvBSJOAVOHYft5aE+g/Rk1MX5cHJJ0l+xVX9+xP0RE9udJ49 ck+vVgiMvHbBxc7Qd4AK5kc9Ci1kHUnCdNf5heCd8DuRGNkxSYhFruaNkpLkFwxsxsOa 6mTb7TQWUAM4ZWEouX1fYoEM8Ew1+aIvKmRzw07AWk1Y2gv1KNglxnBHN4yyGEFGp3lh ecnSV10e7NKo7lDfXVWWqQuoJiLxQvzUv4/mDlWibQfQdL0I3V1AxDhgUdIIJoqywwE3 6faxM1jcq7BdMco6JZcV+wkorOTcfw9xZw03MfSO+ECpRz16ifQsI0cpyBQadj7Vmi9b /28w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=X9zSYqjam8RVI6Am09DPui0fzdWbaJDNlRdm4L/A/1Q=; b=sSgEhpFhDmlc/wT6legyoNAC0u95AbOkjTevaR/kGEcqC1OVRUkJcv60+31mmaQ7Nj k8qM/Oz9T42+WlJ1rduw79t9pbcLz9C8Fy9cRdf2H9DbgYnKesUiM+hdJT0/rV38QZmZ 0tWXCXp1HgEGb3ECqwGWyRRtc29jvdoUh0tvA6u5Hi5bwHxKUo75I8AG9IKGN/baQvyV Xk1r1lYkI7r4Q9o5adaBucgAl0j+HTxfsEoYL+CzsUx6183Gs+VSzXCy8MgVHWujM/Pd OIaYslvbupabdieOZyUqoc5WAxQDTj2V/tAC3no6LFeSiqgjJeXbLLdTn0aOiBYjlX+f lk7g== X-Gm-Message-State: ALQs6tCso19wOYdfD8iCwlwrIueatpX6kMI2DMZpi1zUoLJpFCRQjobx a/9hYJlr7hhDEh4s1TzTRhsixD4tfV66efyYMIFFKg== X-Google-Smtp-Source: AIpwx4+Rn+zNf+IZzJ0IC5RnKKt10gXsfkspFG632S2T/0e6NWpUqo7/o0zj0inTUmPH1PmUtF/Ga/XbEW3yGH+b5vY= X-Received: by 10.55.9.211 with SMTP id 202mr305824qkj.183.1522437805586; Fri, 30 Mar 2018 12:23:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.195.204 with HTTP; Fri, 30 Mar 2018 12:23:25 -0700 (PDT) In-Reply-To: References: From: Vincenzo Maffione Date: Fri, 30 Mar 2018 21:23:25 +0200 Message-ID: Subject: Re: netmap: How the buf_num of buffers is used by driver and monitor app To: Ming Fu Cc: "freebsd-net@freebsd.org" , Giuseppe Lettieri Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Mar 2018 19:23:26 -0000 Hi, 2018-03-30 6:00 GMT+02:00 Ming Fu : > Hi, > > I am wondering about the netmap module parameters buf_num. The default > buf_num 163840 seems fairly big compare to intel 10G ixgbe of max 4096 > queue size. A full queue of the interface can only use a very small portion > of the buf_num. Can the netmap enabled driver like ixgbe use more buffers > than the "ethtool -G" configured rx/tx queue size? > No, netmap will just the same number of buffers as configured with ethtool -G. The default buf_num is just a default that covers many things that you can do. First of all you would need 4096 for each TX and RX ring (so typically 2*8*4096). Then you need additional 2*4096 buffers for the host rings. If you create netmap pipes like 'netmap:eth0{1' you need more buffers for the pipe TX and RX ring. If you know how many netmap buffers you really need for your purpose you can just lower buf_num to the minimun number that won't cause your application to fail with ENOMEM. > > I need to feed packages captured from netmap device to a few traffic > monitors on the same box. The monitor application attach to the device as > netmap monitor with the /r at the end of device name. My question is if the > primary read of the device calls poll(), the netmap buffer is synced with > the kernel. Can the other monitor application still access the packet that > the primary read just returned to the kernel? If one of the monitor on the > netmap device is slow, will it cause trouble for the primary reader and > other monitors? How can I cache a lot of packets in the buffer in case one > of the monitor application had a temporary slow down? > If you use copy-based monitor (no 'z') each monitor will receive an independent copy of the traffic going through monitored rings. So I think if one of the monitoring applications slows down there should be no effect on the others. If you use zerocopy monitor, the same intercepted buffer is passed between the various monitoring applications (and the main application), so I think if one stops or slows down, then all the monitors will stop. There is some explanation in the comment here https://github.com/luigirizzo/netmap/blob/master/sys/dev/netmap/netmap_monitor.c#L30-L69 Feel free to open a ticket on github ( https://github.com/luigirizzo/netmap/issues) if you have a more specific question. Cheers, Vincenzo > > Thanks, > Ming > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione