From owner-freebsd-net@freebsd.org Sun Nov 1 15:59:41 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E975A235A8 for ; Sun, 1 Nov 2015 15:59:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 898B81DB1 for ; Sun, 1 Nov 2015 15:59:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA1Fxfrq021032 for ; Sun, 1 Nov 2015 15:59:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 203922] The kern.ipc.acceptqueue limit is too low Date: Sun, 01 Nov 2015 15:59:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: needs-qa, patch, performance X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: keywords flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2015 15:59:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203922 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |needs-qa, performance Flags| |mfc-stable9?, mfc-stable10? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@freebsd.org Sun Nov 1 21:00:11 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11751A23E04 for ; Sun, 1 Nov 2015 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 055031CDC for ; Sun, 1 Nov 2015 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA1L0AYR057724 for ; Sun, 1 Nov 2015 21:00:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201511012100.tA1L0AYR057724@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-net@FreeBSD.org Subject: Problem reports for freebsd-net@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 01 Nov 2015 21:00:10 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2015 21:00:11 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 200221 | em0 watchdog timeout under load New | 203175 | Daily kernel crashes in tcp_twclose
Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 740D4A244F2 for ; Sun, 1 Nov 2015 21:22:11 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id 5A0A01D01 for ; Sun, 1 Nov 2015 21:22:11 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id 586913320E67; Sun, 1 Nov 2015 21:22:11 +0000 (UTC) Date: Sun, 1 Nov 2015 21:22:11 +0000 To: freebsd-net@freebsd.org From: "mmoll (Michael Moll)" Reply-to: D1944+325+8925873bdc96dfc2@reviews.freebsd.org Subject: [Differential] [Commented On] D1944: PF and VIMAGE fixes Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFY2goM= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2015 21:22:11 -0000 mmoll added a subscriber: mmoll. mmoll added a comment. what's the status here? REVISION DETAIL https://reviews.freebsd.org/D1944 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: nvass-gmx.com, bz, trociny, kristof, gnn, zec, rodrigc, glebius, eri Cc: mmoll, javier_ovi_yahoo.com, farrokhi, julian, robak, freebsd-virtualization-list, freebsd-pf-list, freebsd-net-list From owner-freebsd-net@freebsd.org Mon Nov 2 21:07:41 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A844EA24A0E for ; Mon, 2 Nov 2015 21:07:41 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id 8C92F101C for ; Mon, 2 Nov 2015 21:07:41 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id 86BAD147B80; Mon, 2 Nov 2015 21:07:41 +0000 (UTC) Date: Mon, 2 Nov 2015 21:07:41 +0000 To: freebsd-net@freebsd.org From: "rodrigc (Craig Rodrigues)" Reply-to: D1944+325+8925873bdc96dfc2@reviews.freebsd.org Subject: [Differential] [Commented On] D1944: PF and VIMAGE fixes Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFY30J0= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2015 21:07:41 -0000 rodrigc added a comment. @mmoll : It would be nice if @glebius could review this patch. He previously committed some patches I committed to FreeBSD which attempted to fix this problem, so he has an interest in this area. REVISION DETAIL https://reviews.freebsd.org/D1944 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: nvass-gmx.com, bz, trociny, kristof, gnn, zec, rodrigc, glebius, eri Cc: mmoll, javier_ovi_yahoo.com, farrokhi, julian, robak, freebsd-virtualization-list, freebsd-pf-list, freebsd-net-list From owner-freebsd-net@freebsd.org Mon Nov 2 21:18:20 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40E17A24CAD for ; Mon, 2 Nov 2015 21:18:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2DDEE14EA for ; Mon, 2 Nov 2015 21:18:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA2LIKQF099246 for ; Mon, 2 Nov 2015 21:18:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 202983] ixv driver in 11.0-CURRENT(10.1 & 10.2 RELEASE) doesn't pass traffic using XEN hypervisor(AWS EC2) Date: Mon, 02 Nov 2015 21:18:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: jeffrey.e.pieper@intel.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2015 21:18:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202983 --- Comment #2 from Jeff Pieper --- We are able to reproduce this with a KVM hypervisor. This occurs when the PF has MTU set to 9000 and the VF has MTU set to default (1500). We are investigating. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@freebsd.org Tue Nov 3 13:14:49 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A71BA2416A for ; Tue, 3 Nov 2015 13:14:49 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id D42FD120D for ; Tue, 3 Nov 2015 13:14:47 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.20,238,1444654800"; d="gif'147?scan'147,208,147";a="14515267" Received: from gsp-ex03.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.19]) by iport1.cc.swin.edu.au with ESMTP; 04 Nov 2015 00:14:45 +1100 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.2]) by gsp-ex03.ds.swin.edu.au ([169.254.3.127]) with mapi id 14.03.0248.002; Wed, 4 Nov 2015 00:14:45 +1100 From: Rasool Al-Saadi To: "freebsd-net@freebsd.org" Subject: Timing issue with Dummynet on high kernel timer interrupt Thread-Topic: Timing issue with Dummynet on high kernel timer interrupt Thread-Index: AdEWOVE5DoAlkzqPQWaBZ8ln4esVdw== Date: Tue, 3 Nov 2015 13:14:45 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [136.186.126.11] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 13:14:49 -0000 Hello all, While we were implementing new AQMs (Codel, PIE, FQ-Codel and FQ-PIE) in Fr= eeBSD targeting ipfw+Dummynet, we noticed timing issue with Dummynet when w= e increase kernel timer interrupt Hz (changing the value of kern.hz in /boo= t/loader.conf) to 3000 or higher. The issue is that spikes (upward and down= ward) appear in RTT when we use FreeBSD+ipfw+Dummynet as a router with high= Hz value. We did simple experiment (configuration listed below) with six runs and Hz= =3D{1000, 2000, 3000, 3250, 3500, 4000} to demonstrate the issue. I have at= tached two RTT vs Time graphs for our experiments. You can see in these gra= phs everything is smooth when Hz ~ <=3D3000, and the issue starts to appear= when Hz ~>=3D3250. As part of figuring out why this happen, I found that removing C_DIRECT_EXE= C and C_HARDCLOCK flags from callout_reset_sbt() call(in dn_reschedule() fu= nction) causes the issue with high kernel interrupt hz (>3000 in my case) t= o disappear even when kern.hz=3D10000. I did another two experiment (kern.hz=3D10000, with and without the flags m= entioned above) and I logged dn_cfg.curr_time from dummynet_task() (after t= he adjustments code). Then I calculated the difference (delta) between curr= _time[i] and curr_time[i-1] of the log file (the logged dn_cfg.curr_time). = I found that when kern.hz=3D10000 and C_DIRECT_EXEC and C_HARDCLOCK flags a= re set, the delta values vary from 0 to ~600, while when C_DIRECT_EXEC and= C_HARDCLOCK flags are unset, the delta values vary from 1 to ~80. It seems= like the high jumps of delta values when the flags are set causes the issu= e. I have attached a graph that compares delta values for the two experimen= ts. Does anyone have thoughts on what we can test next to narrow down the root-= cause of these unusual timing jumps? Our configurations: Router PC: Intel(R) Core(TM)2 Duo @ 2.33GHz processer, 4GiB RAM. FreeBSD 10.1 (also I tried FreeBSD11-CURRENT-r289420). =20 ipfw+Dummynet: 10mbit traffic shaping , emulated delay 30ms (RTT=3D60ms), q= ueue size is 100 slices, droptail Hosts: Linux 3.17 Traffic: One TCP flow, iperf, TCP NewReno NICs: Intel PRO/1000 GT Testbed experiments under the control of TEACUP (http://caia.swin.edu.au/to= ols/teacup/) Regards, Rasool Al-Saadi From owner-freebsd-net@freebsd.org Tue Nov 3 13:31:58 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7247A2464E for ; Tue, 3 Nov 2015 13:31:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 98E791C99 for ; Tue, 3 Nov 2015 13:31:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 38BA11FE023; Tue, 3 Nov 2015 14:31:54 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Rasool Al-Saadi , "freebsd-net@freebsd.org" References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> From: Hans Petter Selasky Message-ID: <5638B7B5.3030802@selasky.org> Date: Tue, 3 Nov 2015 14:33:41 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 13:31:58 -0000 On 11/03/15 14:14, Rasool Al-Saadi wrote: > Does anyone have thoughts on what we can test next to narrow down the root-cause of these unusual timing jumps? You might also want to test the "projects/hps_head" branch, which uses a bit different callout implementation. --HPS From owner-freebsd-net@freebsd.org Tue Nov 3 14:04:40 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA64DA24DAE for ; Tue, 3 Nov 2015 14:04:40 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3755F1D9D for ; Tue, 3 Nov 2015 14:04:40 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id 079B3A24DAA; Tue, 3 Nov 2015 14:04:40 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E27DFA24DA7; Tue, 3 Nov 2015 14:04:39 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ED001D59; Tue, 3 Nov 2015 14:04:38 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id tA3E4aYE027892; Tue, 3 Nov 2015 06:04:36 -0800 (PST) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id tA3E4aAh027891; Tue, 3 Nov 2015 06:04:36 -0800 (PST) (envelope-from david) Date: Tue, 3 Nov 2015 06:04:36 -0800 From: David Wolfskill To: ipfw@freebsd.org Cc: current@freebsd.org, net@freebsd.org Subject: panic: refcount inconsistency: found: 0 total: 1 Message-ID: <20151103140436.GJ21127@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , ipfw@freebsd.org, current@freebsd.org, net@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="C7PTD44AewjTsiSV" Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 14:04:41 -0000 --C7PTD44AewjTsiSV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable This was on my laptop; yesterday, it built & booted: FreeBSD g1-252.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #230 r2902= 70M/290270:1100085: Mon Nov 2 05:03:07 PST 2015 root@g1-252.catwhisker= =2Eorg:/common/S4/obj/usr/src/sys/CANARY amd64 OK; today, after building: FreeBSD localhost 11.0-CURRENT FreeBSD 11.0-CURRENT #231 r290334M/290334:1= 100086: Tue Nov 3 04:51:24 PST 2015 root@g1-252.catwhisker.org:/common= /S4/obj/usr/src/sys/CANARY amd64 I tried booting it, and during the transition to multi-user mode, once ipfw was being invoked, I got the above-cited panic. Circumvention was to leave it disconnected from a network (turn off the WiFi switch, in my case), so we don't get a chance to use the network. I was able to get a dump by explicitly typing "call doadump" -- an earlier attempt at "panic" didn't capture one. Stack trace: #0 doadump (textdump=3D0) at pcpu.h:221 221 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=3D0) at pcpu.h:221 #1 0xffffffff8037b6b6 in db_fncall (dummy1=3D,=20 dummy2=3D, dummy3=3D,=20 dummy4=3D) at /usr/src/sys/ddb/db_command.c:568 #2 0xffffffff8037b14e in db_command (cmd_table=3D0x0) at /usr/src/sys/ddb/db_command.c:440 #3 0xffffffff8037aee4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 #4 0xffffffff8037d97b in db_trap (type=3D, code=3D0) at /usr/src/sys/ddb/db_main.c:251 #5 0xffffffff80a270f3 in kdb_trap (type=3D3, code=3D0, tf=3D) at /usr/src/sys/kern/subr_kdb.c:654 #6 0xffffffff80db6668 in trap (frame=3D0xfffffe060bdde1d0) at /usr/src/sys/amd64/amd64/trap.c:549 #7 0xffffffff80d961f7 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:234 #8 0xffffffff80a267db in kdb_enter (why=3D0xffffffff812a5566 "panic",=20 msg=3D0x80
) at cpufunc.h:63 #9 0xffffffff809ea01f in vpanic (fmt=3D,=20 ap=3D) at /usr/src/sys/kern/kern_shutdown.c:750 #10 0xffffffff809e9e76 in kassert_panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:647 #11 0xffffffff80c2a788 in ipfw_rewrite_rule_uidx (chain=3D0xffffffff81be531= 0,=20 ci=3D0xfffffe060bdde4b8) at /usr/src/sys/netpfil/ipfw/ip_fw_table.c:3395 #12 0xffffffff80c267c3 in commit_rules (chain=3D0xffffffff81be5310,=20 rci=3D0xfffffe060bdde4b8, count=3D1) at /usr/src/sys/netpfil/ipfw/ip_fw_sockopt.c:678 #13 0xffffffff80c25d80 in add_rules (chain=3D0xffffffff81be5310,=20 op3=3D, sd=3D) at /usr/src/sys/netpfil/ipfw/ip_fw_sockopt.c:2594 #14 0xffffffff80c232f4 in ipfw_ctl3 (sopt=3D0xfffffe060bdde920) at /usr/src/sys/netpfil/ipfw/ip_fw_sockopt.c:3242 #15 0xffffffff80b3d8b1 in rip_ctloutput (so=3D,=20 sopt=3D0xfffffe060bdde920) at /usr/src/sys/netinet/raw_ip.c:588 #16 0xffffffff80a72bc6 in sogetopt (so=3D0xfffff80009e658b8,=20 sopt=3D0xfffffe060bdde920) at /usr/src/sys/kern/uipc_socket.c:2731 #17 0xffffffff80a7729e in kern_getsockopt (td=3D0xfffff800098119a0,=20 s=3D, level=3D,=20 name=3D, val=3D, valseg=3D464= ,=20 valsize=3D0xfffffe060bdde98c) at /usr/src/sys/kern/uipc_syscalls.c:1540 #18 0xffffffff80a771a0 in sys_getsockopt (td=3D0xfffff800098119a0,=20 uap=3D0xfffffe060bddea40) at /usr/src/sys/kern/uipc_syscalls.c:1486 #19 0xffffffff80db7519 in amd64_syscall (td=3D0xfffff800098119a0, traced=3D= 0) at subr_syscall.c:140 #20 0xffffffff80d964db in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:394 #21 0x0000000800b2cbea in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb)=20 I've copied the vmcore.z & core.txt.7 to ; gzipped copies are also available: Index of /~david/FreeBSD/head/ipfw Icon Name Last modified Size Description _____________________________________________________________________ [PARENTDIR] Parent Directory - [TXT] core.txt.7 2015-11-03 05:22 155K [ ] core.txt.7.gz 2015-11-03 05:22 35K [ ] vmcore.7 2015-11-03 05:22 528M [ ] vmcore.7.gz 2015-11-03 05:22 45M _____________________________________________________________________ I'll start taking a closer look at recent changes (e.g., in src/sys/netpfil/ipfw), but I'm not really all that familiar with the code. Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --C7PTD44AewjTsiSV Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWOL70XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7YqwP/3ctxamCUq5Xt5CxCle4HVSG LL3gi+6WEC6FbYMO7SjKdHw/OREkja/mBkNDcI76ps7W+o3xMNW0YwMX5/Wr/aNR T7KTGNkTDK6MP04DimtMscQXqTYVLerd3hOKNoGw/vrGUQZwYbhO9jBlVAoBP2Ax aobqivKD5ajHngRnOKfdGQKucfx75mKNbwypej0w6GnDo0vhkETfwvsYSFIdOvUD JFPSXq5raGJKpjazE6tXvxFi+zN/XHQ2CnYni/1Q8pbuNc17TMBIA73zC0sDbTp4 9hiA22/G7EdAfC+PpRNfx2afwd39yNx5zlTAAKXzpA+12HLRVhX2q3y5xaMDlrCK r82csuRGX+lq0w//I6S2zMKolx9fxj6GOQrg38uAdjgwNaywRdm5V9es9KfmYlBO BTLLY1szyYZOzql8GeVI7D2zXC2jxhZZ8YCWOlJtrvLhcrYYI5tjUdQd88HQSIan Irn9Vrf6cMJwrFw7qiZIzPrCi1QTknkFaq5ybVAA84TG0YA4XGWhC3zeVLpUFMKp v4lRI/tAPhtAShu/zA8u58nqVUJj9Nvg/U2K4JS+nPI2NTrKEOVIeh1LpGM3lZS2 hUTjqnHZNBEBCXgvxcPjNkwL5noT+DUWX8V5fNpnCT8mWCXPYIHge6cLpW0jSO0r kCVxbpPFuJXMOz2KhldA =Xz+w -----END PGP SIGNATURE----- --C7PTD44AewjTsiSV-- From owner-freebsd-net@freebsd.org Tue Nov 3 15:15:41 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2FC9A25DA5 for ; Tue, 3 Nov 2015 15:15:41 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 731D31312 for ; Tue, 3 Nov 2015 15:15:41 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 6D817A25DA2; Tue, 3 Nov 2015 15:15:41 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B4B9A25D9F; Tue, 3 Nov 2015 15:15:41 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward15h.cmail.yandex.net (forward15h.cmail.yandex.net [IPv6:2a02:6b8:0:f35::a0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEC0D130F; Tue, 3 Nov 2015 15:15:40 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from web30h.yandex.ru (web30h.yandex.ru [IPv6:2a02:6b8:0:f05::40]) by forward15h.cmail.yandex.net (Yandex) with ESMTP id 6EEB921380; Tue, 3 Nov 2015 18:15:37 +0300 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web30h.yandex.ru (Yandex) with ESMTP id 5787E26C27EE; Tue, 3 Nov 2015 18:15:36 +0300 (MSK) Received: by web30h.yandex.ru with HTTP; Tue, 03 Nov 2015 18:15:35 +0300 From: Alexander V. Chernikov Envelope-From: melifaro@ipfw.ru To: David Wolfskill , "ipfw@freebsd.org" Cc: "current@freebsd.org" , "net@freebsd.org" In-Reply-To: <20151103140436.GJ21127@albert.catwhisker.org> References: null <20151103140436.GJ21127@albert.catwhisker.org> Subject: Re: panic: refcount inconsistency: found: 0 total: 1 MIME-Version: 1.0 Message-Id: <693401446563735@web30h.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Tue, 03 Nov 2015 18:15:35 +0300 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=koi8-r X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 15:15:41 -0000 03.11.2015, 17:05, "David Wolfskill" : > This was on my laptop; yesterday, it built & booted: > > FreeBSD g1-252.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #230 r290270M/290270:1100085: Mon Nov 2 05:03:07 PST 2015 root@g1-252.catwhisker.org:/common/S4/obj/usr/src/sys/CANARY amd64 > > OK; today, after building: > > FreeBSD localhost 11.0-CURRENT FreeBSD 11.0-CURRENT #231 r290334M/290334:1100086: Tue Nov 3 04:51:24 PST 2015 root@g1-252.catwhisker.org:/common/S4/obj/usr/src/sys/CANARY amd64 > > I tried booting it, and during the transition to multi-user mode, > once ipfw was being invoked, I got the above-cited panic. Circumvention > was to leave it disconnected from a network (turn off the WiFi > switch, in my case), so we don't get a chance to use the network. It is most probably related with r290334. Would you mind reverting it and checking if ipfw works correctly ? > > I was able to get a dump by explicitly typing "call doadump" -- an > earlier attempt at "panic" didn't capture one. Stack trace: > > #0 doadump (textdump=0) at pcpu.h:221 > 221 pcpu.h: No such file or directory. > ššššššššin pcpu.h > (kgdb) #0 doadump (textdump=0) at pcpu.h:221 > #1 0xffffffff8037b6b6 in db_fncall (dummy1=, > ššššdummy2=, dummy3=, > ššššdummy4=) at /usr/src/sys/ddb/db_command.c:568 > #2 0xffffffff8037b14e in db_command (cmd_table=0x0) > ššššat /usr/src/sys/ddb/db_command.c:440 > #3 0xffffffff8037aee4 in db_command_loop () > ššššat /usr/src/sys/ddb/db_command.c:493 > #4 0xffffffff8037d97b in db_trap (type=, code=0) > ššššat /usr/src/sys/ddb/db_main.c:251 > #5 0xffffffff80a270f3 in kdb_trap (type=3, code=0, tf=) > ššššat /usr/src/sys/kern/subr_kdb.c:654 > #6 0xffffffff80db6668 in trap (frame=0xfffffe060bdde1d0) > ššššat /usr/src/sys/amd64/amd64/trap.c:549 > #7 0xffffffff80d961f7 in calltrap () > ššššat /usr/src/sys/amd64/amd64/exception.S:234 > #8 0xffffffff80a267db in kdb_enter (why=0xffffffff812a5566 "panic", > ššššmsg=0x80
) at cpufunc.h:63 > #9 0xffffffff809ea01f in vpanic (fmt=, > ššššap=) at /usr/src/sys/kern/kern_shutdown.c:750 > #10 0xffffffff809e9e76 in kassert_panic (fmt=) > ššššat /usr/src/sys/kern/kern_shutdown.c:647 > #11 0xffffffff80c2a788 in ipfw_rewrite_rule_uidx (chain=0xffffffff81be5310, > ššššci=0xfffffe060bdde4b8) at /usr/src/sys/netpfil/ipfw/ip_fw_table.c:3395 > #12 0xffffffff80c267c3 in commit_rules (chain=0xffffffff81be5310, > ššššrci=0xfffffe060bdde4b8, count=1) > ššššat /usr/src/sys/netpfil/ipfw/ip_fw_sockopt.c:678 > #13 0xffffffff80c25d80 in add_rules (chain=0xffffffff81be5310, > ššššop3=, sd=) > ššššat /usr/src/sys/netpfil/ipfw/ip_fw_sockopt.c:2594 > #14 0xffffffff80c232f4 in ipfw_ctl3 (sopt=0xfffffe060bdde920) > ššššat /usr/src/sys/netpfil/ipfw/ip_fw_sockopt.c:3242 > #15 0xffffffff80b3d8b1 in rip_ctloutput (so=, > ššššsopt=0xfffffe060bdde920) at /usr/src/sys/netinet/raw_ip.c:588 > #16 0xffffffff80a72bc6 in sogetopt (so=0xfffff80009e658b8, > ššššsopt=0xfffffe060bdde920) at /usr/src/sys/kern/uipc_socket.c:2731 > #17 0xffffffff80a7729e in kern_getsockopt (td=0xfffff800098119a0, > ššššs=, level=, > ššššname=, val=, valseg=464, > ššššvalsize=0xfffffe060bdde98c) at /usr/src/sys/kern/uipc_syscalls.c:1540 > #18 0xffffffff80a771a0 in sys_getsockopt (td=0xfffff800098119a0, > ššššuap=0xfffffe060bddea40) at /usr/src/sys/kern/uipc_syscalls.c:1486 > #19 0xffffffff80db7519 in amd64_syscall (td=0xfffff800098119a0, traced=0) > ššššat subr_syscall.c:140 > #20 0xffffffff80d964db in Xfast_syscall () > ššššat /usr/src/sys/amd64/amd64/exception.S:394 > #21 0x0000000800b2cbea in ?? () > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > (kgdb) > > I've copied the vmcore.z & core.txt.7 to > ; gzipped > copies are also available: > > ššššššššššššššššššššIndex of /~david/FreeBSD/head/ipfw > > šIcon Name Last modified Size Description > šš_____________________________________________________________________ > š[PARENTDIR] Parent Directory - > š[TXT] core.txt.7 2015-11-03 05:22 155K > š[ ] core.txt.7.gz 2015-11-03 05:22 35K > š[ ] vmcore.7 2015-11-03 05:22 528M > š[ ] vmcore.7.gz 2015-11-03 05:22 45M > šš_____________________________________________________________________ > > I'll start taking a closer look at recent changes (e.g., in > src/sys/netpfil/ipfw), but I'm not really all that familiar with > the code. > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Those who would murder in the name of God or prophet are blasphemous cowards. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. From owner-freebsd-net@freebsd.org Tue Nov 3 16:39:54 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20C5FA24474 for ; Tue, 3 Nov 2015 16:39:54 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DF20717ED for ; Tue, 3 Nov 2015 16:39:53 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id D9107A24471; Tue, 3 Nov 2015 16:39:53 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE154A2446B; Tue, 3 Nov 2015 16:39:53 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A25417E8; Tue, 3 Nov 2015 16:39:53 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id tA3Gdqki029110; Tue, 3 Nov 2015 08:39:52 -0800 (PST) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id tA3GdqFK029109; Tue, 3 Nov 2015 08:39:52 -0800 (PST) (envelope-from david) Date: Tue, 3 Nov 2015 08:39:52 -0800 From: David Wolfskill To: "Alexander V. Chernikov" Cc: "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" Subject: Re: panic: refcount inconsistency: found: 0 total: 1 Message-ID: <20151103163952.GP21127@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , "Alexander V. Chernikov" , "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" References: <20151103140436.GJ21127@albert.catwhisker.org> <693401446563735@web30h.yandex.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="r/w8vo2lxBmCPGjQ" Content-Disposition: inline In-Reply-To: <693401446563735@web30h.yandex.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 16:39:54 -0000 --r/w8vo2lxBmCPGjQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 03, 2015 at 06:15:35PM +0300, Alexander V. Chernikov wrote: > ... > > I tried booting it, and during the transition to multi-user mode, > > once ipfw was being invoked, I got the above-cited panic. Circumvention > > was to leave it disconnected from a network (turn off the WiFi > > switch, in my case), so we don't get a chance to use the network. > It is most probably related with r290334. Would you mind reverting it and= checking if ipfw works correctly ? > .... OK; after: Script started on Tue Nov 3 08:22:37 2015 g1-252(10.2-S)[1] (cd /S4/usr/src/ && svn diff -c 290334 >/tmp/r290334)^M g1-252(10.2-S)[2] (cd /S4/usr/src/ && svn patch --reverse-diff /tmp/r290334= )^M U sys/netpfil/ipfw/ip_fw_sockopt.c U sys/netpfil/ipfw/ip_fw_table.c g1-252(10.2-S)[3] exit followed by a "make buildkernel", I now have: FreeBSD localhost 11.0-CURRENT FreeBSD 11.0-CURRENT #232 r290334M/290334:1= 100086: Tue Nov 3 08:27:20 PST 2015 root@localhost:/common/S4/obj/usr/= src/sys/CANARY amd64 and localhost(11.0-C)[3] ifconfig wlan0=20 wlan0: flags=3D8843 metric 0 mtu 15= 00 ether 00:24:d6:7a:03:ce inet 198.135.49.33 netmask 0xfffffc00 broadcast 198.135.51.255=20 =2E.. as I type, and IPFW is isn use at a *guess*, we'll need a bit more effort to keep "found" and "ci->object_opcodes" in sync, at least by the time the KASSERT on src/sys/netpfil/ipfw/ip_fw_table.c:3395 looks at the values. Thanks! Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --r/w8vo2lxBmCPGjQ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWOONYXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7VRIP/3i6WSUOZHXNnuYmo3Rj5PFK j9X1xP4PRIEYk5nKrrevEEvm7P4BfQ3snhK5dBPXbHwweIjYnzk0wdet38MKKX1u FSK+gJ0/asN0F2kCmudkLyZ3Es63ZdMMEKNRTrVyJQuVejSLLXCPfUJkEs9MSj/K H/TrqhRFMX3yw6PDuN7mumqVeiawylTKz3Cb5qFNcrJSRiY+1WxQRNQ5QJQCGSXx zo3ecXqmzjMm0eijrYV7TyIJ0HJKKa4vIsWu0wpejoBhBP8rJLyXwY26QZxS9WBE jvCApt2zkutYPN2uoq7w6V+Fsk6lN/t5TbQvtneUI2qMHjprRhfJ531i95RCYd98 wWaLe35WUXAA4OdE+zocscd8JF4+2Gn3Ug/Io5c+1IjKY1Tpgm2euQyWy+VD4DJ2 CzlCpwm+HN9REhK1iHfvXeEqT6LEQWd9KS14/AbUzXmXtyq9e/1OAWz/kPPERDQU 6mwSFGeoC+DDGrHyD+OQEbqLOeq+wWPy0dql8n53Y1CDuG/wthTjsxHvnF/4uANw Mv3P8JRVUVHLKscqCIAjqgW9W2wY+VaTG8LUS2cx0drEAlmoamXs+oD+P54HP4tk VDXMIYSJ8UQ3N3KXRTlkMFcGKiQQxososZaTpQiuhG9mEIXQN+WvnzmggoaPo2PU LqUrt3IGlvNmx80daG7F =Tmh8 -----END PGP SIGNATURE----- --r/w8vo2lxBmCPGjQ-- From owner-freebsd-net@freebsd.org Tue Nov 3 21:10:06 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53664A25C96 for ; Tue, 3 Nov 2015 21:10:06 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 382F21033 for ; Tue, 3 Nov 2015 21:10:06 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id 30C92A25C90; Tue, 3 Nov 2015 21:10:06 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FF63A25C8B; Tue, 3 Nov 2015 21:10:06 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 75BDE102E; Tue, 3 Nov 2015 21:10:04 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id tA3LA1L6031313; Tue, 3 Nov 2015 13:10:01 -0800 (PST) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id tA3LA16e031312; Tue, 3 Nov 2015 13:10:01 -0800 (PST) (envelope-from david) Date: Tue, 3 Nov 2015 13:10:01 -0800 From: David Wolfskill To: "Alexander V. Chernikov" , "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" Subject: Re: panic: refcount inconsistency: found: 0 total: 1 Message-ID: <20151103211001.GT21127@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , "Alexander V. Chernikov" , "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" References: <20151103140436.GJ21127@albert.catwhisker.org> <693401446563735@web30h.yandex.ru> <20151103163952.GP21127@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="lpUp1egui7PDlNtH" Content-Disposition: inline In-Reply-To: <20151103163952.GP21127@albert.catwhisker.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 21:10:06 -0000 --lpUp1egui7PDlNtH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 03, 2015 at 08:39:52AM -0800, David Wolfskill wrote: > On Tue, Nov 03, 2015 at 06:15:35PM +0300, Alexander V. Chernikov wrote: > > ... > > > I tried booting it, and during the transition to multi-user mode, > > > once ipfw was being invoked, I got the above-cited panic. Circumventi= on > > > was to leave it disconnected from a network (turn off the WiFi > > > switch, in my case), so we don't get a chance to use the network. > > It is most probably related with r290334. Would you mind reverting it a= nd checking if ipfw works correctly ? > > .... >=20 > ... [Reverting r290334 gets things working again -- dhw] >=20 > at a *guess*, we'll need a bit more effort to keep "found" and > "ci->object_opcodes" in sync, at least by the time the KASSERT on > src/sys/netpfil/ipfw/ip_fw_table.c:3395 looks at the values. > ... So... looking at the code a bit (and bearing in mind that I still am unfamiliar with said code, and I hack more than I "write" code)... here's the original bit of src/sys/netpfil/ipfw/ip_fw_table.c in question, with the modification from r290334 shown: =2E.. /* * Finds and bumps refcount for objects referenced by given @rule. * Auto-creates non-existing tables. * Fills in @oib array with userland/kernel indexes. * * Returns 0 on success. */ static int ref_rule_objects(struct ip_fw_chain *ch, struct ip_fw *rule, struct rule_check_info *ci, struct obj_idx *oib, struct tid_info *ti) { =2E.. if (error !=3D 0) { /* Unref everything we have already done */ unref_oib_objects(ch, rule->cmd, oib, pidx); IPFW_UH_WUNLOCK(ch); return (error); } IPFW_UH_WUNLOCK(ch); found =3D pidx - oib; KASSERT(found =3D=3D ci->object_opcodes, ("refcount inconsistency: found: %d total: %d", found, ci->object_opcodes)); =20 /* Perform auto-creation for non-existing objects */ if (numnew !=3D 0) error =3D create_objects_compat(ch, rule->cmd, oib, pidx, t= i); =20 + /* Calculate real number of dynamic objects */ + ci->object_opcodes =3D (uint16_t)(pidx - oib); + return (error); } =2E.. The added code to "Calculate real number of dynamic objects" is apparently setting ci->object_opcodes to the same value that "found" was just set to (pidx - oib -- cast to uint16_t in the case of the added code). But that appears to come a bit late for the KASSERT. Moving the KASSERT relative to the added code (so the KASSERT comes after) would be an option, but I'm not sure it makes sense to actually do the KASSERT unless we have some reason to believe that the difference between pidx and oib might change in the interval represented by a couple lines of code AND we have not coded to handle that situation any more "gracefully" than to panic. So... with the change from r290334, what's the point of the KASSERT? Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --lpUp1egui7PDlNtH Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWOSKpXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7M3QP/RwN2RNOPwqO6jVxTs4fJa5e Z+53TPcDIL1EAtGS9/s5ENOduCEFFbDH/4rBg7HhP7mtRtpucWINRcel8/XF/ydU hRy+NAr/IncpTU/wWuq+af0FpwPiBWERSK92mneWoihrgpGyeDAYRhYftbkg9OkE OKlVYvXnZt6KPciC1ib6JZXvKKrQMKROHrjNmjnMUQe+q1ZuqvuW5oikTB8yesJa 1zQqUMDFdenIrzVv/9nINbCjAZtVNhdbsSfg2OfEnv6WpZE12DJkIKKlNvOH8w5n cfrfLeaAvHHif/CUUb8NO15E8D0GdXmDkAUUKi82BEldB337o6lbxWOffBlAym2L iY2Z0FUiUgCo04FMMlARKBu1ipuWMUSB6H4D15r28bdCXQWiprpT7fSZiBTph5jv fJnu/mkxTEZfsste7JfJJkGPG8EOEY2QhnYeDkdyvrDvXcDUHeuSySPHcea3SW1B t7w2gvjONqmInQ/pD3pF1N5BD8pNKPu6qIfFujNrl/TDBdsihjYNcZIn4VwfM5KU gocdPXhusTjbzO/BkWXqs1SddcZtrQHnY6WZKmKK5tusQuhujOlmy+NJlx6ZjU8A fY07aYdRouIbu2MUPbrCeeyNY/axh3NTPZXnA8U3goUbHtGMqijTUxBhiWYzNOPG VEg8C/HMAia4t4HrsEGV =B1g6 -----END PGP SIGNATURE----- --lpUp1egui7PDlNtH-- From owner-freebsd-net@freebsd.org Tue Nov 3 22:25:45 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42081A2508A for ; Tue, 3 Nov 2015 22:25:45 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1C1131034 for ; Tue, 3 Nov 2015 22:25:45 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 16021A25086; Tue, 3 Nov 2015 22:25:45 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 152F7A25084; Tue, 3 Nov 2015 22:25:45 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward13j.cmail.yandex.net (forward13j.cmail.yandex.net [5.255.227.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B8B9F1032; Tue, 3 Nov 2015 22:25:44 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp14.mail.yandex.net (smtp14.mail.yandex.net [IPv6:2a02:6b8:0:801:1::13]) by forward13j.cmail.yandex.net (Yandex) with ESMTP id 02F08217BC; Wed, 4 Nov 2015 01:25:39 +0300 (MSK) Received: from smtp14.mail.yandex.net (localhost [127.0.0.1]) by smtp14.mail.yandex.net (Yandex) with ESMTP id 196781B603EF; Wed, 4 Nov 2015 01:25:38 +0300 (MSK) Received: by smtp14.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id EQzKOFqcSY-Pc24eGDs; Wed, 4 Nov 2015 01:25:38 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1446589538; bh=Tx0Fm1r5SO7kVas2ANGmmPSDsQ+/YIUunixGAth8LbQ=; h=Subject:To:References:From:Message-ID:Date:User-Agent: MIME-Version:In-Reply-To:Content-Type; b=ULJ+36GVO2hu6EFCmND5ix/iEiGvhiRFjQp2GK6y5OP2K98LZVvMb7PFWxSqQ+P16 Bdtk9jCNru6sNTnFY48Do1MxR1IeJW5VWeijFtq7WH3wGU4Asz1IIXi3671mRub9aV Xl6qGHtIhlcEF8W5rESVjoc3tSmQOmXWiDeGIzeM= Authentication-Results: smtp14.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-ForeignMX: US Subject: Re: panic: refcount inconsistency: found: 0 total: 1 To: David Wolfskill , "Alexander V. Chernikov" , "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" References: <20151103140436.GJ21127@albert.catwhisker.org> <693401446563735@web30h.yandex.ru> <20151103163952.GP21127@albert.catwhisker.org> <20151103211001.GT21127@albert.catwhisker.org> From: "Andrey V. Elsukov" Message-ID: <563933EE.1050909@yandex.ru> Date: Wed, 4 Nov 2015 01:23:42 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20151103211001.GT21127@albert.catwhisker.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="haPd523UHhOx2ICdN2cukHB5n3APTq374" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 22:25:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --haPd523UHhOx2ICdN2cukHB5n3APTq374 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 04.11.2015 00:10, David Wolfskill wrote: > So... with the change from r290334, what's the point of the KASSERT? Yes, you are right. We changed this code and use it some time, but without INVARIANTS. I just removed this KASSERT in r290345. Can you try this revision? Sorry for the breakage. --=20 WBR, Andrey V. Elsukov --haPd523UHhOx2ICdN2cukHB5n3APTq374 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJWOTPuAAoJEAHF6gQQyKF6kP4H/iymjkU5mWjJ9OsL5FLGaR94 Ai7qyV0gAHvVgl6Wj0Te5uYDoO7XyHSzGSQQwE3EFlfQpouyzveXgAqOQl82KZe3 AZh8DZToiV19S4HT1bgNgb/QKD1wrdEywCCiJ9hZtAOMa0nT4nZAihr7FpPSFkBz V7ML2aKlgUJ1P5GhweHWTblu9jFhthm/E6OzPgvkL8AySEXx+hH2YcLvRS3k1WsC 2NW24HRspWPjkKrd4gCnXCs5j4bTvfQzZ3QW2Kf15ckrnIamvuYngd3X+SrjVZwf DVkOrPcENvAhDAUlF8oTUiH70f3V5p8loDUcBciJbmnzfbMG94DHcmTOusiRtJs= =1JOj -----END PGP SIGNATURE----- --haPd523UHhOx2ICdN2cukHB5n3APTq374-- From owner-freebsd-net@freebsd.org Wed Nov 4 00:30:10 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 054CEA2695F for ; Wed, 4 Nov 2015 00:30:10 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.strugglingcoder.info", Issuer "mail.strugglingcoder.info" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E4DB51787 for ; Wed, 4 Nov 2015 00:30:09 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 41F94E410; Tue, 3 Nov 2015 16:30:03 -0800 (PST) Date: Tue, 3 Nov 2015 16:30:03 -0800 From: hiren panchasara To: Hans Petter Selasky Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" Subject: Re: Timing issue with Dummynet on high kernel timer interrupt Message-ID: <20151104003003.GA90595@strugglingcoder.info> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="T4sUOijqQbZv57TR" Content-Disposition: inline In-Reply-To: <5638B7B5.3030802@selasky.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 00:30:10 -0000 --T4sUOijqQbZv57TR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 11/03/15 at 02:33P, Hans Petter Selasky wrote: > On 11/03/15 14:14, Rasool Al-Saadi wrote: > > Does anyone have thoughts on what we can test next to narrow down the r= oot-cause of these unusual timing jumps? >=20 > You might also want to test the "projects/hps_head" branch, which uses a= =20 > bit different callout implementation. Rasool, Nice findings. Thank you for your work. Making ipfw/dummynet work well under higher freq is much needed and very useful. Hans, Does the branch fix (or help) with the problems reported by Rasool? If so, is it possible for you to have a patch that can be applied against -head? Unsure about others but it'd be easier for someone like me to test.= =20 Cheers, Hiren --T4sUOijqQbZv57TR Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWOVGIXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lEMYH/0my9OwMDF8VksN8pVLSaODU i2zQJ5bt0pGWlEVguAVvm9SN99z1DsCOgI8YLmxsIxrhKgK1ner0GeqoXw+D9XxR mXD7Ns7t2eGKpw8xV1rFt42ZfTfCGUUbIAO+vFdYN3MEvMVXITA59jadjgiZFvDP RemXjujDpOqjTJ9Makh5cAeHrCD8RbBRtHcr5QnxCh/PhpS246s2hsVd5UuXdZvh VjdB2Rz4z8P9bofBW4qWxVZL+Ca7zUMXzpLTfYNquSG5/wDnlaOrvr90UB8prEF5 TcJuPWOk3uu9ZOufSTM9gsexN9Mus+/SVKZvhwFLoAKMk7NysGY/pZqv8iGPYzk= =kDV0 -----END PGP SIGNATURE----- --T4sUOijqQbZv57TR-- From owner-freebsd-net@freebsd.org Wed Nov 4 00:40:40 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7E05A26AA4 for ; Wed, 4 Nov 2015 00:40:40 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id 4E6E21BE8 for ; Wed, 4 Nov 2015 00:40:39 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.20,240,1444654800"; d="scan'208";a="14531143" Received: from gsp-ex03.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.19]) by iport1.cc.swin.edu.au with ESMTP; 04 Nov 2015 11:40:39 +1100 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.2]) by gsp-ex03.ds.swin.edu.au ([169.254.3.127]) with mapi id 14.03.0248.002; Wed, 4 Nov 2015 11:40:38 +1100 From: Rasool Al-Saadi To: "freebsd-net@freebsd.org" Subject: RE: Timing issue with Dummynet on high kernel timer interrupt Thread-Topic: Timing issue with Dummynet on high kernel timer interrupt Thread-Index: AdEWOVE5DoAlkzqPQWaBZ8ln4esVdwAXiYQA Date: Wed, 4 Nov 2015 00:40:37 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C0DD112@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.126.11] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 00:40:40 -0000 > I have > attached two RTT vs Time graphs for our experiments.=20 Oops, the list stripped my attachments, here are the images online at https= ://goo.gl/photos/3oxXFvr1T1ZC88PA8 and https://goo.gl/photos/QM1tQL157w6NRd= mE6 > I have attached a graph that compares delta > values for the two experiments. And also this one at https://goo.gl/photos/vBuKE5PskvkwQber9 Regards, Rasool From owner-freebsd-net@freebsd.org Wed Nov 4 02:33:28 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BBDDA25EAD for ; Wed, 4 Nov 2015 02:33:28 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 51FA618C4 for ; Wed, 4 Nov 2015 02:33:28 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4F45AA25EAA; Wed, 4 Nov 2015 02:33:28 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3336BA25EA5; Wed, 4 Nov 2015 02:33:28 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 866DF18BE; Wed, 4 Nov 2015 02:33:26 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id tA42XJmB033986; Tue, 3 Nov 2015 18:33:19 -0800 (PST) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id tA42XJce033985; Tue, 3 Nov 2015 18:33:19 -0800 (PST) (envelope-from david) Date: Tue, 3 Nov 2015 18:33:19 -0800 From: David Wolfskill To: "Andrey V. Elsukov" Cc: "Alexander V. Chernikov" , "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" Subject: Re: panic: refcount inconsistency: found: 0 total: 1 Message-ID: <20151104023319.GZ21127@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , "Andrey V. Elsukov" , "Alexander V. Chernikov" , "ipfw@freebsd.org" , "current@freebsd.org" , "net@freebsd.org" References: <20151103140436.GJ21127@albert.catwhisker.org> <693401446563735@web30h.yandex.ru> <20151103163952.GP21127@albert.catwhisker.org> <20151103211001.GT21127@albert.catwhisker.org> <563933EE.1050909@yandex.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6ysXqiu0yoUmUNJB" Content-Disposition: inline In-Reply-To: <563933EE.1050909@yandex.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 02:33:28 -0000 --6ysXqiu0yoUmUNJB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 04, 2015 at 01:23:42AM +0300, Andrey V. Elsukov wrote: > On 04.11.2015 00:10, David Wolfskill wrote: > > So... with the change from r290334, what's the point of the KASSERT? >=20 > Yes, you are right. We changed this code and use it some time, but > without INVARIANTS. I just removed this KASSERT in r290345. Can you try > this revision? Sorry for the breakage. > .... OK; I first did "svn revert /usr/src/sys/netpfil/ipfw/", then (having saved a copy of r290345's commit message as /tmp/r290345), did "svn patch --strip 1 /tmp/r290345 /usr/src", then built & installed a new kernel, which booted without a panic: FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #233 r290334M/290334:1100086: T= ue Nov 3 17:18:26 PST 2015 root@localhost:/common/S4/obj/usr/src/sys/C= ANARY amd64 I was un the shuttle bus at the time; apparently the DHCP server in the access point wasn't behaving itself, as I got a DHCPNAK from it -- running either head or stable/10. I booted up head here at home, and DHCP works fine: FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #233 r290334M/290334:1100086: T= ue Nov 3 17:18:26 PST 2015 root@localhost:/common/S4/obj/usr/src/sys/C= ANARY amd64 So: looks like awin to me! :-) Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --6ysXqiu0yoUmUNJB Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWOW5vXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7FtgP/1VtwOEkCY2jQQm02mALkh6M vjUaVhdV8EUw3u3Y7ziuuCDEsnSb2raDl0TQDa3pPcRWjRkPGfHpA1Ep1XbIOMLl QvckSq5i9QcKXHBkpxWrm1xXfIvE2tQOM3eJfJWGNH2+eGA5+6qaH1wANqYOEAV3 /oqZixR5KXm9PiowXs/QHWjfxHw2hi3dm51fzwNWurTdtJKvilNeHsdaxiDiL00/ Obnmrv/tIc7OLU/dOWNUtj7TdBjGKeCgUGtsNXI7UoFi9JIE83X8/3VTjvnH1RD5 CyCjXZrPL2Gfu1y7ZAZ0QWiSRY+foug5FKmZcNw68BMljk828eMyBSVLTkbPDsV5 k6QD5ZDZIIFTe3udyPzPUyTjoxEbH/mXONLHlcRt0ZMW4C38hXO6aRhSu+wd/FIH LVOqFB8OlJvWiMqJu5HIQrw53c5tACCbyrxdd3vmr5h3BW009+4bbuzp78juLrKd JKC6YkI9DSHp0Gui9ucQ7fXZZun6VAaUG7NwZnicwPbj2e3FSEHd4/GDIPIh4A7y goTIZFqXuWV2ENGa7tY7k6IdFzA7YEwEA96kiGsbb0IOF4ZF699P4L3l3cuwn9GX welGKUE4+L0zr5eElJ7qfdBr5rSDBAJWGYqWcDHwJYe6v3RWnpnID4aYaa0zCOb/ VY5VDVt0I++PdAcyMRZO =Wrhk -----END PGP SIGNATURE----- --6ysXqiu0yoUmUNJB-- From owner-freebsd-net@freebsd.org Wed Nov 4 07:55:01 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D9F6A26A00 for ; Wed, 4 Nov 2015 07:55:01 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id AF2851592 for ; Wed, 4 Nov 2015 07:54:59 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38943796 for freebsd-net@freebsd.org; Wed, 04 Nov 2015 13:54:56 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA47ss1U000709 for ; Wed, 4 Nov 2015 13:54:56 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA47ssVu000708 for freebsd-net@freebsd.org; Wed, 4 Nov 2015 13:54:54 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Wed, 4 Nov 2015 13:54:54 +0600 From: Victor Sudakov To: freebsd-net@freebsd.org Subject: tap(4) and host-only networking between host and guest Message-ID: <20151104075454.GA99850@admin.sibptus.tomsk.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 07:55:01 -0000 Colleagues, I am experimenting with bhyve which uses tap(4) for network access. I don't want to bridge tap0 with any of the hosts's real NICs. How can I create a private network just between the host and the guest? I even tried to create a lo1 interface on the host and bridge it with tap0 but lo1 would not add as a bridge member even if I adjust its MTU. root@vas:/etc # ifconfig bridge0 create root@vas:/etc # ifconfig lo1 mtu 1500 up root@vas:/etc # ifconfig bridge0 addm tap0 root@vas:/etc # ifconfig bridge0 addm lo1 ifconfig: BRDGADD lo1: Invalid argument root@vas:/etc # root@vas:/etc # ifconfig bridge0 bridge0: flags=8802 metric 0 mtu 1500 ether 02:8f:fd:16:50:00 nd6 options=9 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0 member: tap0 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 root@vas:/etc # Any help is appreciated. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Wed Nov 4 10:42:45 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98ED5A26763 for ; Wed, 4 Nov 2015 10:42:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8520513C6 for ; Wed, 4 Nov 2015 10:42:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA4Agj3O084733 for ; Wed, 4 Nov 2015 10:42:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 203630] [Hyper-V] [nat] [tcp] 10.2 NAT bug in TCP stack or hyperv netsvc driver Date: Wed, 04 Nov 2015 10:42:43 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: weh@microsoft.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 10:42:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203630 Wei Hu changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162011|0 |1 is obsolete| | Attachment #162111|0 |1 is obsolete| | --- Comment #14 from Wei Hu --- Created attachment 162763 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=162763&action=edit Fix a checksum offloading bug in Hyper-V netvsc driver. Do not calculate TCP checksum when the receiving bits in csum_flags are set. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-net@freebsd.org Wed Nov 4 10:44:50 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B312A26846 for ; Wed, 4 Nov 2015 10:44:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 87DA7185D for ; Wed, 4 Nov 2015 10:44:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA4Aiowa087877 for ; Wed, 4 Nov 2015 10:44:50 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 203630] [Hyper-V] [nat] [tcp] 10.2 NAT bug in TCP stack or hyperv netsvc driver Date: Wed, 04 Nov 2015 10:44:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: weh@microsoft.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 10:44:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203630 --- Comment #15 from Wei Hu --- Sorry for the late response. We still cannot reproduce the issue, but another customer reported the same issue and found a bug in the Hyper-V checksum path. Attached is a patch to fix this issue. Please apply on a clean 10.2 kernel, rebuild and see if this fixes the problem you are seeing. Let us know if this works or not. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-net@freebsd.org Wed Nov 4 12:50:39 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1E39A24269 for ; Wed, 4 Nov 2015 12:50:39 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD962142B for ; Wed, 4 Nov 2015 12:50:39 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-229-78.lns20.per1.internode.on.net [121.45.229.78]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id tA4CoXjj047388 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 4 Nov 2015 04:50:36 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: tap(4) and host-only networking between host and guest To: Victor Sudakov , freebsd-net@freebsd.org References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> From: Julian Elischer Message-ID: <5639FF12.1020109@freebsd.org> Date: Wed, 4 Nov 2015 20:50:26 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151104075454.GA99850@admin.sibptus.tomsk.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 12:50:40 -0000 On 11/4/15 3:54 PM, Victor Sudakov wrote: > Colleagues, > > I am experimenting with bhyve which uses tap(4) for network access. > > I don't want to bridge tap0 with any of the hosts's real NICs. How can > I create a private network just between the host and the guest? you are thinking too hard! tap IS the interface.. ifconfig tap0 $address... and in the VM, ifconfig vtnet0 ${some_other_address} > > I even tried to create a lo1 interface on the host and bridge it with > tap0 but lo1 would not add as a bridge member even if I adjust its MTU. > > > root@vas:/etc # ifconfig bridge0 create > root@vas:/etc # ifconfig lo1 mtu 1500 up > root@vas:/etc # ifconfig bridge0 addm tap0 > root@vas:/etc # ifconfig bridge0 addm lo1 > ifconfig: BRDGADD lo1: Invalid argument > root@vas:/etc # > root@vas:/etc # ifconfig bridge0 > bridge0: flags=8802 metric 0 mtu 1500 > ether 02:8f:fd:16:50:00 > nd6 options=9 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0 > member: tap0 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > root@vas:/etc # > > Any help is appreciated. > From owner-freebsd-net@freebsd.org Wed Nov 4 13:12:37 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A005A2454F for ; Wed, 4 Nov 2015 13:12:37 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id C24D71D9A; Wed, 4 Nov 2015 13:12:35 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38944240; Wed, 04 Nov 2015 19:12:31 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA4DCUAo003479; Wed, 4 Nov 2015 19:12:31 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA4DCUYm003478; Wed, 4 Nov 2015 19:12:30 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Wed, 4 Nov 2015 19:12:30 +0600 From: Victor Sudakov To: Julian Elischer Cc: freebsd-net@freebsd.org Subject: Re: tap(4) and host-only networking between host and guest Message-ID: <20151104131230.GA1117@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5639FF12.1020109@freebsd.org> Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 13:12:37 -0000 Julian Elischer wrote: > > > > I am experimenting with bhyve which uses tap(4) for network access. > > > > I don't want to bridge tap0 with any of the hosts's real NICs. How can > > I create a private network just between the host and the guest? > you are thinking too hard! > > tap IS the interface.. > > ifconfig tap0 $address... > and in the VM, ifconfig vtnet0 ${some_other_address} Thank you, Julian! It works. I felt I was missing something obvious. And if I need the VMs to talk to one another, I think I can bridge several tapN devices on the host. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Wed Nov 4 13:49:21 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB5D4A267B6 for ; Wed, 4 Nov 2015 13:49:21 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70E891AD5 for ; Wed, 4 Nov 2015 13:49:20 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 8639B200345 for ; Wed, 4 Nov 2015 14:43:57 +0100 (CET) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 76B90405884 for ; Wed, 4 Nov 2015 14:43:57 +0100 (CET) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id 4D9BE405882 for ; Wed, 4 Nov 2015 14:43:57 +0100 (CET) Received: from arc.aei.uni-hannover.de ([10.117.15.110]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6HF1691) with ESMTP id 2015110414435443-370865 ; Wed, 4 Nov 2015 14:43:54 +0100 Date: Wed, 4 Nov 2015 14:43:54 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-net@freebsd.org Subject: strange nfs/rsync stalls Message-Id: <20151104144354.acccf3c7315a6009264d755d@aei.mpg.de> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 04.11.2015 14:43:54, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 04.11.2015 14:43:56, Serialize complete at 04.11.2015 14:43:56 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2015.11.4.133616 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1000_LESS 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, BODY_SIZE_900_999 0, NO_CTA_URI_FOUND 0, NO_URI_FOUND 0, NO_URI_HTTPS 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_START 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 13:49:21 -0000 Hi all, I see some weird (I think) NFS issue here: I have a NFS server running 10.2-RELEASE exporting a directory to a NFS client running 10.2-STABLE (as of yesterday, tried that update from 10.1 to see if it would fix this issue, but it stayed with me). I use rsync on the client to copy data from local disks to nfs. This appears to work for a few minutes, then the rsync process gets stuck in state "NEWNFS". Messages "newnfs server servername:/data/systems: not responding" start appearing and get repeated over and over. What I think is strange: I cannot see any issue apart from these rsync jobs getting stuck. I can still access the nfs-mounted data on the client just fine in a different shell. I can read, write, do whatever I like, but the rsync job never moves on. I have a couple of other machines that use the same server in the same way, none of them ever showed this behaviour. Are there any suggestions how to debug/fix this? cu Gerrit From owner-freebsd-net@freebsd.org Wed Nov 4 14:13:28 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 567E7A26CB3 for ; Wed, 4 Nov 2015 14:13:28 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E1A641852 for ; Wed, 4 Nov 2015 14:13:27 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id tA4ED9ju067847 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 4 Nov 2015 15:13:09 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: vas@mpeks.tomsk.su Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id tA4ECwSA069608 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 4 Nov 2015 21:12:59 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: tap(4) and host-only networking between host and guest To: Victor Sudakov References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> Cc: freebsd-net@freebsd.org From: Eugene Grosbein Message-ID: <563A1265.7010208@grosbein.net> Date: Wed, 4 Nov 2015 21:12:53 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151104131230.GA1117@admin.sibptus.tomsk.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 14:13:28 -0000 04.11.2015 20:12, Victor Sudakov ΠΏΠΈΡˆΠ΅Ρ‚: > And if I need the VMs to talk to one another, I think I can bridge > several tapN devices on the host. You could also just use broader IP network and single tap interface for several VMs. From owner-freebsd-net@freebsd.org Wed Nov 4 15:34:22 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50DDFA26ABE for ; Wed, 4 Nov 2015 15:34:22 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id B81791D6A for ; Wed, 4 Nov 2015 15:34:20 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38944460; Wed, 04 Nov 2015 21:34:18 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA4FYF7e004647; Wed, 4 Nov 2015 21:34:16 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA4FYF4h004646; Wed, 4 Nov 2015 21:34:15 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Wed, 4 Nov 2015 21:34:15 +0600 From: Victor Sudakov To: Eugene Grosbein Cc: freebsd-net@freebsd.org Subject: Re: tap(4) and host-only networking between host and guest Message-ID: <20151104153415.GB1117@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <563A1265.7010208@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <563A1265.7010208@grosbein.net> Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 15:34:22 -0000 Eugene Grosbein wrote: > > > And if I need the VMs to talk to one another, I think I can bridge > > several tapN devices on the host. > > You could also just use broader IP network and single tap interface > for several VMs. Do you mean to say I can run several bhyve instances on the same tap0 device? Like, with the identical "-s 3,virtio-net,tap0" parameter? tap(4) says that tapN is an exclusive-open device. And when I tried it out, the second bhyve instances said "open of tap device /dev/tap0 failed", which was kind of expected. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Wed Nov 4 16:54:58 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8979A26CD4 for ; Wed, 4 Nov 2015 16:54:58 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from smtp-out.sentor.se (smtp-out.sentor.se [176.124.225.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9FA7E1069 for ; Wed, 4 Nov 2015 16:54:58 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from localhost (localhost [127.0.0.1]) by farmermaggot.shire.sentor.se (Postfix) with ESMTP id C9C96B61D235 for ; Wed, 4 Nov 2015 17:44:56 +0100 (CET) Date: Wed, 4 Nov 2015 17:44:56 +0100 (CET) From: elof2@sentor.se To: freebsd-net Subject: netstat -B "Recv" Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 16:54:58 -0000 Hi! Question: What do the Recv column in 'netstat -B' show? I thought it was tha amount of packets received, but appaently not so. I send 2000000 packets from a tcpreplay machine to a receiving machine. I do it a few times. On the receiver I see: netstat -in Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 ix1 1500 0c:c4:7a:58:e2:3d 6000000 0 0 0 0 0 and then netstat -in Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 ix1 1500 0c:c4:7a:58:e2:3d 8000000 0 0 0 0 0 So 6000000 has increased to 8000000. Good. However, 'netstat -B' show: Pid Netif Flags Recv Drop Match Sblen Hblen Command 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump How can the "Recv" be *lower* than "Match"? 1996862 < 2000000. For every new run (fast and slow) I get the same results, slightly less than 2000000 Recv. What am I missing? /Elof From owner-freebsd-net@freebsd.org Wed Nov 4 18:45:10 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B238A2042E for ; Wed, 4 Nov 2015 18:45:10 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id A941B18F9 for ; Wed, 4 Nov 2015 18:45:08 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38944822 for freebsd-net@freebsd.org; Thu, 05 Nov 2015 00:45:07 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA4Ij4Rr006373 for ; Thu, 5 Nov 2015 00:45:06 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA4Ij4PW006372 for freebsd-net@freebsd.org; Thu, 5 Nov 2015 00:45:04 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Thu, 5 Nov 2015 00:45:03 +0600 From: Victor Sudakov To: freebsd-net@freebsd.org Subject: Re: tap(4) and host-only networking between host and guest Message-ID: <20151104184503.GC1117@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151104131230.GA1117@admin.sibptus.tomsk.ru> Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 18:45:10 -0000 Victor Sudakov wrote: > Julian Elischer wrote: > > > > > > I am experimenting with bhyve which uses tap(4) for network access. > > > > > > I don't want to bridge tap0 with any of the hosts's real NICs. How can > > > I create a private network just between the host and the guest? > > you are thinking too hard! > > > > tap IS the interface.. > > > > ifconfig tap0 $address... > > and in the VM, ifconfig vtnet0 ${some_other_address} > > Thank you, Julian! It works. I felt I was missing something obvious. For some reason, after a guest is shutdown or rebooted, the IP address on the host's tap0 interface is deleted. It's kind of inconvenient. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Wed Nov 4 19:20:38 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC595A20BC2 for ; Wed, 4 Nov 2015 19:20:38 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail2.jnielsen.net (webmail2.jnielsen.net [50.114.224.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "webmail2.jnielsen.net", Issuer "freebsdsolutions.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E4421A38 for ; Wed, 4 Nov 2015 19:20:38 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from [10.10.1.196] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by webmail2.jnielsen.net (8.15.2/8.15.1) with ESMTPSA id tA4J7thT006184 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 4 Nov 2015 12:07:57 -0700 (MST) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail2.jnielsen.net: Host office.betterlinux.com [199.58.199.60] claimed to be [10.10.1.196] Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: tap(4) and host-only networking between host and guest From: John Nielsen In-Reply-To: <20151104184503.GC1117@admin.sibptus.tomsk.ru> Date: Wed, 4 Nov 2015 12:07:55 -0700 Cc: freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> To: Victor Sudakov X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 19:20:38 -0000 On Nov 4, 2015, at 11:45 AM, Victor Sudakov wrote: > Victor Sudakov wrote: >> Julian Elischer wrote: >>>>=20 >>>> I am experimenting with bhyve which uses tap(4) for network access. >>>>=20 >>>> I don't want to bridge tap0 with any of the hosts's real NICs. How = can >>>> I create a private network just between the host and the guest? >>> you are thinking too hard! >>>=20 >>> tap IS the interface.. >>>=20 >>> ifconfig tap0 $address... >>> and in the VM, ifconfig vtnet0 ${some_other_address} >>=20 >> Thank you, Julian! It works. I felt I was missing something obvious.=20= >=20 > For some reason, after a guest is shutdown or rebooted, the IP address > on the host's tap0 interface is deleted. >=20 > It's kind of inconvenient.=20 What I have done in this scenario is create a bridge interface, assign = the host=E2=80=99s IP to the bridge, and add the tap as a member to the = bridge.= From owner-freebsd-net@freebsd.org Wed Nov 4 19:34:52 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8F01A20F44 for ; Wed, 4 Nov 2015 19:34:52 +0000 (UTC) (envelope-from csjp@sqrt.ca) Received: from mx01.sqrt.ca (i4hg.x.rootbsd.net [208.79.81.26]) by mx1.freebsd.org (Postfix) with ESMTP id 0BFB012CF for ; Wed, 4 Nov 2015 19:34:51 +0000 (UTC) (envelope-from csjp@sqrt.ca) Received: from [192.168.0.2] (unknown [24.114.96.106]) by mx01.sqrt.ca (Postfix) with ESMTPSA id D9830B828; Wed, 4 Nov 2015 14:28:10 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.0 \(3094\)) Subject: Re: netstat -B "Recv" From: Christian Peron In-Reply-To: Date: Wed, 4 Nov 2015 13:28:50 -0600 Cc: freebsd-net Content-Transfer-Encoding: quoted-printable Message-Id: <04EF5880-800D-4A52-8CD1-DAF87EB6E37A@sqrt.ca> References: To: elof2@sentor.se X-Mailer: Apple Mail (2.3094) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 19:34:53 -0000 Your assumptions are correct, recv count is the number of packets which = are received by the bpf peer, match count is the number of packets which = matched the bpf filter that is active.. this shouldn=E2=80=99t be = happening. Are you running netstat during the tcpreplay run or after it = completed? Also can you give me more information on what specifically = mon0 is? > On Nov 4, 2015, at 10:44 AM, elof2@sentor.se wrote: >=20 >=20 > Hi! >=20 > Question: > What do the Recv column in 'netstat -B' show? >=20 > I thought it was tha amount of packets received, but appaently not so. >=20 > I send 2000000 packets from a tcpreplay machine to a receiving = machine. > I do it a few times. >=20 > On the receiver I see: > netstat -in > Name Mtu Network Address Ipkts Ierrs Idrop = Opkts Oerrs Coll > ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 = 0 0 0 > ix1 1500 0c:c4:7a:58:e2:3d 6000000 0 0 = 0 0 0 >=20 > and then > netstat -in > Name Mtu Network Address Ipkts Ierrs Idrop = Opkts Oerrs Coll > ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 = 0 0 0 > ix1 1500 0c:c4:7a:58:e2:3d 8000000 0 0 = 0 0 0 >=20 > So 6000000 has increased to 8000000. Good. >=20 >=20 > However, 'netstat -B' show: > Pid Netif Flags Recv Drop Match Sblen Hblen Command > 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump >=20 > How can the "Recv" be *lower* than "Match"? > 1996862 < 2000000. >=20 >=20 > For every new run (fast and slow) I get the same results, slightly = less than 2000000 Recv. >=20 > What am I missing? >=20 > /Elof > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Nov 4 19:41:46 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3E88A2601F for ; Wed, 4 Nov 2015 19:41:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0D8BD185E for ; Wed, 4 Nov 2015 19:41:45 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA19745 for ; Wed, 04 Nov 2015 21:41:37 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Zu3w9-0004z9-Eu for freebsd-net@freebsd.org; Wed, 04 Nov 2015 21:41:37 +0200 To: "freebsd-net@freebsd.org" From: Andriy Gapon Subject: who uses this port? Message-ID: <563A5F39.7010906@FreeBSD.org> Date: Wed, 4 Nov 2015 21:40:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 19:41:46 -0000 $ sockstat -l | fgrep 631 ? ? ? ? tcp4 127.0.0.1:631 *:* $ nc -l 127.0.0.1 631 nc: Address already in use -- Andriy Gapon From owner-freebsd-net@freebsd.org Wed Nov 4 19:44:08 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B6DFA26163 for ; Wed, 4 Nov 2015 19:44:08 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from mail-in6.apple.com (mail-out6.apple.com [17.151.62.28]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 486D11A0D; Wed, 4 Nov 2015 19:44:07 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from relay5.apple.com (relay5.apple.com [17.128.113.88]) by mail-in6.apple.com (Apple Secure Mail Relay) with SMTP id 17.28.19062.0006A365; Wed, 4 Nov 2015 11:44:00 -0800 (PST) X-AuditID: 11973e15-f79976d000004a76-64-563a6000103f Received: from [17.149.226.107] (Unknown_Domain [17.149.226.107]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by relay5.apple.com (Apple SCV relay) with SMTP id 7E.43.07593.0006A365; Wed, 4 Nov 2015 11:44:00 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: who uses this port? From: Charles Swiger In-Reply-To: <563A5F39.7010906@FreeBSD.org> Date: Wed, 4 Nov 2015 11:44:00 -0800 Cc: "freebsd-net@freebsd.org" Content-Transfer-Encoding: 7bit Message-Id: <6B1F31BD-863E-45E3-BD48-71B2E3702203@mac.com> References: <563A5F39.7010906@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.3096.5) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBLMWRmVeSWpSXmKPExsUi2FAYocuQYBVm8L/BxmLSq052iw872pkc mDxmfJrPEsAYxWWTkpqTWZZapG+XwJXxddITpoJDzBVN8y6zNjAeZOpi5OSQEDCRWPrrIpQt JnHh3nq2LkYuDiGBvYwSK+7fhyv6tfk1K0RiOpPEjYn/WEASzAJaEjf+vQQr4hUwkGi7eoYR xBYWUJCYtWofUJyDg01ATWLCRB6QMKeAtsS3P4eYQWwWARWJW2vfg5UwC5hL/LhhDjFRXmL7 2znMEBOtJCYemMYOYgsBbbq06hxYXERASWL+foi4BFD9z5UNUGe+ZZVYspZ9AqPQLCTHzUJy 3CwkKxYwMq9iFMpNzMzRzcwz00ssKMhJ1UvOz93ECAra6XaiOxjPrLI6xCjAwajEw3uD0TJM iDWxrLgy9xCjNAeLkjhvVJRVmJBAemJJanZqakFqUXxRaU5q8SFGJg5OqQbGN/5lC8x0fpxa +VblVYvI08YPD9jf99zNsCpRubV+lqLyxXOJqitXMLK0/YngWq7ssqE07+tbbd//lzU2OAdp CNz8X63Edene17pq19+p0stXMMVJq22+LHixovTf+Tw9UcNzK710ChuCj0uLvF3TsZLjx7d2 eZbT3Uorfkc5cf0WEKrnDvRQYinOSDTUYi4qTgQANlq/QDsCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprDLMWRmVeSWpSXmKPExsUiOPVRti5DglWYwZ/3lhaTXnWyW3zY0c7k wOQx49N8lgDGKC6blNSczLLUIn27BK6Mr5OeMBUcYq5omneZtYHxIFMXIyeHhICJxK/Nr1kh bDGJC/fWs3UxcnEICUxnkrgx8R8LSIJZQEvixr+XYA28AgYSbVfPMILYwgIKErNW7QOKc3Cw CahJTJjIAxLmFNCW+PbnEDOIzSKgInFr7XuwEmYBc4kfN8whJspLbH87hxliopXExAPT2EFs IaBNl1adA4uLCChJzN8PEZcAqv+5soFpAiP/LCQHzUJy0CwkYxcwMq9iFChKzUmsNNVLLCjI SdVLzs/dxAgKs4bCiB2M/5dZHWIU4GBU4uG9wWgZJsSaWFZcmXuIUYKDWUmEd56yVZgQb0pi ZVVqUX58UWlOavEhRmkOFiVx3tKt5mFCAumJJanZqakFqUUwWSYOTqkGxnsLXwn3r8j4y/P2 yj3zeLbkVc1Hyvedqwg8WXb887xPIbOMj36Rm7V2/rrPR31el9YyO3blyyoLJS3bJvm5rf7W A1HPqXbLZyUfluph9P7fI2OZeuLsN65Lp1TbD7XJRcg+3aC43kNvgQyv8INjz6dMYq+YcyJO 3rhr9skj74W/fSwM3Pxzlb0SS3FGoqEWc1FxIgA5jUIfLwIAAA== X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 19:44:08 -0000 On Nov 4, 2015, at 11:40 AM, Andriy Gapon wrote: > $ sockstat -l | fgrep 631 > ? ? ? ? tcp4 127.0.0.1:631 *:* > > $ nc -l 127.0.0.1 631 > nc: Address already in use That's the IPP port, commonly grabbed by CUPS and other printing software: % grep 631 /etc/services | head -2 ipp 631/tcp # Internet Printing Protocol ipp 631/udp # Internet Printing Protocol Regards, -- -Chuck From owner-freebsd-net@freebsd.org Wed Nov 4 19:44:14 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5747CA26176 for ; Wed, 4 Nov 2015 19:44:14 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F4011A53; Wed, 4 Nov 2015 19:44:14 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by obbww6 with SMTP id ww6so22008172obb.0; Wed, 04 Nov 2015 11:44:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=eOgI5XU1nyPzDQXvky/SA6InacntRMPJ9TmiNZNc/Ns=; b=vacTZgkdyhKO4l/D9lSIOZX4oXN2DkPh8xeUPcrMKO1fUUqmEuEkiUeO6x8+XcFVct 0WN39Pb2SzW5SsTKQs/Gtg9uow0kxc4fCjt1OGtPvstc2JUtz4Su+hUYh/dUI+/d4MRW DJoIpcuXtDobPQVfPSgJFZMaY1ZACZ0RAjqS08ai3BgA61iH5J+V0qvATDELCAp+wqsT 1WNbTeVGN5Geq0YFDKFtNWLmuSXhGPld0ycqm2XQD8EILWkCeZinCwZHZ9nQfTSJ1eZx 0ULg9RxRNBneEMUO7NhGKTBhgBkAj6GzcCTpYkcztdJrdO8zCP7QM56azQT092CUlNX4 EChA== MIME-Version: 1.0 X-Received: by 10.60.33.202 with SMTP id t10mr2031585oei.66.1446666253537; Wed, 04 Nov 2015 11:44:13 -0800 (PST) Received: by 10.76.8.4 with HTTP; Wed, 4 Nov 2015 11:44:13 -0800 (PST) In-Reply-To: <563A5F39.7010906@FreeBSD.org> References: <563A5F39.7010906@FreeBSD.org> Date: Wed, 4 Nov 2015 11:44:13 -0800 Message-ID: Subject: Re: who uses this port? From: Freddie Cash To: Andriy Gapon Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 19:44:14 -0000 CUPS uses port 631 by default. On Wed, Nov 4, 2015 at 11:40 AM, Andriy Gapon wrote: > $ sockstat -l | fgrep 631 > ? ? ? ? tcp4 127.0.0.1:631 *:* > > $ nc -l 127.0.0.1 631 > nc: Address already in use > > -- > Andriy Gapon > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Freddie Cash fjwcash@gmail.com From owner-freebsd-net@freebsd.org Wed Nov 4 19:44:46 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 907C5A261B9 for ; Wed, 4 Nov 2015 19:44:46 +0000 (UTC) (envelope-from jhellenthal@dataix.net) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA131B45 for ; Wed, 4 Nov 2015 19:44:46 +0000 (UTC) (envelope-from jhellenthal@dataix.net) Received: by ioll68 with SMTP id l68so66558996iol.3 for ; Wed, 04 Nov 2015 11:44:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=lq7iTVjHkPyUaM/30i8M5711zH5mMySCGVgyIZRL3bU=; b=H0tT0gD1A3SFIcXpZZAO57+ORT+1xxGNSC7xphRCzDllq4G8zPFOB0xFPLBES1xf/B 0xw5c//Ujkjq57AZDiLrYF8YhsRPxjnqy6+CRa9npI6oGPCbZw+BfKOcpzzpwpSYKn2w B6ibQ7CErMty2AAmL/89j+MElBZ2B7AGOMc2k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=lq7iTVjHkPyUaM/30i8M5711zH5mMySCGVgyIZRL3bU=; b=gsgBJ9mkUnVd3BHcJtYtmwYApJrGf0TJYs2mD+a0XyTFYk68j8xATr3Cfwu10UmLVL JgFraHvkjA4j46MlGW97LCzJxBprEtAlbMJJ6REMqh0yE7sR66OFNtW70IPnZmFKa3R1 H5mT3BdgNFF8L5RrHu8OAAPEI8aLauHk4rUAvGgUSWeo8GfFGJLRUrB9Xye/GlIJPpG4 h56E7yGVFuZ5yTOqfrYUVxE16o86QZNnRHUwrN0/f4J8qVXL5rglfX1rylM0LLn1dwVc 3ao6Id1qknFmk6GY5SzOorophaiMK/eZxJAZgwgHd/D5c+0USbD8uKszkwM/L+FDGGAV zqVA== X-Gm-Message-State: ALoCoQn2SBmnxqe9zWSmBP6S1W86rSebBJIl7R0LzpOkBFkIWKDWOjpIR5ASGy7WNeCwhiwk4ISM X-Received: by 10.107.15.229 with SMTP id 98mr5288525iop.63.1446666285740; Wed, 04 Nov 2015 11:44:45 -0800 (PST) Received: from [172.31.32.13] (cpe-65-26-238-24.wi.res.rr.com. [65.26.238.24]) by smtp.gmail.com with ESMTPSA id k12sm9343584igt.2.2015.11.04.11.44.45 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 04 Nov 2015 11:44:45 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: who uses this port? From: Jason Hellenthal In-Reply-To: <563A5F39.7010906@FreeBSD.org> Date: Wed, 4 Nov 2015 13:44:44 -0600 Cc: "freebsd-net@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <1CC53BC3-1D5E-4149-9674-CBB8309C414C@dataix.net> References: <563A5F39.7010906@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 19:44:46 -0000 Think I=E2=80=99ve seen this happen once or twice by rpcbind on startup. = By default though its usually a CUPS printing daemon. Besides that got any ng_sockets open ? > On Nov 4, 2015, at 13:40, Andriy Gapon wrote: >=20 > $ sockstat -l | fgrep 631 > ? ? ? ? tcp4 127.0.0.1:631 *:* >=20 > $ nc -l 127.0.0.1 631 > nc: Address already in use >=20 > --=20 > Andriy Gapon > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Jason Hellenthal JJH48-ARIN From owner-freebsd-net@freebsd.org Wed Nov 4 22:25:48 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CD7AA26701 for ; Wed, 4 Nov 2015 22:25:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6FC5E1F07 for ; Wed, 4 Nov 2015 22:25:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA4MPm0A024824 for ; Wed, 4 Nov 2015 22:25:48 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197198] bxe driver failed to init NIC Date: Wed, 04 Nov 2015 22:25:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mg@fork.pl X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 22:25:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197198 mg@fork.pl changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mg@fork.pl --- Comment #2 from mg@fork.pl --- Same here on 10.2 (amd64, generic, livedvd) for bxe0 ERROR: Invalid VLAN (57005) ERROR: Enumerated function 0 is marked as hidden Found 10GBase-CX4 media Using defaults for TSO: 65518/35/2048 Ethernet address: de:ad:de:ad:de:ad MSI-X vectors Requested 3 and Allocated 3 for bxe1 ERROR: md_mode=SD functions 1 and 3 have the same ovlan (0) ERROR: No Ethernet address programmed! Found 10GBase-CX4 media Using defaults for TSO: 65518/35/2048 MSI-X vectors Requested 3 and Allocated 3 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@freebsd.org Wed Nov 4 23:07:26 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE57BA26D72 for ; Wed, 4 Nov 2015 23:07:26 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EC9C1FDB for ; Wed, 4 Nov 2015 23:07:26 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: by wmeg8 with SMTP id g8so54077988wme.1 for ; Wed, 04 Nov 2015 15:07:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4wylJZ7yOnyACbxw4hD+eMsDUh+ntNuQNtxRBlnhjK4=; b=DQXm0DuMftue7PLawCTlgMbYUOlN9NUhmLU49ELAV7hPcox3jIVZaU8b2m5G1RONGE 3qRJL+6QiBu/d+pKkmxW4z91dluaHy+9b70gBrHJC2Z6nosynpGYTBTumfWlYq0E9z41 fi5Y9EnB3dmgxF/VaDyG+BEpHvLfZ9vw0uxBnmYpyUgG2Ja8jZ7TQoDWJJnSwTjMLs/9 6bq1Z8UC2JSriMuFDotj/rFNRMcf8zlyl2nKkkD4qD2D3+ydf4M4VdiCSK3w86uRpga7 05VgD/1vKJIaVJDJpsZhihaFtSUgPZgHF2aPiZ9+iFCAoRU0NzB8/47vTszUqOfTrSjY 6/pA== MIME-Version: 1.0 X-Received: by 10.28.54.197 with SMTP id y66mr26706024wmh.31.1446678444812; Wed, 04 Nov 2015 15:07:24 -0800 (PST) Received: by 10.27.1.139 with HTTP; Wed, 4 Nov 2015 15:07:24 -0800 (PST) In-Reply-To: <20151104184503.GC1117@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> Date: Wed, 4 Nov 2015 15:07:24 -0800 Message-ID: Subject: Re: tap(4) and host-only networking between host and guest From: Neel Natu To: Victor Sudakov Cc: freebsd-net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 23:07:26 -0000 Hi Victor, On Wed, Nov 4, 2015 at 10:45 AM, Victor Sudakov wrote: > Victor Sudakov wrote: >> Julian Elischer wrote: >> > > >> > > I am experimenting with bhyve which uses tap(4) for network access. >> > > >> > > I don't want to bridge tap0 with any of the hosts's real NICs. How can >> > > I create a private network just between the host and the guest? >> > you are thinking too hard! >> > >> > tap IS the interface.. >> > >> > ifconfig tap0 $address... >> > and in the VM, ifconfig vtnet0 ${some_other_address} >> >> Thank you, Julian! It works. I felt I was missing something obvious. > > For some reason, after a guest is shutdown or rebooted, the IP address > on the host's tap0 interface is deleted. > Try using 'vmnet0' instead of 'tap0'. It will retain the IP address even after a guest reboot/shutdown. best Neel > It's kind of inconvenient. > > -- > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > sip:sudakov@sibptus.tomsk.ru > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Nov 4 23:44:24 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24EFDA26418 for ; Wed, 4 Nov 2015 23:44:24 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id B08BC117B for ; Wed, 4 Nov 2015 23:44:23 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.20,245,1444654800"; d="scan'208";a="14570228" Received: from gsp-ex03.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.19]) by iport1.cc.swin.edu.au with ESMTP; 05 Nov 2015 10:44:16 +1100 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.2]) by gsp-ex03.ds.swin.edu.au ([169.254.3.127]) with mapi id 14.03.0248.002; Thu, 5 Nov 2015 10:44:15 +1100 From: Rasool Al-Saadi To: Hans Petter Selasky , "freebsd-net@freebsd.org" Subject: RE: Timing issue with Dummynet on high kernel timer interrupt Thread-Topic: Timing issue with Dummynet on high kernel timer interrupt Thread-Index: AdEWOVE5DoAlkzqPQWaBZ8ln4esVd///TXiA//0UVtA= Date: Wed, 4 Nov 2015 23:44:14 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> In-Reply-To: <5638B7B5.3030802@selasky.org> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.126.11] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2015 23:44:24 -0000 On Wednesday, 4 November 2015 12:34 AM, Hans Petter Selasky wrote: > On 11/03/15 14:14, Rasool Al-Saadi wrote: > > Does anyone have thoughts on what we can test next to narrow down the > root-cause of these unusual timing jumps? >=20 > You might also want to test the "projects/hps_head" branch, which uses a = bit > different callout implementation. Thanks Hans for your suggestion. I have tried "projects/hps_head" branch and the result is better (number of= spikes is less than in the master branch). However, the problem still exis= ts on the same timer interrupt frequencies (>3000 in my case). You can see = in this graph https://goo.gl/photos/C2Mqx4xhMQuzxWnz6 the RTT spikes still = there. Do you have any further suggestions?=20 Regards, Rasool > --HPS > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Nov 5 00:06:11 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B019AA26BDB for ; Thu, 5 Nov 2015 00:06:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 5501A1CB8 for ; Thu, 5 Nov 2015 00:06:10 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:O4BAJxbhODJB4wtkPiRQ9Yb/LSx+4OfEezUN459isYplN5qZpcm8bnLW6fgltlLVR4KTs6sC0LqL9fu5EjVRqb+681k8M7V0HycfjssXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aJBzzOEJPK/jvHcaK1oLsh730oMSYOlQArQH+SI0xBS3+lR/WuMgSjNkqAYcK4TyNnEF1ff9Lz3hjP1OZkkW0zM6x+Jl+73YY4Kp5pIYTGZn9Kos1V6ZZEHwFp2AzrJnkuAPZTBfJ5WYRUmM+mxdJRQ3d41f2U8GinDH9s79H2SKZdej/RrMwVDHqu71uQRTrjCoCHyM+/3zajtRwyqlS9kHy7ydjypLZNdnGfMF1ebnQKIsX X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DWAwDJnDpW/61jaINehA4sAUIGvW0BDYFeFwqFJ0oCgXYUAQEBAQEBAQGBCYIuggcBAQEDAQEBASAEJyALEAIBCBgCAg0ZAgInAQkmAgQIBwQBHASIBQgNsRWRAgEBAQEBAQEDAQEBAQEBAQEXBIEBhVOEfoQ7AQEFgzOBQwWOEYg3hR2FH4RBh2SKMIhRAh8BAUKEIiA0B4NsOoEHAQEB X-IronPort-AV: E=Sophos;i="5.20,245,1444708800"; d="scan'208";a="249300968" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 04 Nov 2015 19:05:01 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 095E215F55D; Wed, 4 Nov 2015 19:05:00 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 4Oyubks-akNW; Wed, 4 Nov 2015 19:05:00 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 0334015F565; Wed, 4 Nov 2015 19:04:59 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id zp73noTHgB9Y; Wed, 4 Nov 2015 19:04:59 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 8BE0B15F55D; Wed, 4 Nov 2015 19:04:59 -0500 (EST) Date: Wed, 4 Nov 2015 19:04:59 -0500 (EST) From: Rick Macklem To: Gerrit =?utf-8?B?S8O8aG4=?= Cc: freebsd-net@freebsd.org Message-ID: <1569514731.73706862.1446681899411.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20151104144354.acccf3c7315a6009264d755d@aei.mpg.de> References: <20151104144354.acccf3c7315a6009264d755d@aei.mpg.de> Subject: Re: strange nfs/rsync stalls MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: strange nfs/rsync stalls Thread-Index: cL59sYuzEHXL83gixW2iLMRLk2zwGA== X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 00:06:11 -0000 Gerritt Kuhn wrote: > Hi all, > > I see some weird (I think) NFS issue here: > I have a NFS server running 10.2-RELEASE exporting a directory to a NFS > client running 10.2-STABLE (as of yesterday, tried that update from 10.1 > to see if it would fix this issue, but it stayed with me). I use rsync on > the client to copy data from local disks to nfs. > This appears to work for a few minutes, then the rsync process gets stuck > in state "NEWNFS". Messages "newnfs server servername:/data/systems: not > responding" start appearing and get repeated over and over. > > What I think is strange: I cannot see any issue apart from these rsync > jobs getting stuck. I can still access the nfs-mounted data on the client > just fine in a different shell. I can read, write, do whatever I like, but > the rsync job never moves on. > I have a couple of other machines that use the same server in the same way, > none of them ever showed this behaviour. Are there any suggestions how to > debug/fix this? > You didn't mention what kind of net interface you are using. 1 - disable TSO if you haven't already done so. 2 - try capturing packets when it is stuck and nothing else is being done on the mount point and see if any traffic is being sent to the server. # tcpdump -s 0 -w .pcap host - run on the client for a little while should do it. --> Look at .pcap in wireshark, since it knows NFS, unlike tcpdump. 3 - Do "nfsstat -e -c" on the client a few times when it is stuck and see what RPC counts are increasing. (Again when nothing else is being done on the mount.) Hopefully, the above will give you some idea of what is going on. (Btw, all processes stuck in "newnfs" tells you is that they are waiting for an NFS client vnode lock held by some other thread.) Good luck with it, rick > > cu > Gerrit > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Thu Nov 5 02:31:21 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26B8AA26B09 for ; Thu, 5 Nov 2015 02:31:21 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id 847F41041 for ; Thu, 5 Nov 2015 02:31:20 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38945540; Thu, 05 Nov 2015 08:31:17 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA52VFcL010331; Thu, 5 Nov 2015 08:31:17 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA52VFhS010330; Thu, 5 Nov 2015 08:31:15 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Thu, 5 Nov 2015 08:31:15 +0600 From: Victor Sudakov To: Neel Natu Cc: freebsd-net Subject: Re: tap(4) and host-only networking between host and guest Message-ID: <20151105023115.GA10139@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 02:31:21 -0000 Neel Natu wrote: > >> Julian Elischer wrote: > >> > > > >> > > I am experimenting with bhyve which uses tap(4) for network access. > >> > > > >> > > I don't want to bridge tap0 with any of the hosts's real NICs. How can > >> > > I create a private network just between the host and the guest? > >> > you are thinking too hard! > >> > > >> > tap IS the interface.. > >> > > >> > ifconfig tap0 $address... > >> > and in the VM, ifconfig vtnet0 ${some_other_address} > >> > >> Thank you, Julian! It works. I felt I was missing something obvious. > > > > For some reason, after a guest is shutdown or rebooted, the IP address > > on the host's tap0 interface is deleted. > > > > Try using 'vmnet0' instead of 'tap0'. It will retain the IP address > even after a guest reboot/shutdown. Thanks, I see now vmnet is documented in tap(4). -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Thu Nov 5 02:33:28 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D24BA26C58 for ; Thu, 5 Nov 2015 02:33:28 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id E391C12A8 for ; Thu, 5 Nov 2015 02:33:27 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38945528; Thu, 05 Nov 2015 08:33:26 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA52XNGP010356; Thu, 5 Nov 2015 08:33:23 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA52XNJv010355; Thu, 5 Nov 2015 08:33:23 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Thu, 5 Nov 2015 08:33:23 +0600 From: Victor Sudakov To: John Nielsen Cc: freebsd-net@freebsd.org Subject: Re: tap(4) and host-only networking between host and guest Message-ID: <20151105023323.GB10139@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 02:33:28 -0000 John Nielsen wrote: > >>>> > >>>> I am experimenting with bhyve which uses tap(4) for network access. > >>>> > >>>> I don't want to bridge tap0 with any of the hosts's real NICs. How can > >>>> I create a private network just between the host and the guest? > >>> you are thinking too hard! > >>> > >>> tap IS the interface.. > >>> > >>> ifconfig tap0 $address... > >>> and in the VM, ifconfig vtnet0 ${some_other_address} > >> > >> Thank you, Julian! It works. I felt I was missing something obvious. > > > > For some reason, after a guest is shutdown or rebooted, the IP address > > on the host's tap0 interface is deleted. > > > > It's kind of inconvenient. > > What I have done in this scenario is create a bridge interface, > assign the host???s IP to the bridge, and add the tap as a member to > the bridge. That's what I have eventually done too. This way several guests can share a LAN emulation, and a DHCP server can listen on bridge0. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Thu Nov 5 07:20:42 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0543A267A6 for ; Thu, 5 Nov 2015 07:20:42 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-yk0-x22b.google.com (mail-yk0-x22b.google.com [IPv6:2607:f8b0:4002:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A215B15BE; Thu, 5 Nov 2015 07:20:42 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by ykba4 with SMTP id a4so116364623ykb.3; Wed, 04 Nov 2015 23:20:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=U2sGfVRILzmLvq88XtBF8/Zf334ntYNLS//GpzDtKPA=; b=z51we7ZrC5FXYrrJwdF9me5/AH6DC/5pBWxBOKuNAeXd2iIwfEcg983bcGEZr6+1YK u4keHozq4kluFsOVpwsQ481gJJMaA6E9tYGmoX1LoIuYifT8/K8mV2+cZhLRkz7S4UlU ZBg/QEEQZvcvNtrbUWJAFUO2OhtWVtksWAqCQTK6oDphgD48sDF+2BGiLW6XIfvkcKtU dg6ilHHhyjaCShJFNLcOQul3bijhpiuG5m+oOYz7lumCcHUZPcfecy7h0TRS5gAcsQ3i 49n9Dsu8F9YYM0s/g1ykZ9enaaDHMp8ZqXCcrZ5RjnKjrBxE8+NCmScZdZ3ydi5DWEz8 AjOQ== MIME-Version: 1.0 X-Received: by 10.31.166.209 with SMTP id p200mr5690603vke.26.1446708041764; Wed, 04 Nov 2015 23:20:41 -0800 (PST) Received: by 10.31.77.130 with HTTP; Wed, 4 Nov 2015 23:20:41 -0800 (PST) In-Reply-To: <563A5F39.7010906@FreeBSD.org> References: <563A5F39.7010906@FreeBSD.org> Date: Thu, 5 Nov 2015 15:20:41 +0800 Message-ID: Subject: Re: who uses this port? From: Ben Woods To: Andriy Gapon Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 07:20:43 -0000 On Wednesday, 4 November 2015, Andriy Gapon wrote: > $ sockstat -l | fgrep 631 > ? ? ? ? tcp4 127.0.0.1:631 *:* > > $ nc -l 127.0.0.1 631 > nc: Address already in use > I'm more curious as to why sockstat gives you question marks instead of the proper process details. Any ideas? Regards, Ben -- -- From: Benjamin Woods woodsb02@gmail.com From owner-freebsd-net@freebsd.org Thu Nov 5 07:46:38 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EB88A26E0C for ; Thu, 5 Nov 2015 07:46:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 67CFB13D7 for ; Thu, 5 Nov 2015 07:46:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA29972; Thu, 05 Nov 2015 09:46:34 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZuFFi-0005rO-1o; Thu, 05 Nov 2015 09:46:34 +0200 Subject: Re: who uses this port? To: Ben Woods , "freebsd-net@freebsd.org" References: <563A5F39.7010906@FreeBSD.org> From: Andriy Gapon Message-ID: <563B0922.9030805@FreeBSD.org> Date: Thu, 5 Nov 2015 09:45:38 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 07:46:38 -0000 On 05/11/2015 09:20, Ben Woods wrote: > On Wednesday, 4 November 2015, Andriy Gapon > wrote: > > $ sockstat -l | fgrep 631 > ? ? ? ? tcp4 127.0.0.1:631 > *:* > > $ nc -l 127.0.0.1 631 > nc: Address already in use > > > > I'm more curious as to why sockstat gives you question marks instead of the > proper process details. Any ideas? Yeah, I should have stated my question more accurately. What you are asking is what I actually intended to ask. I was debugging a problem of cupsd not being able to bind to its port after a restart. Eventually I had to reboot the affected system. -- Andriy Gapon From owner-freebsd-net@freebsd.org Thu Nov 5 08:02:23 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0D64A25432 for ; Thu, 5 Nov 2015 08:02:23 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5387C1DCE for ; Thu, 5 Nov 2015 08:02:22 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id tA582Adc071213 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 5 Nov 2015 09:02:11 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: vas@mpeks.tomsk.su Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id tA5822A9075379; Thu, 5 Nov 2015 15:02:02 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: tap(4) and host-only networking between host and guest To: Victor Sudakov References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <563A1265.7010208@grosbein.net> <20151104153415.GB1117@admin.sibptus.tomsk.ru> Cc: freebsd-net@freebsd.org From: Eugene Grosbein Message-ID: <563B0CFA.301@grosbein.net> Date: Thu, 5 Nov 2015 15:02:02 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20151104153415.GB1117@admin.sibptus.tomsk.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, T_DATE_IN_FUTURE_96_Q autolearn=no version=3.3.2 X-Spam-Report: * 0.0 T_DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 08:02:23 -0000 On 04.11.2015 22:34, Victor Sudakov wrote: > tap(4) says that tapN is an exclusive-open device. And when I tried it > out, the second bhyve instances said "open of tap device /dev/tap0 failed", > which was kind of expected. Hmm, false memory... I have missed "exclusive" part. From owner-freebsd-net@freebsd.org Thu Nov 5 09:51:28 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEC23A26BF3 for ; Thu, 5 Nov 2015 09:51:28 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 66A41188B for ; Thu, 5 Nov 2015 09:51:28 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 135931FE023; Thu, 5 Nov 2015 10:51:23 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Rasool Al-Saadi , "freebsd-net@freebsd.org" References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> From: Hans Petter Selasky Message-ID: <563B2703.5080402@selasky.org> Date: Thu, 5 Nov 2015 10:53:07 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> Content-Type: multipart/mixed; boundary="------------010409060107000901010706" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 09:51:28 -0000 This is a multi-part message in MIME format. --------------010409060107000901010706 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 11/05/15 00:44, Rasool Al-Saadi wrote: > > On Wednesday, 4 November 2015 12:34 AM, Hans Petter Selasky wrote: >> On 11/03/15 14:14, Rasool Al-Saadi wrote: >>> Does anyone have thoughts on what we can test next to narrow down the >> root-cause of these unusual timing jumps? >> >> You might also want to test the "projects/hps_head" branch, which uses a bit >> different callout implementation. > > Thanks Hans for your suggestion. > I have tried "projects/hps_head" branch and the result is better (number of spikes is less than in the master branch). However, the problem still exists on the same timer interrupt frequencies (>3000 in my case). You can see in this graph https://goo.gl/photos/C2Mqx4xhMQuzxWnz6 the RTT spikes still there. > > Do you have any further suggestions? > Hi, If the jitter is in the xx milliseconds range, like your graph shows, my guess is that td_owepreemt is not set when we return from the dummynet() callback. See attached patch. Else you might want to try to remove the C_HARDCLOCK flag from callout_reset_sbt() in ip_dummynet.c. --HPS --------------010409060107000901010706 Content-Type: text/x-patch; name="ip_dummynet.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ip_dummynet.c.diff" Index: sys/netpfil/ipfw/ip_dummynet.c =================================================================== --- sys/netpfil/ipfw/ip_dummynet.c (revision 290134) +++ sys/netpfil/ipfw/ip_dummynet.c (working copy) @@ -78,6 +78,8 @@ static struct task dn_task; static struct taskqueue *dn_tq = NULL; +#include + static void dummynet(void *arg) { @@ -84,6 +86,9 @@ (void)arg; /* UNUSED */ taskqueue_enqueue_fast(dn_tq, &dn_task); + + /* ensure taskqueue gets running */ + sched_preempt(curthread); } void --------------010409060107000901010706-- From owner-freebsd-net@freebsd.org Thu Nov 5 09:58:51 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94525A26DEC for ; Thu, 5 Nov 2015 09:58:51 +0000 (UTC) (envelope-from katoon@sfc.wide.ad.jp) Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [IPv6:2001:200:0:8803::53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2F0031B36; Thu, 5 Nov 2015 09:58:51 +0000 (UTC) (envelope-from katoon@sfc.wide.ad.jp) Received: from mail-io0-f173.google.com (mail-io0-f173.google.com [209.85.223.173]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 1D79B278216; Thu, 5 Nov 2015 18:58:48 +0900 (JST) Received: by ioc74 with SMTP id 74so18582183ioc.2; Thu, 05 Nov 2015 01:58:46 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.107.30.10 with SMTP id e10mr9996613ioe.43.1446717526446; Thu, 05 Nov 2015 01:58:46 -0800 (PST) Received: by 10.79.79.73 with HTTP; Thu, 5 Nov 2015 01:58:46 -0800 (PST) In-Reply-To: References: <201509050053.t850rh9P071595@gw.catspoiler.org> Date: Thu, 5 Nov 2015 18:58:46 +0900 Message-ID: Subject: Re: default ECN settings From: Midori Kato To: "K. Macy" Cc: Don Lewis , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 09:58:51 -0000 Hi Macy and Don, I am Midori. Too late to catch up this topic but this topic is interesting to me. Linux separates inbound and outbound ecn operation while RFC 3168 says that making hosts fail during the negotiation without ecn configuration. I think FreeBSD is probably able to distinguish inbound and outbound with cc_var flag as well. I like to try to work this. If the sender like to use ECN, behaving as ECN receiver is good for the TCP connection. Regards, -- Midori 2015-09-05 10:05 GMT+09:00 K. Macy : > On Fri, Sep 4, 2015 at 5:53 PM, Don Lewis wrote: > > On 4 Sep, K. Macy wrote: > >> By default ECN is completely disabled on FreeBSD. On Linux the default > >> is to disable it outbound (not request it) but enable it inbound > >> (accept new connections asking for it). Is there a good reason to only > >> set ECN_PERMIT on inbound connections if the system is doing ECN on > >> outbound connections? > > > > Not that I can think of. The risk in enabling ECN for outbound > > connections is that some connection attempts can fail, especially if you > > are attempting to connect to some old and oddball device. That should > > not be a risk for inbound connections since those devices won't be > > requesting ECN. > > Even with 'oddball' devices the stack is configured to retry ECN n > times where n defaults to 1 and then revert to not requesting ECN > support. Thus connections would take longer on 'oddball' devices. The > solution that *I* would choose for that would be to track ECN support > in the host cache. The first connection to a new host would always try > ECN and in the event that that failed all subsequent connection > attempts would not try ECN. To me this seems like the most robust > compromise. However, I don't yet have enough information to say how > much benefit this would confer. > > > Seems like we should be defaulting ECN on for inbound connections, > > though we currently can't control the two directions separately. > > That is a straightforward change. > > > Cheers. > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Thu Nov 5 09:59:11 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E6E6A26E0D for ; Thu, 5 Nov 2015 09:59:11 +0000 (UTC) (envelope-from matthew@freebsd.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 994ED1BF1 for ; Thu, 5 Nov 2015 09:59:10 +0000 (UTC) (envelope-from matthew@freebsd.org) Received: from ox-dell39.ox.adestra.com (no-reverse-dns.metronet-uk.com [85.199.232.226] (may be forged)) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.2/8.15.2) with ESMTPSA id tA59wv8R072743 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 5 Nov 2015 09:59:03 GMT (envelope-from matthew@freebsd.org) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=freebsd.org DKIM-Filter: OpenDKIM Filter v2.10.3 smtp.infracaninophile.co.uk tA59wv8R072743 Authentication-Results: smtp.infracaninophile.co.uk/tA59wv8R072743; dkim=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host no-reverse-dns.metronet-uk.com [85.199.232.226] (may be forged) claimed to be ox-dell39.ox.adestra.com Subject: Re: who uses this port? To: freebsd-net@freebsd.org References: <563A5F39.7010906@FreeBSD.org> <563B0922.9030805@FreeBSD.org> From: Matthew Seaman X-Enigmail-Draft-Status: N1110 Message-ID: <563B285A.4070808@freebsd.org> Date: Thu, 5 Nov 2015 09:58:50 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <563B0922.9030805@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="uXmnUAsrS9N9jHJ7ebB1BGOj3L8cuvP5K" X-Virus-Scanned: clamav-milter 0.98.7 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 09:59:11 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --uXmnUAsrS9N9jHJ7ebB1BGOj3L8cuvP5K Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 05/11/2015 09:20, Ben Woods wrote: >> I'm more curious as to why sockstat gives you question marks instead o= f the >> > proper process details. Any ideas? This indicates a connection that is in the process of closing down but the system still hasn't cleared up all the connection state yet. It's normal. You might find that adjusting the net.inet.tcp.fast_finwait2_recycle sysctl makes these old entries get cleared more promptly. Cheers, Matthew =09 --uXmnUAsrS9N9jHJ7ebB1BGOj3L8cuvP5K Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJWOyhgAAoJEABRPxDgqeTnXNEP/2ckKpF6zgmJ7Rw+N+ktlhue oJRXEofETLn0QSND4eSMU33WfKloga7yYon3Cfaq2MOQSYyL73ob8IwC3E/+pYtY ZUEAdD+CKQWvfTLIx9XhtqTpsugxYkfhMuxJ9ZSVpPuwXxPGufaVRA720O2oxwkU sfWHFF7SB4Tzcl4rZNM4W5DLWkyoS02w5t4SSnrMFpNsHFt+YZ6hafY2TSBhoKgX 74T1XC/T2/bQBDNggYKTtl2+WpWoZYjzUU9OtZOgjmXmkQrmJ1bAIYA4JN6tyswu 0TMaQiFBSQqpe0apjvg09DkgWYqg6M7ic8blMlriurFnYUCsUMYHasV/FjBl5imb cploVgtE5/TiPz1plXBpAmg9rZuu+xzyZThGiNn7cKe5fPCUDnlDJzX7kTN7EjYC f8dc13NTdWo9LBvZVEpIOYrJ1pbC5VaoalVf7c74jVaazbBYqTnNp7aDd0c7OhQE 3iTRXQ1Z3hLTcYQCFZmR+AfXz2MfnLwSQsCSH5/+cqYVod7Jj1la4JCoMxUuB1p+ 8vZRdRCI29GwEN8wz3y06HZol8/VudjKpQ0Nd+S+ztUBA6Y9ugJX2gSvByGzGHzd Zyf/0GpcTy5iaiZJyBs5EEArxosHy70ASe97nTfar+uYlWXGvT1ak4giCfXe2k3O NZn3Nnbu+BqyzVGWShhy =9Zhm -----END PGP SIGNATURE----- --uXmnUAsrS9N9jHJ7ebB1BGOj3L8cuvP5K-- From owner-freebsd-net@freebsd.org Thu Nov 5 10:22:46 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7FA67A27455 for ; Thu, 5 Nov 2015 10:22:46 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 51FE41AE8 for ; Thu, 5 Nov 2015 10:22:45 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-229-78.lns20.per1.internode.on.net [121.45.229.78]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id tA5AMXHe050970 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 5 Nov 2015 02:22:36 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: tap(4) and host-only networking between host and guest To: Victor Sudakov , Neel Natu References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> <20151105023115.GA10139@admin.sibptus.tomsk.ru> Cc: freebsd-net From: Julian Elischer Message-ID: <563B2DE4.2060307@freebsd.org> Date: Thu, 5 Nov 2015 18:22:28 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151105023115.GA10139@admin.sibptus.tomsk.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 10:22:46 -0000 On 11/5/15 10:31 AM, Victor Sudakov wrote: > Neel Natu wrote: >>>> Julian Elischer wrote: >>>>>> I am experimenting with bhyve which uses tap(4) for network access. >>>>>> >>>>>> I don't want to bridge tap0 with any of the hosts's real NICs. How can >>>>>> I create a private network just between the host and the guest? >>>>> you are thinking too hard! >>>>> >>>>> tap IS the interface.. >>>>> >>>>> ifconfig tap0 $address... >>>>> and in the VM, ifconfig vtnet0 ${some_other_address} >>>> Thank you, Julian! It works. I felt I was missing something obvious. >>> For some reason, after a guest is shutdown or rebooted, the IP address >>> on the host's tap0 interface is deleted. >>> >> Try using 'vmnet0' instead of 'tap0'. It will retain the IP address >> even after a guest reboot/shutdown. > Thanks, I see now vmnet is documented in tap(4). I didn't know that... Ok, so now you are an expert, can we count on you to answer all future questions on this topic? ;-) > From owner-freebsd-net@freebsd.org Thu Nov 5 11:39:56 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 904FBA26F57 for ; Thu, 5 Nov 2015 11:39:56 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from smtp-out.sentor.se (smtp-out.sentor.se [176.124.225.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34FC91901 for ; Thu, 5 Nov 2015 11:39:56 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from localhost (localhost [127.0.0.1]) by farmermaggot.shire.sentor.se (Postfix) with ESMTP id B5BB1B61D235; Thu, 5 Nov 2015 12:39:52 +0100 (CET) Date: Thu, 5 Nov 2015 12:39:52 +0100 (CET) From: elof2@sentor.se To: Christian Peron cc: freebsd-net Subject: Re: netstat -B "Recv" In-Reply-To: <04EF5880-800D-4A52-8CD1-DAF87EB6E37A@sqrt.ca> Message-ID: References: <04EF5880-800D-4A52-8CD1-DAF87EB6E37A@sqrt.ca> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 11:39:56 -0000 Hi Christian. Sorry, I was in a bit of a hurry when I wrote the below. Here's more information: FreeBSD 10.1 amd64 (on both boxes). In rc.conf on the receiver I have: ifconfig_ix0="up -arp -lro" ifconfig_ix1="up -arp -lro" cloned_interfaces="bridge0" ifconfig_bridge0="up addm ix0 -discover ix0 -learn ix0 private ix0 \ addm ix1 -discover ix1 -learn ix1 private ix1 \ maxaddr 1 -arp monitor name mon0" So 'mon0' is a monitor interface consisting of ix0 and ix1. During my test, ix0 has no carrier, only ix1 is in use. On the sending box I run tcpreplay to send 2000000 packets for each run. Naturally I run the netstat commands on the receiving machine after the tcpreplay on the other machine has finished, not during. :-) I reboot to start over from scratch. I start a sniffer: # tcpdump -nli mon0 -w /dev/null # netstat -in Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 ix1 1500 0c:c4:7a:58:e2:3d 0 0 0 0 0 0 mon0 1500 02:74:e9:56:9c:00 0 0 0 0 0 0 # netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1422 mon0 p--s--- 0 0 0 0 0 tcpdump I send 2000000 packets. The receiver now show: # netstat -in Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 ix1 1500 0c:c4:7a:58:e2:3d 2000000 0 0 0 0 0 mon0 1500 02:74:e9:56:9c:00 2000000 0 0 0 0 0 # netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1422 mon0 p--s--- 1999851 0 2000000 0 0 tcpdump I press ctrl-c in tcpdump and it says: 2000000 packets captured 1999851 packets received by filter 0 packets dropped by kernel So the BPF stats from both netstat and tcpdump look quite strange. :-) I start a new tcpdump and send 2000000 new packets. I now run 'netstat -B' a few times during the transfer, and one can see that the diffing packets are evenly distributed over time: Pid Netif Flags Recv Drop Match Sblen Hblen Command 1557 mon0 p--s--- 0 0 0 0 0 tcpdump 1557 mon0 p--s--- 383905 0 383923 1993540 0 tcpdump 1557 mon0 p--s--- 651010 0 651038 1112814 0 tcpdump 1557 mon0 p--s--- 944090 0 944128 2883633 0 tcpdump 1557 mon0 p--s--- 1125245 0 1125292 2034033 0 tcpdump 1557 mon0 p--s--- 1319456 0 1319514 467992 0 tcpdump 1557 mon0 p--s--- 1516545 0 1516618 4183543 0 tcpdump 1557 mon0 p--s--- 1760527 0 1760608 1365257 0 tcpdump 1557 mon0 p--s--- 1999908 0 2000000 2870940 0 tcpdump 1557 mon0 p--s--- 1999908 0 2000000 0 0 tcpdump 1557 mon0 p--s--- 1999908 0 2000000 0 0 tcpdump ctrl-c on tcpdump says: 2000000 packets captured 1999908 packets received by filter 0 packets dropped by kernel I run a tcpdump directly on ix1, instead of the bridge, and try again. Same thing, so the bridge has nothing to do with it: Pid Netif Flags Recv Drop Match Sblen Hblen Command 1819 ix1 p--s--- 1999904 0 2000000 0 0 tcpdump ^C 2000000 packets captured 1999904 packets received by filter 0 packets dropped by kernel I rebooted it again, started tcpdump on ix1 and received 2000000 packets. By accident I forgot to kill other processes that might have an impact on my tests. Maybe that was a good thing, because 'netstat -B' now show: Pid Netif Flags Recv Drop Match Sblen Hblen Command 921 mon0 p--s--- 2000000 76390 2000000 0 0 suricata 1358 ix1 p--s--- 1999897 0 2000000 0 0 tcpdump So the BPF stats for suricata is showing 2000000 received and 2000000 matched (and a few drops) as it should. Just to see if there would be any difference, I tried running tcpdump linebuffered but throwing away the output: # tcpdump -nli ix1 > /dev/null # netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 921 mon0 p--s--- 3999997 214356 4000000 0 0 suricata 1521 ix1 p--s--- 1999875 46200 2000000 0 0 tcpdump ^C 1953800 packets captured 1999875 packets received by filter 46200 packets dropped by kernel Nope, same-same. (I don't know what I was expecting) Since tcpdump is the one showing odd values I test with tshark (dumpcap) instead: Pid Netif Flags Recv Drop Match Sblen Hblen Command 921 mon0 p--s--- 5999996 473528 6000000 0 0 suricata 1624 ix1 p--s--- 1999920 0 2000000 0 0 dumpcap Nope same problem there. Tshark received 2000000 packets but for some reason, the BPF stats show 1999920 received packets. Also, this run suricata also had the same problem, so maybe it was just a coincidence that it showed correct values that first run. I try again, this time with ngrep as the sniffer: # ngrep -d ix1 > /dev/null Pid Netif Flags Recv Drop Match Sblen Hblen Command 921 mon0 p--s--- 7999950 766788 8000000 0 0 suricata 1730 ix1 p--s--- 1999862 0 0 0 0 ngrep Ngrep also show too few recv packets (and zero matches for some reason). Suricata recv-stats had problems this run as well, so it must have been a fluke that it got correct values above. Here's some ix stats, in case they should reveal something interesting: # netstat -in Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 ix1 1500 0c:c4:7a:58:e2:3d 12000000 0 0 0 0 0 mon0 1500 02:74:e9:56:9c:00 11999998 0 0 0 0 0 # sysctl -a | egrep "ix\.|ixgbe" | grep -v "ix\.0" | more device ixgbe hw.ix.enable_aim: 1 hw.ix.max_interrupt_rate: 31250 hw.ix.rx_process_limit: 256 hw.ix.tx_process_limit: 256 hw.ix.enable_msix: 1 hw.ix.num_queues: 8 hw.ix.txd: 2048 hw.ix.rxd: 2048 dev.ix.%parent: dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.15 dev.ix.1.%driver: ix dev.ix.1.%location: slot=0 function=1 dev.ix.1.%pnpinfo: vendor=0x8086 device=0x1528 subvendor=0x15d9 subdevice=0x0734 class=0x020000 dev.ix.1.%parent: pci1 dev.ix.1.fc: 3 dev.ix.1.enable_aim: 1 dev.ix.1.advertise_speed: 0 dev.ix.1.ts: 0 dev.ix.1.dropped: 0 dev.ix.1.mbuf_defrag_failed: 0 dev.ix.1.watchdog_events: 0 dev.ix.1.link_irq: 2 dev.ix.1.queue0.interrupt_rate: 500000 dev.ix.1.queue0.irqs: 1500836 dev.ix.1.queue0.txd_head: 0 dev.ix.1.queue0.txd_tail: 0 dev.ix.1.queue0.tso_tx: 0 dev.ix.1.queue0.no_tx_dma_setup: 0 dev.ix.1.queue0.no_desc_avail: 0 dev.ix.1.queue0.tx_packets: 0 dev.ix.1.queue0.rxd_head: 1704 dev.ix.1.queue0.rxd_tail: 1703 dev.ix.1.queue0.rx_packets: 1500840 dev.ix.1.queue0.rx_bytes: 676084080 dev.ix.1.queue0.rx_copies: 959400 dev.ix.1.queue0.lro_queued: 0 dev.ix.1.queue0.lro_flushed: 0 dev.ix.1.queue1.interrupt_rate: 500000 dev.ix.1.queue1.irqs: 971514 dev.ix.1.queue1.txd_head: 0 dev.ix.1.queue1.txd_tail: 0 dev.ix.1.queue1.tso_tx: 0 dev.ix.1.queue1.no_tx_dma_setup: 0 dev.ix.1.queue1.no_desc_avail: 0 dev.ix.1.queue1.tx_packets: 0 dev.ix.1.queue1.rxd_head: 768 dev.ix.1.queue1.rxd_tail: 767 dev.ix.1.queue1.rx_packets: 971520 dev.ix.1.queue1.rx_bytes: 347213640 dev.ix.1.queue1.rx_copies: 663480 dev.ix.1.queue1.lro_queued: 0 dev.ix.1.queue1.lro_flushed: 0 dev.ix.1.queue2.interrupt_rate: 500000 dev.ix.1.queue2.irqs: 2178838 dev.ix.1.queue2.txd_head: 0 dev.ix.1.queue2.txd_tail: 0 dev.ix.1.queue2.tso_tx: 0 dev.ix.1.queue2.no_tx_dma_setup: 0 dev.ix.1.queue2.no_desc_avail: 0 dev.ix.1.queue2.tx_packets: 0 dev.ix.1.queue2.rxd_head: 1816 dev.ix.1.queue2.rxd_tail: 1815 dev.ix.1.queue2.rx_packets: 2178840 dev.ix.1.queue2.rx_bytes: 1161300000 dev.ix.1.queue2.rx_copies: 1239240 dev.ix.1.queue2.lro_queued: 0 dev.ix.1.queue2.lro_flushed: 0 dev.ix.1.queue3.interrupt_rate: 500000 dev.ix.1.queue3.irqs: 2644559 dev.ix.1.queue3.txd_head: 0 dev.ix.1.queue3.txd_tail: 0 dev.ix.1.queue3.tso_tx: 0 dev.ix.1.queue3.no_tx_dma_setup: 0 dev.ix.1.queue3.no_desc_avail: 0 dev.ix.1.queue3.tx_packets: 0 dev.ix.1.queue3.rxd_head: 592 dev.ix.1.queue3.rxd_tail: 591 dev.ix.1.queue3.rx_packets: 2644560 dev.ix.1.queue3.rx_bytes: 1756521840 dev.ix.1.queue3.rx_copies: 1032240 dev.ix.1.queue3.lro_queued: 0 dev.ix.1.queue3.lro_flushed: 0 dev.ix.1.queue4.interrupt_rate: 500000 dev.ix.1.queue4.irqs: 1001156 dev.ix.1.queue4.txd_head: 0 dev.ix.1.queue4.txd_tail: 0 dev.ix.1.queue4.tso_tx: 0 dev.ix.1.queue4.no_tx_dma_setup: 0 dev.ix.1.queue4.no_desc_avail: 0 dev.ix.1.queue4.tx_packets: 0 dev.ix.1.queue4.rxd_head: 1736 dev.ix.1.queue4.rxd_tail: 1735 dev.ix.1.queue4.rx_packets: 1001160 dev.ix.1.queue4.rx_bytes: 566511600 dev.ix.1.queue4.rx_copies: 527520 dev.ix.1.queue4.lro_queued: 0 dev.ix.1.queue4.lro_flushed: 0 dev.ix.1.queue5.interrupt_rate: 500000 dev.ix.1.queue5.irqs: 1141199 dev.ix.1.queue5.txd_head: 0 dev.ix.1.queue5.txd_tail: 0 dev.ix.1.queue5.tso_tx: 0 dev.ix.1.queue5.no_tx_dma_setup: 0 dev.ix.1.queue5.no_desc_avail: 0 dev.ix.1.queue5.tx_packets: 0 dev.ix.1.queue5.rxd_head: 464 dev.ix.1.queue5.rxd_tail: 463 dev.ix.1.queue5.rx_packets: 1141200 dev.ix.1.queue5.rx_bytes: 759726240 dev.ix.1.queue5.rx_copies: 564600 dev.ix.1.queue5.lro_queued: 0 dev.ix.1.queue5.lro_flushed: 0 dev.ix.1.queue6.interrupt_rate: 500000 dev.ix.1.queue6.irqs: 1361040 dev.ix.1.queue6.txd_head: 0 dev.ix.1.queue6.txd_tail: 0 dev.ix.1.queue6.tso_tx: 0 dev.ix.1.queue6.no_tx_dma_setup: 0 dev.ix.1.queue6.no_desc_avail: 0 dev.ix.1.queue6.tx_packets: 0 dev.ix.1.queue6.rxd_head: 1168 dev.ix.1.queue6.rxd_tail: 1167 dev.ix.1.queue6.rx_packets: 1361040 dev.ix.1.queue6.rx_bytes: 772066560 dev.ix.1.queue6.rx_copies: 482640 dev.ix.1.queue6.lro_queued: 0 dev.ix.1.queue6.lro_flushed: 0 dev.ix.1.queue7.interrupt_rate: 500000 dev.ix.1.queue7.irqs: 1200826 dev.ix.1.queue7.txd_head: 0 dev.ix.1.queue7.txd_tail: 0 dev.ix.1.queue7.tso_tx: 0 dev.ix.1.queue7.no_tx_dma_setup: 0 dev.ix.1.queue7.no_desc_avail: 0 dev.ix.1.queue7.tx_packets: 0 dev.ix.1.queue7.rxd_head: 712 dev.ix.1.queue7.rxd_tail: 711 dev.ix.1.queue7.rx_packets: 1200840 dev.ix.1.queue7.rx_bytes: 744130800 dev.ix.1.queue7.rx_copies: 660720 dev.ix.1.queue7.lro_queued: 0 dev.ix.1.queue7.lro_flushed: 0 dev.ix.1.mac_stats.crc_errs: 0 dev.ix.1.mac_stats.ill_errs: 0 dev.ix.1.mac_stats.byte_errs: 0 dev.ix.1.mac_stats.short_discards: 0 dev.ix.1.mac_stats.local_faults: 7 dev.ix.1.mac_stats.remote_faults: 2 dev.ix.1.mac_stats.rec_len_errs: 0 dev.ix.1.mac_stats.xon_txd: 0 dev.ix.1.mac_stats.xon_recvd: 0 dev.ix.1.mac_stats.xoff_txd: 0 dev.ix.1.mac_stats.xoff_recvd: 0 dev.ix.1.mac_stats.total_octets_rcvd: 6831554760 dev.ix.1.mac_stats.good_octets_rcvd: 6831554760 dev.ix.1.mac_stats.total_pkts_rcvd: 12000000 dev.ix.1.mac_stats.good_pkts_rcvd: 12000000 dev.ix.1.mac_stats.mcast_pkts_rcvd: 12480 dev.ix.1.mac_stats.bcast_pkts_rcvd: 33720 dev.ix.1.mac_stats.rx_frames_64: 305160 dev.ix.1.mac_stats.rx_frames_65_127: 4237560 dev.ix.1.mac_stats.rx_frames_128_255: 2372520 dev.ix.1.mac_stats.rx_frames_256_511: 252720 dev.ix.1.mac_stats.rx_frames_512_1023: 1553160 dev.ix.1.mac_stats.rx_frames_1024_1522: 3278880 dev.ix.1.mac_stats.recv_undersized: 0 dev.ix.1.mac_stats.recv_fragmented: 0 dev.ix.1.mac_stats.recv_oversized: 0 dev.ix.1.mac_stats.recv_jabberd: 0 dev.ix.1.mac_stats.management_pkts_rcvd: 0 dev.ix.1.mac_stats.management_pkts_drpd: 0 dev.ix.1.mac_stats.checksum_errs: 0 dev.ix.1.mac_stats.good_octets_txd: 0 dev.ix.1.mac_stats.total_pkts_txd: 0 dev.ix.1.mac_stats.good_pkts_txd: 0 dev.ix.1.mac_stats.bcast_pkts_txd: 0 dev.ix.1.mac_stats.mcast_pkts_txd: 0 dev.ix.1.mac_stats.management_pkts_txd: 0 dev.ix.1.mac_stats.tx_frames_64: 0 dev.ix.1.mac_stats.tx_frames_65_127: 0 dev.ix.1.mac_stats.tx_frames_128_255: 0 dev.ix.1.mac_stats.tx_frames_256_511: 0 dev.ix.1.mac_stats.tx_frames_512_1023: 0 dev.ix.1.mac_stats.tx_frames_1024_1522: 0 PS: I'm not using zerocopy yet (that's the next step and part of the reason for my tests in the first place). # sysctl -a | grep bpf device bpf net.bpf.maxinsns: 512 net.bpf.zerocopy_enable: 0 net.bpf.optimize_writers: 0 net.bpf.bufsize: 4194304 net.bpf.maxbufsize: 2138815488 I hope this information help you understand what is going on. /Elof On Wed, 4 Nov 2015, Christian Peron wrote: > Your assumptions are correct, recv count is the number of packets which are received by the bpf peer, match count is the number of packets which matched the bpf filter that is active.. this shouldn’t be happening. Are you running netstat during the tcpreplay run or after it completed? Also can you give me more information on what specifically mon0 is? > > >> On Nov 4, 2015, at 10:44 AM, elof2@sentor.se wrote: >> >> >> Hi! >> >> Question: >> What do the Recv column in 'netstat -B' show? >> >> I thought it was tha amount of packets received, but appaently not so. >> >> I send 2000000 packets from a tcpreplay machine to a receiving machine. >> I do it a few times. >> >> On the receiver I see: >> netstat -in >> Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll >> ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 >> ix1 1500 0c:c4:7a:58:e2:3d 6000000 0 0 0 0 0 >> >> and then >> netstat -in >> Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll >> ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 0 0 >> ix1 1500 0c:c4:7a:58:e2:3d 8000000 0 0 0 0 0 >> >> So 6000000 has increased to 8000000. Good. >> >> >> However, 'netstat -B' show: >> Pid Netif Flags Recv Drop Match Sblen Hblen Command >> 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump >> >> How can the "Recv" be *lower* than "Match"? >> 1996862 < 2000000. >> >> >> For every new run (fast and slow) I get the same results, slightly less than 2000000 Recv. >> >> What am I missing? >> >> /Elof >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Nov 5 12:09:48 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64425A25F6B for ; Thu, 5 Nov 2015 12:09:48 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id 8C6B41AB6; Thu, 5 Nov 2015 12:09:46 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.98.5_1 for FreeBSD at relay2.tomsk.ru Received: from admin.sibptus.TOMSK.ru ([212.73.125.240] verified) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 38946626; Thu, 05 Nov 2015 18:09:44 +0600 Received: from admin.sibptus.TOMSK.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7) with ESMTP id tA5C9gXf016395; Thu, 5 Nov 2015 18:09:44 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.TOMSK.ru (8.14.9/8.14.7/Submit) id tA5C9gLw016394; Thu, 5 Nov 2015 18:09:42 +0600 (NOVT) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.TOMSK.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Thu, 5 Nov 2015 18:09:42 +0600 From: Victor Sudakov To: Julian Elischer Cc: Neel Natu , freebsd-net Subject: Re: tap(4) and host-only networking between host and guest Message-ID: <20151105120942.GC16353@admin.sibptus.tomsk.ru> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> <20151105023115.GA10139@admin.sibptus.tomsk.ru> <563B2DE4.2060307@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <563B2DE4.2060307@freebsd.org> Organization: OAO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 12:09:48 -0000 Julian Elischer wrote: [dd] > >>> For some reason, after a guest is shutdown or rebooted, the IP address > >>> on the host's tap0 interface is deleted. > >>> > >> Try using 'vmnet0' instead of 'tap0'. It will retain the IP address > >> even after a guest reboot/shutdown. > > Thanks, I see now vmnet is documented in tap(4). > I didn't know that... > Ok, so now you are an expert, can we count on you to answer all future > questions on this topic? ;-) > Oh, it's at http://victor-sudakov.dreamwidth.org/356635.html Sorry it's in Russian :-) -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:sudakov@sibptus.tomsk.ru From owner-freebsd-net@freebsd.org Thu Nov 5 12:31:06 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C78EA2371C for ; Thu, 5 Nov 2015 12:31:06 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward13h.cmail.yandex.net (forward13h.cmail.yandex.net [IPv6:2a02:6b8:0:f35::9e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28B5B14F4 for ; Thu, 5 Nov 2015 12:31:06 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from web29h.yandex.ru (web29h.yandex.ru [84.201.187.163]) by forward13h.cmail.yandex.net (Yandex) with ESMTP id 5FE7C2144A; Thu, 5 Nov 2015 15:31:01 +0300 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web29h.yandex.ru (Yandex) with ESMTP id D90B82FC23D2; Thu, 5 Nov 2015 15:31:00 +0300 (MSK) Received: by web29h.yandex.ru with HTTP; Thu, 05 Nov 2015 15:31:00 +0300 From: Alexander V. Chernikov Envelope-From: melifaro@ipfw.ru To: "elof2@sentor.se" , freebsd-net In-Reply-To: References: null Subject: Re: netstat -B "Recv" MIME-Version: 1.0 Message-Id: <111891446726660@web29h.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Thu, 05 Nov 2015 15:31:00 +0300 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=koi8-r X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 12:31:06 -0000 04.11.2015, 19:55, "elof2@sentor.se" : > Hi! > > Question: > What do the Recv column in 'netstat -B' show? > > I thought it was tha amount of packets received, but appaently not so. > > I send 2000000 packets from a tcpreplay machine to a receiving machine. > I do it a few times. > > On the receiver I see: > netstat -in > Name Mtu Network Address Ipkts Ierrs Idrop Opkts > Oerrs Coll > ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 > 0 0 > ix1 1500 0c:c4:7a:58:e2:3d 6000000 0 0 0 > 0 0 > > and then > netstat -in > Name Mtu Network Address Ipkts Ierrs Idrop Opkts > Oerrs Coll > ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 > 0 0 > ix1 1500 0c:c4:7a:58:e2:3d 8000000 0 0 0 > 0 0 > > So 6000000 has increased to 8000000. Good. > > However, 'netstat -B' show: > šššPid Netif Flags Recv Drop Match Sblen Hblen Command > 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump > > How can the "Recv" be *lower* than "Match"? > 1996862 < 2000000. > > For every new run (fast and slow) I get the same results, slightly less > than 2000000 Recv. > > What am I missing? Well, "Recv" is read from d->bd_rcount which is not per-cpu counter and is incrementing unlocked. On the other hand, "Match" increases when filter returned match condition and we (w)locked bpf descriptor, so this one is accurate. > > /Elof > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Nov 5 14:51:37 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86778A26FB9 for ; Thu, 5 Nov 2015 14:51:37 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from smtp-out.sentor.se (smtp-out.sentor.se [176.124.225.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 474211536; Thu, 5 Nov 2015 14:51:37 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from localhost (localhost [127.0.0.1]) by farmermaggot.shire.sentor.se (Postfix) with ESMTP id 55AC0B61D235; Thu, 5 Nov 2015 15:51:33 +0100 (CET) Date: Thu, 5 Nov 2015 15:51:33 +0100 (CET) From: "elof2@sentor.se" To: "Alexander V. Chernikov" cc: freebsd-net Subject: Re: netstat -B "Recv" In-Reply-To: <111891446726660@web29h.yandex.ru> Message-ID: References: null <111891446726660@web29h.yandex.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-ID: Content-Type: TEXT/PLAIN; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 14:51:37 -0000 On Thu, 5 Nov 2015, Alexander V. Chernikov wrote: > > > 04.11.2015, 19:55, "elof2@sentor.se" : >> Hi! >> >> Question: >> What do the Recv column in 'netstat -B' show? >> >> I thought it was tha amount of packets received, but appaently not so. >> >> I send 2000000 packets from a tcpreplay machine to a receiving machine. >> I do it a few times. >> >> On the receiver I see: >> netstat -in >> Name Mtu Network Address Ipkts Ierrs Idrop Opkts >> Oerrs Coll >> ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 >> 0 0 >> ix1 1500 0c:c4:7a:58:e2:3d 6000000 0 0 0 >> 0 0 >> >> and then >> netstat -in >> Name Mtu Network Address Ipkts Ierrs Idrop Opkts >> Oerrs Coll >> ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 >> 0 0 >> ix1 1500 0c:c4:7a:58:e2:3d 8000000 0 0 0 >> 0 0 >> >> So 6000000 has increased to 8000000. Good. >> >> However, 'netstat -B' show: >>    Pid Netif Flags Recv Drop Match Sblen Hblen Command >> 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump >> >> How can the "Recv" be *lower* than "Match"? >> 1996862 < 2000000. >> >> For every new run (fast and slow) I get the same results, slightly less >> than 2000000 Recv. >> >> What am I missing? > Well, "Recv" is read from d->bd_rcount which is not per-cpu counter and is incrementing unlocked. > On the other hand, "Match" increases when filter returned match condition and we (w)locked bpf descriptor, so this one is accurate. Ah. Thanks. Will you make a bugzilla out of this? Or should I? Or is it not interesting enough to fix? /Elof From owner-freebsd-net@freebsd.org Thu Nov 5 15:00:48 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 892A7A2707A for ; Thu, 5 Nov 2015 15:00:48 +0000 (UTC) (envelope-from barney@databus.com) Received: from pit.databus.com (Databus-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:80b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 532AB1A90; Thu, 5 Nov 2015 15:00:48 +0000 (UTC) (envelope-from barney@databus.com) Received: by pit.databus.com (Postfix, from userid 202) id 965B65AC0; Thu, 5 Nov 2015 10:00:46 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=databus.com; s=20140217; t=1446735646; bh=iTSPnlhTZ4bqKcpISfRInXX4/UH5/rPvoVUZiyTrpjw=; l=938; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=L5VODmSl48d17/D054AzEglHpkvfRzvL6Kk+4qZLauU4ZrTO6pPm1bA9DYuVKVsap yMFBqL1UE02/hDPhd3e8PM+mMW2foZ9/EUzTUNfKYrbJUsTmPEPk37mu5axWaMgNnk bqzW2MH30LH9b+4NB5qTztE8iWLko8J7WrjLjjQf5Jey9cZKPa4RjqkZTmZ2HDG5l5 9bQpap2iF0SJgNwaLcg97ouGygYFswOND3QKJhi2j9wzDxG7tznPA/hX0WMqo7L5FW vFS3cHtS2qNRvJwTV0kWzYAM0Jh8Buc/hRQ/6bLLmEPtOfnSmWz7znoF62LbTv1Jsc 04z9IVuuatePw== Date: Thu, 5 Nov 2015 10:00:46 -0500 From: Barney Wolff To: Andriy Gapon Cc: Ben Woods , "freebsd-net@freebsd.org" Subject: Re: who uses this port? Message-ID: <20151105150046.GA69422@pit.databus.com> References: <563A5F39.7010906@FreeBSD.org> <563B0922.9030805@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <563B0922.9030805@FreeBSD.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 15:00:48 -0000 On Thu, Nov 05, 2015 at 09:45:38AM +0200, Andriy Gapon wrote: > On 05/11/2015 09:20, Ben Woods wrote: > > On Wednesday, 4 November 2015, Andriy Gapon > > wrote: > > > > $ sockstat -l | fgrep 631 > > ? ? ? ? tcp4 127.0.0.1:631 > > *:* > > > > $ nc -l 127.0.0.1 631 > > nc: Address already in use > > > > > > > > I'm more curious as to why sockstat gives you question marks instead of the > > proper process details. Any ideas? > > Yeah, I should have stated my question more accurately. What you are asking is > what I actually intended to ask. > > I was debugging a problem of cupsd not being able to bind to its port after a > restart. Eventually I had to reboot the affected system. Might be mountd. I once saw it bind to the imaps port, also not good. Forcing it on a port I don't use cured that, but I have a suspicion that the real solution lies in the rc order. From owner-freebsd-net@freebsd.org Thu Nov 5 15:01:24 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85ABEA27154 for ; Thu, 5 Nov 2015 15:01:24 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from smtp-out.sentor.se (smtp-out.sentor.se [176.124.225.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D0B91BB2 for ; Thu, 5 Nov 2015 15:01:23 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from localhost (localhost [127.0.0.1]) by farmermaggot.shire.sentor.se (Postfix) with ESMTP id 2EEE0B61D235 for ; Thu, 5 Nov 2015 16:01:22 +0100 (CET) Date: Thu, 5 Nov 2015 16:01:22 +0100 (CET) From: elof2@sentor.se To: freebsd-net Subject: ngrep/ixgbe bpf bug Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 15:01:24 -0000 Hi all! Why don't ngrep work on ix interfaces? It shows nice values if I sniff on other interfaces, e.g. igb0 and bridge0: # ngrep -d igb0 "q" ip I see some matching packets. Everything looks good. # netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1800 igb0 p--s--- 135 0 129 380 0 ngrep The BPF stats show Recv and Match values. Good. # ngrep -d bridge0 "q" ip I see some matching packets. Good. # netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1901 bridge0 p--s--- 661897 0 659170 425606 0 ngrep Again, the BPF stats show Recv and Match values. Good. However, if I sniff on an ix interface: # ngrep -d ix1 "q" ip I get no matching packets! # netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1816 ix1 p--s--- 45340 0 0 0 0 ngrep ^^^ ...and the BPF stats always show zero Matches. Bug in the ixgbe driver or in ngrep? /Elof From owner-freebsd-net@freebsd.org Thu Nov 5 15:38:11 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69BFEA277A4 for ; Thu, 5 Nov 2015 15:38:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 729E31E32; Thu, 5 Nov 2015 15:38:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA06994; Thu, 05 Nov 2015 17:38:07 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZuMc1-0006O9-OM; Thu, 05 Nov 2015 17:38:05 +0200 Subject: Re: who uses this port? To: Matthew Seaman , freebsd-net@FreeBSD.org References: <563A5F39.7010906@FreeBSD.org> <563B0922.9030805@FreeBSD.org> <563B285A.4070808@freebsd.org> From: Andriy Gapon Message-ID: <563B77A5.4060403@FreeBSD.org> Date: Thu, 5 Nov 2015 17:37:09 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <563B285A.4070808@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 15:38:11 -0000 On 05/11/2015 11:58, Matthew Seaman wrote: > On 05/11/2015 09:20, Ben Woods wrote: >>> I'm more curious as to why sockstat gives you question marks instead of the >>>> proper process details. Any ideas? > > This indicates a connection that is in the process of closing down but > the system still hasn't cleared up all the connection state yet. It's > normal. You might find that adjusting the > net.inet.tcp.fast_finwait2_recycle sysctl makes these old entries get > cleared more promptly. The thing is that the system was in that state for about 10 minutes after which I grew impatient and rebooted it. To me 10 minutes seem like more than plenty of time even for the slow algorithm. -- Andriy Gapon From owner-freebsd-net@freebsd.org Thu Nov 5 16:25:44 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A967A263F4 for ; Thu, 5 Nov 2015 16:25:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 236D51A5D for ; Thu, 5 Nov 2015 16:25:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA5GPiHh058247 for ; Thu, 5 Nov 2015 16:25:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 203630] [Hyper-V] [nat] [tcp] 10.2 NAT bug in TCP stack or hyperv netsvc driver Date: Thu, 05 Nov 2015 16:25:42 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: onyx@netfusion.fr X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 16:25:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203630 --- Comment #16 from Eddy --- Hi and thank you for the patch. I just applied it on a clean 10.2 kernel and made the same tests as before. It seems to solve the issue! I wait for it to be officially included in the next official updates. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-net@freebsd.org Thu Nov 5 16:52:20 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16D55A26AE7 for ; Thu, 5 Nov 2015 16:52:20 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A1222192C; Thu, 5 Nov 2015 16:52:19 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id tA5Gq50V072614 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 5 Nov 2015 17:52:07 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: avg@FreeBSD.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id tA5Gpwp3081072; Thu, 5 Nov 2015 23:51:59 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: who uses this port? To: Andriy Gapon , "freebsd-net@freebsd.org" References: <563A5F39.7010906@FreeBSD.org> From: Eugene Grosbein Message-ID: <563B892E.3040706@grosbein.net> Date: Thu, 5 Nov 2015 23:51:58 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <563A5F39.7010906@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 16:52:20 -0000 On 05.11.2015 02:40, Andriy Gapon wrote: > $ sockstat -l | fgrep 631 > ? ? ? ? tcp4 127.0.0.1:631 *:* > > $ nc -l 127.0.0.1 631 > nc: Address already in use That may be nlockmgr, as for my system: # sockstat -4lP tcp | fgrep \? ? ? ? ? tcp4 *:698 *:* # rpcinfo -p | grep 698 100021 0 tcp 698 nlockmgr 100021 1 tcp 698 nlockmgr 100021 3 tcp 698 nlockmgr 100021 4 tcp 698 nlockmgr From owner-freebsd-net@freebsd.org Thu Nov 5 16:54:07 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89B1AA26B45 for ; Thu, 5 Nov 2015 16:54:07 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FC7F1A29; Thu, 5 Nov 2015 16:54:06 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id tA5GrxvI072623 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 5 Nov 2015 17:54:00 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: woodsb02@gmail.com Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id tA5Grr4c081085; Thu, 5 Nov 2015 23:53:53 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: who uses this port? To: Ben Woods , Andriy Gapon References: <563A5F39.7010906@FreeBSD.org> Cc: "freebsd-net@freebsd.org" From: Eugene Grosbein Message-ID: <563B89A1.7090109@grosbein.net> Date: Thu, 5 Nov 2015 23:53:53 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, T_DATE_IN_FUTURE_96_Q autolearn=no version=3.3.2 X-Spam-Report: * 0.0 T_DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 16:54:07 -0000 On 05.11.2015 14:20, Ben Woods wrote: > On Wednesday, 4 November 2015, Andriy Gapon wrote: > >> $ sockstat -l | fgrep 631 >> ? ? ? ? tcp4 127.0.0.1:631 *:* >> >> $ nc -l 127.0.0.1 631 >> nc: Address already in use >> > > > I'm more curious as to why sockstat gives you question marks instead of the > proper process details. Any ideas? That's for sockets opened by kernel level such as NFS/RPC. From owner-freebsd-net@freebsd.org Thu Nov 5 17:06:56 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 037D7A26FBA for ; Thu, 5 Nov 2015 17:06:56 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D3521790 for ; Thu, 5 Nov 2015 17:06:55 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id tA5H6esL072671 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 5 Nov 2015 18:06:41 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: vas@mpeks.tomsk.su Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id tA5H6YaZ081238; Fri, 6 Nov 2015 00:06:34 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: tap(4) and host-only networking between host and guest To: Victor Sudakov , freebsd-net@freebsd.org References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> From: Eugene Grosbein Message-ID: <563B8C9A.7070104@grosbein.net> Date: Fri, 6 Nov 2015 00:06:34 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20151104184503.GC1117@admin.sibptus.tomsk.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, T_DATE_IN_FUTURE_96_Q autolearn=no version=3.3.2 X-Spam-Report: * 0.0 T_DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 17:06:56 -0000 On 05.11.2015 01:45, Victor Sudakov wrote: > For some reason, after a guest is shutdown or rebooted, the IP address > on the host's tap0 interface is deleted. > > It's kind of inconvenient. Yes, it is. And there is a solution: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165174 From owner-freebsd-net@freebsd.org Fri Nov 6 00:09:07 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA169A25328 for ; Fri, 6 Nov 2015 00:09:07 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id 6121F17B2 for ; Fri, 6 Nov 2015 00:09:06 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.20,249,1444654800"; d="scan'208";a="14612044" Received: from gsp-ex01.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.17]) by iport1.cc.swin.edu.au with ESMTP; 06 Nov 2015 11:09:00 +1100 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.2]) by gsp-ex01.ds.swin.edu.au ([169.254.1.85]) with mapi id 14.03.0248.002; Fri, 6 Nov 2015 11:09:00 +1100 From: Rasool Al-Saadi To: Hans Petter Selasky , "freebsd-net@freebsd.org" Subject: RE: Timing issue with Dummynet on high kernel timer interrupt Thread-Topic: Timing issue with Dummynet on high kernel timer interrupt Thread-Index: AdEWOVE5DoAlkzqPQWaBZ8ln4esVd///TXiA//0UVtCABdKzgP/+Z6Xg Date: Fri, 6 Nov 2015 00:08:59 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> In-Reply-To: <563B2703.5080402@selasky.org> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.126.11] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 00:09:07 -0000 On Thursday, 5 November 2015 8:53 PM, Hans Petter Selasky wrote: >=20 > On 11/05/15 00:44, Rasool Al-Saadi wrote: > > > > On Wednesday, 4 November 2015 12:34 AM, Hans Petter Selasky wrote: > >> On 11/03/15 14:14, Rasool Al-Saadi wrote: > >>> Does anyone have thoughts on what we can test next to narrow down > >>> the > >> root-cause of these unusual timing jumps? > >> > >> You might also want to test the "projects/hps_head" branch, which > >> uses a bit different callout implementation. > > > > Thanks Hans for your suggestion. > > I have tried "projects/hps_head" branch and the result is better (numbe= r > of spikes is less than in the master branch). However, the problem still = exists > on the same timer interrupt frequencies (>3000 in my case). You can see i= n > this graph https://goo.gl/photos/C2Mqx4xhMQuzxWnz6 the RTT spikes still > there. > > > > Do you have any further suggestions? > > >=20 > Hi, >=20 > If the jitter is in the xx milliseconds range, like your graph shows, my = guess is > that td_owepreemt is not set when we return from the dummynet() > callback. See attached patch. The patch doesn't solve the problem. I tried it on the master and hps branc= hes. > Else you might want to try to remove the C_HARDCLOCK flag from > callout_reset_sbt() in ip_dummynet.c. Removing C_HARDCLOCK reduces the problem but doesn't solve it completely. = However, removing C_DIRECT_EXEC instead solves the problem (but occasiona= lly very small spike(s) appears in high hz values). =20 I mentioned in my first email that removing these flags makes the issue to = disappear. But what the effects of removing these flags? If it cause timing= issue to Dummynet, why we should use them? Regards, Rasool > --HPS From owner-freebsd-net@freebsd.org Fri Nov 6 00:36:39 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1AF7A25A58 for ; Fri, 6 Nov 2015 00:36:39 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5389415B9 for ; Fri, 6 Nov 2015 00:36:38 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B9ABB28411; Fri, 6 Nov 2015 01:36:28 +0100 (CET) Received: from illbsd.quip.test (ip-89-177-49-111.net.upcbroadband.cz [89.177.49.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 269D628422; Fri, 6 Nov 2015 01:36:27 +0100 (CET) Message-ID: <563BF60B.5090201@quip.cz> Date: Fri, 06 Nov 2015 01:36:27 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Eugene Grosbein , Victor Sudakov , freebsd-net@freebsd.org Subject: Re: tap(4) and host-only networking between host and guest References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> <563B8C9A.7070104@grosbein.net> In-Reply-To: <563B8C9A.7070104@grosbein.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 00:36:39 -0000 Eugene Grosbein wrote on 11/05/2015 18:06: > On 05.11.2015 01:45, Victor Sudakov wrote: > >> For some reason, after a guest is shutdown or rebooted, the IP address >> on the host's tap0 interface is deleted. >> >> It's kind of inconvenient. > > Yes, it is. And there is a solution: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165174 I don't understand why such useful patches are left uncommited and without any comments in PR for years. Thank you for the link! Miroslav Lachman From owner-freebsd-net@freebsd.org Fri Nov 6 00:52:57 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67A1EA25EA2 for ; Fri, 6 Nov 2015 00:52:57 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.strugglingcoder.info", Issuer "mail.strugglingcoder.info" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 469F21E3E; Fri, 6 Nov 2015 00:52:56 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 0935C10B5BD; Thu, 5 Nov 2015 16:52:56 -0800 (PST) Date: Thu, 5 Nov 2015 16:52:56 -0800 From: hiren panchasara To: Midori Kato Cc: "K. Macy" , "freebsd-net@freebsd.org" , Don Lewis Subject: Re: default ECN settings Message-ID: <20151106005256.GE69928@strugglingcoder.info> References: <201509050053.t850rh9P071595@gw.catspoiler.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="w/VI3ydZO+RcZ3Ux" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 00:52:57 -0000 --w/VI3ydZO+RcZ3Ux Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 11/05/15 at 06:58P, Midori Kato wrote: > Hi Macy and Don, >=20 > I am Midori. Too late to catch up this topic but this topic is interesting > to me. > Linux separates inbound and outbound ecn operation while RFC 3168 says th= at > making hosts fail during the negotiation without ecn configuration. >=20 > I think FreeBSD is probably able to distinguish inbound and outbound with > cc_var flag as well. > I like to try to work this. If the sender like to use ECN, behaving as ECN > receiver is good for the TCP connection. >=20 > Regards, > -- Midori >=20 >=20 > 2015-09-05 10:05 GMT+09:00 K. Macy : >=20 > > On Fri, Sep 4, 2015 at 5:53 PM, Don Lewis wrote: > > > On 4 Sep, K. Macy wrote: > > >> By default ECN is completely disabled on FreeBSD. On Linux the defau= lt > > >> is to disable it outbound (not request it) but enable it inbound > > >> (accept new connections asking for it). Is there a good reason to on= ly > > >> set ECN_PERMIT on inbound connections if the system is doing ECN on > > >> outbound connections? > > > > > > Not that I can think of. The risk in enabling ECN for outbound > > > connections is that some connection attempts can fail, especially if = you > > > are attempting to connect to some old and oddball device. That should > > > not be a risk for inbound connections since those devices won't be > > > requesting ECN. > > > > Even with 'oddball' devices the stack is configured to retry ECN n > > times where n defaults to 1 and then revert to not requesting ECN > > support. Thus connections would take longer on 'oddball' devices. The > > solution that *I* would choose for that would be to track ECN support > > in the host cache. The first connection to a new host would always try > > ECN and in the event that that failed all subsequent connection > > attempts would not try ECN. To me this seems like the most robust > > compromise. However, I don't yet have enough information to say how > > much benefit this would confer. ECN is a good thing to have and I think that we should support it if an incoming connection requests it. I also like this approach suggested by Kip for implementation. > > > > > Seems like we should be defaulting ECN on for inbound connections, > > > though we currently can't control the two directions separately. > > > > That is a straightforward change. Just to clarify, with/after this change, the default behavior would be: enabled on inbound and disabled on outbound. And we should also have a way to disable ecn completely on both directions. Cheers, Hiren --w/VI3ydZO+RcZ3Ux Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWO/nkXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lVPIH/jSQSWqjVUAWbXYy2V6Q9IFN vYjMwvaRGSfu5tAc3qGXEgXuU4+eM0CdMsmIVXH5SC+7vwpR74IOpPq5ao/cot9J 0j3WRRUkVo3Aqo4Ag9bNW5VzoUSZ6Bgoa9UfAPH8wZlKTfyWc1JHyQXncInZhHHz rewyiyKoIxmSr4q8pLnRSMwQkrRfUmdMZtCVoUJuXBG87DmBzLeIMkJs86C/EAfg bveBLWKxt+5xt1Ub/uJPf5o6gi8aJfhodNmXzYtcS9DJZbB4JN1yV2mBOsddB8Fw TRDRjqF21WrW01bWtz9PliZq5SddNlHgTfKTadsa9eTV4qmTqIZDS9+hWjvqQs0= =VCtp -----END PGP SIGNATURE----- --w/VI3ydZO+RcZ3Ux-- From owner-freebsd-net@freebsd.org Fri Nov 6 01:00:00 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 179FDA25FB6 for ; Fri, 6 Nov 2015 01:00:00 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mail-yk0-x22d.google.com (mail-yk0-x22d.google.com [IPv6:2607:f8b0:4002:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C36E6106A for ; Fri, 6 Nov 2015 00:59:59 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: by ykek133 with SMTP id k133so164037064yke.2 for ; Thu, 05 Nov 2015 16:59:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=XpniBC421hULVUwFEv4yBCJHoUrIs02hBFH/jGg5vls=; b=Nn7+MxF4dTvE0YnVP0D46g38GuSSpKPMeUbHgBfuPIuCjuvlyyAHZ9ePS0zsltW+Tp 5ioEueEQ8ltn13pGaAk7JioPEHe062QDxPfEBoSz2CBt+5IQ3iTFKU+0bHbbwGyrOUES lFOKnIGDKqm7yujpg4EvtGKzAt8mtl0icPz8010SdfeM3F9dRF+EeZDt6J1fTUJn6zOO NfyP4mvh5upJvaue+osTzA626adxsSlvfJBXfMzHV6W6h2btKyFxXfX3dW1pd0aedYrE RBxrOaXj3zK0HyxSPDJj5up89loUNqT/XE4v1ozjc8aKNYAeQleDJBC2wPQysLTQyOZs DKMg== MIME-Version: 1.0 X-Received: by 10.129.96.87 with SMTP id u84mr4166725ywb.81.1446771598861; Thu, 05 Nov 2015 16:59:58 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.37.95.9 with HTTP; Thu, 5 Nov 2015 16:59:58 -0800 (PST) In-Reply-To: <563BF60B.5090201@quip.cz> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> <563B8C9A.7070104@grosbein.net> <563BF60B.5090201@quip.cz> Date: Thu, 5 Nov 2015 16:59:58 -0800 X-Google-Sender-Auth: wc4GYw31DuC5PZMboigOVfosOO4 Message-ID: Subject: Re: tap(4) and host-only networking between host and guest From: Craig Rodrigues To: Miroslav Lachman <000.fbsd@quip.cz> Cc: Eugene Grosbein , Victor Sudakov , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 01:00:00 -0000 On Thu, Nov 5, 2015 at 4:36 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: > Eugene Grosbein wrote on 11/05/2015 18:06: > >> Yes, it is. And there is a solution: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165174 >> > > I don't understand why such useful patches are left uncommited and without > any comments in PR for years. > > This is not official project policy, but in future, I recommend that for submitting patches to the project, people try to use the steps at: https://wiki.freebsd.org/CodeReview to submit the patch to Phabricator. As a FreeBSD developer, I personally find that Phabricator is easier to work with patches thatn Bugzilla. Other FreeBSD developers may disagree with me, but more FreeBSD developers are starting to use Phabricator. -- Craig From owner-freebsd-net@freebsd.org Fri Nov 6 07:09:25 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C025DA27C77 for ; Fri, 6 Nov 2015 07:09:25 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 727F01450 for ; Fri, 6 Nov 2015 07:09:24 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id E3134200126 for ; Fri, 6 Nov 2015 08:09:15 +0100 (CET) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id D5399405882 for ; Fri, 6 Nov 2015 08:09:15 +0100 (CET) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id B353F405881 for ; Fri, 6 Nov 2015 08:09:15 +0100 (CET) Received: from arc.aei.uni-hannover.de ([10.117.15.110]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6HF1691) with ESMTP id 2015110608090531-374689 ; Fri, 6 Nov 2015 08:09:05 +0100 Date: Fri, 6 Nov 2015 08:09:05 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-net@freebsd.org Subject: Re: strange nfs/rsync stalls Message-Id: <20151106080905.cb3369337a481b23dc894539@aei.mpg.de> In-Reply-To: <3546_1446644979_563A0CF3_3546_724_1_20151104144354.acccf3c7315a6009264d755d@aei.mpg.de> References: <3546_1446644979_563A0CF3_3546_724_1_20151104144354.acccf3c7315a6009264d755d@aei.mpg.de> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 06.11.2015 08:09:05, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 06.11.2015 08:09:15, Serialize complete at 06.11.2015 08:09:15 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1 X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2015.11.6.70015 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1000_LESS 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, BODY_SIZE_900_999 0, NO_URI_HTTPS 0, REFERENCES 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __REFERENCES 0, __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 07:09:25 -0000 On Wed, 4 Nov 2015 14:43:54 +0100 Gerrit K=FChn wrote about strange nfs/rsync stalls: > I have a couple of other machines that use the same server in the same > way, none of them ever showed this behaviour. Are there any > suggestions how to debug/fix this? Some more pointers from my side: I see a similar issue when using rsync over ssh, so this does not appear to be a nfs issue. However, even when running rsync with -vvv and debug turned on, I cannot find any hard reason for the lockups. They appear to happen erratically, not at a fixed file or such. When transferring over ssh, I can press CTRL-C to make the syncing go on until the next lockup happens. I also tried to swap client and server (i.e., the machine where I start rsync), and that runs without any issue... which leaves me even more puzzled. Any hints are still very welcome. cu Gerrit From owner-freebsd-net@freebsd.org Fri Nov 6 08:09:35 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9ACFFA26BF9 for ; Fri, 6 Nov 2015 08:09:35 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 147CE197A; Fri, 6 Nov 2015 08:09:33 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id tA689Otu075713 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 6 Nov 2015 09:09:26 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: rodrigc@FreeBSD.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id tA689Dd2089655; Fri, 6 Nov 2015 15:09:14 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: tap(4) and host-only networking between host and guest To: Craig Rodrigues , Miroslav Lachman <000.fbsd@quip.cz> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> <563B8C9A.7070104@grosbein.net> <563BF60B.5090201@quip.cz> Cc: FreeBSD Net , Victor Sudakov From: Eugene Grosbein Message-ID: <563C6029.5010109@grosbein.net> Date: Fri, 6 Nov 2015 15:09:13 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 08:09:35 -0000 On 06.11.2015 07:59, Craig Rodrigues wrote: > On Thu, Nov 5, 2015 at 4:36 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: > >> Eugene Grosbein wrote on 11/05/2015 18:06: >> >>> Yes, it is. And there is a solution: >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165174 >>> >> >> I don't understand why such useful patches are left uncommited and without >> any comments in PR for years. >> >> > This is not official project policy, but in future, I recommend that for > submitting patches to the > project, people try to use the steps at: https://wiki.freebsd.org/CodeReview > to submit the patch to Phabricator. > > As a FreeBSD developer, I personally find that Phabricator is easier to > work with patches thatn Bugzilla. > > Other FreeBSD developers may disagree with me, but more FreeBSD developers > are starting to use Phabricator. I've just read https://wiki.freebsd.org/CodeReview and registered at Phabricator. It seems more developer-centric (it needs commit-messages etc.) and over-complicated for non-developer reporting small bug and supplying small fix. From owner-freebsd-net@freebsd.org Fri Nov 6 08:42:40 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C6DFA27451 for ; Fri, 6 Nov 2015 08:42:40 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E12211806 for ; Fri, 6 Nov 2015 08:42:39 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id BFDE61FE023; Fri, 6 Nov 2015 09:42:36 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Rasool Al-Saadi , "freebsd-net@freebsd.org" References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> From: Hans Petter Selasky Message-ID: <563C6864.2090907@selasky.org> Date: Fri, 6 Nov 2015 09:44:20 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> Content-Type: multipart/mixed; boundary="------------010202050408090608050403" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 08:42:40 -0000 This is a multi-part message in MIME format. --------------010202050408090608050403 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 11/06/15 01:08, Rasool Al-Saadi wrote: > > On Thursday, 5 November 2015 8:53 PM, Hans Petter Selasky wrote: >> >> On 11/05/15 00:44, Rasool Al-Saadi wrote: >>> >>> On Wednesday, 4 November 2015 12:34 AM, Hans Petter Selasky wrote: >>>> On 11/03/15 14:14, Rasool Al-Saadi wrote: >>>>> Does anyone have thoughts on what we can test next to narrow down >>>>> the >>>> root-cause of these unusual timing jumps? >>>> >>>> You might also want to test the "projects/hps_head" branch, which >>>> uses a bit different callout implementation. >>> >>> Thanks Hans for your suggestion. >>> I have tried "projects/hps_head" branch and the result is better (number >> of spikes is less than in the master branch). However, the problem still exists >> on the same timer interrupt frequencies (>3000 in my case). You can see in >> this graph https://goo.gl/photos/C2Mqx4xhMQuzxWnz6 the RTT spikes still >> there. >>> >>> Do you have any further suggestions? >>> >> >> Hi, >> >> If the jitter is in the xx milliseconds range, like your graph shows, my guess is >> that td_owepreemt is not set when we return from the dummynet() >> callback. See attached patch. > > The patch doesn't solve the problem. I tried it on the master and hps branches. > >> Else you might want to try to remove the C_HARDCLOCK flag from >> callout_reset_sbt() in ip_dummynet.c. > > Removing C_HARDCLOCK reduces the problem but doesn't solve it completely. However, removing C_DIRECT_EXEC instead solves the problem (but occasionally very small spike(s) appears in high hz values). > I mentioned in my first email that removing these flags makes the issue to disappear. But what the effects of removing these flags? If it cause timing issue to Dummynet, why we should use them? > Hi, The C_DIRECT_EXEC flag reduces task switching overhead, that you don't have to wakeup a thread to wakeup the dummynet worker thread. It affects timing. Here is one more patch you can try. See attachment. It restarts the timeout from within the timer callback, instead of when the worker thread is executing. --HPS --------------010202050408090608050403 Content-Type: text/x-patch; name="ipfw_reschedule.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ipfw_reschedule.diff" Index: sys/netpfil/ipfw/ip_dn_io.c =================================================================== --- sys/netpfil/ipfw/ip_dn_io.c (revision 290134) +++ sys/netpfil/ipfw/ip_dn_io.c (working copy) @@ -712,7 +712,7 @@ } DN_BH_WUNLOCK(); - dn_reschedule(); + //dn_reschedule(); if (q.head != NULL) dummynet_send(q.head); CURVNET_RESTORE(); Index: sys/netpfil/ipfw/ip_dummynet.c =================================================================== --- sys/netpfil/ipfw/ip_dummynet.c (revision 290134) +++ sys/netpfil/ipfw/ip_dummynet.c (working copy) @@ -84,6 +84,7 @@ (void)arg; /* UNUSED */ taskqueue_enqueue_fast(dn_tq, &dn_task); + dn_reschedule(); } void --------------010202050408090608050403-- From owner-freebsd-net@freebsd.org Fri Nov 6 08:50:32 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BA20A27653 for ; Fri, 6 Nov 2015 08:50:32 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1127C1BE1 for ; Fri, 6 Nov 2015 08:50:32 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by lbbkw15 with SMTP id kw15so47084129lbb.0 for ; Fri, 06 Nov 2015 00:50:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Qxtb/vNt0Kh2uCRI/MEah6TrxjPGP7K0H5pN/4CCoc4=; b=qkzrJ0+uJdxKbVCg7XDX1puvpJ3dpcnxxgj6CU5+mLePkbZHfbdtgHfb1ofi3a9IsL 46cx+1gHJ26a6RwCQLQLPyMhl16uPTD4xkbzRx8liLFzfwdSopSXTgM1LQxlXezOKwEM gUuHWBHjUv37K9x7uyyOORa6lZ92JtvYxGbD3cQ8f/4u2KXCYtEgZrg2MbsW8cNDswwb DPJyFuN+OaAOeCAuT2xmLQDTP+6uDfpPtqu1Tvb6J//mIhT9yV8T+zz760LXBPazPUzh /LViFBQWJLeiKWAnZvkUcVd792te3Rd0dvnKs2OPrh//a+YyQlwnpx70jRc5MxhvSTAK 35Vg== MIME-Version: 1.0 X-Received: by 10.112.180.230 with SMTP id dr6mr6179523lbc.72.1446799830227; Fri, 06 Nov 2015 00:50:30 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.78.3 with HTTP; Fri, 6 Nov 2015 00:50:30 -0800 (PST) In-Reply-To: <563C6864.2090907@selasky.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> Date: Fri, 6 Nov 2015 09:50:30 +0100 X-Google-Sender-Auth: Zv6ExREoqnR0XtdvrePTnCWegBc Message-ID: Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Luigi Rizzo To: Hans Petter Selasky Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 08:50:32 -0000 On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky wrote: > On 11/06/15 01:08, Rasool Al-Saadi wrote: >> >> >> On Thursday, 5 November 2015 8:53 PM, Hans Petter Selasky wrote: >>> >>> >>> On 11/05/15 00:44, Rasool Al-Saadi wrote: ... >> Removing C_HARDCLOCK reduces the problem but doesn't solve it completely. >> However, removing C_DIRECT_EXEC instead solves the problem (but >> occasionally very small spike(s) appears in high hz values). >> I mentioned in my first email that removing these flags makes the issue to >> disappear. But what the effects of removing these flags? If it cause timing >> issue to Dummynet, why we should use them? >> > > Hi, > > The C_DIRECT_EXEC flag reduces task switching overhead, that you don't have > to wakeup a thread to wakeup the dummynet worker thread. It affects timing. Hans, thanks for the explanation. Can you clarify the behaviour of C_DIRECT_EXEC ? Does this mean that the task is run within some common thread instead of a dedicated one ? If so, for this type of task (dummynet may run at high rate and use a significant amount of cpu time) it may be a good idea to remove C_DIRECT_EXEC altogether. cheers luigi From owner-freebsd-net@freebsd.org Fri Nov 6 09:51:09 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF90EA26C9A for ; Fri, 6 Nov 2015 09:51:09 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 919391DB4 for ; Fri, 6 Nov 2015 09:51:09 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id A45E71FE023; Fri, 6 Nov 2015 10:51:00 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" From: Hans Petter Selasky Message-ID: <563C786C.1050305@selasky.org> Date: Fri, 6 Nov 2015 10:52:44 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 09:51:09 -0000 On 11/06/15 09:50, Luigi Rizzo wrote: > On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky wrote: >> On 11/06/15 01:08, Rasool Al-Saadi wrote: >>> >>> >>> On Thursday, 5 November 2015 8:53 PM, Hans Petter Selasky wrote: >>>> >>>> >>>> On 11/05/15 00:44, Rasool Al-Saadi wrote: > ... >>> Removing C_HARDCLOCK reduces the problem but doesn't solve it completely. >>> However, removing C_DIRECT_EXEC instead solves the problem (but >>> occasionally very small spike(s) appears in high hz values). >>> I mentioned in my first email that removing these flags makes the issue to >>> disappear. But what the effects of removing these flags? If it cause timing >>> issue to Dummynet, why we should use them? >>> >> >> Hi, >> >> The C_DIRECT_EXEC flag reduces task switching overhead, that you don't have >> to wakeup a thread to wakeup the dummynet worker thread. It affects timing. > > Hans, > thanks for the explanation. > > Can you clarify the behaviour of C_DIRECT_EXEC ? > Does this mean that the task is run within some common > thread instead of a dedicated one ? Hi Luigi, C_DIRECT_EXEC means that the timer callback is executed directly from the fast interrupt filter of the timer or IPI. > > If so, for this type of task (dummynet may run at high rate > and use a significant amount of cpu time) it may be a good > idea to remove C_DIRECT_EXEC altogether. The ipfw dummynet code is not run from the timer callback. It is run from a taskqueue. I suspect there is likely a bug somewhere. At the moment it is not clear to me if there is a bug in the callout subsystem, that the when the timer is started with 1 tick delay it doesn't kick in until after 50ms or so at HZ=4000. Or if the dummynet's task is doing a lot of work for 50ms. I think we need some more information to nail this one. --HPS > > cheers > luigi > From owner-freebsd-net@freebsd.org Fri Nov 6 10:08:48 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86AF9A270BE for ; Fri, 6 Nov 2015 10:08:48 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BE2D14BC for ; Fri, 6 Nov 2015 10:08:48 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by lfbn126 with SMTP id n126so70116780lfb.2 for ; Fri, 06 Nov 2015 02:08:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ZlyNXFdGCjJrMVff1Xd6ifZBdByq1DQotOUQUOAoseo=; b=aaeFLzQ8an/qds2UrbaXGYQ8z14wzDUBrNkcOnO5AekPo/frCzxag5p6AKtYEKq9WC lymz9b7Si9BPmKpJlex4/a377lrxlLGU+WgWBm8xgyf7GHS2X48IceeCBmsdAg1w2YkQ QiBBPXjZgZ+KMOI2e2eCpReMRkYpviYjjgqrZIzOmMP9HF0uv/dxshnDnlv3Q8fcs2EO xXqeaSIwhZqEGxN+OSmDHVaGeNxaXmM9ivpt+LtgM6D6MF8I7wF1Zap/sDht8JO5hL0Q gN/q9+4L+pq5L8Wd+1u4ar7rW3/22JeE8SC1vV4ELAUjQoGX2hifOEOXKyVeoMahkqRh FQ9w== MIME-Version: 1.0 X-Received: by 10.25.165.206 with SMTP id o197mr3848469lfe.83.1446804525049; Fri, 06 Nov 2015 02:08:45 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.78.3 with HTTP; Fri, 6 Nov 2015 02:08:44 -0800 (PST) In-Reply-To: <563C786C.1050305@selasky.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> Date: Fri, 6 Nov 2015 11:08:44 +0100 X-Google-Sender-Auth: n3TpOh307Q1LmuCChCSyJdVdF_k Message-ID: Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Luigi Rizzo To: Hans Petter Selasky Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 10:08:48 -0000 On Fri, Nov 6, 2015 at 10:52 AM, Hans Petter Selasky wrote: > On 11/06/15 09:50, Luigi Rizzo wrote: >> >> On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky >> wrote: ... >>> Hi, >>> >>> The C_DIRECT_EXEC flag reduces task switching overhead, that you don't >>> have >>> to wakeup a thread to wakeup the dummynet worker thread. It affects >>> timing. >> >> >> Hans, >> thanks for the explanation. >> >> Can you clarify the behaviour of C_DIRECT_EXEC ? >> Does this mean that the task is run within some common >> thread instead of a dedicated one ? > > > Hi Luigi, > > C_DIRECT_EXEC means that the timer callback is executed directly from the > fast interrupt filter of the timer or IPI. > >> >> If so, for this type of task (dummynet may run at high rate >> and use a significant amount of cpu time) it may be a good >> idea to remove C_DIRECT_EXEC altogether. > > > The ipfw dummynet code is not run from the timer callback. It is run from a > taskqueue. I suspect there is likely a bug somewhere. At the moment it is > not clear to me if there is a bug in the callout subsystem, that the when > the timer is started with 1 tick delay it doesn't kick in until after 50ms > or so at HZ=4000. Or if the dummynet's task is doing a lot of work for 50ms. > I think we need some more information to nail this one. It certainly does not run for 50ms, but it might occasionally keep the thread busy for some 10-50us (I doubt it is longer than that) and possibly cause the reschedule request to fall into the interval where it should actually run. So if your theory is correct, it may well be that the callout system sees the request "in the past" (possibly as a result as some incorrect wraparound, or undefined behaviour on integer wraps) and then the event is only recovered when the callout wheel (or whatever is the underlying implementation) happens to go again through the entry. What is so magic in the values we see (400 or 600 or 40ms) i have no idea. cheers luigi > --HPS > >> >> cheers >> luigi >> > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Fri Nov 6 12:21:51 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69D0CA27974 for ; Fri, 6 Nov 2015 12:21:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 552791D5F for ; Fri, 6 Nov 2015 12:21:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA6CLp9E025113 for ; Fri, 6 Nov 2015 12:21:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 156226] [lagg]: failover does not announce the failover to switch Date: Fri, 06 Nov 2015 12:21:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: feature, needs-patch, needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: keywords cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 12:21:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=156226 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |needs-patch CC| |smh@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@freebsd.org Fri Nov 6 12:40:53 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05B55A26107 for ; Fri, 6 Nov 2015 12:40:53 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA11F14A7; Fri, 6 Nov 2015 12:40:52 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 4CF5E153401; Fri, 6 Nov 2015 13:40:38 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UMZcc2Ik5kGS; Fri, 6 Nov 2015 13:40:07 +0100 (CET) Received: from [192.168.101.176] (vpn.ecoracks.nl [31.223.170.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 26F24153413; Fri, 6 Nov 2015 13:40:07 +0100 (CET) Subject: Re: tap(4) and host-only networking between host and guest To: Eugene Grosbein , Craig Rodrigues , Miroslav Lachman <000.fbsd@quip.cz> References: <20151104075454.GA99850@admin.sibptus.tomsk.ru> <5639FF12.1020109@freebsd.org> <20151104131230.GA1117@admin.sibptus.tomsk.ru> <20151104184503.GC1117@admin.sibptus.tomsk.ru> <563B8C9A.7070104@grosbein.net> <563BF60B.5090201@quip.cz> <563C6029.5010109@grosbein.net> Cc: FreeBSD Net , Victor Sudakov From: Willem Jan Withagen Message-ID: <563C9FA6.9040909@digiware.nl> Date: Fri, 6 Nov 2015 13:40:06 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <563C6029.5010109@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 12:40:53 -0000 On 6-11-2015 09:09, Eugene Grosbein wrote: > On 06.11.2015 07:59, Craig Rodrigues wrote: >> On Thu, Nov 5, 2015 at 4:36 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: >> >>> Eugene Grosbein wrote on 11/05/2015 18:06: >>> >>>> Yes, it is. And there is a solution: >>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165174 >>>> >>> >>> I don't understand why such useful patches are left uncommited and without >>> any comments in PR for years. >>> >>> >> This is not official project policy, but in future, I recommend that for >> submitting patches to the >> project, people try to use the steps at: https://wiki.freebsd.org/CodeReview >> to submit the patch to Phabricator. >> >> As a FreeBSD developer, I personally find that Phabricator is easier to >> work with patches thatn Bugzilla. >> >> Other FreeBSD developers may disagree with me, but more FreeBSD developers >> are starting to use Phabricator. > > I've just read https://wiki.freebsd.org/CodeReview and registered at Phabricator. > It seems more developer-centric (it needs commit-messages etc.) > and over-complicated for non-developer reporting small bug and supplying small fix. I will register there, but I have the same feeling as Eugene. It feels like quite more hassle for committing a bugreports, and or a small 2 line fixer. On the other hand, if that is where a lot of the discussion on fixes and patches is moving to..... It needs to be advertised as such. --WjW From owner-freebsd-net@freebsd.org Fri Nov 6 15:02:59 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78284A28F7F for ; Fri, 6 Nov 2015 15:02:59 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39CBA14DC for ; Fri, 6 Nov 2015 15:02:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4410A1FE023; Fri, 6 Nov 2015 16:02:55 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" From: Hans Petter Selasky Message-ID: <563CC186.9000807@selasky.org> Date: Fri, 6 Nov 2015 16:04:38 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 15:02:59 -0000 On 11/06/15 11:08, Luigi Rizzo wrote: > On Fri, Nov 6, 2015 at 10:52 AM, Hans Petter Selasky wrote: >> On 11/06/15 09:50, Luigi Rizzo wrote: >>> >>> On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky >>> wrote: > ... >>>> Hi, >>>> >>>> The C_DIRECT_EXEC flag reduces task switching overhead, that you don't >>>> have >>>> to wakeup a thread to wakeup the dummynet worker thread. It affects >>>> timing. >>> >>> >>> Hans, >>> thanks for the explanation. >>> >>> Can you clarify the behaviour of C_DIRECT_EXEC ? >>> Does this mean that the task is run within some common >>> thread instead of a dedicated one ? >> >> >> Hi Luigi, >> >> C_DIRECT_EXEC means that the timer callback is executed directly from the >> fast interrupt filter of the timer or IPI. >> >>> >>> If so, for this type of task (dummynet may run at high rate >>> and use a significant amount of cpu time) it may be a good >>> idea to remove C_DIRECT_EXEC altogether. >> >> >> The ipfw dummynet code is not run from the timer callback. It is run from a >> taskqueue. I suspect there is likely a bug somewhere. At the moment it is >> not clear to me if there is a bug in the callout subsystem, that the when >> the timer is started with 1 tick delay it doesn't kick in until after 50ms >> or so at HZ=4000. Or if the dummynet's task is doing a lot of work for 50ms. >> I think we need some more information to nail this one. > > It certainly does not run for 50ms, but it might occasionally keep the > thread busy for some 10-50us (I doubt it is longer than that) > and possibly cause the reschedule request to > fall into the interval where it should actually run. > > So if your theory is correct, it may well be that the callout system > sees the request "in the past" (possibly as a result as some incorrect > wraparound, or undefined behaviour on integer wraps) and then the > event is only recovered when the callout wheel (or whatever is the > underlying implementation) happens to go again through the entry. > > What is so magic in the values we see (400 or 600 or 40ms) i have no idea. > Rasool: It might be worth trying to set: kern.eventtimer.periodic=1 In /boot/loader.conf . Can you test that too? You need to reboot before the setting takes into effect. Luigi: I'm wondering if there is a problem with: cpu_new_callout(a,b,c); --HPS From owner-freebsd-net@freebsd.org Fri Nov 6 15:43:27 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A3EBA27848 for ; Fri, 6 Nov 2015 15:43:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 352E01C13 for ; Fri, 6 Nov 2015 15:43:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA6FhRDG007017 for ; Fri, 6 Nov 2015 15:43:27 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 156226] [lagg]: failover does not announce the failover to switch Date: Fri, 06 Nov 2015 15:43:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: feature, needs-patch, needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: smh@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: smh@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 15:43:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=156226 Steven Hartland changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-net@FreeBSD.org |smh@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@freebsd.org Fri Nov 6 16:26:55 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAA84A28665; Fri, 6 Nov 2015 16:26:55 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B72D1ED0; Fri, 6 Nov 2015 16:26:55 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4989A1FE023; Fri, 6 Nov 2015 17:26:52 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , FreeBSD Current , Alexander Motin From: Hans Petter Selasky Message-ID: <563CD533.2000909@selasky.org> Date: Fri, 6 Nov 2015 17:28:35 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <563CC186.9000807@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 16:26:55 -0000 Hi, I spent some time to write a test application to investigate this issue and I found some irregularities, that when kern.eventtimer.periodic=0, the timer appears to run very irregular. Test software: ============== fetch http://home.selasky.org:8192/privat/callout_test_dummynet.tar.gz tar -zxvf callout_test_dummynet.tar.gz cd callout_test_dummynet make -m /usr/src/share/mk SYSDIR=/usr/src/sys all install Test I: ======= The following settings are placed in /boot/loader.conf and computer rebooted. kern.hz=10000 kern.eventtimer.periodic=1 kldload callout_test ./test.sh debug.total: 32604 -> 0 debug.total: 10021 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10019 -> 0 debug.total: 10021 -> 0 debug.total: 10020 -> 0 debug.total: 10021 -> 0 debug.total: 10019 -> 0 debug.total: 10020 -> 0 debug.total: 10018 -> 0 debug.total: 10021 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 debug.total: 10020 -> 0 Test II: ======== The following settings are placed in /boot/loader.conf and computer rebooted. kern.hz=10000 kern.eventtimer.periodic=0 kldload callout_test ./test.sh debug.total: 20337 -> 0 debug.total: 10091 -> 0 debug.total: 10018 -> 1 debug.total: 10041 -> 0 debug.total: 10019 -> 0 debug.total: 10644 -> 0 debug.total: 10014 -> 1 debug.total: 10020 -> 0 debug.total: 10019 -> 0 debug.total: 10644 -> 0 debug.total: 10014 -> 0 debug.total: 10644 -> 0 debug.total: 10640 -> 0 debug.total: 10210 -> 0 debug.total: 10015 -> 0 debug.total: 10020 -> 1 debug.total: 10020 -> 0 debug.total: 10453 -> 0 debug.total: 10642 -> 0 If you load the computer, like a multi-CPU buildkernel, the value flattens out again. Test III: ========= The following settings are placed in /boot/loader.conf and computer rebooted. kern.hz=1000 kern.eventtimer.periodic=1 kldload callout_test ./test.sh debug.total: 5238 -> 0 debug.total: 1005 -> 0 debug.total: 1003 -> 0 debug.total: 1006 -> 0 debug.total: 1003 -> 0 debug.total: 1003 -> 0 debug.total: 1005 -> 0 debug.total: 1006 -> 0 debug.total: 1003 -> 0 debug.total: 1013 -> 0 debug.total: 1005 -> 0 debug.total: 1006 -> 0 debug.total: 1004 -> 0 debug.total: 1005 -> 0 debug.total: 1003 -> 0 debug.total: 1004 -> 0 debug.total: 1006 -> 0 debug.total: 1004 -> 0 debug.total: 1004 -> 0 debug.total: 1007 -> 0 debug.total: 1003 -> 0 debug.total: 1004 -> 0 The difference between 100021 and 10642 interrupts in a second for test II is quite big, close to 6.5% deviation. Does anyone have any clue if this is the expected behaviour when kern.eventtimer.periodic=1 or not? MAV ? --HPS From owner-freebsd-net@freebsd.org Fri Nov 6 16:43:53 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 558F0A28BEC for ; Fri, 6 Nov 2015 16:43:53 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from erouter6.ore.mailhop.org (erouter6.ore.mailhop.org [54.187.213.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 32D301E1B for ; Fri, 6 Nov 2015 16:43:52 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound3.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Fri, 6 Nov 2015 16:42:48 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id tA6GhnqZ051087; Fri, 6 Nov 2015 09:43:49 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1446828229.91534.417.camel@freebsd.org> Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Ian Lepore To: Hans Petter Selasky , Luigi Rizzo Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , FreeBSD Current , Alexander Motin Date: Fri, 06 Nov 2015 09:43:49 -0700 In-Reply-To: <563CD533.2000909@selasky.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.16.5 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 16:43:53 -0000 On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: > Hi, > > I spent some time to write a test application to investigate this > issue > and I found some irregularities, that when > kern.eventtimer.periodic=0, > the timer appears to run very irregular. > > Test software: > ============== > > fetch > http://home.selasky.org:8192/privat/callout_test_dummynet.tar.gz > tar -zxvf callout_test_dummynet.tar.gz > cd callout_test_dummynet > make -m /usr/src/share/mk SYSDIR=/usr/src/sys all install > > Test I: > ======= > > The following settings are placed in /boot/loader.conf and computer > rebooted. > kern.hz=10000 > kern.eventtimer.periodic=1 > > kldload callout_test > > ./test.sh > debug.total: 32604 -> 0 > debug.total: 10021 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10019 -> 0 > debug.total: 10021 -> 0 > debug.total: 10020 -> 0 > debug.total: 10021 -> 0 > debug.total: 10019 -> 0 > debug.total: 10020 -> 0 > debug.total: 10018 -> 0 > debug.total: 10021 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > debug.total: 10020 -> 0 > > Test II: > ======== > > The following settings are placed in /boot/loader.conf and computer > rebooted. > kern.hz=10000 > kern.eventtimer.periodic=0 > > kldload callout_test > > ./test.sh > debug.total: 20337 -> 0 > debug.total: 10091 -> 0 > debug.total: 10018 -> 1 > debug.total: 10041 -> 0 > debug.total: 10019 -> 0 > debug.total: 10644 -> 0 > debug.total: 10014 -> 1 > debug.total: 10020 -> 0 > debug.total: 10019 -> 0 > debug.total: 10644 -> 0 > debug.total: 10014 -> 0 > debug.total: 10644 -> 0 > debug.total: 10640 -> 0 > debug.total: 10210 -> 0 > debug.total: 10015 -> 0 > debug.total: 10020 -> 1 > debug.total: 10020 -> 0 > debug.total: 10453 -> 0 > debug.total: 10642 -> 0 > > If you load the computer, like a multi-CPU buildkernel, the value > flattens out again. > > Test III: > ========= > The following settings are placed in /boot/loader.conf and computer > rebooted. > kern.hz=1000 > kern.eventtimer.periodic=1 > > kldload callout_test > > ./test.sh > debug.total: 5238 -> 0 > debug.total: 1005 -> 0 > debug.total: 1003 -> 0 > debug.total: 1006 -> 0 > debug.total: 1003 -> 0 > debug.total: 1003 -> 0 > debug.total: 1005 -> 0 > debug.total: 1006 -> 0 > debug.total: 1003 -> 0 > debug.total: 1013 -> 0 > debug.total: 1005 -> 0 > debug.total: 1006 -> 0 > debug.total: 1004 -> 0 > debug.total: 1005 -> 0 > debug.total: 1003 -> 0 > debug.total: 1004 -> 0 > debug.total: 1006 -> 0 > debug.total: 1004 -> 0 > debug.total: 1004 -> 0 > debug.total: 1007 -> 0 > debug.total: 1003 -> 0 > debug.total: 1004 -> 0 > > > The difference between 100021 and 10642 interrupts in a second for > test > II is quite big, close to 6.5% deviation. Does anyone have any clue > if > this is the expected behaviour when kern.eventtimer.periodic=1 or > not? > > MAV ? > > --HPS Do the test II results change with this setting? sysctl kern.timecounter.alloweddeviation=0 -- Ian From owner-freebsd-net@freebsd.org Fri Nov 6 16:49:47 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 135E0A28DB2; Fri, 6 Nov 2015 16:49:47 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C165114E8; Fri, 6 Nov 2015 16:49:46 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id C14A91FE023; Fri, 6 Nov 2015 17:49:44 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Ian Lepore , Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , FreeBSD Current , Alexander Motin From: Hans Petter Selasky Message-ID: <563CDA8F.5010901@selasky.org> Date: Fri, 6 Nov 2015 17:51:27 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <1446828229.91534.417.camel@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 16:49:47 -0000 On 11/06/15 17:43, Ian Lepore wrote: > On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: >> Hi, > > Do the test II results change with this setting? > > sysctl kern.timecounter.alloweddeviation=0 > Yes, it looks much better: debug.total: 10013 -> 0 debug.total: 10013 -> 0 debug.total: 10012 -> 0 debug.total: 10012 -> 0 debug.total: 10012 -> 0 debug.total: 10013 -> 0 debug.total: 10012 -> 1 debug.total: 10013 -> 0 debug.total: 10013 -> 0 debug.total: 10013 -> 0 debug.total: 10012 -> 0 debug.total: 10013 -> 0 debug.total: 10013 -> 0 --HPS From owner-freebsd-net@freebsd.org Fri Nov 6 16:55:50 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C399A26049; Fri, 6 Nov 2015 16:55:50 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD84B1CDE; Fri, 6 Nov 2015 16:55:49 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 893BF1FE023; Fri, 6 Nov 2015 17:55:46 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Ian Lepore , Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin , FreeBSD Current From: Hans Petter Selasky Message-ID: <563CDBF9.3090800@selasky.org> Date: Fri, 6 Nov 2015 17:57:29 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <563CDA8F.5010901@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 16:55:50 -0000 On 11/06/15 17:51, Hans Petter Selasky wrote: > On 11/06/15 17:43, Ian Lepore wrote: >> On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: >>> Hi, > >> >> Do the test II results change with this setting? >> >> sysctl kern.timecounter.alloweddeviation=0 >> > > Yes, it looks much better: > > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 0 > debug.total: 10012 -> 0 > debug.total: 10012 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 1 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 0 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > > --HPS > Thought I still see some unexpected dips, as the test runs for a longer amount of time. Not sure what the cause might be. > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 9844 -> 0 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 0 --HPS From owner-freebsd-net@freebsd.org Fri Nov 6 17:00:34 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 457C1A262F4 for ; Fri, 6 Nov 2015 17:00:34 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 16ECF1574 for ; Fri, 6 Nov 2015 17:00:33 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound2.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Fri, 6 Nov 2015 17:01:05 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id tA6H0UPJ051152; Fri, 6 Nov 2015 10:00:30 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1446829230.91534.425.camel@freebsd.org> Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Ian Lepore To: Hans Petter Selasky , Luigi Rizzo Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , FreeBSD Current , Alexander Motin Date: Fri, 06 Nov 2015 10:00:30 -0700 In-Reply-To: <563CDA8F.5010901@selasky.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.16.5 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 17:00:34 -0000 On Fri, 2015-11-06 at 17:51 +0100, Hans Petter Selasky wrote: > On 11/06/15 17:43, Ian Lepore wrote: > > On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: > > > Hi, > > > > > Do the test II results change with this setting? > > > > sysctl kern.timecounter.alloweddeviation=0 > > > > Yes, it looks much better: > > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 0 > debug.total: 10012 -> 0 > debug.total: 10012 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 1 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > debug.total: 10012 -> 0 > debug.total: 10013 -> 0 > debug.total: 10013 -> 0 > > --HPS This isn't the first time that the alloweddeviation feature has led people (including me in the past) to think there is a timing bug. I think the main purpose of the feature is to help save battery power on laptops by clustering nearby scheduled wakeups to all happen at the same time and then allow for longer sleeps between each wakeup. I've been wondering lately whether this might also be behind the unexplained "load average is always 0.60" problem people have noticed on some systems. If load average is calculated by sampling what work is happening when a timer interrupt fires, and the system is working hard to ensure that a timer interrupt only happens when there is actual work to do, you'd end up with statistics reporting that there is work being done most of the time when it took a sample. Maybe it would make sense to only enable the feature by default on battery-powered systems? -- Ian From owner-freebsd-net@freebsd.org Fri Nov 6 17:23:08 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DED7A26B4E for ; Fri, 6 Nov 2015 17:23:08 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from erouter6.ore.mailhop.org (erouter6.ore.mailhop.org [54.187.213.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2CE7A1519 for ; Fri, 6 Nov 2015 17:23:07 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound3.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Fri, 6 Nov 2015 17:22:04 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id tA6HN5r1051171; Fri, 6 Nov 2015 10:23:05 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1446830585.91534.435.camel@freebsd.org> Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Ian Lepore To: Hans Petter Selasky , Luigi Rizzo Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin , FreeBSD Current Date: Fri, 06 Nov 2015 10:23:05 -0700 In-Reply-To: <563CDBF9.3090800@selasky.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <563CDBF9.3090800@selasky.org> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.16.5 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 17:23:08 -0000 On Fri, 2015-11-06 at 17:57 +0100, Hans Petter Selasky wrote: > On 11/06/15 17:51, Hans Petter Selasky wrote: > > On 11/06/15 17:43, Ian Lepore wrote: > > > On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: > > > > Hi, > > > > > > > > Do the test II results change with this setting? > > > > > > sysctl kern.timecounter.alloweddeviation=0 > > > > > > > Yes, it looks much better: > > > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 1 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > > > --HPS > > > > Thought I still see some unexpected dips, as the test runs for a > longer > amount of time. Not sure what the cause might be. > > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 9844 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 0 > > --HPS Is it possible your machine is occasionally falling into a deeper sleep state (C3 or whatever)? I think it can take hundreds of microseconds to wake up from some of the deeper power-saving modes. If not power-saving, just plain old system-is-busy could lead to the reduced counts occasionally, since each callout is scheduled as a delta from the current actual wakeup time, not a delta from the current scheduled wakeup time (i.e., there is a phase shift in the firing of events over time). Raising the priority of the test thread into the realtime range might help with system-busy variation. It's also a bit iffy that you're using eventtimers to measure the performance of eventtimers (by using sleep 1 in the script). That explains the +12/13 count every second, not the reduced counts unless we speculate that sleep is returning early (which I have NEVER caught it doing). If you have a PPS source available, that can make a good way to sleep based on an off-box timing signal and avoid using a clock to measure itself. But I doubt it would make a big difference in this case. -- Ian From owner-freebsd-net@freebsd.org Fri Nov 6 18:47:04 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 603EEA282ED; Fri, 6 Nov 2015 18:47:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 1B1B11C2B; Fri, 6 Nov 2015 18:47:03 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id 0DC5E1A482D; Sat, 7 Nov 2015 05:46:53 +1100 (AEDT) Date: Sat, 7 Nov 2015 05:46:53 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ian Lepore cc: Hans Petter Selasky , Luigi Rizzo , Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin , FreeBSD Current Subject: Re: Timing issue with Dummynet on high kernel timer interrupt In-Reply-To: <1446829230.91534.425.camel@freebsd.org> Message-ID: <20151107050810.M3139@besplex.bde.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <1446829230.91534.425.camel@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=cK4dyQqN c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=kj9zAlcOel0A:10 a=nlnVIGYx78Tj1a5HvFcA:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 18:47:04 -0000 On Fri, 6 Nov 2015, Ian Lepore wrote: > On Fri, 2015-11-06 at 17:51 +0100, Hans Petter Selasky wrote: >> On 11/06/15 17:43, Ian Lepore wrote: >>> On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: >>>> Hi, >> >>> >>> Do the test II results change with this setting? >>> >>> sysctl kern.timecounter.alloweddeviation=0 >> >> Yes, it looks much better: >> >> debug.total: 10013 -> 0 >> debug.total: 10013 -> 0 >> ... > This isn't the first time that the alloweddeviation feature has led > people (including me in the past) to think there is a timing bug. I > think the main purpose of the feature is to help save battery power on > laptops by clustering nearby scheduled wakeups to all happen at the > same time and then allow for longer sleeps between each wakeup. I was trying to remember the flag for turning off that "feature". It gives the bizarre behaviour that on an old system with a timer resolution of 10 msec, "time sleep 1" sleeps for 1 second with an average error of < 10 msec, but with a timer resolution of 1 msec for hardclock and finer for short timeouts, "time sleep 1" sleeps for an average of an extra 30 msec (worst case 1.069 seconds IIRC). Thus high resolution timers give much lower resolution for medium-sized timeouts. (For "sleep 10", the average error is again 30 msec but this is relatively smaller, and for "sleep .001" the average error must be less than 1 msec to work at all, though it is likely to be relatively large.) > I've been wondering lately whether this might also be behind the > unexplained "load average is always 0.60" problem people have noticed > on some systems. If load average is calculated by sampling what work > is happening when a timer interrupt fires, and the system is working > hard to ensure that a timer interrupt only happens when there is actual > work to do, you'd end up with statistics reporting that there is work > being done most of the time when it took a sample. I use HZ = 100 and haven't seen this. Strangely, HZ = 100 gives the same 69 msec max error for "sleep 1" as HZ = 1000. Schedulers should mostly use the actual thread runtimes to avoid sampling biases. That might even be faster. But it doesn't work so well for the load average, or at all for resource usages that are averages, or for the usr/sys/intr splitting of the runtime. It is good enough for scheduling since the splitting is not need for scheduling. Bruce From owner-freebsd-net@freebsd.org Fri Nov 6 19:15:50 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A47A3A2892E; Fri, 6 Nov 2015 19:15:50 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6269B1D3B; Fri, 6 Nov 2015 19:15:50 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by iodd200 with SMTP id d200so131936605iod.0; Fri, 06 Nov 2015 11:15:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mtkFGAUVN2PuqJ3htuK+pKgdLYxSJCPo8ZqwZyvPZYw=; b=CUBsyTTxDFzxbEOFlNXaUdeMeMJcn/G+PZPTmHni1phlP7W2ucGxyJKowXi/zucc5T a3RHQ6wjqJG3DVrNtAZsILIEkusBVSOliMvZnxO7adPyFxFn6WXqGD26kiwWzE9KRK0W iCO99app/GpDxoi61mIGHXQzzp07Cvk+Jc0jtj9+XeH6EsOIWIyIN01QIcuOHpGC9Sq1 9V9XMnoq9jHjdOnCx/V413dBG4d7A5ko9k6P+Rx8ZoAtfQFIN8SCZGmBRDnj7ODi5tKX BNmjytW1xig2nGIYnuxUbSevKR4zTqsg/ERcd6LRl2HhZr0YLPiZNGqCLab4HuZUItWr LRhA== MIME-Version: 1.0 X-Received: by 10.107.152.2 with SMTP id a2mr16038481ioe.123.1446837349881; Fri, 06 Nov 2015 11:15:49 -0800 (PST) Received: by 10.36.217.196 with HTTP; Fri, 6 Nov 2015 11:15:49 -0800 (PST) In-Reply-To: <1446830585.91534.435.camel@freebsd.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <563CDBF9.3090800@selasky.org> <1446830585.91534.435.camel@freebsd.org> Date: Fri, 6 Nov 2015 11:15:49 -0800 Message-ID: Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Adrian Chadd To: Ian Lepore Cc: Hans Petter Selasky , Luigi Rizzo , Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin , FreeBSD Current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 19:15:50 -0000 Ideally there'd be both behaviours: * You'd specify whether a timer/sleep needs to be exact or can withstand some jitter (which is what linux provides); and * You can communicate to the kernel its aggressiveness for coalescing wakeups. Teaching powerd to flip on/off a sysctl for this isn't that tricky. I remember seeing someone else in #bsdmips having issues figuring out sleep-in-kernel durations and it being exactly this stuff (and it being you, Ian, that figured it out) - so I'd like to at least fix it in -head this time around! Also - I thought the other issue was that precise callouts (versus the default for callouts now which is "fuzzy" unless you specify precision) were still jittering. Is that the case? Thanks, -adrian From owner-freebsd-net@freebsd.org Fri Nov 6 20:22:39 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6FF9BA28697 for ; Fri, 6 Nov 2015 20:22:39 +0000 (UTC) (envelope-from csjp@sqrt.ca) Received: from mx01.sqrt.ca (i4hg.x.rootbsd.net [208.79.81.26]) by mx1.freebsd.org (Postfix) with ESMTP id DEB0E1965; Fri, 6 Nov 2015 20:22:37 +0000 (UTC) (envelope-from csjp@sqrt.ca) Received: from [172.22.20.171] (unknown [207.164.135.98]) by mx01.sqrt.ca (Postfix) with ESMTPSA id A3ED6B828; Fri, 6 Nov 2015 15:21:39 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.0 \(3094\)) Subject: Re: netstat -B "Recv" From: Christian Peron In-Reply-To: Date: Fri, 6 Nov 2015 14:22:29 -0600 Cc: "Alexander V. Chernikov" , freebsd-net Content-Transfer-Encoding: quoted-printable Message-Id: <884C10B2-9C17-44C6-81AC-1C56548FE31D@sqrt.ca> References: <111891446726660@web29h.yandex.ru> To: elof2@sentor.se X-Mailer: Apple Mail (2.3094) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 20:22:39 -0000 It needs to get fixed.. let me generate a patch for you and you can test = it. > On Nov 5, 2015, at 8:51 AM, elof2@sentor.se wrote: >=20 > On Thu, 5 Nov 2015, Alexander V. Chernikov wrote: >=20 >>=20 >>=20 >> 04.11.2015, 19:55, "elof2@sentor.se" : >>> Hi! >>>=20 >>> Question: >>> What do the Recv column in 'netstat -B' show? >>>=20 >>> I thought it was tha amount of packets received, but appaently not = so. >>>=20 >>> I send 2000000 packets from a tcpreplay machine to a receiving = machine. >>> I do it a few times. >>>=20 >>> On the receiver I see: >>> netstat -in >>> Name Mtu Network Address Ipkts Ierrs Idrop Opkts >>> Oerrs Coll >>> ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 >>> 0 0 >>> ix1 1500 0c:c4:7a:58:e2:3d 6000000 0 0 0 >>> 0 0 >>>=20 >>> and then >>> netstat -in >>> Name Mtu Network Address Ipkts Ierrs Idrop Opkts >>> Oerrs Coll >>> ix0 1500 0c:c4:7a:58:e2:3c 0 0 0 0 >>> 0 0 >>> ix1 1500 0c:c4:7a:58:e2:3d 8000000 0 0 0 >>> 0 0 >>>=20 >>> So 6000000 has increased to 8000000. Good. >>>=20 >>> However, 'netstat -B' show: >>> Pid Netif Flags Recv Drop Match Sblen Hblen Command >>> 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump >>>=20 >>> How can the "Recv" be *lower* than "Match"? >>> 1996862 < 2000000. >>>=20 >>> For every new run (fast and slow) I get the same results, slightly = less >>> than 2000000 Recv. >>>=20 >>> What am I missing? >> Well, "Recv" is read from d->bd_rcount which is not per-cpu counter = and is incrementing unlocked. >> On the other hand, "Match" increases when filter returned match = condition and we (w)locked bpf descriptor, so this one is accurate. >=20 > Ah. Thanks. >=20 >=20 > Will you make a bugzilla out of this? Or should I? Or is it not = interesting enough to fix? >=20 > /Elof > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Fri Nov 6 20:42:28 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A194A28A78 for ; Fri, 6 Nov 2015 20:42:28 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5ACC41314; Fri, 6 Nov 2015 20:42:28 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id tA6KgJbX024298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 6 Nov 2015 12:42:19 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id tA6KgHd6024297; Fri, 6 Nov 2015 12:42:17 -0800 (PST) (envelope-from jmg) Date: Fri, 6 Nov 2015 12:42:17 -0800 From: John-Mark Gurney To: Adrian Chadd Cc: Ian Lepore , Hans Petter Selasky , "freebsd-net@freebsd.org" , Alexander Motin , FreeBSD Current , Rasool Al-Saadi , Luigi Rizzo Subject: Re: Timing issue with Dummynet on high kernel timer interrupt Message-ID: <20151106204217.GI65715@funkthat.com> References: <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <563CDBF9.3090800@selasky.org> <1446830585.91534.435.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Fri, 06 Nov 2015 12:42:19 -0800 (PST) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 20:42:28 -0000 Adrian Chadd wrote this message on Fri, Nov 06, 2015 at 11:15 -0800: > Ideally there'd be both behaviours: > > * You'd specify whether a timer/sleep needs to be exact or can > withstand some jitter (which is what linux provides); and Isn't that what the precision argument in callout is for? See callout_reset_sbt(9): The sbt, pr, and flags arguments provide more control over the scheduled time including support for higher resolution times, specifying the precision of the scheduled time, and setting an absolute deadline instead of a relative timeout. The callout is scheduled to execute in a time window which begins at the time specified in sbt and extends for the amount of time specified in pr. If sbt specifies a time in the past, the window is adjusted to start at the current time. A non-zero value for pr allows the callout subsystem to coalesce callouts scheduled close to each other into fewer timer interrupts, reducing processing overhead and power consumption. These flags may be specified to adjust the interpretation of sbt and pr: -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@freebsd.org Fri Nov 6 20:58:33 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5B7BA28E52 for ; Fri, 6 Nov 2015 20:58:33 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE34D1FA1 for ; Fri, 6 Nov 2015 20:58:33 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound1.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Fri, 6 Nov 2015 20:58:31 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id tA6KwPNL051469; Fri, 6 Nov 2015 13:58:25 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1446843506.91534.443.camel@freebsd.org> Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Ian Lepore To: Adrian Chadd Cc: Hans Petter Selasky , Luigi Rizzo , Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin Date: Fri, 06 Nov 2015 13:58:26 -0700 In-Reply-To: References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <563CDBF9.3090800@selasky.org> <1446830585.91534.435.camel@freebsd.org> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.16.5 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 20:58:33 -0000 On Fri, 2015-11-06 at 11:15 -0800, Adrian Chadd wrote: > Ideally there'd be both behaviours: > > * You'd specify whether a timer/sleep needs to be exact or can > withstand some jitter (which is what linux provides); and > * You can communicate to the kernel its aggressiveness for coalescing > wakeups. > We already implement exactly both of these things (the former in args to scheduling callouts, the latter in the form of the sysctl I referenced earlier which lets you set the "agressivness for coalescing"). The problem with the former is very little code actually uses the flavor of callout scheduling that lets you specify precision. The problem with the latter is that its default value is to allow enough deviation that people think the system is misbehaving. -- Ian From owner-freebsd-net@freebsd.org Fri Nov 6 21:15:51 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00974A2814D for ; Fri, 6 Nov 2015 21:15:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B849D1972; Fri, 6 Nov 2015 21:15:50 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by igpw7 with SMTP id w7so46687300igp.0; Fri, 06 Nov 2015 13:15:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Wa/UIBmAo1fZqNSbdI19VY6zw+9xoMNhblbSqb6c+10=; b=Cu1Yx9V/jGMvtj0DwBilP8kXlu0u6jz7JWVaaZMOsCrnsd4rlNKgkoK0ydWjMHmcOI Ll7Ur2xFuqe7qeiz/VjPVudi2aHXpnFf6P7kW3hvInaIRej/e3b0tLWj4o9TJ01VX/n4 94VZ+QA1HoEraiy68GFWLwTHFcn2DM5frNTrxq0Trt9PiXEw/rplPHJMTYpgkyjXtfkg hpgb4o+5qOVyw96naRNINOdn0Qo9oEN9qGF7GNGPE4xSroPTGPVttegqjrwkXomq0KpL iWTiwTEEt5GSC4oqTMg6RtDL4mKAa5Hmm6L6Zhit+QHA65LFYXME+YuLEEhkpWWvfz8K NARA== MIME-Version: 1.0 X-Received: by 10.50.178.141 with SMTP id cy13mr10520811igc.61.1446844550283; Fri, 06 Nov 2015 13:15:50 -0800 (PST) Received: by 10.36.217.196 with HTTP; Fri, 6 Nov 2015 13:15:50 -0800 (PST) In-Reply-To: <1446843506.91534.443.camel@freebsd.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <563CDBF9.3090800@selasky.org> <1446830585.91534.435.camel@freebsd.org> <1446843506.91534.443.camel@freebsd.org> Date: Fri, 6 Nov 2015 13:15:50 -0800 Message-ID: Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Adrian Chadd To: Ian Lepore Cc: Hans Petter Selasky , Luigi Rizzo , Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 21:15:51 -0000 On 6 November 2015 at 12:58, Ian Lepore wrote: > On Fri, 2015-11-06 at 11:15 -0800, Adrian Chadd wrote: >> Ideally there'd be both behaviours: >> >> * You'd specify whether a timer/sleep needs to be exact or can >> withstand some jitter (which is what linux provides); and >> * You can communicate to the kernel its aggressiveness for coalescing >> wakeups. >> > > We already implement exactly both of these things (the former in args > to scheduling callouts, the latter in the form of the sysctl I > referenced earlier which lets you set the "agressivness for > coalescing"). > > The problem with the former is very little code actually uses the > flavor of callout scheduling that lets you specify precision. The > problem with the latter is that its default value is to allow enough > deviation that people think the system is misbehaving. Right. What about sleep, though? (eg userland calling sleep, usleep, etc.) Would the original poster problem be fixed just by setting the exact precision? -adrian From owner-freebsd-net@freebsd.org Sat Nov 7 01:51:39 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D359A2570C for ; Sat, 7 Nov 2015 01:51:39 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id 7E0D7106C for ; Sat, 7 Nov 2015 01:51:37 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.20,255,1444654800"; d="scan'208";a="14644137" Received: from gsp-ex03.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.19]) by iport1.cc.swin.edu.au with ESMTP; 07 Nov 2015 12:51:29 +1100 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.2]) by gsp-ex03.ds.swin.edu.au ([169.254.3.127]) with mapi id 14.03.0248.002; Sat, 7 Nov 2015 12:51:29 +1100 From: Rasool Al-Saadi To: Hans Petter Selasky , Luigi Rizzo CC: "freebsd-net@freebsd.org" Subject: RE: Timing issue with Dummynet on high kernel timer interrupt Thread-Topic: Timing issue with Dummynet on high kernel timer interrupt Thread-Index: AdEWOVE5DoAlkzqPQWaBZ8ln4esVd///TXiA//0UVtCABdKzgP/+Z6XggAMXeACAAAG5AIAAEWQAgAAEeACAAFKsAP/+mCjA Date: Sat, 7 Nov 2015 01:51:29 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> In-Reply-To: <563CC186.9000807@selasky.org> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.126.11] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Nov 2015 01:51:39 -0000 On Saturday, 7 November 2015 2:05 AM, Hans Petter Selasky wrote: > On 11/06/15 11:08, Luigi Rizzo wrote: > > On Fri, Nov 6, 2015 at 10:52 AM, Hans Petter Selasky > wrote: > >> On 11/06/15 09:50, Luigi Rizzo wrote: > >>> > >>> On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky > >>> > >>> wrote: > > ... > >>>> Hi, > >>>> > >>>> The C_DIRECT_EXEC flag reduces task switching overhead, that you > >>>> don't have to wakeup a thread to wakeup the dummynet worker > thread. > >>>> It affects timing. > >>> > >>> > >>> Hans, > >>> thanks for the explanation. > >>> > >>> Can you clarify the behaviour of C_DIRECT_EXEC ? > >>> Does this mean that the task is run within some common thread > >>> instead of a dedicated one ? > >> > >> > >> Hi Luigi, > >> > >> C_DIRECT_EXEC means that the timer callback is executed directly from > >> the fast interrupt filter of the timer or IPI. > >> > >>> > >>> If so, for this type of task (dummynet may run at high rate and use > >>> a significant amount of cpu time) it may be a good idea to remove > >>> C_DIRECT_EXEC altogether. > >> > >> > >> The ipfw dummynet code is not run from the timer callback. It is run > >> from a taskqueue. I suspect there is likely a bug somewhere. At the > >> moment it is not clear to me if there is a bug in the callout > >> subsystem, that the when the timer is started with 1 tick delay it > >> doesn't kick in until after 50ms or so at HZ=3D4000. Or if the dummyne= t's > task is doing a lot of work for 50ms. > >> I think we need some more information to nail this one. > > > > It certainly does not run for 50ms, but it might occasionally keep the > > thread busy for some 10-50us (I doubt it is longer than that) and > > possibly cause the reschedule request to fall into the interval where > > it should actually run. > > > > So if your theory is correct, it may well be that the callout system > > sees the request "in the past" (possibly as a result as some incorrect > > wraparound, or undefined behaviour on integer wraps) and then the > > event is only recovered when the callout wheel (or whatever is the > > underlying implementation) happens to go again through the entry. > > > > What is so magic in the values we see (400 or 600 or 40ms) i have no id= ea. > > >=20 > Rasool: >=20 > It might be worth trying to set: >=20 > kern.eventtimer.periodic=3D1 >=20 > In /boot/loader.conf . Can you test that too? >=20 > You need to reboot before the setting takes into effect. Hans, Yes, this solves the problem! I will do more checking when I am near my testbed. Thanks for your effort and time! Cheers, Rasool >=20 > Luigi: >=20 > I'm wondering if there is a problem with: >=20 > cpu_new_callout(a,b,c); >=20 > --HPS >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Sat Nov 7 06:19:42 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B21FA28363 for ; Sat, 7 Nov 2015 06:19:42 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail105.syd.optusnet.com.au (mail105.syd.optusnet.com.au [211.29.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id 525631E39 for ; Sat, 7 Nov 2015 06:19:41 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail105.syd.optusnet.com.au (Postfix) with ESMTPS id EA9CF1045F37; Sat, 7 Nov 2015 17:19:29 +1100 (AEDT) Date: Sat, 7 Nov 2015 17:19:29 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Rasool Al-Saadi cc: Hans Petter Selasky , Luigi Rizzo , "freebsd-net@freebsd.org" Subject: RE: Timing issue with Dummynet on high kernel timer interrupt In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> Message-ID: <20151107162915.A893@besplex.bde.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=R6/+YolX c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=kj9zAlcOel0A:10 a=IpulGqAEFL8BaEeDC60A:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Nov 2015 06:19:42 -0000 On Sat, 7 Nov 2015, Rasool Al-Saadi wrote: > On Saturday, 7 November 2015 2:05 AM, Hans Petter Selasky wrote: >> ... >> It might be worth trying to set: >> >> kern.eventtimer.periodic=1 >> >> In /boot/loader.conf . Can you test that too? >> >> You need to reboot before the setting takes into effect. You don't need to reboot for the setting to take effect if you make the setting in the correct way. kern.eventtimer.periodic is a bogus as a tunable since it is not needed for booting. If it were needed for booting, then it would be documented in some man page related to booting, but it is only documented in its log message and in eventtimer(4). Similarly for other eventtimer tunables. FreeBSD has too man of these knobs which no one knows about. You can't run sysctl -da | grep foo at the loader prompt to search for interesting ones, or kenv | grep foo later to find all the possibilities. kern.eventtimer.timer is an example of a non-bogus tunable. It might be needed for booting if there are too many choices and the default choice doesn't work. But this is unusable for booting in practice since it is not documented in any man page related to booting. Also, its implementation is messier than the other eventtimer tunables. It is still a TUNABLE_INT() and a SYSCTL_PROC(), while the others were converted to SYSCTL_[U]INT() with CTLFLAG_RWTUN. It needs the SYSCTL_PROC() to work without rebooting. kern.eventtimer.timer is also changeable after booting using a SYSCTL_PROC() and has the messy implementation from not using CTLFLAG_RWTUN. I don't know if CTLFLAG_RWTUN with SYSCTL_PROC() actually works for all types, but it is used with CTLFLAG_STRING for kern.corefile. This is another bogus undocumented tunable. It is better documented as a sysctl than most since it is old so it is documented in sysctl(8). It is not documented as a tunable there of course. It is also not documented as a tunable in core(5). Also, the idletick boolean variable unsigned and its SYSCTL_UINT() matches it. This bug is missing for the periodic boolean variable. Bruce From owner-freebsd-net@freebsd.org Sat Nov 7 06:33:43 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1683BA285FB for ; Sat, 7 Nov 2015 06:33:43 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1AE931354 for ; Sat, 7 Nov 2015 06:33:41 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id tA76XVXT065879; Sat, 7 Nov 2015 17:33:31 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 7 Nov 2015 17:33:31 +1100 (EST) From: Ian Smith To: Rasool Al-Saadi cc: Hans Petter Selasky , Luigi Rizzo , "freebsd-net@freebsd.org" , Bruce Evans Subject: RE: Timing issue with Dummynet on high kernel timer interrupt In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> Message-ID: <20151107155144.I12989@sola.nimnet.asn.au> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Nov 2015 06:33:43 -0000 On Sat, 7 Nov 2015 01:51:29 +0000, Rasool Al-Saadi wrote: > On Saturday, 7 November 2015 2:05 AM, Hans Petter Selasky wrote: > > On 11/06/15 11:08, Luigi Rizzo wrote: > > > On Fri, Nov 6, 2015 at 10:52 AM, Hans Petter Selasky > > wrote: > > >> On 11/06/15 09:50, Luigi Rizzo wrote: > > >>> > > >>> On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky > > >>> > > >>> wrote: > > > ... > > >>>> Hi, > > >>>> > > >>>> The C_DIRECT_EXEC flag reduces task switching overhead, that you > > >>>> don't have to wakeup a thread to wakeup the dummynet worker > > thread. > > >>>> It affects timing. > > >>> > > >>> > > >>> Hans, > > >>> thanks for the explanation. > > >>> > > >>> Can you clarify the behaviour of C_DIRECT_EXEC ? > > >>> Does this mean that the task is run within some common thread > > >>> instead of a dedicated one ? > > >> > > >> > > >> Hi Luigi, > > >> > > >> C_DIRECT_EXEC means that the timer callback is executed directly from > > >> the fast interrupt filter of the timer or IPI. > > >> > > >>> > > >>> If so, for this type of task (dummynet may run at high rate and use > > >>> a significant amount of cpu time) it may be a good idea to remove > > >>> C_DIRECT_EXEC altogether. > > >> > > >> > > >> The ipfw dummynet code is not run from the timer callback. It is run > > >> from a taskqueue. I suspect there is likely a bug somewhere. At the > > >> moment it is not clear to me if there is a bug in the callout > > >> subsystem, that the when the timer is started with 1 tick delay it > > >> doesn't kick in until after 50ms or so at HZ=4000. Or if the dummynet's > > task is doing a lot of work for 50ms. > > >> I think we need some more information to nail this one. > > > > > > It certainly does not run for 50ms, but it might occasionally keep the > > > thread busy for some 10-50us (I doubt it is longer than that) and > > > possibly cause the reschedule request to fall into the interval where > > > it should actually run. > > > > > > So if your theory is correct, it may well be that the callout system > > > sees the request "in the past" (possibly as a result as some incorrect > > > wraparound, or undefined behaviour on integer wraps) and then the > > > event is only recovered when the callout wheel (or whatever is the > > > underlying implementation) happens to go again through the entry. > > > > > > What is so magic in the values we see (400 or 600 or 40ms) i have no idea. > > > > > > > Rasool: > > > > It might be worth trying to set: > > > > kern.eventtimer.periodic=1 > > > > In /boot/loader.conf . Can you test that too? > > > > You need to reboot before the setting takes into effect. I've been trying to study and document the behaviour that gives rise to these '0.6' load averages often seen at even 99.9-100% idle on laptops especially, and here (Lenovo X200 2.4GHz Core2Duo on recent stable/9) it depends entirely on which eventtimer is selected. Here the issue is with HPET - whether in one-shot or periodic mode - while LAPIC, in either mode, shows 0.00 LAs after 15 minutes. However LAPIC consumes ~40% more power, running ~20% hotter on battery as it only drops to (~100%) C2 state, whereas HPET runs over 95% in C3 in the same idle scenario. A friend reports the same (0.6-0.7) LAs on a Digital Ocean 'droplet' on one core of an 8-core Xeon under KVM using LAPIC, and Jeremy Chadwick kicked this off in stable@ (re 10.x) showing ~0.15 LAs on bare metal, also with LAPIC, so it does appear there may be some deeper issues. I have however, seen no obvious[*] problem switching between eventtimers and en/disabling .periodic on the fly by sysctl, and observing immediate effects via top -SCHP, vmstat -w, and especially systat -vm. Recently I've been playing with kern.eventtimer.idletick as well, and in the next few days hope to collate results of my tests into something coherent. [*] except that it's necessary to remove then reapply AC power to have dev.cpu.N.cx_usage properly reflect the C-state stats, especially when switching to LAPIC from HPET on the fly, which is no doubt officially unsupported, though it does instantly 'fix' the load average issue. I also notice that with HPET, the timer interrupts themselves, whether as few as 60-70/s idle in one-shot mode or ~2000/s in periodic, appear in systat -vm or vmstat -w10 'Int' column, whereas with LAPIC or (yes I even tried) i8254, these timer interrupts don't count in that column. I'll try Hans' suggestion of adjusting in loader.conf, rebooting between various options if really needed, though that'll be far more tedious .. [ I've just now seen Bruce's latest, so maybe I haven't been doing the wrong thing playing with these after all? ] I've been struggling to express this clearly over on stable@ against ongoing concensus that the (symptomatic) load average issue is merely 'cosmetic,' but my first thought when seeing $subject issue was to ask about values of sysctls kern.timecounter and kern.eventtimer and for a systat -vm shot or two while running your tests .. I now wish I had. So could you show those now, so your clocks/timers config is clear? Excuse a long rather pent-up rave, but I could be away for a few days. (Kind of amusing that you're trying for best performance at up to 10kHz ticker, while I'm hoping for at least vaguely accurate LAs at idle :) cheers, Ian > Hans, > > Yes, this solves the problem! > I will do more checking when I am near my testbed. > Thanks for your effort and time! > > Cheers, > Rasool > > > > Luigi: > > > > I'm wondering if there is a problem with: > > > > cpu_new_callout(a,b,c); > > > > --HPS From owner-freebsd-net@freebsd.org Sat Nov 7 10:45:43 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74E5DA27B8D for ; Sat, 7 Nov 2015 10:45:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 352F51DD1 for ; Sat, 7 Nov 2015 10:45:42 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id C3B021FE023; Sat, 7 Nov 2015 11:45:33 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Bruce Evans , Rasool Al-Saadi References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> <20151107162915.A893@besplex.bde.org> Cc: Luigi Rizzo , "freebsd-net@freebsd.org" From: Hans Petter Selasky Message-ID: <563DD6B2.8010607@selasky.org> Date: Sat, 7 Nov 2015 11:47:14 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20151107162915.A893@besplex.bde.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Nov 2015 10:45:43 -0000 On 11/07/15 07:19, Bruce Evans wrote: > I don't know if CTLFLAG_RWTUN with SYSCTL_PROC() actually works for all > types, but it is used with CTLFLAG_STRING for kern.corefile. This is > another bogus undocumented tunable. It is better documented as a sysctl > than most since it is old so it is documented in sysctl(8). It is not > documented as a tunable there of course. It is also not documented as > a tunable in core(5). Hi, SYSCTL_PROC() works with RWTUN from /boot/loader.conf, as long as the procedure callback handles early calls during boot. --HPS From owner-freebsd-net@freebsd.org Sat Nov 7 11:26:10 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EE37A2750C for ; Sat, 7 Nov 2015 11:26:10 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail105.syd.optusnet.com.au (mail105.syd.optusnet.com.au [211.29.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id 054601336 for ; Sat, 7 Nov 2015 11:26:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail105.syd.optusnet.com.au (Postfix) with ESMTPS id 6935D104592A; Sat, 7 Nov 2015 22:26:07 +1100 (AEDT) Date: Sat, 7 Nov 2015 22:26:06 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ian Smith cc: Rasool Al-Saadi , Hans Petter Selasky , Luigi Rizzo , "freebsd-net@freebsd.org" , Bruce Evans Subject: RE: Timing issue with Dummynet on high kernel timer interrupt In-Reply-To: <20151107155144.I12989@sola.nimnet.asn.au> Message-ID: <20151107220450.X1692@besplex.bde.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> <20151107155144.I12989@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=cK4dyQqN c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=kj9zAlcOel0A:10 a=JGmdEZ7-IIC20yNyzecA:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Nov 2015 11:26:10 -0000 On Sat, 7 Nov 2015, Ian Smith wrote: > ... > I also notice that with HPET, the timer interrupts themselves, whether > as few as 60-70/s idle in one-shot mode or ~2000/s in periodic, appear > in systat -vm or vmstat -w10 'Int' column, whereas with LAPIC or (yes I > even tried) i8254, these timer interrupts don't count in that column. Interrupt reporting is mostly broken for "fast" interrupts. E.g., systat -v counts them in the Interrupts column (separate and Total) but not in the Int summary; vmstat -m doesn't count them anywhere (it shows the Int summary). vmstat -i shows them much like the Interrupts column in systat -v (except it actually displays them all when there are too man CPUs or Interrupts to fit in systat -v). I use COUNT_IPIS and COUNT_XINVLTLB. The interrupts counted by these are actually fast. These mess up the systat -v display by giving too many active interrupts to display for just a couple of CPU. systat -v tends to display these in preference to more interesting interrupts. vmstat -i displays these too. "fast" and fast interrupts should be counted separately and are probaly best left out of the Int summary. They should also be displayed separately. > I'll try Hans' suggestion of adjusting in loader.conf, rebooting between > various options if really needed, though that'll be far more tedious .. > [ I've just now seen Bruce's latest, so maybe I haven't been doing the > wrong thing playing with these after all? ] The eventtimer choice is readonly, so rebooting seems to be required to change it. Too much like Windows. I often miss being able to change hz. Event timers now do much more complicated changes than that. They implement a sort of dynamic hz, but still use the boot-time hz as a default. Bruce From owner-freebsd-net@freebsd.org Sat Nov 7 11:51:26 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60916A27BB7 for ; Sat, 7 Nov 2015 11:51:26 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 10BCC10AA for ; Sat, 7 Nov 2015 11:51:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 7BF28783992; Sat, 7 Nov 2015 22:51:15 +1100 (AEDT) Date: Sat, 7 Nov 2015 22:51:14 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Hans Petter Selasky cc: Bruce Evans , Rasool Al-Saadi , Luigi Rizzo , "freebsd-net@freebsd.org" Subject: Re: Timing issue with Dummynet on high kernel timer interrupt In-Reply-To: <563DD6B2.8010607@selasky.org> Message-ID: <20151107223816.D1837@besplex.bde.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E1B79@gsp-ex02.ds.swin.edu.au> <20151107162915.A893@besplex.bde.org> <563DD6B2.8010607@selasky.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=R4L+YolX c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=kj9zAlcOel0A:10 a=0G27pUAqrrk60t1OlWMA:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Nov 2015 11:51:26 -0000 On Sat, 7 Nov 2015, Hans Petter Selasky wrote: > On 11/07/15 07:19, Bruce Evans wrote: >> I don't know if CTLFLAG_RWTUN with SYSCTL_PROC() actually works for all >> types, but it is used with CTLFLAG_STRING for kern.corefile. This is >> another bogus undocumented tunable. It is better documented as a sysctl >> than most since it is old so it is documented in sysctl(8). It is not >> documented as a tunable there of course. It is also not documented as >> a tunable in core(5). > > SYSCTL_PROC() works with RWTUN from /boot/loader.conf, as long as the > procedure callback handles early calls during boot. Hmm, that would be messy if it wants to do hardware stuff like sysctl_kern_eventtimer_periodic() does. Timercounters only have 1 MI tunable (sysctl_kern_timecounter_adjprecision). It has a (cold) check to avoid doing too much. Bruce