From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 03:27:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 055ADC67 for ; Sun, 28 Dec 2014 03:27:26 +0000 (UTC) Received: from omicron.logn.net (omicron.logn.net [72.10.118.7]) by mx1.freebsd.org (Postfix) with ESMTP id D88BB6455C for ; Sun, 28 Dec 2014 03:27:25 +0000 (UTC) Received: from [10.3.73.86] (unknown [10.3.73.86]) by omicron.logn.net (Postfix) with ESMTPSA id 0C5004A0082 for ; Sat, 27 Dec 2014 22:19:42 -0500 (EST) From: Jason Healy Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: IPv6 routes leaking between FIBs? Message-Id: Date: Sat, 27 Dec 2014 22:19:42 -0500 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 03:27:26 -0000 Hello, Trying out FreeBSD for the first time to build a firewall box that=92s = multi-core and runs PF. I=92m very interested in the FIB code, as it = lines up well with the way my core networking equipment works and should = allow me to route traffic on an interface that=92s logically separate = from the management interfaces. I=92ve been playing for a bit with the FIB features, but I=92m getting = hung up on IPv6. I=92m trying to set up two interfaces on my box to = each have a different FIB, and to not leak routes between the = interfaces: # sysctl net.add_addr_allfibs=3D0 # ifconfig em1 inet 192.0.2.1/24 fib 1 # ifconfig em1 inet6 2001:db8:dead:beef::1/64 fib 1 # ifconfig em2 inet 203.0.113.1/24 fib 2 # ifconfig em2 inet6 2001:db8:cafe:babe::1/64 fib 2 If I then check the routing tables for each FIB, here=92s what I get: # setfib -F 1 netstat -rn Routing tables (fib: 1) Internet: Destination Gateway Flags Netif Expire 192.0.2.0/24 link#2 U em1 192.0.2.1 link#2 UHS lo0 Internet6: Destination Gateway Flags = Netif Expire 2001:db8:cafe:babe::/64 link#3 U = em2 2001:db8:dead:beef::/64 link#2 U = em1 2001:db8:dead:beef::1 link#2 UHS = lo0 fe80::%em1/64 link#2 U = em1 fe80::a00:27ff:fef6:162a%em1 link#2 UHS = lo0 fe80::%em2/64 link#3 U = em2 fe80::%lo0/64 link#5 U = lo0 # setfib -F 2 netstat -rn Routing tables (fib: 2) Internet: Destination Gateway Flags Netif Expire 203.0.113.0/24 link#3 U em2 203.0.113.1 link#3 UHS lo0 Internet6: Destination Gateway Flags = Netif Expire 2001:db8:cafe:babe::/64 link#3 U = em2 2001:db8:cafe:babe::1 link#3 UHS = lo0 2001:db8:dead:beef::/64 link#2 U = em1 fe80::%em1/64 link#2 U = em1 fe80::%em2/64 link#3 U = em2 fe80::a00:27ff:fe62:d267%em2 link#3 UHS = lo0 fe80::%lo0/64 link#5 U = lo0 Note that as expected, the IPv4 routes are constrained to their FIB = (192.0.2.0 to FIB 1 and 203.0.113.0 to FIB 2). However, the IPv6 routes = (deadbeef and cafebabe) leak between the FIBs; both prefixes that I add = are listed in both FIBs (as well as the link-local stuff). According to: = https://www.freebsd.org/news/status/report-2012-01-2012-03.html#Multi-FIB:= -IPv6-Support-and-Other-Enhancements IPv6 parity is claimed for the FIB code, so I=92m not sure if I=92m = doing it wrong, or if there=92s a problem with the FIB code and IPv6 = routes. Thanks in advance for any help or clarification! Jason= From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 10:17:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D038197 for ; Sun, 28 Dec 2014 10:17:02 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EC25315FD for ; Sun, 28 Dec 2014 10:17:01 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id B1FB725D37C3; Sun, 28 Dec 2014 10:16:57 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 93EFBC7706D; Sun, 28 Dec 2014 10:16:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id zLw8KAnWvOYy; Sun, 28 Dec 2014 10:16:54 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:9857:437e:f6b9:544d] (unknown [IPv6:fde9:577b:c1a9:4410:9857:437e:f6b9:544d]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 391C5C77058; Sun, 28 Dec 2014 10:16:53 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: IPv6 routes leaking between FIBs? From: "Bjoern A. Zeeb" In-Reply-To: Date: Sun, 28 Dec 2014 10:16:21 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jason Healy X-Mailer: Apple Mail (2.1993) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 10:17:02 -0000 > On 28 Dec 2014, at 03:19 , Jason Healy wrote: >=20 > Hello, >=20 > Trying out FreeBSD for the first time to build a firewall box that=92s = multi-core and runs PF. I=92m very interested in the FIB code, as it = lines up well with the way my core networking equipment works and should = allow me to route traffic on an interface that=92s logically separate = from the management interfaces. >=20 > I=92ve been playing for a bit with the FIB features, but I=92m getting = hung up on IPv6. I=92m trying to set up two interfaces on my box to = each have a different FIB, and to not leak routes between the = interfaces: >=20 > # sysctl net.add_addr_allfibs=3D0 > # ifconfig em1 inet 192.0.2.1/24 fib 1 > # ifconfig em1 inet6 2001:db8:dead:beef::1/64 fib 1 > # ifconfig em2 inet 203.0.113.1/24 fib 2 > # ifconfig em2 inet6 2001:db8:cafe:babe::1/64 fib 2 >=20 > If I then check the routing tables for each FIB, here=92s what I get: >=20 > # setfib -F 1 netstat -rn >=20 > Routing tables (fib: 1) >=20 > Internet: > Destination Gateway Flags Netif Expire > 192.0.2.0/24 link#2 U em1 > 192.0.2.1 link#2 UHS lo0 >=20 > Internet6: > Destination Gateway Flags = Netif Expire > 2001:db8:cafe:babe::/64 link#3 U = em2 > 2001:db8:dead:beef::/64 link#2 U = em1 > 2001:db8:dead:beef::1 link#2 UHS = lo0 > fe80::%em1/64 link#2 U = em1 > fe80::a00:27ff:fef6:162a%em1 link#2 UHS = lo0 > fe80::%em2/64 link#3 U = em2 > fe80::%lo0/64 link#5 U = lo0 >=20 >=20 > # setfib -F 2 netstat -rn >=20 > Routing tables (fib: 2) >=20 > Internet: > Destination Gateway Flags Netif Expire > 203.0.113.0/24 link#3 U em2 > 203.0.113.1 link#3 UHS lo0 >=20 > Internet6: > Destination Gateway Flags = Netif Expire > 2001:db8:cafe:babe::/64 link#3 U = em2 > 2001:db8:cafe:babe::1 link#3 UHS = lo0 > 2001:db8:dead:beef::/64 link#2 U = em1 > fe80::%em1/64 link#2 U = em1 > fe80::%em2/64 link#3 U = em2 > fe80::a00:27ff:fe62:d267%em2 link#3 UHS = lo0 > fe80::%lo0/64 link#5 U = lo0 >=20 >=20 > Note that as expected, the IPv4 routes are constrained to their FIB = (192.0.2.0 to FIB 1 and 203.0.113.0 to FIB 2). However, the IPv6 routes = (deadbeef and cafebabe) leak between the FIBs; both prefixes that I add = are listed in both FIBs (as well as the link-local stuff). >=20 > According to: >=20 > = https://www.freebsd.org/news/status/report-2012-01-2012-03.html#Multi-FIB:= -IPv6-Support-and-Other-Enhancements >=20 > IPv6 parity is claimed for the FIB code, so I=92m not sure if I=92m = doing it wrong, or if there=92s a problem with the FIB code and IPv6 = routes. >=20 > Thanks in advance for any help or clarification! People simply broke it (again). Please file a bug report. You may = mention that there are regression test scripts in src/tools/ somewhere = to test all the cases for IPv6. =97=20 Bjoern A. Zeeb Charles Haddon Spurgeon: "Friendship is one of the sweetest joys of life. Many might have failed beneath the bitterness of their trial had they not found a friend." From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 19:07:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F39C252 for ; Sun, 28 Dec 2014 19:07:46 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6224F115C for ; Sun, 28 Dec 2014 19:07:46 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id sBSJ7kKH083104 for ; Sun, 28 Dec 2014 19:07:46 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id sBSJ7kgY083103; Sun, 28 Dec 2014 19:07:46 GMT (envelope-from root) Date: Sun, 28 Dec 2014 19:07:46 +0000 To: freebsd-net@freebsd.org From: "kibab (Ilya Bakulin)" Subject: [Differential] [Request, 6 lines] D1388: IP6: Turned on verbose logging for fragment handling code Message-ID: X-Priority: 3 Thread-Topic: D1388: IP6: Turned on verbose logging for fragment handling code X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Thread-Index: NzQ2MGZiNWU2MzVkYmUyNmI0Mzg5NzkzYTJj X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 19:07:46 -0000 kibab created this revision. kibab added a reviewer: bz. kibab added a subscriber: freebsd-net. REVISION SUMMARY Implement behavior suggested by RFC5722: drop the fragment queue entirely if the system receives a duplicate fragment TEST PLAN Send a fragmented packet to the system, then send one of the fragments once again. One can use frag6 tests from the OpenBSD regression suite. BRANCH provost_pfrag REVISION DETAIL https://reviews.freebsd.org/D1388 AFFECTED FILES sys/netinet6/frag6.c To: kibab, bz Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 19:17:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36C299AE for ; Sun, 28 Dec 2014 19:17:59 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 17CFE1318 for ; Sun, 28 Dec 2014 19:17:59 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id sBSJHw8T093976 for ; Sun, 28 Dec 2014 19:17:58 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id sBSJHwYi093975; Sun, 28 Dec 2014 19:17:58 GMT (envelope-from root) Date: Sun, 28 Dec 2014 19:17:58 +0000 To: freebsd-net@freebsd.org From: "kibab (Ilya Bakulin)" Subject: [Differential] [Commented On] D1388: IP6: Turned on verbose logging for fragment handling code Message-ID: X-Priority: 3 Thread-Topic: D1388: IP6: Turned on verbose logging for fragment handling code X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NzQ2MGZiNWU2MzVkYmUyNmI0Mzg5NzkzYTJjIFSgV2Y= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 19:17:59 -0000 kibab added a comment. I have added the wrong revision to the review! This is the correct one, I couldn't edit the patch itself :-( --- a/sys/netinet6/frag6.c +++ b/sys/netinet6/frag6.c @@ -63,7 +63,7 @@ static void frag6_enq(struct ip6asfrag *, struct ip6asfrag *); static void frag6_deq(struct ip6asfrag *); static void frag6_insque(struct ip6q *, struct ip6q *); static void frag6_remque(struct ip6q *); -static void frag6_freef(struct ip6q *); +static void frag6_freef(struct ip6q *, int); static struct mtx ip6qlock; /* @@ -459,6 +459,7 @@ frag6_input(struct mbuf **mp, int *offp, int proto) i, ip6_sprintf(ip6buf, &q6->ip6q_src)); #endif free(ip6af, M_FTABLE); + frag6_freef(q6, 0); goto dropfrag; } } @@ -471,6 +472,7 @@ frag6_input(struct mbuf **mp, int *offp, int proto) i, ip6_sprintf(ip6buf, &q6->ip6q_src)); #endif free(ip6af, M_FTABLE); + frag6_freef(q6, 0); goto dropfrag; } } @@ -603,7 +605,7 @@ insert: * associated datagrams. */ void -frag6_freef(struct ip6q *q6) +frag6_freef(struct ip6q *q6, int sendicmp) { struct ip6asfrag *af6, *down6; @@ -620,7 +622,7 @@ frag6_freef(struct ip6q *q6) * Return ICMP time exceeded error for the 1st fragment. * Just free other fragments. */ - if (af6->ip6af_off == 0) { + if (af6->ip6af_off == 0 && sendicmp == 1) { struct ip6_hdr *ip6; /* adjust pointer */ @@ -719,7 +721,7 @@ frag6_slowtimo(void) if (q6->ip6q_prev->ip6q_ttl == 0) { IP6STAT_INC(ip6s_fragtimeout); /* XXX in6_ifstat_inc(ifp, ifs6_reass_fail) */ - frag6_freef(q6->ip6q_prev); + frag6_freef(q6->ip6q_prev, 1); } } /* @@ -731,7 +733,7 @@ frag6_slowtimo(void) V_ip6q.ip6q_prev) { IP6STAT_INC(ip6s_fragoverflow); /* XXX in6_ifstat_inc(ifp, ifs6_reass_fail) */ - frag6_freef(V_ip6q.ip6q_prev); + frag6_freef(V_ip6q.ip6q_prev, 1); } CURVNET_RESTORE(); } @@ -757,7 +759,7 @@ frag6_drain(void) while (V_ip6q.ip6q_next != &V_ip6q) { IP6STAT_INC(ip6s_fragdropped); /* XXX in6_ifstat_inc(ifp, ifs6_reass_fail) */ - frag6_freef(V_ip6q.ip6q_next); + frag6_freef(V_ip6q.ip6q_next, 1); } CURVNET_RESTORE(); } REVISION DETAIL https://reviews.freebsd.org/D1388 To: kibab, bz Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 19:26:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88A77CB3 for ; Sun, 28 Dec 2014 19:26:29 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67DCA64456 for ; Sun, 28 Dec 2014 19:26:29 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id sBSJQS5v003170 for ; Sun, 28 Dec 2014 19:26:28 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id sBSJQSLW003169; Sun, 28 Dec 2014 19:26:28 GMT (envelope-from root) Date: Sun, 28 Dec 2014 19:26:28 +0000 To: freebsd-net@freebsd.org From: "kibab (Ilya Bakulin)" Subject: [Differential] [Updated, 20 lines] D1388: IP6: Turned on verbose logging for fragment handling code Message-ID: X-Priority: 3 Thread-Topic: D1388: IP6: Turned on verbose logging for fragment handling code X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NzQ2MGZiNWU2MzVkYmUyNmI0Mzg5NzkzYTJjIFSgWWQ= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 19:26:29 -0000 kibab updated this revision to Diff 2902. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1388?vs=2901&id=2902 BRANCH provost_pfrag REVISION DETAIL https://reviews.freebsd.org/D1388 AFFECTED FILES sys/netinet6/frag6.c To: kibab, bz Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 19:28:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 420FFD53 for ; Sun, 28 Dec 2014 19:28:15 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2497664494 for ; Sun, 28 Dec 2014 19:28:15 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id sBSJSECG004669 for ; Sun, 28 Dec 2014 19:28:14 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id sBSJSE0T004668; Sun, 28 Dec 2014 19:28:14 GMT (envelope-from root) Date: Sun, 28 Dec 2014 19:28:14 +0000 To: freebsd-net@freebsd.org From: "kibab (Ilya Bakulin)" Subject: [Differential] [Updated, 14 lines] D1388: IP6: Turned on verbose logging for fragment handling code Message-ID: <2010ba5e37f25883a802a0b8cded01b3@localhost.localdomain> X-Priority: 3 Thread-Topic: D1388: IP6: Turned on verbose logging for fragment handling code X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NzQ2MGZiNWU2MzVkYmUyNmI0Mzg5NzkzYTJjIFSgWc4= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 19:28:15 -0000 kibab updated this revision to Diff 2903. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1388?vs=2902&id=2903 REVISION DETAIL https://reviews.freebsd.org/D1388 AFFECTED FILES sys/netinet6/frag6.c To: kibab, bz Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sun Dec 28 19:30:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C2EFED4 for ; Sun, 28 Dec 2014 19:30:52 +0000 (UTC) Received: from olymp.kibab.com (olymp6.kibab.com [IPv6:2a01:4f8:160:84c1::2]) by mx1.freebsd.org (Postfix) with ESMTP id D2B41644BF for ; Sun, 28 Dec 2014 19:30:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 olymp.kibab.com 9F20E75917 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=bakulin.de; s=default; t=1419795049; bh=UcUn/8LPhqcMF4GXFyqNO4Vl5TO82hh8aCdDspevRU4=; h=Date:From:To:CC:Subject:References:In-Reply-To; b=jnWQS+wTq0l/Rr3xraau5znEVbLlwRg0cjHAJSvKNYYZVwGDfFGN2av9tkkGZo0fH ZsHzmthK88QfLrFUV7qDIhLvDeSMG0pmCUFb9/XqWPzQS13cXBa0j2mzzDZ0ZyOow4 E6jUhoEvf8vr4I7j5O5EF7JoPynaSUfg1/ODjacs= Message-ID: <54A05A69.607@bakulin.de> Date: Sun, 28 Dec 2014 20:30:49 +0100 From: Ilya Bakulin MIME-Version: 1.0 To: =?UTF-8?B?56We5piO6YGU5ZOJ?= Subject: Re: IPv6 fragments handling References: <5495FAE5.8090707@bakulin.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Dec 2014 19:30:52 -0000 On 22.12.14, 17:59, =E7=A5=9E=E6=98=8E=E9=81=94=E5=93=89 wrote: > At Sat, 20 Dec 2014 23:40:37 +0100, > Ilya Bakulin wrote: > >> But what we do is just silently discarding the overlapping segment, se= e [2]. >> When using PF with fragment reassembly, the behavior changes to what R= FC >> says >> and the packet is completely dropped. >> >> There is no security issue with current behavior, because the already >> received >> part is never overwritten, but following RFC a bit closer would be nic= e. >> >> Maybe we should fix the stack to drop such packets? > That would be a nice cleanup (the current implementation you cited > seems to be written way before RFC5722, so it's not surprising it > doesn't follow the latest recommendation). >> [1] https://tools.ietf.org/html/rfc5722#section-4 >> [2] https://github.com/freebsd/freebsd/blob/master/sys/netinet6/frag6.= c#L443 > -- > JINMEI, Tatuya > Hi Tatuya, thank you for your feedback. I have created a diff [1] that implements the change. [1] https://reviews.freebsd.org/D1388 --=20 Regards, Ilya Bakulin From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 06:29:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C50BA63D for ; Mon, 29 Dec 2014 06:29:13 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D62CBD2 for ; Mon, 29 Dec 2014 06:29:13 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-239-243.lns20.per1.internode.on.net [121.45.239.243]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sBT6T02s065074 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 28 Dec 2014 22:29:04 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54A0F4A7.5020502@freebsd.org> Date: Mon, 29 Dec 2014 14:28:55 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Jason Healy , freebsd-net@freebsd.org Subject: Re: IPv6 routes leaking between FIBs? References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 06:29:14 -0000 On 12/28/14 11:19 AM, Jason Healy wrote: > Hello, > > Trying out FreeBSD for the first time to build a firewall box thats multi-core and runs PF. Im very interested in the FIB code, as it lines up well with the way my core networking equipment works and should allow me to route traffic on an interface thats logically separate from the management interfaces. to some extent this is what it was written for.. teh fib code was written for Ironport/Cisco for separating the management port from the data ports onn their appliances, however the VNET code that came later is an even cleaner way of doing it and FIBs were only used by Ironport because VNET was not yet available. Have you tried vnet jails for interface isolation? > Ive been playing for a bit with the FIB features, but Im getting hung up on IPv6. Im trying to set up two interfaces on my box to each have a different FIB, and to not leak routes between the interfaces: > > # sysctl net.add_addr_allfibs=0 > # ifconfig em1 inet 192.0.2.1/24 fib 1 > # ifconfig em1 inet6 2001:db8:dead:beef::1/64 fib 1 > # ifconfig em2 inet 203.0.113.1/24 fib 2 > # ifconfig em2 inet6 2001:db8:cafe:babe::1/64 fib 2 > > If I then check the routing tables for each FIB, heres what I get: > > # setfib -F 1 netstat -rn > > Routing tables (fib: 1) > > Internet: > Destination Gateway Flags Netif Expire > 192.0.2.0/24 link#2 U em1 > 192.0.2.1 link#2 UHS lo0 > > Internet6: > Destination Gateway Flags Netif Expire > 2001:db8:cafe:babe::/64 link#3 U em2 > 2001:db8:dead:beef::/64 link#2 U em1 > 2001:db8:dead:beef::1 link#2 UHS lo0 > fe80::%em1/64 link#2 U em1 > fe80::a00:27ff:fef6:162a%em1 link#2 UHS lo0 > fe80::%em2/64 link#3 U em2 > fe80::%lo0/64 link#5 U lo0 > > > # setfib -F 2 netstat -rn > > Routing tables (fib: 2) > > Internet: > Destination Gateway Flags Netif Expire > 203.0.113.0/24 link#3 U em2 > 203.0.113.1 link#3 UHS lo0 > > Internet6: > Destination Gateway Flags Netif Expire > 2001:db8:cafe:babe::/64 link#3 U em2 > 2001:db8:cafe:babe::1 link#3 UHS lo0 > 2001:db8:dead:beef::/64 link#2 U em1 > fe80::%em1/64 link#2 U em1 > fe80::%em2/64 link#3 U em2 > fe80::a00:27ff:fe62:d267%em2 link#3 UHS lo0 > fe80::%lo0/64 link#5 U lo0 > > > Note that as expected, the IPv4 routes are constrained to their FIB (192.0.2.0 to FIB 1 and 203.0.113.0 to FIB 2). However, the IPv6 routes (deadbeef and cafebabe) leak between the FIBs; both prefixes that I add are listed in both FIBs (as well as the link-local stuff). > > According to: > > https://www.freebsd.org/news/status/report-2012-01-2012-03.html#Multi-FIB:-IPv6-Support-and-Other-Enhancements > > IPv6 parity is claimed for the FIB code, so Im not sure if Im doing it wrong, or if theres a problem with the FIB code and IPv6 routes. > > Thanks in advance for any help or clarification! > > Jason > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 07:08:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82296BE5 for ; Mon, 29 Dec 2014 07:08:20 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 535592218 for ; Mon, 29 Dec 2014 07:08:20 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-239-243.lns20.per1.internode.on.net [121.45.239.243]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sBT78Fxm065191 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 28 Dec 2014 23:08:18 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54A0FDD9.4090009@freebsd.org> Date: Mon, 29 Dec 2014 15:08:09 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Nikolay Denev , "freebsd-net@freebsd.org" Subject: Re: setfib and RSTs References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 07:08:20 -0000 On 12/26/14 10:41 PM, Nikolay Denev wrote: > Hi, > > I have a process (bittorrent client) running in a non-default fib and using > a VPN for default gateway: > > from /etc/rc.local : > > /usr/sbin/setfib 1 route add $vpn_provider 10.0.0.1 > /usr/sbin/setfib 1 /usr/local/sbin/openvpn --config > /usr/local/etc/openvpn/provider.ovpn > /usr/sbin/setfib 1 /usr/sbin/service transmission onestart > > Then openvpn installs default gateway in fib 1 to point to the tun(4) > interface. > > Stil, I'm seeing RST packets from the bittorrent client process to be sent > not via the tunnel, but to the default gateway of the lan which seems > wrong. As if when the kernel generates the RST it does not take into > account the FIB of the socket? it's possible that you are correct. I checked that RST and other generated packets used the FIB for the session if it existed when they are generated, but I don't know what they do when a single unexpected packet enters..You may need toset the fib in received packets using either the ipfw setfib command or the ifconfig fib command. The RST should then use the same FIB to respond. let me know.. > Any ideas? > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 13:11:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 642AF139; Mon, 29 Dec 2014 13:11:39 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D915E66A75; Mon, 29 Dec 2014 13:11:38 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id r20so23658726wiv.0; Mon, 29 Dec 2014 05:11:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Qd0IsHta1sdoRSw5LjXcmcU2A6BayKZTBSHtE6Xqz/M=; b=WOtcKFh9lII+sOdTC9ZO5wkVoTiX01BlL6Ajnl+Ni/gT6v8DPbDfmLYZgQVVFD83yO MFzfJ93hULCdMe+fJ8yoredGPCRByalaGgbMZz8e8zkfAKkY5LfQqL5Q69FGEruNFtgr 3UgHtsl/Tql8wZ9lh5UOTYD3/JSS4J9jUm7Uj/yDY4PljjZBJx0Fz+i+C5cnjDrR7hLU DkSlPJp7bHdbhsd9ysVQQCtGzzFsWinJXx9lj7xHmUZ0Mz1xNEcRLJrtFNE2ZT8mcUrp frMkUvOn7FljJi2csFsZV1Qo2lZMLFSKA/xRKD6BJFy03PqoqSjeFKgunEZkfjv+Tm+i 1ENg== MIME-Version: 1.0 X-Received: by 10.194.57.43 with SMTP id f11mr106285986wjq.6.1419858697254; Mon, 29 Dec 2014 05:11:37 -0800 (PST) Received: by 10.27.177.218 with HTTP; Mon, 29 Dec 2014 05:11:37 -0800 (PST) In-Reply-To: <54A0FDD9.4090009@freebsd.org> References: <54A0FDD9.4090009@freebsd.org> Date: Mon, 29 Dec 2014 14:11:37 +0100 Message-ID: Subject: Re: setfib and RSTs From: Nikolay Denev To: Julian Elischer Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 13:11:39 -0000 On Mon, Dec 29, 2014 at 8:08 AM, Julian Elischer wrote: > On 12/26/14 10:41 PM, Nikolay Denev wrote: > >> Hi, >> >> I have a process (bittorrent client) running in a non-default fib and >> using >> a VPN for default gateway: >> >> from /etc/rc.local : >> >> /usr/sbin/setfib 1 route add $vpn_provider 10.0.0.1 >> /usr/sbin/setfib 1 /usr/local/sbin/openvpn --config >> /usr/local/etc/openvpn/provider.ovpn >> /usr/sbin/setfib 1 /usr/sbin/service transmission onestart >> >> Then openvpn installs default gateway in fib 1 to point to the tun(4) >> interface. >> >> Stil, I'm seeing RST packets from the bittorrent client process to be sent >> not via the tunnel, but to the default gateway of the lan which seems >> wrong. As if when the kernel generates the RST it does not take into >> account the FIB of the socket? >> > it's possible that you are correct. > I checked that RST and other generated packets used the FIB for the > session if it existed when they are generated, > but I don't know what they do when a single unexpected packet enters..You > may need toset the fib in received packets using either the ipfw setfib > command or the ifconfig fib command. The RST should then use the same FIB > to respond. > > let me know.. > > > Any ideas? >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> > The weird thing was that the RST was a response to a SYN-ACK. What I saw was the transmission bittorent daemon to try to connect to some peer over tun2 (the openvpn connection), so it sends the SYN, the peer responds with SYN-ACK, then I see RST back to the peer, but not over the tun2, but over lagg0 (as originally mentioned). Here 10.8.0.31 is the tun2 address, and XX.XX.XX.XX is one of the peers that I was able to capture this happening. (both SYN-ACK and RST's are retransmitted, as the RSTs get blocked by the pf running on the default gw as it does not have state for them, as the VPN is a running on a host inside the network) # Captured on tun2: [14:03][ndenev@macbook:~]%tcpdump -vnr 1.pcap host XX.XX.XX.XX reading from file 1.pcap, link-type NULL (BSD loopback) 20:43:39.623920 IP (tos 0x0, ttl 64, id 12450, offset 0, flags [DF], proto TCP (6), length 60) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [S], cksum 0xace8 (correct), seq 2985341180, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val 515982338 ecr 0], length 0 20:43:39.912558 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5a3d (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207923992 ecr 515982338,nop,wscale 7], length 0 20:43:41.132751 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x590a (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207924299 ecr 515982338,nop,wscale 7], length 0 20:43:43.141474 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5716 (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207924799 ecr 515982338,nop,wscale 7], length 0 20:43:47.129657 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x532e (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207925799 ecr 515982338,nop,wscale 7], length 0 20:43:55.133735 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x4b5e (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207927799 ecr 515982338,nop,wscale 7], length 0 20:44:11.134635 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x3bbe (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207931799 ecr 515982338,nop,wscale 7], length 0 20:45:11.290740 IP (tos 0x0, ttl 64, id 61675, offset 0, flags [DF], proto TCP (6), length 60) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [S], cksum 0xdbfb (correct), seq 2733189104, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val 516074004 ecr 0], length 0 20:45:11.595504 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x81f1 (correct), seq 2072870049, ack 2733189105, win 28960, options [mss 1368,sackOK,TS val 207946912 ecr 516074004,nop,wscale 7], length 0 20:45:12.748574 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x80cf (correct), seq 2072870049, ack 2733189105, win 28960, options [mss 1368,sackOK,TS val 207947202 ecr 516074004,nop,wscale 7], length 0 20:45:14.746692 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x7edb (correct), seq 2072870049, ack 2733189105, win 28960, options [mss 1368,sackOK,TS val 207947702 ecr 516074004,nop,wscale 7], length 0 # Captured on lagg0: [14:04][ndenev@macbook:~]%tcpdump -vnr 2.pcap host XX.XX.XX.XX reading from file 2.pcap, link-type EN10MB (Ethernet) 20:43:39.912570 IP (tos 0x0, ttl 64, id 12524, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->a2c)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:41.132761 IP (tos 0x0, ttl 64, id 13085, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->7fb)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:43.141485 IP (tos 0x0, ttl 64, id 14702, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->1aa)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:47.129669 IP (tos 0x0, ttl 64, id 16915, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->f904)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:55.133748 IP (tos 0x0, ttl 64, id 23534, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->df29)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:44:11.134645 IP (tos 0x0, ttl 64, id 30066, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->c5a5)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:45:11.595514 IP (tos 0x0, ttl 64, id 62112, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->4877)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:12.748583 IP (tos 0x0, ttl 64, id 62741, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->4602)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:14.746703 IP (tos 0x0, ttl 64, id 63874, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->4195)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:18.754657 IP (tos 0x0, ttl 64, id 65318, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->3bf1)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:26.759522 IP (tos 0x0, ttl 64, id 3473, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->2d87)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:42.758524 IP (tos 0x0, ttl 64, id 10814, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->10da)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 [14:04][ndenev@macbook:~]% I'll try the IPFW workaround, I'm not seeing this at the moment (looks like it was related to how many active torrents there were, as now there are none). --Nikolay From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 13:42:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 187B0518 for ; Mon, 29 Dec 2014 13:42:54 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED4762041 for ; Mon, 29 Dec 2014 13:42:53 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id sBTDgrar045286 for ; Mon, 29 Dec 2014 13:42:53 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id sBTDgrtY045285; Mon, 29 Dec 2014 13:42:53 GMT (envelope-from root) Date: Mon, 29 Dec 2014 13:42:53 +0000 To: freebsd-net@freebsd.org From: "ae (Andrey V. Elsukov)" Subject: [Differential] [Changed Subscribers] D1388: IP6: Turned on verbose logging for fragment handling code Message-ID: <5a940340da4cf65b14d247129c4e864c@localhost.localdomain> X-Priority: 3 Thread-Topic: D1388: IP6: Turned on verbose logging for fragment handling code X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NzQ2MGZiNWU2MzVkYmUyNmI0Mzg5NzkzYTJjIFShWl0= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 13:42:54 -0000 ae added a subscriber: ae. ae added a comment. I think you need to adjust some comments in frag6_input(). REVISION DETAIL https://reviews.freebsd.org/D1388 To: kibab, bz Cc: ae, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 16:03:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B46F5D36; Mon, 29 Dec 2014 16:03:23 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 490CE64B78; Mon, 29 Dec 2014 16:03:23 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id b13so19251990wgh.3; Mon, 29 Dec 2014 08:03:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=KMu8GeBgovWQGaDxF5gLmrBLloTIHkb/LVqhqH8QlJA=; b=xuMKleET1eSK7J9Rnn5hUqV/Qci2xPQVxS2G3QsHI3SQUsU7+P2px/95EYoPDnIpCU EA/YJFwfsrMgUkBwYxL5qvufJ9rE3erAxQJ1Q6uhqT77A2jvBNBwEm9DeEi60pFj5msJ F4wdX7w5yIhkro8TlqRVBFEuwe4ly3ils3tDeVEA4ClVy1tpdjZFDviIPJHr7s5SipKy uS3VmAUM5brDXL1/5sLnNpHFtNB26xpEf6Pm3SZYR2SyVV8kMXAEkp1+xlmS/t+x2YMs Ehfhy37u8ZUOD1zefcHVBThp+j7Ye6LPjg8sNtxoxCH7BVAmhrv7nbWn0ysU1X0Gj6P/ 3HcA== MIME-Version: 1.0 X-Received: by 10.194.201.137 with SMTP id ka9mr114341265wjc.66.1419869001724; Mon, 29 Dec 2014 08:03:21 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.17.129 with HTTP; Mon, 29 Dec 2014 08:03:21 -0800 (PST) In-Reply-To: References: Date: Mon, 29 Dec 2014 09:03:21 -0700 X-Google-Sender-Auth: JkbB3dSmz4MjomxTmhBXfFfmSh8 Message-ID: Subject: Re: IPv6 routes leaking between FIBs? From: Alan Somers To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , Jason Healy X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 16:03:23 -0000 On Sun, Dec 28, 2014 at 3:16 AM, Bjoern A. Zeeb wrote: > >> On 28 Dec 2014, at 03:19 , Jason Healy wrote: >> >> Hello, >> >> Trying out FreeBSD for the first time to build a firewall box that=E2=80= =99s multi-core and runs PF. I=E2=80=99m very interested in the FIB code, = as it lines up well with the way my core networking equipment works and sho= uld allow me to route traffic on an interface that=E2=80=99s logically sepa= rate from the management interfaces. >> >> I=E2=80=99ve been playing for a bit with the FIB features, but I=E2=80= =99m getting hung up on IPv6. I=E2=80=99m trying to set up two interfaces = on my box to each have a different FIB, and to not leak routes between the = interfaces: >> >> # sysctl net.add_addr_allfibs=3D0 >> # ifconfig em1 inet 192.0.2.1/24 fib 1 >> # ifconfig em1 inet6 2001:db8:dead:beef::1/64 fib 1 >> # ifconfig em2 inet 203.0.113.1/24 fib 2 >> # ifconfig em2 inet6 2001:db8:cafe:babe::1/64 fib 2 >> >> If I then check the routing tables for each FIB, here=E2=80=99s what I g= et: >> >> # setfib -F 1 netstat -rn >> >> Routing tables (fib: 1) >> >> Internet: >> Destination Gateway Flags Netif Expire >> 192.0.2.0/24 link#2 U em1 >> 192.0.2.1 link#2 UHS lo0 >> >> Internet6: >> Destination Gateway Flags = Netif Expire >> 2001:db8:cafe:babe::/64 link#3 U = em2 >> 2001:db8:dead:beef::/64 link#2 U = em1 >> 2001:db8:dead:beef::1 link#2 UHS = lo0 >> fe80::%em1/64 link#2 U = em1 >> fe80::a00:27ff:fef6:162a%em1 link#2 UHS = lo0 >> fe80::%em2/64 link#3 U = em2 >> fe80::%lo0/64 link#5 U = lo0 >> >> >> # setfib -F 2 netstat -rn >> >> Routing tables (fib: 2) >> >> Internet: >> Destination Gateway Flags Netif Expire >> 203.0.113.0/24 link#3 U em2 >> 203.0.113.1 link#3 UHS lo0 >> >> Internet6: >> Destination Gateway Flags = Netif Expire >> 2001:db8:cafe:babe::/64 link#3 U = em2 >> 2001:db8:cafe:babe::1 link#3 UHS = lo0 >> 2001:db8:dead:beef::/64 link#2 U = em1 >> fe80::%em1/64 link#2 U = em1 >> fe80::%em2/64 link#3 U = em2 >> fe80::a00:27ff:fe62:d267%em2 link#3 UHS = lo0 >> fe80::%lo0/64 link#5 U = lo0 >> >> >> Note that as expected, the IPv4 routes are constrained to their FIB (192= .0.2.0 to FIB 1 and 203.0.113.0 to FIB 2). However, the IPv6 routes (deadb= eef and cafebabe) leak between the FIBs; both prefixes that I add are liste= d in both FIBs (as well as the link-local stuff). >> >> According to: >> >> https://www.freebsd.org/news/status/report-2012-01-2012-03.html#Multi-F= IB:-IPv6-Support-and-Other-Enhancements >> >> IPv6 parity is claimed for the FIB code, so I=E2=80=99m not sure if I=E2= =80=99m doing it wrong, or if there=E2=80=99s a problem with the FIB code a= nd IPv6 routes. >> >> Thanks in advance for any help or clarification! > > > People simply broke it (again). Please file a bug report. You may ment= ion that there are regression test scripts in src/tools/ somewhere to test = all the cases for IPv6. Sounds like those tests need to be merged into the ATF tests at tests/sys/netinet/fibs_test.sh so they'll run continuously. -Alan From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 17:20:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8842ADDD; Mon, 29 Dec 2014 17:20:06 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3AB9037DF; Mon, 29 Dec 2014 17:20:05 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id B85E425D389C; Mon, 29 Dec 2014 17:20:01 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id D9A26C7709D; Mon, 29 Dec 2014 17:20:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id VYKiXuLA6S25; Mon, 29 Dec 2014 17:19:59 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (orange-tun0-ula.sbone.de [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id AC125C7706F; Mon, 29 Dec 2014 17:19:58 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: IPv6 routes leaking between FIBs? From: "Bjoern A. Zeeb" In-Reply-To: Date: Mon, 29 Dec 2014 17:19:57 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <76630B88-9845-414B-A90E-1469B4C2660A@FreeBSD.org> References: To: Alan Somers X-Mailer: Apple Mail (2.1993) Cc: FreeBSD Net , Jason Healy X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 17:20:06 -0000 > On 29 Dec 2014, at 16:03 , Alan Somers wrote: >=20 > On Sun, Dec 28, 2014 at 3:16 AM, Bjoern A. Zeeb = wrote: >>=20 >> People simply broke it (again). Please file a bug report. You may = mention that there are regression test scripts in src/tools/ somewhere = to test all the cases for IPv6. >=20 > Sounds like those tests need to be merged into the ATF tests at > tests/sys/netinet/fibs_test.sh so they'll run continuously. Probably but they also need multiple machines (or network stacks), = access to privileged services (ifconfig, ipfw, =E2=80=A6), and I have no = clue how to do all this with ATF. Be my guest :-) =E2=80=94=20 Bjoern A. Zeeb Charles Haddon Spurgeon: "Friendship is one of the sweetest joys of life. Many might have failed beneath the bitterness of their trial had they not found a friend." From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 17:34:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 425832FF for ; Mon, 29 Dec 2014 17:34:29 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F0963A47 for ; Mon, 29 Dec 2014 17:34:29 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id sBTHYSHJ084487 for ; Mon, 29 Dec 2014 17:34:28 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id sBTHYSVd084486; Mon, 29 Dec 2014 17:34:28 GMT (envelope-from root) Date: Mon, 29 Dec 2014 17:34:28 +0000 To: freebsd-net@freebsd.org From: "bz (Bjoern A. Zeeb)" Subject: [Differential] [Updated] D1388: IP6: Turned on verbose logging for fragment handling code Message-ID: <6e70dff155965dfc2339d8ac7fc48a73@localhost.localdomain> X-Priority: 3 Thread-Topic: D1388: IP6: Turned on verbose logging for fragment handling code X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NzQ2MGZiNWU2MzVkYmUyNmI0Mzg5NzkzYTJjIFShkKQ= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 17:34:29 -0000 bz added a comment. I somehow would expect a comment to be updated somewhere referencing RFC5722? Appart from that no objections though I have only skimmed through and not properly reviewed this. REVISION DETAIL https://reviews.freebsd.org/D1388 To: kibab, bz Cc: ae, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 17:42:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90C515BA; Mon, 29 Dec 2014 17:42:05 +0000 (UTC) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36A453C9B; Mon, 29 Dec 2014 17:42:05 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id h11so22691064wiw.13; Mon, 29 Dec 2014 09:42:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=LvEAJZFIHSMUqMoWqIbTdgfPqFW4QTecCAYn+YkzcBs=; b=T4uxrM1m3GT9uK/RaxDi77hVRva/MbVsZsv1qghB7IABsPIXcB+cshgk1/DuecPYUM k03rcISRpTcKZvdOSAS9QhqXS5rv2eucvLY2yL9oiLWK4Y1IjAUVygGltzCWUebrjiYc cFkyQgyTK08eN3J/5ttcFhC4Tei8COVXFNqQ3FedIU45xZqWtRYIrHXHRdOHloCUMJjc TtlcC205pM+TWDaZLTRJ1QsnTXAoICYBiQkCM7sDI0AfznPwXJXeHSq7x9D7ljX+tb6s oMDFtvhpHK6x0cpH3g8FdJLS6RgJ2wVoQCZEcVDu1zI6oIrJZ70B/0A8svJAR6z2muPT QhiQ== MIME-Version: 1.0 X-Received: by 10.194.205.138 with SMTP id lg10mr82678019wjc.130.1419874923214; Mon, 29 Dec 2014 09:42:03 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.17.129 with HTTP; Mon, 29 Dec 2014 09:42:03 -0800 (PST) In-Reply-To: <76630B88-9845-414B-A90E-1469B4C2660A@FreeBSD.org> References: <76630B88-9845-414B-A90E-1469B4C2660A@FreeBSD.org> Date: Mon, 29 Dec 2014 10:42:03 -0700 X-Google-Sender-Auth: yj3c3chBCq1Kpo4u3DM3IsBtbU4 Message-ID: Subject: Re: IPv6 routes leaking between FIBs? From: Alan Somers To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , Jason Healy X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 17:42:05 -0000 On Mon, Dec 29, 2014 at 10:19 AM, Bjoern A. Zeeb wrote: > >> On 29 Dec 2014, at 16:03 , Alan Somers wrote: >> >> On Sun, Dec 28, 2014 at 3:16 AM, Bjoern A. Zeeb wrote: >>> >>> People simply broke it (again). Please file a bug report. You may me= ntion that there are regression test scripts in src/tools/ somewhere to tes= t all the cases for IPv6. >> >> Sounds like those tests need to be merged into the ATF tests at >> tests/sys/netinet/fibs_test.sh so they'll run continuously. > > Probably but they also need multiple machines (or network stacks), access= to privileged services (ifconfig, ipfw, =E2=80=A6), and I have no clue how= to do all this with ATF. Be my guest :-) I will -- the day that my boss tells me that we need to ship IPv6. Until then I can't justify the time required. It looks like the local tests in tools/test/netfibs/initiator.sh should be easy to convert. Could the others be done with two tap(4) interfaces on the same host? -Alan From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 17:59:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23C1EABC; Mon, 29 Dec 2014 17:59:29 +0000 (UTC) Received: from omicron.logn.net (omicron.logn.net [72.10.118.7]) by mx1.freebsd.org (Postfix) with ESMTP id 01A1764031; Mon, 29 Dec 2014 17:59:28 +0000 (UTC) Received: from [10.3.73.86] (unknown [10.3.73.86]) by omicron.logn.net (Postfix) with ESMTPSA id CCA294A0082; Mon, 29 Dec 2014 12:59:21 -0500 (EST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: IPv6 routes leaking between FIBs? From: Jason Healy In-Reply-To: <54A0F4A7.5020502@freebsd.org> Date: Mon, 29 Dec 2014 12:59:21 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54A0F4A7.5020502@freebsd.org> To: Julian Elischer X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 17:59:29 -0000 On Dec 29, 2014, at 1:28 AM, Julian Elischer wrote: > to some extent this is what it was written for.. teh fib code was = written for Ironport/Cisco for separating the management port from the = data ports onn their appliances, however the VNET code that came later = is an even cleaner way of doing it and FIBs were only used by Ironport = because VNET was not yet available. Have you tried vnet jails for = interface isolation? I freely admit that I haven=92t. I=92m just coming over to FreeBSD and = while I=92m aware of jails, I thought of them more as service isolation = than for routing. I=92m searching around for a moment, and I=92m not 100% sure this is = going to work for my use case. Can you confirm that jails would be the = most appropriate way to solve my problem? These are the major = requirements: - A router/firewall that will perform NAT from an internal RFC1918 = space to public IPv4, as well as stateful firewalling of IPv6 packets = passed to it. - 3 interfaces: 1) Transit interface (10g, packets to/from PF are received/sent on = this interface) 2) PFsync (to connect to a second box for active-active PF) 3) Management (LAN side only) - Separate routing tables for the transit and management interfaces, so = that the transit interface can have a default route that is distinct = from that of the management network. It sounds to me that if I ran this as a jail, I=92d need to throw the = 10g transit interface and the pfsync interface into the jail, and leave = the management interface on the host. I=92d probably need to run PF in = the jail as well? Or are we just using the jail to isolate the routing = tables, and I=92d still run PF on the host? I=92m happy to provide more details on the setup in case there=92s a = better way to architect this. I=92m a Debian/OpenBSD guy, so I=92m = sorry if I don=92t have all the terminology sorted out yet... I will still file a bug against the FIB code, as it sounds like that=92s = not working as intended/designed. Thanks, Jason From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 18:47:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99FD5513; Mon, 29 Dec 2014 18:47:10 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22F271918; Mon, 29 Dec 2014 18:47:10 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x13so19571465wgg.19; Mon, 29 Dec 2014 10:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Lpc32/eYs60nhbCUH77/H7mlNpPSyA7es1cmPrKShnc=; b=EAnWfnvXlCBW1GW9X9UgcQFb08ZcneonYaHIpLh+UHY3S8zu5vij019rUkdwPBSrJr 7c0NXXMgOOMhZ3gTobrZJ9AIUolyOg9sOYClE4Bp6CSJunxuFIRaNaZzP+f3JM4c9Lph o11oOlZVx3zqsEG3YCkWyEpimlYSAIUOCK/i2bcy8fIQIvsGlVZsIbCpzJlg7UOs20QI jOOHYpTSCbXuPVCrngJHsawVi2Xza92TWAOjNjGBBp6Wg3Wmo8DGKmDQ5CrJHoP29P2B +kJAxH81KDq2C6N/E7bafjlMDHrhXslmukSwoXESQTwQLAQW5E/si82k1oh7ts8jHXEn LdHA== MIME-Version: 1.0 X-Received: by 10.180.87.36 with SMTP id u4mr97077712wiz.20.1419878828626; Mon, 29 Dec 2014 10:47:08 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.106.195 with HTTP; Mon, 29 Dec 2014 10:47:08 -0800 (PST) In-Reply-To: References: <54A0FDD9.4090009@freebsd.org> Date: Mon, 29 Dec 2014 10:47:08 -0800 X-Google-Sender-Auth: JJC7FnuXucDhlUgrf6ejAF_ocJI Message-ID: Subject: Re: setfib and RSTs From: Adrian Chadd To: Nikolay Denev Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 18:47:10 -0000 Have you filed a PR about this? This seems like it's pretty important to fix. (For other non-torrent reasons too; like people wishing to use this for things like tor browsing.) -adrian On 29 December 2014 at 05:11, Nikolay Denev wrote: > On Mon, Dec 29, 2014 at 8:08 AM, Julian Elischer wrote: > >> On 12/26/14 10:41 PM, Nikolay Denev wrote: >> >>> Hi, >>> >>> I have a process (bittorrent client) running in a non-default fib and >>> using >>> a VPN for default gateway: >>> >>> from /etc/rc.local : >>> >>> /usr/sbin/setfib 1 route add $vpn_provider 10.0.0.1 >>> /usr/sbin/setfib 1 /usr/local/sbin/openvpn --config >>> /usr/local/etc/openvpn/provider.ovpn >>> /usr/sbin/setfib 1 /usr/sbin/service transmission onestart >>> >>> Then openvpn installs default gateway in fib 1 to point to the tun(4) >>> interface. >>> >>> Stil, I'm seeing RST packets from the bittorrent client process to be sent >>> not via the tunnel, but to the default gateway of the lan which seems >>> wrong. As if when the kernel generates the RST it does not take into >>> account the FIB of the socket? >>> >> it's possible that you are correct. >> I checked that RST and other generated packets used the FIB for the >> session if it existed when they are generated, >> but I don't know what they do when a single unexpected packet enters..You >> may need toset the fib in received packets using either the ipfw setfib >> command or the ifconfig fib command. The RST should then use the same FIB >> to respond. >> >> let me know.. >> >> >> Any ideas? >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> >> > > The weird thing was that the RST was a response to a SYN-ACK. > What I saw was the transmission bittorent daemon to try to connect to some > peer over tun2 (the openvpn connection), > so it sends the SYN, the peer responds with SYN-ACK, then I see RST back to > the peer, but not over the tun2, but over lagg0 (as originally mentioned). > > Here 10.8.0.31 is the tun2 address, and XX.XX.XX.XX is one of the peers > that I was able to capture this happening. (both SYN-ACK and RST's are > retransmitted, as the RSTs get blocked by the pf running on the default gw > as it does not have state for them, as the VPN is a running on a host > inside the network) > > # Captured on tun2: > > [14:03][ndenev@macbook:~]%tcpdump -vnr 1.pcap host XX.XX.XX.XX > reading from file 1.pcap, link-type NULL (BSD loopback) > 20:43:39.623920 IP (tos 0x0, ttl 64, id 12450, offset 0, flags [DF], proto > TCP (6), length 60) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [S], cksum 0xace8 (correct), > seq 2985341180, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val > 515982338 ecr 0], length 0 > 20:43:39.912558 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5a3d > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207923992 ecr 515982338,nop,wscale 7], length 0 > 20:43:41.132751 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x590a > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207924299 ecr 515982338,nop,wscale 7], length 0 > 20:43:43.141474 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5716 > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207924799 ecr 515982338,nop,wscale 7], length 0 > 20:43:47.129657 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x532e > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207925799 ecr 515982338,nop,wscale 7], length 0 > 20:43:55.133735 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x4b5e > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207927799 ecr 515982338,nop,wscale 7], length 0 > 20:44:11.134635 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x3bbe > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207931799 ecr 515982338,nop,wscale 7], length 0 > 20:45:11.290740 IP (tos 0x0, ttl 64, id 61675, offset 0, flags [DF], proto > TCP (6), length 60) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [S], cksum 0xdbfb (correct), > seq 2733189104, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val > 516074004 ecr 0], length 0 > 20:45:11.595504 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x81f1 > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > 1368,sackOK,TS val 207946912 ecr 516074004,nop,wscale 7], length 0 > 20:45:12.748574 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x80cf > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > 1368,sackOK,TS val 207947202 ecr 516074004,nop,wscale 7], length 0 > 20:45:14.746692 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x7edb > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > 1368,sackOK,TS val 207947702 ecr 516074004,nop,wscale 7], length 0 > > # Captured on lagg0: > > [14:04][ndenev@macbook:~]%tcpdump -vnr 2.pcap host XX.XX.XX.XX > reading from file 2.pcap, link-type EN10MB (Ethernet) > 20:43:39.912570 IP (tos 0x0, ttl 64, id 12524, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->a2c)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:41.132761 IP (tos 0x0, ttl 64, id 13085, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->7fb)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:43.141485 IP (tos 0x0, ttl 64, id 14702, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->1aa)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:47.129669 IP (tos 0x0, ttl 64, id 16915, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->f904)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:55.133748 IP (tos 0x0, ttl 64, id 23534, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->df29)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:44:11.134645 IP (tos 0x0, ttl 64, id 30066, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->c5a5)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:45:11.595514 IP (tos 0x0, ttl 64, id 62112, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->4877)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:12.748583 IP (tos 0x0, ttl 64, id 62741, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->4602)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:14.746703 IP (tos 0x0, ttl 64, id 63874, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->4195)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:18.754657 IP (tos 0x0, ttl 64, id 65318, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->3bf1)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:26.759522 IP (tos 0x0, ttl 64, id 3473, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->2d87)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:42.758524 IP (tos 0x0, ttl 64, id 10814, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->10da)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > [14:04][ndenev@macbook:~]% > > > I'll try the IPFW workaround, I'm not seeing this at the moment (looks like > it was related to how many active torrents there were, as now there are > none). > > --Nikolay > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 19:17:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10F0BFCA for ; Mon, 29 Dec 2014 19:17:50 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BCBE61DCD for ; Mon, 29 Dec 2014 19:17:49 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-239-243.lns20.per1.internode.on.net [121.45.239.243]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sBTJHiAm067593 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 29 Dec 2014 11:17:46 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54A1A8D2.9080704@freebsd.org> Date: Tue, 30 Dec 2014 03:17:38 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Jason Healy Subject: Re: IPv6 routes leaking between FIBs? References: <54A0F4A7.5020502@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 19:17:50 -0000 On 12/30/14 1:59 AM, Jason Healy wrote: > On Dec 29, 2014, at 1:28 AM, Julian Elischer wrote: > >> to some extent this is what it was written for.. teh fib code was written for Ironport/Cisco for separating the management port from the data ports onn their appliances, however the VNET code that came later is an even cleaner way of doing it and FIBs were only used by Ironport because VNET was not yet available. Have you tried vnet jails for interface isolation? > I freely admit that I havent. Im just coming over to FreeBSD and while Im aware of jails, I thought of them more as service isolation than for routing. > > Im searching around for a moment, and Im not 100% sure this is going to work for my use case. Can you confirm that jails would be the most appropriate way to solve my problem? These are the major requirements: > > - A router/firewall that will perform NAT from an internal RFC1918 space to public IPv4, as well as stateful firewalling of IPv6 packets passed to it. > > - 3 interfaces: > 1) Transit interface (10g, packets to/from PF are received/sent on this interface) > 2) PFsync (to connect to a second box for active-active PF) > 3) Management (LAN side only) the only hitch may be the pfsync stuff.. I have no idea about how virtualised that is and I never use pf..or pfsync. Basically you can assign a complatly separate network stack to teh management interface, (or the other ones) and run whatever the appliation is in the jail.. it's still possible to communicate with the jailed processes using shared files or fifos, but they have a completely separate network stack and are therefore completely unaware of the management interface. Each jail (if enabled with vnet option) has itsl own interfaces, routing tables, firewall(s) etc. > - Separate routing tables for the transit and management interfaces, so that the transit interface can have a default route that is distinct from that of the management network. > > It sounds to me that if I ran this as a jail, Id need to throw the 10g transit interface and the pfsync interface into the jail, and leave the management interface on the host. Id probably need to run PF in the jail as well? Or are we just using the jail to isolate the routing tables, and Id still run PF on the host? > > Im happy to provide more details on the setup in case theres a better way to architect this. Im a Debian/OpenBSD guy, so Im sorry if I dont have all the terminology sorted out yet... > > I will still file a bug against the FIB code, as it sounds like thats not working as intended/designed. > > Thanks, > > Jason > > > > From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 19:34:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54CD2592; Mon, 29 Dec 2014 19:34:13 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D026A31C7; Mon, 29 Dec 2014 19:34:12 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 3C4A225D3A42; Mon, 29 Dec 2014 19:34:09 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 469B3C7709D; Mon, 29 Dec 2014 19:34:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id JQgFjP2hO-HG; Mon, 29 Dec 2014 19:34:06 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (orange-tun0-ula.sbone.de [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 6F44CC7706F; Mon, 29 Dec 2014 19:34:05 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: IPv6 routes leaking between FIBs? From: "Bjoern A. Zeeb" In-Reply-To: <54A1A8D2.9080704@freebsd.org> Date: Mon, 29 Dec 2014 19:34:03 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54A0F4A7.5020502@freebsd.org> <54A1A8D2.9080704@freebsd.org> To: Julian Elischer X-Mailer: Apple Mail (2.1993) Cc: freebsd-net@freebsd.org, Jason Healy X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 19:34:13 -0000 > On 29 Dec 2014, at 19:17 , Julian Elischer wrote: >=20 > On 12/30/14 1:59 AM, Jason Healy wrote: >> On Dec 29, 2014, at 1:28 AM, Julian Elischer = wrote: >>=20 >>> to some extent this is what it was written for.. teh fib code was = written for Ironport/Cisco for separating the management port from the = data ports onn their appliances, however the VNET code that came later = is an even cleaner way of doing it and FIBs were only used by Ironport = because VNET was not yet available. Have you tried vnet jails for = interface isolation? >> I freely admit that I haven=92t. I=92m just coming over to FreeBSD = and while I=92m aware of jails, I thought of them more as service = isolation than for routing. >>=20 >> I=92m searching around for a moment, and I=92m not 100% sure this is = going to work for my use case. Can you confirm that jails would be the = most appropriate way to solve my problem? These are the major = requirements: >>=20 >> - A router/firewall that will perform NAT from an internal RFC1918 = space to public IPv4, as well as stateful firewalling of IPv6 packets = passed to it. >>=20 >> - 3 interfaces: >> 1) Transit interface (10g, packets to/from PF are received/sent on = this interface) >> 2) PFsync (to connect to a second box for active-active PF) >> 3) Management (LAN side only) > the only hitch may be the pfsync stuff.. I have no idea about how = virtualised that is and I never use pf..or pfsync. pf and VNETs are a cause for panic at the moment; don=92t go that route = (yet). > Basically you can assign a complatly separate network stack to teh = management interface, (or the other ones) > and run whatever the appliation is in the jail.. it's still possible = to communicate with the jailed processes using shared files or fifos, = but they have a completely separate network stack and are therefore = completely unaware of the management interface. > Each jail (if enabled with vnet option) has itsl own interfaces, = routing tables, firewall(s) etc. >=20 >=20 >=20 >> - Separate routing tables for the transit and management interfaces, = so that the transit interface can have a default route that is distinct = from that of the management network. >>=20 >> It sounds to me that if I ran this as a jail, I=92d need to throw the = 10g transit interface and the pfsync interface into the jail, and leave = the management interface on the host. I=92d probably need to run PF in = the jail as well? Or are we just using the jail to isolate the routing = tables, and I=92d still run PF on the host? >>=20 >> I=92m happy to provide more details on the setup in case there=92s a = better way to architect this. I=92m a Debian/OpenBSD guy, so I=92m = sorry if I don=92t have all the terminology sorted out yet... >>=20 >> I will still file a bug against the FIB code, as it sounds like = that=92s not working as intended/designed. >>=20 >> Thanks, >>=20 >> Jason >>=20 >>=20 >>=20 >>=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" =97=20 Bjoern A. Zeeb Charles Haddon Spurgeon: "Friendship is one of the sweetest joys of life. Many might have failed beneath the bitterness of their trial had they not found a friend." From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 20:18:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 390EF3BE; Mon, 29 Dec 2014 20:18:32 +0000 (UTC) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2A703807; Mon, 29 Dec 2014 20:18:31 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id l13so12536762iga.2; Mon, 29 Dec 2014 12:18:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=mdJp3g2HKW6+1BmHDlinsFScaEda5wRBCglSSiTG1nY=; b=yuiqV6BELSw6ilo/SaPCXoPtjFozBrZWuwRjJNS+sACXoWC7+zz0W1jRUQihVXfKBQ bLhivC9BcuKQxGfmYOBxJyaMP9FWv0L3xwOCSlRl8+cyXsJzpfugVaC8i6d4+/5AZ7ir qvqUioPQ+BHMgqc/5KIf4JcjdVLEq7q98qVTYYrEUpQo3ijYhaekfL8OlAijVnCcZe/D bEhQS4vpDSMF7SO5RC+Gr/t7YevA8vMLDvMsPTXEi6d/2v/+BEn/DbSkzGUMyq3zVg0H bVijzY3H/QIGlVAJtdATmb2hJbFZAXrNmrxq4u4PylZBtext8OSES8/7hCL9sAoGQamh Qu1A== X-Received: by 10.50.28.20 with SMTP id x20mr47763593igg.36.1419884311439; Mon, 29 Dec 2014 12:18:31 -0800 (PST) MIME-Version: 1.0 Sender: mr.kodiak@gmail.com Received: by 10.64.0.171 with HTTP; Mon, 29 Dec 2014 12:18:01 -0800 (PST) In-Reply-To: <36413168.TE9MLvZCzL@kde4.my.domain> References: <36413168.TE9MLvZCzL@kde4.my.domain> From: Bryan Venteicher Date: Mon, 29 Dec 2014 14:18:01 -0600 X-Google-Sender-Auth: zJd5_1WbQ-8-s4Joh3VKhkRCy1s Message-ID: Subject: Re: SIOCSVH, SIOCGVH ioctl(2) and virtio ethernet driver To: Oleg Ginzburg Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net@freebsd.org, Bryan Venteicher X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 20:18:32 -0000 On Fri, Dec 26, 2014 at 8:09 AM, Oleg Ginzburg wrote: > is it possible to use the carp(4) protocol with > vtnet(4) interfaces ( which is used, for example, in bhyve(8) ) > Currently, the standard carp init operation causes an SIOCGVH error: > > /sbin/ifconfig vtnet0 vhid 1 advskew 100 pass pass 10.10.10.10/24 alias > ifconfig: SIOCGVH: Protocol not supported > > > You probably don't have the carp(4) module loaded. From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 21:03:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F12AE1AD; Mon, 29 Dec 2014 21:03:40 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58E832188; Mon, 29 Dec 2014 21:03:40 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id n3so23027385wiv.7; Mon, 29 Dec 2014 13:03:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WbUraqnLVjV4AruQPzLzN+bD3hNwU2gz6jYCrvHkEWY=; b=CCogxt9r28SRJpna+ND5tD0PjPpx6Bu4eI1IqCHQT0w9Dbfui7HrbCAUCwRRbxRqID LSoYqvqM5ZGJ7dHrsrUbke03sKkdRn9QMf68xcaC/H013XpRhQLER6rnzo4lpyuA87gZ ckqLW/fC5ztJaECf7tteKvoBWfEPigE+vCrGqLr+ZDju512osX+HCqO3VjA6sV0rmYAk Y/BmeupTVUdZZPIk1z0LtDUQoa22wwvJsSoNwc6IAP58vh1911xiCcz6pVdkmJEM4Gee UwM6GeNK6PomoxU+9OY6w0B0IWU04wCQKd/UhodNypyIoRejel+jeSHGMttjwywuFnWB cXsQ== MIME-Version: 1.0 X-Received: by 10.180.76.144 with SMTP id k16mr98285280wiw.3.1419887018836; Mon, 29 Dec 2014 13:03:38 -0800 (PST) Received: by 10.27.177.218 with HTTP; Mon, 29 Dec 2014 13:03:38 -0800 (PST) In-Reply-To: References: <54A0FDD9.4090009@freebsd.org> Date: Mon, 29 Dec 2014 22:03:38 +0100 Message-ID: Subject: Re: setfib and RSTs From: Nikolay Denev To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 21:03:41 -0000 On Mon, Dec 29, 2014 at 7:47 PM, Adrian Chadd wrote: > Have you filed a PR about this? This seems like it's pretty important to > fix. > > (For other non-torrent reasons too; like people wishing to use this > for things like tor browsing.) > > > -adrian > > > On 29 December 2014 at 05:11, Nikolay Denev wrote: > > On Mon, Dec 29, 2014 at 8:08 AM, Julian Elischer > wrote: > > > >> On 12/26/14 10:41 PM, Nikolay Denev wrote: > >> > >>> Hi, > >>> > >>> I have a process (bittorrent client) running in a non-default fib and > >>> using > >>> a VPN for default gateway: > >>> > >>> from /etc/rc.local : > >>> > >>> /usr/sbin/setfib 1 route add $vpn_provider 10.0.0.1 > >>> /usr/sbin/setfib 1 /usr/local/sbin/openvpn --config > >>> /usr/local/etc/openvpn/provider.ovpn > >>> /usr/sbin/setfib 1 /usr/sbin/service transmission onestart > >>> > >>> Then openvpn installs default gateway in fib 1 to point to the tun(4) > >>> interface. > >>> > >>> Stil, I'm seeing RST packets from the bittorrent client process to be > sent > >>> not via the tunnel, but to the default gateway of the lan which seems > >>> wrong. As if when the kernel generates the RST it does not take into > >>> account the FIB of the socket? > >>> > >> it's possible that you are correct. > >> I checked that RST and other generated packets used the FIB for the > >> session if it existed when they are generated, > >> but I don't know what they do when a single unexpected packet > enters..You > >> may need toset the fib in received packets using either the ipfw setfib > >> command or the ifconfig fib command. The RST should then use the same > FIB > >> to respond. > >> > >> let me know.. > >> > >> > >> Any ideas? > >>> _______________________________________________ > >>> freebsd-net@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >>> > >>> > >>> > >> > > > > The weird thing was that the RST was a response to a SYN-ACK. > > What I saw was the transmission bittorent daemon to try to connect to > some > > peer over tun2 (the openvpn connection), > > so it sends the SYN, the peer responds with SYN-ACK, then I see RST back > to > > the peer, but not over the tun2, but over lagg0 (as originally > mentioned). > > > > Here 10.8.0.31 is the tun2 address, and XX.XX.XX.XX is one of the peers > > that I was able to capture this happening. (both SYN-ACK and RST's are > > retransmitted, as the RSTs get blocked by the pf running on the default > gw > > as it does not have state for them, as the VPN is a running on a host > > inside the network) > > > > # Captured on tun2: > > > > [14:03][ndenev@macbook:~]%tcpdump -vnr 1.pcap host XX.XX.XX.XX > > reading from file 1.pcap, link-type NULL (BSD loopback) > > 20:43:39.623920 IP (tos 0x0, ttl 64, id 12450, offset 0, flags [DF], > proto > > TCP (6), length 60) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [S], cksum 0xace8 > (correct), > > seq 2985341180, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val > > 515982338 ecr 0], length 0 > > 20:43:39.912558 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5a3d > > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > > 1368,sackOK,TS val 207923992 ecr 515982338,nop,wscale 7], length 0 > > 20:43:41.132751 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x590a > > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > > 1368,sackOK,TS val 207924299 ecr 515982338,nop,wscale 7], length 0 > > 20:43:43.141474 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5716 > > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > > 1368,sackOK,TS val 207924799 ecr 515982338,nop,wscale 7], length 0 > > 20:43:47.129657 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x532e > > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > > 1368,sackOK,TS val 207925799 ecr 515982338,nop,wscale 7], length 0 > > 20:43:55.133735 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x4b5e > > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > > 1368,sackOK,TS val 207927799 ecr 515982338,nop,wscale 7], length 0 > > 20:44:11.134635 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x3bbe > > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > > 1368,sackOK,TS val 207931799 ecr 515982338,nop,wscale 7], length 0 > > 20:45:11.290740 IP (tos 0x0, ttl 64, id 61675, offset 0, flags [DF], > proto > > TCP (6), length 60) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [S], cksum 0xdbfb > (correct), > > seq 2733189104, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val > > 516074004 ecr 0], length 0 > > 20:45:11.595504 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x81f1 > > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > > 1368,sackOK,TS val 207946912 ecr 516074004,nop,wscale 7], length 0 > > 20:45:12.748574 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x80cf > > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > > 1368,sackOK,TS val 207947202 ecr 516074004,nop,wscale 7], length 0 > > 20:45:14.746692 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto > TCP > > (6), length 60) > > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x7edb > > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > > 1368,sackOK,TS val 207947702 ecr 516074004,nop,wscale 7], length 0 > > > > # Captured on lagg0: > > > > [14:04][ndenev@macbook:~]%tcpdump -vnr 2.pcap host XX.XX.XX.XX > > reading from file 2.pcap, link-type EN10MB (Ethernet) > > 20:43:39.912570 IP (tos 0x0, ttl 64, id 12524, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->a2c)!) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x978a), seq 2985341181, win 0, length 0 > > 20:43:41.132761 IP (tos 0x0, ttl 64, id 13085, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->7fb)!) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x978a), seq 2985341181, win 0, length 0 > > 20:43:43.141485 IP (tos 0x0, ttl 64, id 14702, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->1aa)!) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x978a), seq 2985341181, win 0, length 0 > > 20:43:47.129669 IP (tos 0x0, ttl 64, id 16915, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->f904)!) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x978a), seq 2985341181, win 0, length 0 > > 20:43:55.133748 IP (tos 0x0, ttl 64, id 23534, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->df29)!) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x978a), seq 2985341181, win 0, length 0 > > 20:44:11.134645 IP (tos 0x0, ttl 64, id 30066, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->c5a5)!) > > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x978a), seq 2985341181, win 0, length 0 > > 20:45:11.595514 IP (tos 0x0, ttl 64, id 62112, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->4877)!) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x2cb1), seq 2733189105, win 0, length 0 > > 20:45:12.748583 IP (tos 0x0, ttl 64, id 62741, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->4602)!) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x2cb1), seq 2733189105, win 0, length 0 > > 20:45:14.746703 IP (tos 0x0, ttl 64, id 63874, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->4195)!) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x2cb1), seq 2733189105, win 0, length 0 > > 20:45:18.754657 IP (tos 0x0, ttl 64, id 65318, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->3bf1)!) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x2cb1), seq 2733189105, win 0, length 0 > > 20:45:26.759522 IP (tos 0x0, ttl 64, id 3473, offset 0, flags [DF], proto > > TCP (6), length 40, bad cksum 0 (->2d87)!) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x2cb1), seq 2733189105, win 0, length 0 > > 20:45:42.758524 IP (tos 0x0, ttl 64, id 10814, offset 0, flags [DF], > proto > > TCP (6), length 40, bad cksum 0 (->10da)!) > > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 > (incorrect > > -> 0x2cb1), seq 2733189105, win 0, length 0 > > [14:04][ndenev@macbook:~]% > > > > > > I'll try the IPFW workaround, I'm not seeing this at the moment (looks > like > > it was related to how many active torrents there were, as now there are > > none). > > > > --Nikolay > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > No, no PR yet, but I will file one. I wanted to collect some more data first. So, I've did some dtrace digging : [20:54][root@nas:~]#cat reset.d #!/usr/sbin/dtrace -s fbt:kernel:tcp_dropwithreset:entry { printf("reason %d fib %d src_port %d dst_port %d", args[4], args[2] ? args[2]->t_inpcb->inp_inc.inc_fibnum : -1, ntohs(args[1]->th_sport), ntohs(args[1]->th_dport)); /* stack(); */ } [20:54][root@nas:~]#./reset.d dtrace: script './reset.d' matched 1 probe CPU ID FUNCTION:NAME 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 51413 dst_port 47011 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 6881 dst_port 47080 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 16881 dst_port 47009 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 36269 dst_port 47090 0 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 16881 dst_port 47092 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 53263 dst_port 46960 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 22658 dst_port 47079 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 54158 dst_port 47070 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 59547 dst_port 46980 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 16881 dst_port 47092 ^C 1 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 27890 dst_port 47100 0 7849 tcp_dropwithreset:entry reason 3 fib -1 src_port 51413 dst_port 47099 [20:54][root@nas:~]# The port numbers here match RST packets that I'm seeing with tcpdump in another window. reason 3 is BANDLIM_RST_CLOSEDPORT (from icmp_var.h) Looking at tcp_input.c I see that there are cases where the INPCB does not exists, and from what I see this is how the FIB gets determined. Also here I see that tcp_dropwithreset() is called with tcpcb set to NULL, so probably this is why the FIB is not found. Why this is happening, I have no idea yet. --Nikolay From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 21:48:21 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C4E2B07 for ; Mon, 29 Dec 2014 21:48:21 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13AD617C9 for ; Mon, 29 Dec 2014 21:48:21 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id sBTLmKj7088852 for ; Mon, 29 Dec 2014 21:48:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194672] [carp] Changing advskew to 0 from another value doesn't work Date: Mon, 29 Dec 2014 21:48:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RC2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: garga@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 21:48:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194672 Renato Botelho changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |garga@FreeBSD.org Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --- Comment #1 from Renato Botelho --- Looking into sys/netinet/ip_carp.c at line 1706: if (carpr.carpr_advskew > 0) { if (carpr.carpr_advskew >= 255) { error = EINVAL; break; } sc->sc_advskew = carpr.carpr_advskew; } It silently accepts 0 as a good value (no error returned) but it doesn't set sc->sc_advskew. Is it expected to work this way? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 21:59:59 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4BAAEE9 for ; Mon, 29 Dec 2014 21:59:59 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AC3B01AC8 for ; Mon, 29 Dec 2014 21:59:59 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id sBTLxxth028462 for ; Mon, 29 Dec 2014 21:59:59 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194672] [carp] Changing advskew to 0 from another value doesn't work Date: Mon, 29 Dec 2014 21:59:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RC2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: garga@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 21:59:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194672 --- Comment #2 from Renato Botelho --- Created attachment 151093 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=151093&action=edit Fix set advskew back to 0 The attached patch make it possible to set advskew back to 0 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 23:51:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFFE59E3 for ; Mon, 29 Dec 2014 23:51:47 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A1F372E5C for ; Mon, 29 Dec 2014 23:51:47 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 5BBC225D3891; Mon, 29 Dec 2014 23:51:44 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 4CDA8C7709D; Mon, 29 Dec 2014 23:51:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 1RQm9EGcCQgb; Mon, 29 Dec 2014 23:51:41 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:8080:1cb9:8dcd:f166] (unknown [IPv6:fde9:577b:c1a9:4410:8080:1cb9:8dcd:f166]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 6AE7DC7706F; Mon, 29 Dec 2014 23:51:40 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: setfib and RSTs From: "Bjoern A. Zeeb" In-Reply-To: Date: Mon, 29 Dec 2014 23:51:08 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net> References: <54A0FDD9.4090009@freebsd.org> To: Nikolay Denev X-Mailer: Apple Mail (2.1993) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 23:51:48 -0000 > On 29 Dec 2014, at 21:03 , Nikolay Denev wrote: >=20 > No, no PR yet, but I will file one. I wanted to collect some more data > first. >=20 > So, I've did some dtrace digging : >=20 > [20:54][root@nas:~]#cat reset.d > #!/usr/sbin/dtrace -s >=20 > fbt:kernel:tcp_dropwithreset:entry > { > printf("reason %d fib %d src_port %d dst_port %d", args[4], args[2] = ? > args[2]->t_inpcb->inp_inc.inc_fibnum : -1, ntohs(args[1]->th_sport), > ntohs(args[1]->th_dport)); > /* stack(); */ > } > =E2=80=A6 > The port numbers here match RST packets that I'm seeing with tcpdump = in > another window. > reason 3 is BANDLIM_RST_CLOSEDPORT (from icmp_var.h) > Looking at tcp_input.c I see that there are cases where the INPCB does = not > exists, and from what I see this is how the FIB gets determined. > Also here I see that tcp_dropwithreset() is called with tcpcb set to = NULL, > so probably this is why the FIB is not found. >=20 > Why this is happening, I have no idea yet. Could you also check for the mbuf *m and the fibnum from the pkthdr = there? It might be even more interesting to see this for tcp_respond and the = following ip_output as well, in case you want to keep state in the d = script; otherwise just tcp_dropwithreset and/or tcp_respond should be = fine. Usually I would expect for the tcp_dropwithreset case that inp will be = NULL in tcp_respond, the mbuf *m and th will be valid and thus the FIB = number from the incoming mbuf would be re-used as the mbuf will be = re-used, but for that the mbuf needs to be properly tagged on receive. /bz =E2=80=94=20 Bjoern A. Zeeb Charles Haddon Spurgeon: "Friendship is one of the sweetest joys of life. Many might have failed beneath the bitterness of their trial had they not found a friend." From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 00:28:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEBE5A13 for ; Tue, 30 Dec 2014 00:28:58 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5745B1607 for ; Tue, 30 Dec 2014 00:28:58 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x13so20092872wgg.33 for ; Mon, 29 Dec 2014 16:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xz3fGDIhlQsGwBFpzqfqL/jqaCWiKFIf0axA1uKB2wI=; b=rPbHHaJW+Z6knBFg57Z8YZieBTjSCUgKeNpvsvSzwC8L495aPCCz3z/4LFgYx3Rhsh iAW8LS6ZBdTkI0jgCUpAb33zhgwDnZ+yLr5ptFWDdB8NxCgF9RNcAMiKAedzdoFUYeSN 0A55wSoBNMiacSwwFRSq9/uVVTsPFrZaTIL6KVUYG66DzqFxLTPKRyASQY2H9JsS8SR4 lmR9iYwcrBV+bfSZS8HlIsFk4TpD8a0C7shx/tU7Rdq0Wj7fRlajpVrQndadhvwfj7ea 3VjGOmLVXCwjCfHu3dPJH0gz1/+JPQJyZLY0zU0cn+VJaYWuHf+pMgYa0qgW1FkfjUaw VRgw== MIME-Version: 1.0 X-Received: by 10.194.200.1 with SMTP id jo1mr118086119wjc.64.1419899336800; Mon, 29 Dec 2014 16:28:56 -0800 (PST) Received: by 10.27.177.218 with HTTP; Mon, 29 Dec 2014 16:28:56 -0800 (PST) In-Reply-To: <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net> References: <54A0FDD9.4090009@freebsd.org> <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net> Date: Tue, 30 Dec 2014 01:28:56 +0100 Message-ID: Subject: Re: setfib and RSTs From: Nikolay Denev To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 00:28:58 -0000 On Tue, Dec 30, 2014 at 12:51 AM, Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > > > On 29 Dec 2014, at 21:03 , Nikolay Denev wrote: > > > > No, no PR yet, but I will file one. I wanted to collect some more data > > first. > > > > So, I've did some dtrace digging : > > > > [20:54][root@nas:~]#cat reset.d > > #!/usr/sbin/dtrace -s > > > > fbt:kernel:tcp_dropwithreset:entry > > { > > printf("reason %d fib %d src_port %d dst_port %d", args[4], args[2] = ? > > args[2]->t_inpcb->inp_inc.inc_fibnum : -1, ntohs(args[1]->th_sport), > > ntohs(args[1]->th_dport)); > > /* stack(); */ > > } > > =E2=80=A6 > > > The port numbers here match RST packets that I'm seeing with tcpdump in > > another window. > > reason 3 is BANDLIM_RST_CLOSEDPORT (from icmp_var.h) > > Looking at tcp_input.c I see that there are cases where the INPCB does > not > > exists, and from what I see this is how the FIB gets determined. > > Also here I see that tcp_dropwithreset() is called with tcpcb set to > NULL, > > so probably this is why the FIB is not found. > > > > Why this is happening, I have no idea yet. > > Could you also check for the mbuf *m and the fibnum from the pkthdr there= ? > > It might be even more interesting to see this for tcp_respond and the > following ip_output as well, in case you want to keep state in the d > script; otherwise just tcp_dropwithreset and/or tcp_respond should be fi= ne. > > Usually I would expect for the tcp_dropwithreset case that inp will be > NULL in tcp_respond, the mbuf *m and th will be valid and thus the FIB > number from the incoming mbuf would be re-used as the mbuf will be re-use= d, > but for that the mbuf needs to be properly tagged on receive. > > /bz > > =E2=80=94 > Bjoern A. Zeeb Charles Haddon Spurgeon: > "Friendship is one of the sweetest joys of life. Many might have failed > beneath the bitterness of their trial had they not found a friend." > > If I got this right i see FIB 0 here: fbt:kernel:tcp_dropwithreset:entry { this->mbuf =3D (struct mbuf *)args[0]; printf("reason %d fib %d src_port %d dst_port %d", args[4], this->mbuf->M_dat.MH.MH_pkthdr.fibnum, ntohs(args[1]->th_sport), ntohs(args[1]->th_dport)); } I've did some tracing of the the transmission-daemon, and I saw that after it calls connect() on the sockets (which are non-blocking), the connect returns with EINPROGRESS (as expected), but in most cases after about 20-60us close() is called. (not sure why it's doing this, but this is what I saw) So, by that time the SYN-ACK from the remote end is received the socket is already gone and RST is sent using default FIB, which does seem to be the correct behaviour. I've now set the fib for the tun interface with "ifconfig tun2 fib 1", and I see the RSTs sent back over the tunnel. --Nikolay From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 01:02:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38DC737B; Tue, 30 Dec 2014 01:02:47 +0000 (UTC) Received: from omicron.logn.net (omicron.logn.net [72.10.118.7]) by mx1.freebsd.org (Postfix) with ESMTP id 1256A1C95; Tue, 30 Dec 2014 01:02:46 +0000 (UTC) Received: from [10.3.73.86] (unknown [10.3.73.86]) by omicron.logn.net (Postfix) with ESMTPSA id 7E7C94A0082; Mon, 29 Dec 2014 20:02:45 -0500 (EST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: IPv6 routes leaking between FIBs? From: Jason Healy In-Reply-To: Date: Mon, 29 Dec 2014 20:02:45 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54A0F4A7.5020502@freebsd.org> <54A1A8D2.9080704@freebsd.org> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 01:02:47 -0000 On Dec 29, 2014, at 2:34 PM, Bjoern A. Zeeb = wrote: > pf and VNETs are a cause for panic at the moment; don=92t go that = route (yet). Good to know. With that in mind, I think my best workaround for now is to disable IPv6 = on the management interface, leaving the transit interface as the only = one with a v6 address assigned. This effectively isolates it from the = rest of the box, and I=92ll just have to manage the box itself via v4 = for the time being until the v6 fibs get fixed. Meanwhile, I=92ve created PR 196361 to track the underlying issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D196361 I=92ll keep working to spin up PF on the box and I=92ll let you know if = I bump into any other issues. Thanks for the guidance, Jason From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 14:07:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0551B3F8 for ; Tue, 30 Dec 2014 14:07:40 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D1C93ED5 for ; Tue, 30 Dec 2014 14:07:39 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id gd6so12803147lab.17 for ; Tue, 30 Dec 2014 06:07:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=PxBgnlvUHsjfwz7sLSsd9NjrsPkJj/RsO+iwsW6h5Lw=; b=yGK/Ks61jmi4gZEKyqbkmxQFskFpTMv6PbuRlzh3/cQBUeR0VgVKjiEqso8EoBEZG6 DesI8Xxyh2ruZELEvnYyoZ9Avp6K0sIEWRSz4fsP0WPqYkyBGbFs7sYz2lvz1/gfbDEW 0QQ8/7EeXdgJT2KqTL3uIeRBLw3FRKwIE6KLdL59TCNFbfjFG9fSLtelBUX4W8+OqD0/ MB3GOxiUJSsUQPa8RQXvy7hcOg2U2ZtUGTnup4pz0NvG7QSx5GYfp8OxhxzoNN7yK/AF VZwV9wDHHxvxefpG7Wl9FMyr2DoQhHkX63O06CL+dkGun+aUiFS6qOyqKXwZcoGPtQMe THMQ== X-Received: by 10.152.228.164 with SMTP id sj4mr59964042lac.98.1419948457506; Tue, 30 Dec 2014 06:07:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 06:06:57 -0800 (PST) From: Carlos Ferreira Date: Tue, 30 Dec 2014 14:06:57 +0000 Message-ID: Subject: Regarding Netmap internal memory allocation. To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 14:07:40 -0000 To Luigi and to whom may be able to help Hello all. Is it possible to reduce the size of the memory buffer allocated by the netmap module? I'm asking this because I was implementing some testing code, using NICs and a Tap device in an OpenWRT VM with 64MB of RAM. Because of the small RAM amount, the nm_open crashed when the program tried to netmap the tap device, after I previously netmapped one NIC successfully. After the crash, I bumped the VM RAM to 256MB and the test program ran well, but not without me noticing that the VM RAM consumption was increased about 90 MB by netmap. Resuming, I want to know if there is a way to reduce the memory buffer allocation, without recompiling the netmap kernel module. Thank you for the attention. -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 15:13:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC1552F1 for ; Tue, 30 Dec 2014 15:13:12 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AB391A6E for ; Tue, 30 Dec 2014 15:13:12 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id ms9so12670711lab.10 for ; Tue, 30 Dec 2014 07:13:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=8cVgunzULr/xQmcyqRnbEwUgpQQ96ip3mQvKBy+gZ0k=; b=HfOG9WB7CDnGaupx44ZrNDCKxUnLTgN9AYl+AoGHyBvYC/eTAzF5f69BNnCyml8ksW AcKuXBoD3QM4gh9hzNNj7CeZihpP3BcteWdewEEzv9o1nWHwpggpKtgUSRre6NMDZuOQ e7l1II2IN6+dSswTK7DxzFfsmFPX8lb4vcdstODNurQQ82frJgWr9v69tG57S1hRHhsC wOaydoeYKVn5QoNYPpBX8M4IeG9PoqR362MPxZmJXGzcb0Qbj7vK2vjsZUShPpKN213x 8O6ws72l45BfN3sHHS0VXcfkaRC4MJHS+l0IR+9koB6U35vsz9gVTAMrVqNFZMLT2Ohg +Fsw== X-Received: by 10.152.45.65 with SMTP id k1mr63095887lam.14.1419952390459; Tue, 30 Dec 2014 07:13:10 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 07:12:30 -0800 (PST) In-Reply-To: References: From: Carlos Ferreira Date: Tue, 30 Dec 2014 15:12:30 +0000 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 15:13:13 -0000 Update: I noticed that the netmap module was still crashing, after changing the OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. The netmap module is now consuming about 350MB of Ram, which for my objectives is just too much... On 30 December 2014 at 14:06, Carlos Ferreira wrote: > To Luigi and to whom may be able to help > > Hello all. > > Is it possible to reduce the size of the memory buffer allocated by the > netmap module? > I'm asking this because I was implementing some testing code, using NICs > and a Tap device in an OpenWRT VM with 64MB of RAM. > Because of the small RAM amount, the nm_open crashed when the program > tried to netmap the tap device, after I previously netmapped one NIC > successfully. > After the crash, I bumped the VM RAM to 256MB and the test program ran > well, but not without me noticing that the VM RAM consumption was > increased about 90 MB by netmap. > > Resuming, I want to know if there is a way to reduce the memory buffer > allocation, without recompiling the netmap kernel module. > > Thank you for the attention. > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 15:38:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C0417A1 for ; Tue, 30 Dec 2014 15:38:11 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF1FD1E29 for ; Tue, 30 Dec 2014 15:38:10 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id p9so12152649lbv.7 for ; Tue, 30 Dec 2014 07:38:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=btCm35mV453xf7KW8R/wv6VT7eK5eK/Q3+G0zaNdLMY=; b=ORqefHXZ7OMD+T3ZH/GY9PncdT7HamVqxIS2hkLBl/BX69N5pM9zzLLA+YJNhWo9DH +RhWI/CLtjrK9MfCf+niOGV5Bq1serfPDaI+izTVJknnf7Ar7MlGiNIYPREsTtQ287V8 iNx13XCWJl/9L25ygZNm2nmfg9n52x+eThjkQ7IDmXe+1hu53qePtRWH9H3mcpFestB3 CIKLk2+80jZvX+aUP+XDoZMkdc7ubcBEnkhFtSNYd8nZyn2gupTGQaSFirgj0AbnVk0V j++PZ7LQMe5ReKbZkl4mS3hKdHKV15P+eD6xG/x2Zt35wNDatlrJgXKTLWfHWvUU844A p2SA== MIME-Version: 1.0 X-Received: by 10.112.125.41 with SMTP id mn9mr48186379lbb.80.1419953888721; Tue, 30 Dec 2014 07:38:08 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.174.169 with HTTP; Tue, 30 Dec 2014 07:38:08 -0800 (PST) In-Reply-To: References: Date: Tue, 30 Dec 2014 16:38:08 +0100 X-Google-Sender-Auth: yvXCC0r1yzLxkHxrZXlpFVFTKlU Message-ID: Subject: Re: Regarding Netmap internal memory allocation. From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 15:38:11 -0000 you can reduce the amount of ram (buffers, mostly) by tweaking the values in netmap_mem2.c :: struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { ... } or you can simply modify the constant netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 to something smaller that suits an openwrt box (in which i am very interested, as I'd like to deploy one of these soon) cheers luigi On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira wrote: > Update: > > I noticed that the netmap module was still crashing, after changing the > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. The > netmap module is now consuming about 350MB of Ram, which for my objectives > is just too much... > > On 30 December 2014 at 14:06, Carlos Ferreira wrote: > >> To Luigi and to whom may be able to help >> >> Hello all. >> >> Is it possible to reduce the size of the memory buffer allocated by the >> netmap module? >> I'm asking this because I was implementing some testing code, using NICs >> and a Tap device in an OpenWRT VM with 64MB of RAM. >> Because of the small RAM amount, the nm_open crashed when the program >> tried to netmap the tap device, after I previously netmapped one NIC >> successfully. >> After the crash, I bumped the VM RAM to 256MB and the test program ran >> well, but not without me noticing that the VM RAM consumption was >> increased about 90 MB by netmap. >> >> Resuming, I want to know if there is a way to reduce the memory buffer >> allocation, without recompiling the netmap kernel module. >> >> Thank you for the attention. >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 15:44:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39FB1930 for ; Tue, 30 Dec 2014 15:44:24 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A4941F43 for ; Tue, 30 Dec 2014 15:44:23 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id z11so12201732lbi.24 for ; Tue, 30 Dec 2014 07:44:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=C999WHNgJxxPx+/EKVW9uF7N3OSEXDJv/8h8AFLJcHw=; b=RlxKqee4/dMSJmzgxxtjiuGKIOPThldTspDFqR4C6jaf8/P1W8gawiIONP8YNg0JMK Ge+Yy8W8KgkXyWbODX6jPnauQ/EoR7opgpdhqwYKFSInPz+5Ia3PttENE1cDVsLFgQDJ OvlbK6JLLHDdb34t5qAhaElCEroVgypM9fdmSZUSj3C6/HX1IaPgfB4ZXsWGtAXEi/y5 yWLAKi/45vis5wb9AClXaOoC9LiSgHrlEIjngPXKFM1n0OyoiVrJ2DmRmZqIcSckDC5U f8/YKN3CAdp/BuS/b4Cq0RSEEFxsJd9GKv1OI+alOo3zwa85BmAzn9q/ke674DS4n96s PABA== X-Received: by 10.152.23.38 with SMTP id j6mr62724667laf.81.1419954261601; Tue, 30 Dec 2014 07:44:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 07:43:41 -0800 (PST) In-Reply-To: References: From: Carlos Ferreira Date: Tue, 30 Dec 2014 15:43:41 +0000 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 15:44:24 -0000 Ok thanks. I was hoping not having to recompile the module, but it's ok. Thank you for the info! On 30 December 2014 at 15:38, Luigi Rizzo wrote: > you can reduce the amount of ram (buffers, mostly) by > tweaking the values in netmap_mem2.c :: > struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { > ... > } > > or you can simply modify the constant > > netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 > > to something smaller that suits an openwrt box > (in which i am very interested, as I'd like to deploy one of these soon) > > cheers > luigi > > > On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira > wrote: > > Update: > > > > I noticed that the netmap module was still crashing, after changing the > > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. > The > > netmap module is now consuming about 350MB of Ram, which for my > objectives > > is just too much... > > > > On 30 December 2014 at 14:06, Carlos Ferreira > wrote: > > > >> To Luigi and to whom may be able to help > >> > >> Hello all. > >> > >> Is it possible to reduce the size of the memory buffer allocated by the > >> netmap module? > >> I'm asking this because I was implementing some testing code, using NICs > >> and a Tap device in an OpenWRT VM with 64MB of RAM. > >> Because of the small RAM amount, the nm_open crashed when the program > >> tried to netmap the tap device, after I previously netmapped one NIC > >> successfully. > >> After the crash, I bumped the VM RAM to 256MB and the test program ran > >> well, but not without me noticing that the VM RAM consumption was > >> increased about 90 MB by netmap. > >> > >> Resuming, I want to know if there is a way to reduce the memory buffer > >> allocation, without recompiling the netmap kernel module. > >> > >> Thank you for the attention. > >> > >> -- > >> > >> Carlos Miguel Ferreira > >> Researcher at Telecommunications Institute > >> Aveiro - Portugal > >> Work E-mail - cmf@av.it.pt > >> Skype & GTalk -> carlosmf.pt@gmail.com > >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> > > > > > > > > -- > > > > Carlos Miguel Ferreira > > Researcher at Telecommunications Institute > > Aveiro - Portugal > > Work E-mail - cmf@av.it.pt > > Skype & GTalk -> carlosmf.pt@gmail.com > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 15:45:31 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C0059CD for ; Tue, 30 Dec 2014 15:45:31 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 733AC1F68 for ; Tue, 30 Dec 2014 15:45:31 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id sBUFjVHM090536 for ; Tue, 30 Dec 2014 15:45:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194672] [carp] Changing advskew to 0 from another value doesn't work Date: Tue, 30 Dec 2014 15:45:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RC2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: garga@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 15:45:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194672 --- Comment #3 from Renato Botelho --- Created attachment 151123 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=151123&action=edit 2nd attempt, fix also advbase check There is one more issue, advbase should be >= 1 and <= 255, it makes proper check and also fix default advbase value on ifconfig to 1 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 15:48:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94633A73 for ; Tue, 30 Dec 2014 15:48:04 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F05BD1FBD for ; Tue, 30 Dec 2014 15:48:03 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id hz20so12942459lab.34 for ; Tue, 30 Dec 2014 07:48:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=sPYK6iWEo+A8aH4TxziI+zZC/sswVbvpoAU8WwfsJek=; b=tXlOupGVka3a5Z7gf3VlxiAiLSW4iIxUHZtk5xqctrEolyK6IN7yG78vQ76OLg5rIa 6gYxCF5Fh2+oOh1Omy+M93cj6B7rlWy0DkfWVMpebJT6Ad5GTG+ZCN19DfXQHHvlpTxP uXwddknN/zeBNoN2RqQJ7ZDxcaH4vYTlvBK12w7qF3dpHcPFRZNP9cF0i8L70EOtLcuj 71UtimxfpdSj/AGYfDIRsuPNHNjdRqYknr8iPfXR8HJ87RcXpH7rBiFUR7rTYbh0PB/X OmQYOC2hUX23U2F4gyniIhBhgWaCoaP4nz0oLkrGO5m+/TciofORt5PLoY6DEhDvGnvr MmpQ== X-Received: by 10.152.23.38 with SMTP id j6mr62740235laf.81.1419954481941; Tue, 30 Dec 2014 07:48:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 07:47:21 -0800 (PST) In-Reply-To: References: From: Carlos Ferreira Date: Tue, 30 Dec 2014 15:47:21 +0000 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 15:48:04 -0000 You mean netmap_mem2.c ? It was there where I found the NETMAP_BUF_MAX_NUM define. On 30 December 2014 at 15:43, Carlos Ferreira wrote: > Ok thanks. I was hoping not having to recompile the module, but it's ok. > Thank you for the info! > > > On 30 December 2014 at 15:38, Luigi Rizzo wrote: > >> you can reduce the amount of ram (buffers, mostly) by >> tweaking the values in netmap_mem2.c :: >> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { >> ... >> } >> >> or you can simply modify the constant >> >> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 >> >> to something smaller that suits an openwrt box >> (in which i am very interested, as I'd like to deploy one of these soon) >> >> cheers >> luigi >> >> >> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira >> wrote: >> > Update: >> > >> > I noticed that the netmap module was still crashing, after changing the >> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. >> The >> > netmap module is now consuming about 350MB of Ram, which for my >> objectives >> > is just too much... >> > >> > On 30 December 2014 at 14:06, Carlos Ferreira >> wrote: >> > >> >> To Luigi and to whom may be able to help >> >> >> >> Hello all. >> >> >> >> Is it possible to reduce the size of the memory buffer allocated by the >> >> netmap module? >> >> I'm asking this because I was implementing some testing code, using >> NICs >> >> and a Tap device in an OpenWRT VM with 64MB of RAM. >> >> Because of the small RAM amount, the nm_open crashed when the program >> >> tried to netmap the tap device, after I previously netmapped one NIC >> >> successfully. >> >> After the crash, I bumped the VM RAM to 256MB and the test program ran >> >> well, but not without me noticing that the VM RAM consumption was >> >> increased about 90 MB by netmap. >> >> >> >> Resuming, I want to know if there is a way to reduce the memory buffer >> >> allocation, without recompiling the netmap kernel module. >> >> >> >> Thank you for the attention. >> >> >> >> -- >> >> >> >> Carlos Miguel Ferreira >> >> Researcher at Telecommunications Institute >> >> Aveiro - Portugal >> >> Work E-mail - cmf@av.it.pt >> >> Skype & GTalk -> carlosmf.pt@gmail.com >> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> > >> > >> > >> > -- >> > >> > Carlos Miguel Ferreira >> > Researcher at Telecommunications Institute >> > Aveiro - Portugal >> > Work E-mail - cmf@av.it.pt >> > Skype & GTalk -> carlosmf.pt@gmail.com >> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 16:08:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0CA910F for ; Tue, 30 Dec 2014 16:08:52 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 34F0A24FA for ; Tue, 30 Dec 2014 16:08:52 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gf13so12472040lab.7 for ; Tue, 30 Dec 2014 08:08:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=d2RBl4s1zzczb+X7Ozyf1CHjTyigFrj6c0+xkoX/wzE=; b=gDznh4tM4Fw6eTyHJXDhUmITd7rBjsOvLbxeh2vcieQ7Pk2dqKpYhXKHHJFW+ZGfDh afOx0JqLPg1H7+O7A3GQU3APl5BZ0Ah4lbgTBWv1Y4vOpJL4WVqeRzYqsgOse2I8pwQ7 PR61dmhEqNTkMasoRxbWMK3vuUmo8br2uOv2HfmLnMvrktM0qyvfSf2hJaZXatAkQSAw E7+8d/3DUed13Wd8atiO5fgeMyMOKTs8XPKwRDRUlwKnmVtvWj9WD/IxCDcZEnQ6bCM+ Nt0xV/T5fmjUGlUW8zsz2LOsrA5gqGhKQbkyI4UDS+L60rQeq2893f2hs46/8SL3KAUB pf/A== X-Received: by 10.112.54.167 with SMTP id k7mr12935518lbp.72.1419955729943; Tue, 30 Dec 2014 08:08:49 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 08:08:09 -0800 (PST) In-Reply-To: References: From: Carlos Ferreira Date: Tue, 30 Dec 2014 16:08:09 +0000 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 16:08:52 -0000 By the way, another question. Is there a way to not compile the code regarding the VALE switch? I'm only interested in using netmap with Tap Devices and NICs, so I was hoping to save some memory. On 30 December 2014 at 15:47, Carlos Ferreira wrote: > You mean netmap_mem2.c ? It was there where I found the NETMAP_BUF_MAX_NUM > define. > > > > On 30 December 2014 at 15:43, Carlos Ferreira > wrote: > >> Ok thanks. I was hoping not having to recompile the module, but it's ok. >> Thank you for the info! >> >> >> On 30 December 2014 at 15:38, Luigi Rizzo wrote: >> >>> you can reduce the amount of ram (buffers, mostly) by >>> tweaking the values in netmap_mem2.c :: >>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { >>> ... >>> } >>> >>> or you can simply modify the constant >>> >>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 >>> >>> to something smaller that suits an openwrt box >>> (in which i am very interested, as I'd like to deploy one of these soon) >>> >>> cheers >>> luigi >>> >>> >>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira >>> wrote: >>> > Update: >>> > >>> > I noticed that the netmap module was still crashing, after changing >>> the >>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. >>> The >>> > netmap module is now consuming about 350MB of Ram, which for my >>> objectives >>> > is just too much... >>> > >>> > On 30 December 2014 at 14:06, Carlos Ferreira >>> wrote: >>> > >>> >> To Luigi and to whom may be able to help >>> >> >>> >> Hello all. >>> >> >>> >> Is it possible to reduce the size of the memory buffer allocated by >>> the >>> >> netmap module? >>> >> I'm asking this because I was implementing some testing code, using >>> NICs >>> >> and a Tap device in an OpenWRT VM with 64MB of RAM. >>> >> Because of the small RAM amount, the nm_open crashed when the program >>> >> tried to netmap the tap device, after I previously netmapped one NIC >>> >> successfully. >>> >> After the crash, I bumped the VM RAM to 256MB and the test program ran >>> >> well, but not without me noticing that the VM RAM consumption was >>> >> increased about 90 MB by netmap. >>> >> >>> >> Resuming, I want to know if there is a way to reduce the memory buffer >>> >> allocation, without recompiling the netmap kernel module. >>> >> >>> >> Thank you for the attention. >>> >> >>> >> -- >>> >> >>> >> Carlos Miguel Ferreira >>> >> Researcher at Telecommunications Institute >>> >> Aveiro - Portugal >>> >> Work E-mail - cmf@av.it.pt >>> >> Skype & GTalk -> carlosmf.pt@gmail.com >>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>> >> >>> > >>> > >>> > >>> > -- >>> > >>> > Carlos Miguel Ferreira >>> > Researcher at Telecommunications Institute >>> > Aveiro - Portugal >>> > Work E-mail - cmf@av.it.pt >>> > Skype & GTalk -> carlosmf.pt@gmail.com >>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>> > _______________________________________________ >>> > freebsd-net@freebsd.org mailing list >>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> >>> -- >>> -----------------------------------------+------------------------------- >>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>> TEL +39-050-2211611 . via Diotisalvi 2 >>> Mobile +39-338-6809875 . 56122 PISA (Italy) >>> -----------------------------------------+------------------------------- >>> >> >> >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 16:10:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81E0C1AD for ; Tue, 30 Dec 2014 16:10:52 +0000 (UTC) Received: from mail-lb0-x22f.google.com (mail-lb0-x22f.google.com [IPv6:2a00:1450:4010:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 002762513 for ; Tue, 30 Dec 2014 16:10:51 +0000 (UTC) Received: by mail-lb0-f175.google.com with SMTP id z11so4457997lbi.6 for ; Tue, 30 Dec 2014 08:10:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=AFOUprc/AT/sUUNnqR1hs42arSFiU93X4XqWf7U2Tow=; b=cu9aLCcmajE0zhaGJJUb9+dmD8UEVc+t2tZodQbFhvy5daFjUDzcBq7pyxYE4zBkJ3 bo55JTMi4gS95FUDtELyFulzci3krfXHRzYd+BDNXSL20jwlu3XHRKW0WYixV31zHYL3 P1vsha1uHVhDUNWu3sETEP9vr6C70Uu5Em6+Uotco8l9lOZ25n2kKl9IbI00JrHVHpzv dvR7fXweFMkTK7igaj7bCqLAAH4VsF9ZStoaZuxUCFkVLA6237+WmzEE5lEdE7t5Mwtk GHf05s7W5xt6d1QLEgFMaxcbocebHvN4t0YYiPH7hLrS40oNMX3DEs5Mm/SzQ322gmSl IGdg== MIME-Version: 1.0 X-Received: by 10.112.219.98 with SMTP id pn2mr59219956lbc.74.1419955849968; Tue, 30 Dec 2014 08:10:49 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.174.169 with HTTP; Tue, 30 Dec 2014 08:10:49 -0800 (PST) In-Reply-To: References: Date: Tue, 30 Dec 2014 17:10:49 +0100 X-Google-Sender-Auth: uwYdHyX4oqsSKyjFmdpl_IvIA98 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 16:10:52 -0000 you don't need to recompile, on linux those values are accessible under /sys/modules/netmal_lin/parameters (on FreeBSD they are sysctl variables) cheers luigi On Tue, Dec 30, 2014 at 4:43 PM, Carlos Ferreira wrote: > Ok thanks. I was hoping not having to recompile the module, but it's ok. > Thank you for the info! > > > On 30 December 2014 at 15:38, Luigi Rizzo wrote: >> >> you can reduce the amount of ram (buffers, mostly) by >> tweaking the values in netmap_mem2.c :: >> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { >> ... >> } >> >> or you can simply modify the constant >> >> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 >> >> to something smaller that suits an openwrt box >> (in which i am very interested, as I'd like to deploy one of these soon) >> >> cheers >> luigi >> >> >> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira >> wrote: >> > Update: >> > >> > I noticed that the netmap module was still crashing, after changing the >> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. >> > The >> > netmap module is now consuming about 350MB of Ram, which for my >> > objectives >> > is just too much... >> > >> > On 30 December 2014 at 14:06, Carlos Ferreira >> > wrote: >> > >> >> To Luigi and to whom may be able to help >> >> >> >> Hello all. >> >> >> >> Is it possible to reduce the size of the memory buffer allocated by the >> >> netmap module? >> >> I'm asking this because I was implementing some testing code, using >> >> NICs >> >> and a Tap device in an OpenWRT VM with 64MB of RAM. >> >> Because of the small RAM amount, the nm_open crashed when the program >> >> tried to netmap the tap device, after I previously netmapped one NIC >> >> successfully. >> >> After the crash, I bumped the VM RAM to 256MB and the test program ran >> >> well, but not without me noticing that the VM RAM consumption was >> >> increased about 90 MB by netmap. >> >> >> >> Resuming, I want to know if there is a way to reduce the memory buffer >> >> allocation, without recompiling the netmap kernel module. >> >> >> >> Thank you for the attention. >> >> >> >> -- >> >> >> >> Carlos Miguel Ferreira >> >> Researcher at Telecommunications Institute >> >> Aveiro - Portugal >> >> Work E-mail - cmf@av.it.pt >> >> Skype & GTalk -> carlosmf.pt@gmail.com >> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> > >> > >> > >> > -- >> > >> > Carlos Miguel Ferreira >> > Researcher at Telecommunications Institute >> > Aveiro - Portugal >> > Work E-mail - cmf@av.it.pt >> > Skype & GTalk -> carlosmf.pt@gmail.com >> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- > > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 16:17:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB06D28A for ; Tue, 30 Dec 2014 16:17:00 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A5802642 for ; Tue, 30 Dec 2014 16:17:00 +0000 (UTC) Received: by mail-la0-f46.google.com with SMTP id q1so12650528lam.5 for ; Tue, 30 Dec 2014 08:16:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=qgpM+gymE6e4Fgc/SUVObnmKkEpQ9IYE1gGdsTR5mvA=; b=Bm1DMbOk6ZEoTjmEVSf2iZOoOIRdFM7obD2n/rgGYFwB+ZFzFAv4IiDYrH9gVgEYMh EwejRb9AWB+9cYS9j959CsFZpVir9FYByzHFsazHbSDVLjms64AmYT+YGjYfo42DifMi /XzaEP69xE6wS+iEWsQpJHHRpYfG+Pi5hrF3+pIx6+tyATXYVFBW7th5bUHmawop6esy F+fEzotE2J+UfWyiQhePxynsgbJqsfFf5Gg6lD896Ht8JILden3T4nhs7czQ5gDRGRIG SYdbDClSEB7W6J4B5PRxb5wzXyw64UGo3W85FhDDFnm8msKQdUYgHKIiV9e8KOv8nUXM OKbQ== MIME-Version: 1.0 X-Received: by 10.152.22.199 with SMTP id g7mr62880682laf.23.1419956218416; Tue, 30 Dec 2014 08:16:58 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.174.169 with HTTP; Tue, 30 Dec 2014 08:16:58 -0800 (PST) In-Reply-To: References: Date: Tue, 30 Dec 2014 17:16:58 +0100 X-Google-Sender-Auth: qNC6X9kZnOIHnhNtOJTdEQzEYss Message-ID: Subject: Re: Regarding Netmap internal memory allocation. From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 16:17:01 -0000 you can #undefine WITH_VALE. But it is only 20K of code (and 150K of data structures, which you can further reduce by lowering NM_BRIDGS). The saving is probably not worth the effort. cheers luigi On Tue, Dec 30, 2014 at 5:08 PM, Carlos Ferreira wrote: > By the way, another question. > Is there a way to not compile the code regarding the VALE switch? I'm only > interested in using netmap with Tap Devices and NICs, so I was hoping to > save some memory. > > On 30 December 2014 at 15:47, Carlos Ferreira wrote: >> >> You mean netmap_mem2.c ? It was there where I found the NETMAP_BUF_MAX_NUM >> define. >> >> >> >> On 30 December 2014 at 15:43, Carlos Ferreira >> wrote: >>> >>> Ok thanks. I was hoping not having to recompile the module, but it's ok. >>> Thank you for the info! >>> >>> >>> On 30 December 2014 at 15:38, Luigi Rizzo wrote: >>>> >>>> you can reduce the amount of ram (buffers, mostly) by >>>> tweaking the values in netmap_mem2.c :: >>>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { >>>> ... >>>> } >>>> >>>> or you can simply modify the constant >>>> >>>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 >>>> >>>> to something smaller that suits an openwrt box >>>> (in which i am very interested, as I'd like to deploy one of these soon) >>>> >>>> cheers >>>> luigi >>>> >>>> >>>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira >>>> wrote: >>>> > Update: >>>> > >>>> > I noticed that the netmap module was still crashing, after changing >>>> > the >>>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer crashed. >>>> > The >>>> > netmap module is now consuming about 350MB of Ram, which for my >>>> > objectives >>>> > is just too much... >>>> > >>>> > On 30 December 2014 at 14:06, Carlos Ferreira >>>> > wrote: >>>> > >>>> >> To Luigi and to whom may be able to help >>>> >> >>>> >> Hello all. >>>> >> >>>> >> Is it possible to reduce the size of the memory buffer allocated by >>>> >> the >>>> >> netmap module? >>>> >> I'm asking this because I was implementing some testing code, using >>>> >> NICs >>>> >> and a Tap device in an OpenWRT VM with 64MB of RAM. >>>> >> Because of the small RAM amount, the nm_open crashed when the program >>>> >> tried to netmap the tap device, after I previously netmapped one NIC >>>> >> successfully. >>>> >> After the crash, I bumped the VM RAM to 256MB and the test program >>>> >> ran >>>> >> well, but not without me noticing that the VM RAM consumption was >>>> >> increased about 90 MB by netmap. >>>> >> >>>> >> Resuming, I want to know if there is a way to reduce the memory >>>> >> buffer >>>> >> allocation, without recompiling the netmap kernel module. >>>> >> >>>> >> Thank you for the attention. >>>> >> >>>> >> -- >>>> >> >>>> >> Carlos Miguel Ferreira >>>> >> Researcher at Telecommunications Institute >>>> >> Aveiro - Portugal >>>> >> Work E-mail - cmf@av.it.pt >>>> >> Skype & GTalk -> carlosmf.pt@gmail.com >>>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>>> >> >>>> > >>>> > >>>> > >>>> > -- >>>> > >>>> > Carlos Miguel Ferreira >>>> > Researcher at Telecommunications Institute >>>> > Aveiro - Portugal >>>> > Work E-mail - cmf@av.it.pt >>>> > Skype & GTalk -> carlosmf.pt@gmail.com >>>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>>> > _______________________________________________ >>>> > freebsd-net@freebsd.org mailing list >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>>> >>>> >>>> -- >>>> >>>> -----------------------------------------+------------------------------- >>>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>>> TEL +39-050-2211611 . via Diotisalvi 2 >>>> Mobile +39-338-6809875 . 56122 PISA (Italy) >>>> >>>> -----------------------------------------+------------------------------- >>> >>> >>> >>> >>> -- >>> >>> Carlos Miguel Ferreira >>> Researcher at Telecommunications Institute >>> Aveiro - Portugal >>> Work E-mail - cmf@av.it.pt >>> Skype & GTalk -> carlosmf.pt@gmail.com >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 17:39:09 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2CD5EB3 for ; Tue, 30 Dec 2014 17:39:09 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A9E2B272 for ; Tue, 30 Dec 2014 17:39:09 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id sBUHd9rk041759 for ; Tue, 30 Dec 2014 17:39:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194672] [carp] Changing advskew to 0 from another value doesn't work Date: Tue, 30 Dec 2014 17:39:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RC2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: garga@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 17:39:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194672 --- Comment #4 from Renato Botelho --- Please ignore second patch, it has issues, will send a new one shortly -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 17:39:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF454F3D for ; Tue, 30 Dec 2014 17:39:29 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11F1F280 for ; Tue, 30 Dec 2014 17:39:29 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id z11so12330973lbi.24 for ; Tue, 30 Dec 2014 09:39:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=k8c9Zo1yufgUxuejYl1USLhauCS0YjreTi6JN/uIJH4=; b=f+edGCDMJokVY9t+p9chQOIlSENQOSvQhabHNtSOV70oBcWtydI6Eka97oq+v1j4G3 MWNfOEARawun8x7roAC1U3Y32lRFY4fSHRvUO1vYbs1uMfN/8ZteLH66EuDY9JNq8Dln E0ujaPqGcsAjGubTO/U8npHzWahOqVxTVqiLTbSeJ4z6wnzUwUgQWKL8xsOvBL4gwWFk MRqVFV/8WqgQ8pQdzt24+8uZFPeupJ8nnGY3UXPdWpGA36rkTaTQAmpWV55kcX3HO2WF g2Tl5MEo31nCk85LJl2kxz/OSDNvxHpwE9+YbAINI8PzS8rczASEc5V1qrglJdl3G6MB jAWg== X-Received: by 10.152.37.7 with SMTP id u7mr63554300laj.74.1419961167128; Tue, 30 Dec 2014 09:39:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 09:38:46 -0800 (PST) In-Reply-To: References: From: Carlos Ferreira Date: Tue, 30 Dec 2014 17:38:46 +0000 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 17:39:29 -0000 Ok, I'm having some trouble in tuning the amount of memory for netmap. I have been following the man page from FreeBSD in other to understand the values at /sys/modules/netmap/parameters for linux but I'm having some trouble in understanding what each value actually means. For the following values: dev.netmap.ring_num: 200 -> Is this the number of rings in the Ring Buffer Pool? dev.netmap.ring_size: 36864 -> Is this value, the number of slots per ring? I'm trying to keep the amount of memory used by netmap as low as 4MB - 8MB since I'm going to use only up to 4 NICs and one TAP. Thanks for the help! On 30 December 2014 at 16:16, Luigi Rizzo wrote: > you can #undefine WITH_VALE. > But it is only 20K of code (and 150K of data structures, which you > can further reduce by lowering NM_BRIDGS). > The saving is probably not worth the effort. > > cheers > luigi > > On Tue, Dec 30, 2014 at 5:08 PM, Carlos Ferreira > wrote: > > By the way, another question. > > Is there a way to not compile the code regarding the VALE switch? I'm > only > > interested in using netmap with Tap Devices and NICs, so I was hoping to > > save some memory. > > > > On 30 December 2014 at 15:47, Carlos Ferreira > wrote: > >> > >> You mean netmap_mem2.c ? It was there where I found the > NETMAP_BUF_MAX_NUM > >> define. > >> > >> > >> > >> On 30 December 2014 at 15:43, Carlos Ferreira > >> wrote: > >>> > >>> Ok thanks. I was hoping not having to recompile the module, but it's > ok. > >>> Thank you for the info! > >>> > >>> > >>> On 30 December 2014 at 15:38, Luigi Rizzo wrote: > >>>> > >>>> you can reduce the amount of ram (buffers, mostly) by > >>>> tweaking the values in netmap_mem2.c :: > >>>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { > >>>> ... > >>>> } > >>>> > >>>> or you can simply modify the constant > >>>> > >>>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 > >>>> > >>>> to something smaller that suits an openwrt box > >>>> (in which i am very interested, as I'd like to deploy one of these > soon) > >>>> > >>>> cheers > >>>> luigi > >>>> > >>>> > >>>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira < > carlosmf.pt@gmail.com> > >>>> wrote: > >>>> > Update: > >>>> > > >>>> > I noticed that the netmap module was still crashing, after changing > >>>> > the > >>>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer > crashed. > >>>> > The > >>>> > netmap module is now consuming about 350MB of Ram, which for my > >>>> > objectives > >>>> > is just too much... > >>>> > > >>>> > On 30 December 2014 at 14:06, Carlos Ferreira < > carlosmf.pt@gmail.com> > >>>> > wrote: > >>>> > > >>>> >> To Luigi and to whom may be able to help > >>>> >> > >>>> >> Hello all. > >>>> >> > >>>> >> Is it possible to reduce the size of the memory buffer allocated by > >>>> >> the > >>>> >> netmap module? > >>>> >> I'm asking this because I was implementing some testing code, using > >>>> >> NICs > >>>> >> and a Tap device in an OpenWRT VM with 64MB of RAM. > >>>> >> Because of the small RAM amount, the nm_open crashed when the > program > >>>> >> tried to netmap the tap device, after I previously netmapped one > NIC > >>>> >> successfully. > >>>> >> After the crash, I bumped the VM RAM to 256MB and the test program > >>>> >> ran > >>>> >> well, but not without me noticing that the VM RAM consumption was > >>>> >> increased about 90 MB by netmap. > >>>> >> > >>>> >> Resuming, I want to know if there is a way to reduce the memory > >>>> >> buffer > >>>> >> allocation, without recompiling the netmap kernel module. > >>>> >> > >>>> >> Thank you for the attention. > >>>> >> > >>>> >> -- > >>>> >> > >>>> >> Carlos Miguel Ferreira > >>>> >> Researcher at Telecommunications Institute > >>>> >> Aveiro - Portugal > >>>> >> Work E-mail - cmf@av.it.pt > >>>> >> Skype & GTalk -> carlosmf.pt@gmail.com > >>>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >>>> >> > >>>> > > >>>> > > >>>> > > >>>> > -- > >>>> > > >>>> > Carlos Miguel Ferreira > >>>> > Researcher at Telecommunications Institute > >>>> > Aveiro - Portugal > >>>> > Work E-mail - cmf@av.it.pt > >>>> > Skype & GTalk -> carlosmf.pt@gmail.com > >>>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >>>> > _______________________________________________ > >>>> > freebsd-net@freebsd.org mailing list > >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>> > To unsubscribe, send any mail to " > freebsd-net-unsubscribe@freebsd.org" > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> > -----------------------------------------+------------------------------- > >>>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > dell'Informazione > >>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >>>> TEL +39-050-2211611 . via Diotisalvi 2 > >>>> Mobile +39-338-6809875 . 56122 PISA (Italy) > >>>> > >>>> > -----------------------------------------+------------------------------- > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> Carlos Miguel Ferreira > >>> Researcher at Telecommunications Institute > >>> Aveiro - Portugal > >>> Work E-mail - cmf@av.it.pt > >>> Skype & GTalk -> carlosmf.pt@gmail.com > >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> > >> > >> > >> > >> -- > >> > >> Carlos Miguel Ferreira > >> Researcher at Telecommunications Institute > >> Aveiro - Portugal > >> Work E-mail - cmf@av.it.pt > >> Skype & GTalk -> carlosmf.pt@gmail.com > >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > > > > > > > > > -- > > > > Carlos Miguel Ferreira > > Researcher at Telecommunications Institute > > Aveiro - Portugal > > Work E-mail - cmf@av.it.pt > > Skype & GTalk -> carlosmf.pt@gmail.com > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 18:02:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AF6D292 for ; Tue, 30 Dec 2014 18:02:51 +0000 (UTC) Received: from email.aon.at (nat-warsl417-01.aon.at [195.3.96.119]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BBF8F837 for ; Tue, 30 Dec 2014 18:02:49 +0000 (UTC) Received: (qmail 4728 invoked from network); 30 Dec 2014 18:02:41 -0000 Received: from unknown (HELO email.aon.at) ([172.18.1.205]) (envelope-sender ) by fallback43.highway.telekom.at (qmail-ldap-1.03) with SMTP for ; 30 Dec 2014 18:02:41 -0000 Received: (qmail 3760 invoked from network); 30 Dec 2014 18:02:32 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on WARSBL604.highway.telekom.at X-Spam-Level: Received: from 194-166-23-182.adsl.highway.telekom.at (HELO gandalf.xyzzy) ([194.166.23.182]) (envelope-sender ) by smarthub77.res.a1.net (qmail-ldap-1.03) with AES256-SHA encrypted SMTP for ; 30 Dec 2014 18:02:32 -0000 X-A1Mail-Track-Id: 1419962552:3734:smarthub77:194.166.23.182:1 Received: from mizar.xyzzy (mizar.xyzzy [192.168.1.19]) by gandalf.xyzzy (8.14.9/8.14.9) with ESMTP id sBUI2U0K003049 for ; Tue, 30 Dec 2014 19:02:32 +0100 (CET) (envelope-from la5lbtyi@aon.at) Message-ID: <54A2E8B6.3070801@aon.at> Date: Tue, 30 Dec 2014 19:02:30 +0100 From: Martin Birgmeier Organization: MBi at home User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Tying down network interfaces Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 18:02:51 -0000 Hi, I have two network interfaces as follows: sis0: port 0xa400-0xa4ff mem 0xd5800000-0xd5800fff irq 9 at device 9.0 on pci0 sis1: port 0x9400-0x94ff mem 0xd4800000-0xd4800fff irq 11 at device 12.0 on pci0 When sis0 breaks down, sis1 gets renumbered as sis0, wreaking havoc (mostly on my brains until I figure out which card is actually affected). How do I tie down these two interfaces so that they always stay as sis0 and sis1, respectively, regardless of which ones are present in the system? - I expect to insert something into /boot/device.hints. -- Martin From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 18:40:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93FDBD45 for ; Tue, 30 Dec 2014 18:40:37 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02E532D13 for ; Tue, 30 Dec 2014 18:40:37 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id w7so12574303lbi.2 for ; Tue, 30 Dec 2014 10:40:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gNf+iTaLWAaNsoe07TmUNlbU4B+3ceY8FmvG4vvpd7o=; b=H/NfgXQH9cw2g4fJ55hmfgWL+hcHyVoFyqxnlV+R81HObJy0ziE5OEyvGQL+j0Nlej PUFLtjs4wgkq5MNm1ikpqrqDehqIJWSFn13C0ieqCrJxjUVwGutuHyr98VCKGAVuEe3u U3O0JhCey8HAdQdTk2zBsaxMgP/r7egtw0lrgENipjtaQOyAgWZ6BPrj0YuY60K6VrxU YRcBbYPi7v5s8VhR7jHxELlwbiC8BMDDEQOlfp6TM7E7r7caReZZnSDX+PacmTKmQ6mK ZjdpJloX0sex+09CschQmNihkJeZXGJqFyBlhS538vOKlQauSFVLvbyhio9nvvipsxpv 273w== MIME-Version: 1.0 X-Received: by 10.112.89.232 with SMTP id br8mr24107864lbb.69.1419964834945; Tue, 30 Dec 2014 10:40:34 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.174.169 with HTTP; Tue, 30 Dec 2014 10:40:34 -0800 (PST) In-Reply-To: References: Date: Tue, 30 Dec 2014 19:40:34 +0100 X-Google-Sender-Auth: bx3Ny4GE4NXwmsOvkh13riWiMbk Message-ID: Subject: Re: Regarding Netmap internal memory allocation. From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 18:40:37 -0000 On Tue, Dec 30, 2014 at 6:38 PM, Carlos Ferreira wrote: > Ok, I'm having some trouble in tuning the amount of memory for netmap. > > I have been following the man page from FreeBSD in other to understand the > values at /sys/modules/netmap/parameters for linux but I'm having some > trouble in understanding what each value actually means. > For the following values: > > dev.netmap.ring_num: 200 -> Is this the number of rings in the Ring Buffer > Pool? yes. For interfaces with a single queue you need 4 rings (rx+tx for the nic and another rx+tx for the host port) > dev.netmap.ring_size: 36864 -> Is this value, the number of slots per ring? this is the size in bytes of each ring. The number of slots is set by the hardware (low end devices as in the openwrt devices will probably use 256 or 512 slots, so 10-12k should suffice. But this is not worth changing. Instead, you should reduce the number of buffers, though 8MB is only 4000 buffers and it is a bit on the low side for 5 ports. However, as far as I know most openwrt devices only have one physical NIC, and a switch implementing various vlans. cheers luigi > > I'm trying to keep the amount of memory used by netmap as low as 4MB - 8MB > since I'm going to use only up to 4 NICs and one TAP. > > Thanks for the help! > > > On 30 December 2014 at 16:16, Luigi Rizzo wrote: >> >> you can #undefine WITH_VALE. >> But it is only 20K of code (and 150K of data structures, which you >> can further reduce by lowering NM_BRIDGS). >> The saving is probably not worth the effort. >> >> cheers >> luigi >> >> On Tue, Dec 30, 2014 at 5:08 PM, Carlos Ferreira >> wrote: >> > By the way, another question. >> > Is there a way to not compile the code regarding the VALE switch? I'm >> > only >> > interested in using netmap with Tap Devices and NICs, so I was hoping to >> > save some memory. >> > >> > On 30 December 2014 at 15:47, Carlos Ferreira >> > wrote: >> >> >> >> You mean netmap_mem2.c ? It was there where I found the >> >> NETMAP_BUF_MAX_NUM >> >> define. >> >> >> >> >> >> >> >> On 30 December 2014 at 15:43, Carlos Ferreira >> >> wrote: >> >>> >> >>> Ok thanks. I was hoping not having to recompile the module, but it's >> >>> ok. >> >>> Thank you for the info! >> >>> >> >>> >> >>> On 30 December 2014 at 15:38, Luigi Rizzo wrote: >> >>>> >> >>>> you can reduce the amount of ram (buffers, mostly) by >> >>>> tweaking the values in netmap_mem2.c :: >> >>>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { >> >>>> ... >> >>>> } >> >>>> >> >>>> or you can simply modify the constant >> >>>> >> >>>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 >> >>>> >> >>>> to something smaller that suits an openwrt box >> >>>> (in which i am very interested, as I'd like to deploy one of these >> >>>> soon) >> >>>> >> >>>> cheers >> >>>> luigi >> >>>> >> >>>> >> >>>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira >> >>>> >> >>>> wrote: >> >>>> > Update: >> >>>> > >> >>>> > I noticed that the netmap module was still crashing, after >> >>>> > changing >> >>>> > the >> >>>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer >> >>>> > crashed. >> >>>> > The >> >>>> > netmap module is now consuming about 350MB of Ram, which for my >> >>>> > objectives >> >>>> > is just too much... >> >>>> > >> >>>> > On 30 December 2014 at 14:06, Carlos Ferreira >> >>>> > >> >>>> > wrote: >> >>>> > >> >>>> >> To Luigi and to whom may be able to help >> >>>> >> >> >>>> >> Hello all. >> >>>> >> >> >>>> >> Is it possible to reduce the size of the memory buffer allocated >> >>>> >> by >> >>>> >> the >> >>>> >> netmap module? >> >>>> >> I'm asking this because I was implementing some testing code, >> >>>> >> using >> >>>> >> NICs >> >>>> >> and a Tap device in an OpenWRT VM with 64MB of RAM. >> >>>> >> Because of the small RAM amount, the nm_open crashed when the >> >>>> >> program >> >>>> >> tried to netmap the tap device, after I previously netmapped one >> >>>> >> NIC >> >>>> >> successfully. >> >>>> >> After the crash, I bumped the VM RAM to 256MB and the test program >> >>>> >> ran >> >>>> >> well, but not without me noticing that the VM RAM consumption was >> >>>> >> increased about 90 MB by netmap. >> >>>> >> >> >>>> >> Resuming, I want to know if there is a way to reduce the memory >> >>>> >> buffer >> >>>> >> allocation, without recompiling the netmap kernel module. >> >>>> >> >> >>>> >> Thank you for the attention. >> >>>> >> >> >>>> >> -- >> >>>> >> >> >>>> >> Carlos Miguel Ferreira >> >>>> >> Researcher at Telecommunications Institute >> >>>> >> Aveiro - Portugal >> >>>> >> Work E-mail - cmf@av.it.pt >> >>>> >> Skype & GTalk -> carlosmf.pt@gmail.com >> >>>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >>>> >> >> >>>> > >> >>>> > >> >>>> > >> >>>> > -- >> >>>> > >> >>>> > Carlos Miguel Ferreira >> >>>> > Researcher at Telecommunications Institute >> >>>> > Aveiro - Portugal >> >>>> > Work E-mail - cmf@av.it.pt >> >>>> > Skype & GTalk -> carlosmf.pt@gmail.com >> >>>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >>>> > _______________________________________________ >> >>>> > freebsd-net@freebsd.org mailing list >> >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> >>>> > To unsubscribe, send any mail to >> >>>> > "freebsd-net-unsubscribe@freebsd.org" >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> >> >>>> >> >>>> -----------------------------------------+------------------------------- >> >>>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> >>>> dell'Informazione >> >>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> >>>> TEL +39-050-2211611 . via Diotisalvi 2 >> >>>> Mobile +39-338-6809875 . 56122 PISA (Italy) >> >>>> >> >>>> >> >>>> -----------------------------------------+------------------------------- >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> >> >>> Carlos Miguel Ferreira >> >>> Researcher at Telecommunications Institute >> >>> Aveiro - Portugal >> >>> Work E-mail - cmf@av.it.pt >> >>> Skype & GTalk -> carlosmf.pt@gmail.com >> >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Carlos Miguel Ferreira >> >> Researcher at Telecommunications Institute >> >> Aveiro - Portugal >> >> Work E-mail - cmf@av.it.pt >> >> Skype & GTalk -> carlosmf.pt@gmail.com >> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > >> > >> > >> > >> > -- >> > >> > Carlos Miguel Ferreira >> > Researcher at Telecommunications Institute >> > Aveiro - Portugal >> > Work E-mail - cmf@av.it.pt >> > Skype & GTalk -> carlosmf.pt@gmail.com >> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- > > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 18:55:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 906886F1 for ; Tue, 30 Dec 2014 18:55:50 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DFC7C2FFE for ; Tue, 30 Dec 2014 18:55:49 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id gm9so12839071lab.26 for ; Tue, 30 Dec 2014 10:55:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=ZhRs5RBh6ybyZ34Z19/YEbK5nHxu6hpShPxe9M+Hfx8=; b=nQ1sl3o0u7yhji7AelO4/T1xuMY+NkrYA1NBQxxCAJxDVPdiS/uMU6AXkIgmY4qsu8 fQh4wd2L9vx5LR417KTqeA3AFDOOoPgtszpLuUWBBosf2r4gFjz+Btt0GMv0IjcIy3SN fHdKUhikwszRUIxgt4OwsTGwh2OuX+vrjlFuixkHduPYJ5zWF4rnm7oq697OwflZCmZM uIFzsLvc9zgAUKft95SuWvfG2SygPNsDs+tHQtByVrauQa2wTtBrbEJvgENaLh4hGLmR /5x10TLsa1bJ6mA7VxNtRwNH2DmJh+DprFZ1uTB32obOm7yTMhaEesAC0/qcjJeYNJCZ gs9g== X-Received: by 10.112.85.11 with SMTP id d11mr63176876lbz.100.1419965747928; Tue, 30 Dec 2014 10:55:47 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.36.215 with HTTP; Tue, 30 Dec 2014 10:55:07 -0800 (PST) In-Reply-To: References: From: Carlos Ferreira Date: Tue, 30 Dec 2014 18:55:07 +0000 Message-ID: Subject: Re: Regarding Netmap internal memory allocation. To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 18:55:50 -0000 Well... due to budget constraints I'm using USB 100Mb ports :) This is for experimental purposes only for now. Btw, can netmap work with wireless interfaces? I believe you once answered this question, but I could not find it in the mail list. On 30 December 2014 at 18:40, Luigi Rizzo wrote: > On Tue, Dec 30, 2014 at 6:38 PM, Carlos Ferreira > wrote: > > Ok, I'm having some trouble in tuning the amount of memory for netmap. > > > > I have been following the man page from FreeBSD in other to understand > the > > values at /sys/modules/netmap/parameters for linux but I'm having some > > trouble in understanding what each value actually means. > > For the following values: > > > > dev.netmap.ring_num: 200 -> Is this the number of rings in the Ring > Buffer > > Pool? > > yes. For interfaces with a single queue you need 4 rings (rx+tx for the > nic and > another rx+tx for the host port) > > > dev.netmap.ring_size: 36864 -> Is this value, the number of slots per > ring? > > this is the size in bytes of each ring. The number of slots is set by the > hardware (low end devices as in the openwrt devices will probably use > 256 or 512 slots, so 10-12k should suffice. But this is not worth changing. > > Instead, you should reduce the number of buffers, though 8MB is only 4000 > buffers and it is a bit on the low side for 5 ports. > > However, as far as I know most openwrt devices only have one physical NIC, > and a switch implementing various vlans. > > cheers > luigi > > > > > I'm trying to keep the amount of memory used by netmap as low as 4MB - > 8MB > > since I'm going to use only up to 4 NICs and one TAP. > > > > Thanks for the help! > > > > > > On 30 December 2014 at 16:16, Luigi Rizzo wrote: > >> > >> you can #undefine WITH_VALE. > >> But it is only 20K of code (and 150K of data structures, which you > >> can further reduce by lowering NM_BRIDGS). > >> The saving is probably not worth the effort. > >> > >> cheers > >> luigi > >> > >> On Tue, Dec 30, 2014 at 5:08 PM, Carlos Ferreira > > >> wrote: > >> > By the way, another question. > >> > Is there a way to not compile the code regarding the VALE switch? I'm > >> > only > >> > interested in using netmap with Tap Devices and NICs, so I was hoping > to > >> > save some memory. > >> > > >> > On 30 December 2014 at 15:47, Carlos Ferreira > >> > wrote: > >> >> > >> >> You mean netmap_mem2.c ? It was there where I found the > >> >> NETMAP_BUF_MAX_NUM > >> >> define. > >> >> > >> >> > >> >> > >> >> On 30 December 2014 at 15:43, Carlos Ferreira > > >> >> wrote: > >> >>> > >> >>> Ok thanks. I was hoping not having to recompile the module, but it's > >> >>> ok. > >> >>> Thank you for the info! > >> >>> > >> >>> > >> >>> On 30 December 2014 at 15:38, Luigi Rizzo > wrote: > >> >>>> > >> >>>> you can reduce the amount of ram (buffers, mostly) by > >> >>>> tweaking the values in netmap_mem2.c :: > >> >>>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { > >> >>>> ... > >> >>>> } > >> >>>> > >> >>>> or you can simply modify the constant > >> >>>> > >> >>>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 > >> >>>> > >> >>>> to something smaller that suits an openwrt box > >> >>>> (in which i am very interested, as I'd like to deploy one of these > >> >>>> soon) > >> >>>> > >> >>>> cheers > >> >>>> luigi > >> >>>> > >> >>>> > >> >>>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira > >> >>>> > >> >>>> wrote: > >> >>>> > Update: > >> >>>> > > >> >>>> > I noticed that the netmap module was still crashing, after > >> >>>> > changing > >> >>>> > the > >> >>>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer > >> >>>> > crashed. > >> >>>> > The > >> >>>> > netmap module is now consuming about 350MB of Ram, which for my > >> >>>> > objectives > >> >>>> > is just too much... > >> >>>> > > >> >>>> > On 30 December 2014 at 14:06, Carlos Ferreira > >> >>>> > > >> >>>> > wrote: > >> >>>> > > >> >>>> >> To Luigi and to whom may be able to help > >> >>>> >> > >> >>>> >> Hello all. > >> >>>> >> > >> >>>> >> Is it possible to reduce the size of the memory buffer allocated > >> >>>> >> by > >> >>>> >> the > >> >>>> >> netmap module? > >> >>>> >> I'm asking this because I was implementing some testing code, > >> >>>> >> using > >> >>>> >> NICs > >> >>>> >> and a Tap device in an OpenWRT VM with 64MB of RAM. > >> >>>> >> Because of the small RAM amount, the nm_open crashed when the > >> >>>> >> program > >> >>>> >> tried to netmap the tap device, after I previously netmapped one > >> >>>> >> NIC > >> >>>> >> successfully. > >> >>>> >> After the crash, I bumped the VM RAM to 256MB and the test > program > >> >>>> >> ran > >> >>>> >> well, but not without me noticing that the VM RAM consumption > was > >> >>>> >> increased about 90 MB by netmap. > >> >>>> >> > >> >>>> >> Resuming, I want to know if there is a way to reduce the memory > >> >>>> >> buffer > >> >>>> >> allocation, without recompiling the netmap kernel module. > >> >>>> >> > >> >>>> >> Thank you for the attention. > >> >>>> >> > >> >>>> >> -- > >> >>>> >> > >> >>>> >> Carlos Miguel Ferreira > >> >>>> >> Researcher at Telecommunications Institute > >> >>>> >> Aveiro - Portugal > >> >>>> >> Work E-mail - cmf@av.it.pt > >> >>>> >> Skype & GTalk -> carlosmf.pt@gmail.com > >> >>>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> >>>> >> > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > -- > >> >>>> > > >> >>>> > Carlos Miguel Ferreira > >> >>>> > Researcher at Telecommunications Institute > >> >>>> > Aveiro - Portugal > >> >>>> > Work E-mail - cmf@av.it.pt > >> >>>> > Skype & GTalk -> carlosmf.pt@gmail.com > >> >>>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> >>>> > _______________________________________________ > >> >>>> > freebsd-net@freebsd.org mailing list > >> >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> >>>> > To unsubscribe, send any mail to > >> >>>> > "freebsd-net-unsubscribe@freebsd.org" > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> > >> >>>> > >> >>>> > -----------------------------------------+------------------------------- > >> >>>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > >> >>>> dell'Informazione > >> >>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >> >>>> TEL +39-050-2211611 . via Diotisalvi 2 > >> >>>> Mobile +39-338-6809875 . 56122 PISA (Italy) > >> >>>> > >> >>>> > >> >>>> > -----------------------------------------+------------------------------- > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> > >> >>> Carlos Miguel Ferreira > >> >>> Researcher at Telecommunications Institute > >> >>> Aveiro - Portugal > >> >>> Work E-mail - cmf@av.it.pt > >> >>> Skype & GTalk -> carlosmf.pt@gmail.com > >> >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> > >> >> Carlos Miguel Ferreira > >> >> Researcher at Telecommunications Institute > >> >> Aveiro - Portugal > >> >> Work E-mail - cmf@av.it.pt > >> >> Skype & GTalk -> carlosmf.pt@gmail.com > >> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> > > >> > > >> > > >> > > >> > -- > >> > > >> > Carlos Miguel Ferreira > >> > Researcher at Telecommunications Institute > >> > Aveiro - Portugal > >> > Work E-mail - cmf@av.it.pt > >> > Skype & GTalk -> carlosmf.pt@gmail.com > >> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > >> > >> > >> > >> -- > >> > -----------------------------------------+------------------------------- > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > dell'Informazione > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >> TEL +39-050-2211611 . via Diotisalvi 2 > >> Mobile +39-338-6809875 . 56122 PISA (Italy) > >> > -----------------------------------------+------------------------------- > > > > > > > > > > -- > > > > Carlos Miguel Ferreira > > Researcher at Telecommunications Institute > > Aveiro - Portugal > > Work E-mail - cmf@av.it.pt > > Skype & GTalk -> carlosmf.pt@gmail.com > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 18:58:47 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED9807AD for ; Tue, 30 Dec 2014 18:58:47 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D49DE304D for ; Tue, 30 Dec 2014 18:58:47 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id sBUIwlU6031550 for ; Tue, 30 Dec 2014 18:58:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194672] [carp] Changing advskew to 0 from another value doesn't work Date: Tue, 30 Dec 2014 18:58:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RC2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: garga@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 18:58:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194672 --- Comment #5 from Renato Botelho --- Created attachment 151128 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=151128&action=edit 3rd version - Fix advskew and advbase checks, set default advbase for new entries This version changes ifconfig to set default advbase to 1 on new entries, and because of that make advbase and advskew checks correct on kernel side -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 19:00:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 071EF858 for ; Tue, 30 Dec 2014 19:00:25 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D33833072 for ; Tue, 30 Dec 2014 19:00:24 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id sBUJ0FZs035342; Tue, 30 Dec 2014 11:00:15 -0800 (PST) (envelope-from bsd-lists@bsdforge.com) To: freebsd-net@freebsd.org, Martin Birgmeier In-Reply-To: <54A2E8B6.3070801@aon.at> References: <54A2E8B6.3070801@aon.at> From: "Chris H" Subject: Re: Tying down network interfaces Date: Tue, 30 Dec 2014 11:00:15 -0800 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 19:00:25 -0000 On Tue, 30 Dec 2014 19:02:30 +0100 Martin Birgmeier wrote > Hi, > > I have two network interfaces as follows: > > sis0: port 0xa400-0xa4ff mem > 0xd5800000-0xd5800fff irq 9 at device 9.0 on pci0 > sis1: port 0x9400-0x94ff mem > 0xd4800000-0xd4800fff irq 11 at device 12.0 on pci0 > > When sis0 breaks down, sis1 gets renumbered as sis0, wreaking havoc > (mostly on my brains until I figure out which card is actually affected). > > How do I tie down these two interfaces so that they always stay as sis0 > and sis1, respectively, regardless of which ones are present in the > system? - I expect to insert something into /boot/device.hints. Just for starters, you might simply issue: ifconfig Which will dump the values of both NIC's (and other net related) But once you've done that, you could issue a: ifconfig sis0 down then watch the blinking lights to help determine which NIC belongs to which number. Not extremely elegant, but will at least help narrow down which NIC, is which. Then in the end, you can allocate your NIC's out of rc.conf(5). --Chris > > -- Martin > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 20:13:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FEBDD79 for ; Tue, 30 Dec 2014 20:13:33 +0000 (UTC) Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41EC91CFD for ; Tue, 30 Dec 2014 20:13:33 +0000 (UTC) Received: by mail-oi0-f50.google.com with SMTP id x69so33627941oia.9 for ; Tue, 30 Dec 2014 12:13:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Cz7++QHZCauhJwn/mqcf3sq8xlp8ewAUOVCDO2tkATI=; b=WEC89dOs3JIZtfxGqys06sU+fioT3MQhq4taYlzA62/9amU3qBGr0h5oc8GKIsFhZp QRIxCi/36bcHPQy0lsP4du7wmw80QT1nxGG9NRiiqW03JNM4h8ZAmZscvnfERrfTO82I Tggc6IrPt5qJyhQpyPkz0xNRB5Ga9MI0V/Up0pPEL3GY9OfIEubyIoOxOgxpBGk7ZT4t Y4y0JOzsinR4fI6R2uLRkfMflVGOGWcV5xADZVOtzUp4FvfpeBmrMDWxIQrNo7g9nH43 Y798CXxSRKs00lWKd7JjdrY7hOCWkSqsFGEO9myZLCznr6OI77wLyamPjmD0KjZ6QsDl yLlw== MIME-Version: 1.0 X-Received: by 10.202.185.69 with SMTP id j66mr35339965oif.86.1419970412707; Tue, 30 Dec 2014 12:13:32 -0800 (PST) Received: by 10.202.76.208 with HTTP; Tue, 30 Dec 2014 12:13:32 -0800 (PST) Received: by 10.202.76.208 with HTTP; Tue, 30 Dec 2014 12:13:32 -0800 (PST) In-Reply-To: <54A2E8B6.3070801@aon.at> References: <54A2E8B6.3070801@aon.at> Date: Tue, 30 Dec 2014 12:13:32 -0800 Message-ID: Subject: Re: Tying down network interfaces From: Freddie Cash To: Martin Birgmeier Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 20:13:33 -0000 On Dec 30, 2014 10:02 AM, "Martin Birgmeier" wrote: > > Hi, > > I have two network interfaces as follows: > > sis0: port 0xa400-0xa4ff mem > 0xd5800000-0xd5800fff irq 9 at device 9.0 on pci0 > sis1: port 0x9400-0x94ff mem > 0xd4800000-0xd4800fff irq 11 at device 12.0 on pci0 > > When sis0 breaks down, sis1 gets renumbered as sis0, wreaking havoc > (mostly on my brains until I figure out which card is actually affected). > > How do I tie down these two interfaces so that they always stay as sis0 > and sis1, respectively, regardless of which ones are present in the > system? - I expect to insert something into /boot/device.hints. There was a recent thread on one of the lists about using devd to name USB Ethernet devices based on their MAC or serial number. Something like that should be useful for naming NICs something constant. There's also a bug report for it with a working solution. Cheers, Freddie From owner-freebsd-net@FreeBSD.ORG Wed Dec 31 10:03:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A8DEF15 for ; Wed, 31 Dec 2014 10:03:41 +0000 (UTC) Received: from email.aon.at (smtpout02.highway.telekom.at [195.3.96.113]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B5FD67255 for ; Wed, 31 Dec 2014 10:03:39 +0000 (UTC) Received: (qmail 16868 invoked from network); 31 Dec 2014 10:03:35 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on WARSBL607.highway.telekom.at X-Spam-Level: Received: from 91-113-62-244.adsl.highway.telekom.at (HELO gandalf.xyzzy) ([91.113.62.244]) (envelope-sender ) by smarthub83.res.a1.net (qmail-ldap-1.03) with AES256-SHA encrypted SMTP; 31 Dec 2014 10:03:35 -0000 X-A1Mail-Track-Id: 1420020215:16842:smarthub83:91.113.62.244:1 Received: from mizar.xyzzy (mizar.xyzzy [192.168.1.19]) by gandalf.xyzzy (8.14.9/8.14.9) with ESMTP id sBVA3YbH004978; Wed, 31 Dec 2014 11:03:35 +0100 (CET) (envelope-from la5lbtyi@aon.at) Message-ID: <54A3C9F6.6000603@aon.at> Date: Wed, 31 Dec 2014 11:03:34 +0100 From: Martin Birgmeier Organization: MBi at home User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Freddie Cash Subject: Re: Tying down network interfaces References: <54A2E8B6.3070801@aon.at> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Dec 2014 10:03:41 -0000 The devices are PCI cards, no USB is involved. I assume that I'd have to add lines similar to hint.sis.0.at="pci0:9:0" to /boot/device.hints, but I am unsure of the correct syntax. See also these old articles (which ultimately seem to have gone unanswered): http://lists.freebsd.org/pipermail/freebsd-questions/2009-January/190453.html , http://lists.freebsd.org/pipermail/freebsd-questions/2009-January/190624.html -- Martin On 12/30/14 21:13, Freddie Cash wrote: > > On Dec 30, 2014 10:02 AM, "Martin Birgmeier" > wrote: > > > > Hi, > > > > I have two network interfaces as follows: > > > > sis0: port 0xa400-0xa4ff mem > > 0xd5800000-0xd5800fff irq 9 at device 9.0 on pci0 > > sis1: port 0x9400-0x94ff mem > > 0xd4800000-0xd4800fff irq 11 at device 12.0 on pci0 > > > > When sis0 breaks down, sis1 gets renumbered as sis0, wreaking havoc > > (mostly on my brains until I figure out which card is actually > affected). > > > > How do I tie down these two interfaces so that they always stay as sis0 > > and sis1, respectively, regardless of which ones are present in the > > system? - I expect to insert something into /boot/device.hints. > > There was a recent thread on one of the lists about using devd to name > USB Ethernet devices based on their MAC or serial number. Something > like that should be useful for naming NICs something constant. > > There's also a bug report for it with a working solution. > > Cheers, > Freddie > From owner-freebsd-net@FreeBSD.ORG Wed Dec 31 10:25:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4753D54C for ; Wed, 31 Dec 2014 10:25:00 +0000 (UTC) Received: from mx.aknet.kg (mx.aknet.kg [212.112.96.8]) by mx1.freebsd.org (Postfix) with ESMTP id 9882B674E4 for ; Wed, 31 Dec 2014 10:24:59 +0000 (UTC) Received: from mx.aknet.kg (localhost.aknet.kg [127.0.0.1]) by mx.aknet.kg (Postfix) with ESMTP id B392C1CDF7; Wed, 31 Dec 2014 16:24:52 +0600 (KGT) Received: (from nobody@localhost) by mx.aknet.kg (8.13.8/8.13.8/Submit) id sBVAOqMF005995; Wed, 31 Dec 2014 16:24:52 +0600 (KGT) (envelope-from info@aknet.kg) X-Authentication-Warning: mx.aknet.kg: nobody set sender to info@aknet.kg using -f To: Subject: Netmap-Ipfw: eats 90-100% of CPU, is it normal behaviour =?UTF-8?Q?=3F?= X-PHP-Originating-Script: 501:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 31 Dec 2014 16:24:51 +0600 From: info@aknet.kg Message-ID: X-Sender: info@aknet.kg User-Agent: Roundcube Webmail/0.7.2 Cc: rizzo@iet.unipi.it X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Dec 2014 10:25:00 -0000 Hello, All ! We tried to use netmap-ipfw in production (as filtering bridge) for traffic sanity and bandwidth limitation. And meet a problem. Will be explaned below. CPU: i5-4690 CPU @ 3.50GHz RAM: 8GB x 1800Mhz NET: Intel DA 520 (2 x 10Gbps) kipfw starts as: /usr/local/netmap-ipfw/kipfw netmap:ix0 netmap:ix1 ruleset: 00100 allow ip from 192.168.254.0/24 to 192.168.254.0/24 00200 allow ip from any to 192.168.0.0/16 - incoming (for customers) traffic goes without touching 00400 pipe 665 udp from 192.168.0.0/16 to any dst-port 6881 00500 pipe 666 tcp from 192.168.0.0/16 to any tcpflags syn 00600 deny tcp from table(25) to any dst-port 25 00700 deny tcp from 192.168.0.0/16 to table(26) dst-port 25 00750 allow ip from 192.168.0.0/16 to any - this rule we have to use (explaned below) 00800 pipe 10 ip from 192.168.0.0/16 to any - main rule for this bridge 65535 allow ip from any to any pipes: # BW for packets with SYN flag and UDP-6881 ${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s ${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s # Outgoing BW for each IP ${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s table 25 has about 100 IP's table 26 has about 15 sub-networks this bridge serves about 25K subscribers with IP's from network: 192.168.0.0/16 current traffic: netstat -bdh -w1 -I ix1 input ix1 output packets errs idrops bytes packets errs bytes colls drops 607K 0 0 753M 452K 0 88M 0 0 603K 0 0 750M 449K 0 87M 0 0 604K 0 0 751M 448K 0 88M 0 0 604K 0 0 747M 452K 0 92M 0 0 all traffic: netstat -bdh -w1 input (Total) output packets errs idrops bytes packets errs bytes colls drops 2M 0 0 1.6G 2M 0 1.6G 0 0 2M 0 0 1.6G 2M 0 1.6G 0 0 current CPU: CPU 0: 31.1% user, 0.0% nice, 56.1% system, 5.1% interrupt, 7.7% idle CPU 1: 0.0% user, 0.0% nice, 0.5% system, 8.2% interrupt, 91.3% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 4.6% interrupt, 95.4% idle CPU 3: 0.0% user, 0.0% nice, 0.5% system, 7.1% interrupt, 92.3% idle THE Question: is it normal for kipfw to eat so much resoures ? 660 root 99 0 873M 325M CPU0 0 272:03 91.46% kipfw Also, the rule #750 I have to place into ruleset, cos without it kipfw begins to use all 100% 00750 allow ip from 192.168.0.0/16 to any 00800 pipe 10 ip from 192.168.0.0/16 to any - this rule is the main for using of this bridge, it assigns the same outgoing bandwidth for each of IP addresses - 5120Kbit/s (5Mbps) # BW for packets with SYN flag and UDP-6881 ${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s ${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s # Outgoing BW for each IP ${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s With working rule #800 after 30-50 mins kipfw begins to use 100% in top -PHS and incoming (for users) traffic downs from 750Mbytes/s (about 6Gbit/s) to 330Mbytes/s (2.6Gbit/s), delay increases from 65ms to 250ms and high percentage of drops. Is it real limit of using netmap-ipfw ? We can give any additional info if it will be usefull to expand limits of kipfw. With regards and happy New Year ! Azamat B. Umurzakov AkNet ISP From owner-freebsd-net@FreeBSD.ORG Wed Dec 31 14:45:30 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01E80DB1 for ; Wed, 31 Dec 2014 14:45:30 +0000 (UTC) Received: from smtp-32.italiaonline.it (smtp-32.italiaonline.it [212.48.25.160]) by mx1.freebsd.org (Postfix) with ESMTP id 8AA4424E6 for ; Wed, 31 Dec 2014 14:45:29 +0000 (UTC) Received: from soth.ventu ([151.41.160.1]) by smtp-32.iol.local with bizsmtp id aEkG1p00S026G0P0YEkHY4; Wed, 31 Dec 2014 15:44:18 +0100 x-libjamoibt: 1601 X-CNFS-Analysis: v=2.1 cv=eaezft0H c=1 sm=1 tr=0 a=ll63ros7JW2os3nJXz0AJQ==:117 a=ll63ros7JW2os3nJXz0AJQ==:17 a=SA30iOlf1jgA:10 a=IkcTkHD0fZMA:10 a=A92cGCtB03wA:10 a=RA6ZSCCb2IdtTfKqRcwA:9 a=QEXdDO2ut3YA:10 a=lvKDMF_JM20A:10 a=77krBdBlvRkA:10 Received: from guardian.ventu (bane.ventu [10.1.2.15]) (authenticated bits=0) by soth.ventu (8.15.1/8.14.7) with ESMTPSA id sBVEhuXN074815 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 31 Dec 2014 15:44:02 +0100 (CET) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.ventu: Host bane.ventu [10.1.2.15] claimed to be guardian.ventu Message-ID: <54A40BAD.2000605@netfence.it> Date: Wed, 31 Dec 2014 15:43:57 +0100 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: net@freebsd.org Subject: Dynamic ipfw rules' top Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.75 on 10.1.2.13 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Dec 2014 14:45:30 -0000 Hello. This might be a strange idea, but does such a thing exist? I mean: is there any tool that can show in real-time which dynamic rules are active, their timers, etc... like top does for processes? bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Wed Dec 31 14:58:36 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 722F6443 for ; Wed, 31 Dec 2014 14:58:36 +0000 (UTC) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 304CE27C9 for ; Wed, 31 Dec 2014 14:58:35 +0000 (UTC) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3kCFxB3rddzZs1; Wed, 31 Dec 2014 15:58:22 +0100 (CET) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id FSC6T-avrZHO; Wed, 31 Dec 2014 15:58:20 +0100 (CET) Received: from tommy.madpilot.net (host3-108-dynamic.50-79-r.retail.telecomitalia.it [79.50.108.3]) by mail.madpilot.net (Postfix) with ESMTPSA; Wed, 31 Dec 2014 15:58:20 +0100 (CET) Message-ID: <54A40F09.5040909@madpilot.net> Date: Wed, 31 Dec 2014 15:58:17 +0100 From: Guido Falsi User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Andrea Venturoli , net@freebsd.org Subject: Re: Dynamic ipfw rules' top References: <54A40BAD.2000605@netfence.it> In-Reply-To: <54A40BAD.2000605@netfence.it> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Dec 2014 14:58:36 -0000 On 12/31/14 15:43, Andrea Venturoli wrote: > Hello. > > This might be a strange idea, but does such a thing exist? > > I mean: is there any tool that can show in real-time which dynamic rules > are active, their timers, etc... like top does for processes? > I'm using the port sysutils/cmdwatch with ipfw, like this: cmdwatch "ipfw -d show | sed -n -e '/## Dynamic rules/,\$p'" It's a little crude but works quite fine. Needs to run as root, obviously. -- Guido Falsi From owner-freebsd-net@FreeBSD.ORG Wed Dec 31 16:57:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FA56C2E for ; Wed, 31 Dec 2014 16:57:47 +0000 (UTC) Received: from mx.aknet.kg (mx.aknet.kg [212.112.96.8]) by mx1.freebsd.org (Postfix) with ESMTP id 9B68439CA for ; Wed, 31 Dec 2014 16:57:46 +0000 (UTC) Received: from mx.aknet.kg (localhost.aknet.kg [127.0.0.1]) by mx.aknet.kg (Postfix) with ESMTP id F29EC1CD56 for ; Wed, 31 Dec 2014 22:57:44 +0600 (KGT) Received: (from nobody@localhost) by mx.aknet.kg (8.13.8/8.13.8/Submit) id sBVGviHm036591; Wed, 31 Dec 2014 22:57:44 +0600 (KGT) (envelope-from info@aknet.kg) X-Authentication-Warning: mx.aknet.kg: nobody set sender to info@aknet.kg using -f To: Subject: Re: Netmap-Ipfw: eats 90-100% of CPU, is it normal behaviour =?UTF-8?Q?=20=3F?= X-PHP-Originating-Script: 501:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 31 Dec 2014 22:57:44 +0600 From: info@aknet.kg In-Reply-To: References: Message-ID: X-Sender: info@aknet.kg User-Agent: Roundcube Webmail/0.7.2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Dec 2014 16:57:47 -0000 Hello, All! In addition to previous info I can say, that netmap-ipfw takes about 95% in top -PHS, even if firewall is fully open: 60 root 100 0 885M 342M CPU0 0 621:31 92.38% kipfw when first rule is "allow ip from any to any" May be it needs more RAM ? currently is 885M (RES 342M) and doesn't increase with load growth. current traffic: input ix1 output packets errs idrops bytes packets errs bytes colls drops 528K 0 0 599M 434K 0 124M 0 0 520K 0 0 590M 430K 0 126M 0 0 531K 0 0 603M 437K 0 128M 0 0 IT Dep AkNet ISP info@aknet.kg писал 2014-12-31 16:24: > Hello, All ! > > We tried to use netmap-ipfw in production (as filtering bridge) for > traffic sanity and bandwidth limitation. > And meet a problem. Will be explaned below. > > CPU: i5-4690 CPU @ 3.50GHz > RAM: 8GB x 1800Mhz > NET: Intel DA 520 (2 x 10Gbps) > > kipfw starts as: > /usr/local/netmap-ipfw/kipfw netmap:ix0 netmap:ix1 > > ruleset: > > 00100 allow ip from 192.168.254.0/24 to 192.168.254.0/24 > 00200 allow ip from any to 192.168.0.0/16 - > incoming (for customers) traffic goes without touching > 00400 pipe 665 udp from 192.168.0.0/16 to any dst-port 6881 > 00500 pipe 666 tcp from 192.168.0.0/16 to any tcpflags syn > 00600 deny tcp from table(25) to any dst-port 25 > 00700 deny tcp from 192.168.0.0/16 to table(26) dst-port 25 > 00750 allow ip from 192.168.0.0/16 to any - this > rule we have to use (explaned below) > 00800 pipe 10 ip from 192.168.0.0/16 to any - main > rule for this bridge > 65535 allow ip from any to any > > pipes: > # BW for packets with SYN flag and UDP-6881 > ${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s > ${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s > # Outgoing BW for each IP > ${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s > > table 25 has about 100 IP's > table 26 has about 15 sub-networks > > this bridge serves about 25K subscribers with IP's from network: > 192.168.0.0/16 > > current traffic: > netstat -bdh -w1 -I ix1 > > input ix1 output > packets errs idrops bytes packets errs bytes colls > drops > 607K 0 0 753M 452K 0 88M 0 > 0 > 603K 0 0 750M 449K 0 87M 0 > 0 > 604K 0 0 751M 448K 0 88M 0 > 0 > 604K 0 0 747M 452K 0 92M 0 > 0 > > all traffic: > netstat -bdh -w1 > > input (Total) output > packets errs idrops bytes packets errs bytes colls > drops > 2M 0 0 1.6G 2M 0 1.6G 0 > 0 > 2M 0 0 1.6G 2M 0 1.6G 0 > 0 > > > current CPU: > CPU 0: 31.1% user, 0.0% nice, 56.1% system, 5.1% interrupt, 7.7% > idle > CPU 1: 0.0% user, 0.0% nice, 0.5% system, 8.2% interrupt, 91.3% > idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 4.6% interrupt, 95.4% > idle > CPU 3: 0.0% user, 0.0% nice, 0.5% system, 7.1% interrupt, 92.3% > idle > > THE Question: > is it normal for kipfw to eat so much resoures ? > > 660 root 99 0 873M 325M CPU0 0 272:03 91.46% kipfw > > Also, the rule #750 I have to place into ruleset, cos without it > kipfw begins to use all 100% > > 00750 allow ip from 192.168.0.0/16 to any > 00800 pipe 10 ip from 192.168.0.0/16 to any - this rule is the main > for using of this bridge, > > it assigns the same outgoing bandwidth for each of IP addresses - > 5120Kbit/s (5Mbps) > > > # BW for packets with SYN flag and UDP-6881 > ${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s > ${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s > # Outgoing BW for each IP > ${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s > > With working rule #800 after 30-50 mins kipfw begins to use 100% in > top -PHS and incoming (for users) traffic downs from 750Mbytes/s > (about 6Gbit/s) to 330Mbytes/s (2.6Gbit/s), delay increases from 65ms > to 250ms and high percentage of drops. > > Is it real limit of using netmap-ipfw ? We can give any additional > info if it will be usefull to expand limits of kipfw. > > With regards and happy New Year ! > > Azamat B. Umurzakov > AkNet ISP > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Jan 1 11:20:07 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 630F2F4B for ; Thu, 1 Jan 2015 11:20:07 +0000 (UTC) Received: from mail14.tpgi.com.au (smtp-out14.tpgi.com.au [220.244.226.124]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.tpg.com.au", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E64C62D06 for ; Thu, 1 Jan 2015 11:20:05 +0000 (UTC) X-TPG-Junk-Status: Message not scanned X-TPG-Antivirus: Passed X-TPG-Abuse: host=[202.161.115.54]; ip=202.161.115.54; date=Thu, 1 Jan 2015 22:03:12 +1100 Received: from fish.ish.com.au (202-161-115-54.static.tpgi.com.au [202.161.115.54] (may be forged)) by mail14.tpgi.com.au (envelope-from ari@ish.com.au) (8.14.3/8.14.3) with ESMTP id t01B3Ama021091 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 1 Jan 2015 22:03:12 +1100 Received: from ip-211.ish.com.au ([203.29.62.211]:60342 helo=ish.com.au) by fish.ish.com.au with esmtp (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1Y6dWx-0006p9-2j for freebsd-net@freebsd.org; Thu, 01 Jan 2015 22:03:04 +1100 Received: from [10.242.2.6] (HELO Aristedess-MacBook-Pro.local) by ish.com.au (CommuniGate Pro SMTP 6.1c1) with ESMTPS id 17945681 for freebsd-net@freebsd.org; Thu, 01 Jan 2015 22:03:03 +1100 Message-ID: <54A52966.9040407@ish.com.au> Date: Thu, 01 Jan 2015 22:03:02 +1100 From: Aristedes Maniatis User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Thunderbird/34.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: CARP vhid: across interfaces? Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jan 2015 11:20:07 -0000 I have two firewalls built with FreeBSD 10.1 which are working nicely. Upstream I have two internet links, one going into each firewall. An IP address is shared between the two firewalls using CARP. Internally, we have another address shared between the firewalls, and set as the default gateway for all devices behind. So far, pretty simple. My question that isn't answered in the FreeBSD handbook is what to do with the vhid. If one of the external interfaces goes down I want everything to fail over to the secondary firewall. But that means the internal and external interfaces should fail over together. Should I be doing that by using a single vhid for all interfaces (does that bind them together to failover?), or by writing a script to detect the failover and then bring down the other interface? Thanks Ari -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A From owner-freebsd-net@FreeBSD.ORG Thu Jan 1 11:22:46 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DFB57143 for ; Thu, 1 Jan 2015 11:22:46 +0000 (UTC) Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com [IPv6:2607:f8b0:4003:c06::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F10B2E5B for ; Thu, 1 Jan 2015 11:22:46 +0000 (UTC) Received: by mail-oi0-f45.google.com with SMTP id x69so38030099oia.4 for ; Thu, 01 Jan 2015 03:22:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=HmNKXTOk2x4onoXsGBpwvkinJoWCiMe9SLB+JBg31aU=; b=TjIYCV67uGooWXc07KuTzY0niJb4H4A6gm91Z9F0FYRFeoAFsEz4VavqC6HWaKxLxT ERVFqwLHVhkrNlWD7LMsiJSlef+la0llmF/npOnfXMAdiWw8kiKhnbfEdxCJoN6EShAY TDwhcEJknKGo5CW/Me7aavaJI8nAWQMJZUKYyxyPJ7aX8oELqa1uUDuwN5aloXfcbqEM W1q4EgIlMc3bhNnvpm01czjKBEmAMujp37BSe4n1ej3dkR7apr1ja+9wToE7E5TLAoGA 8ku685uyxFYrnxJagLqEaWFtc9MIYYHbQX8Dm06bFU8fRLrzQlqZz+viEUpGXbkCQGhx tTbg== MIME-Version: 1.0 X-Received: by 10.60.52.2 with SMTP id p2mr42015466oeo.85.1420111366016; Thu, 01 Jan 2015 03:22:46 -0800 (PST) Received: by 10.202.76.208 with HTTP; Thu, 1 Jan 2015 03:22:45 -0800 (PST) Received: by 10.202.76.208 with HTTP; Thu, 1 Jan 2015 03:22:45 -0800 (PST) In-Reply-To: <54A52966.9040407@ish.com.au> References: <54A52966.9040407@ish.com.au> Date: Thu, 1 Jan 2015 03:22:45 -0800 Message-ID: Subject: Re: CARP vhid: across interfaces? From: Freddie Cash To: Aristedes Maniatis Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jan 2015 11:22:47 -0000 There's a sysctl specifically for this. Not at my computer right now, but the following should make it jump out at you: # sysctl -d | grep carp Cheers, Freddie On Jan 1, 2015 3:20 AM, "Aristedes Maniatis" wrote: > I have two firewalls built with FreeBSD 10.1 which are working nicely. > Upstream I have two internet links, one going into each firewall. An IP > address is shared between the two firewalls using CARP. Internally, we have > another address shared between the firewalls, and set as the default > gateway for all devices behind. > > So far, pretty simple. My question that isn't answered in the FreeBSD > handbook is what to do with the vhid. If one of the external interfaces > goes down I want everything to fail over to the secondary firewall. But > that means the internal and external interfaces should fail over together. > Should I be doing that by using a single vhid for all interfaces (does that > bind them together to failover?), or by writing a script to detect the > failover and then bring down the other interface? > > Thanks > Ari > > > -- > --------------------------> > Aristedes Maniatis > ish > http://www.ish.com.au > Level 1, 30 Wilson Street Newtown 2042 Australia > phone +61 2 9550 5001 fax +61 2 9550 4001 > GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jan 1 11:29:15 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 328BB292 for ; Thu, 1 Jan 2015 11:29:15 +0000 (UTC) Received: from mail14.tpgi.com.au (mail14.tpgi.com.au [203.12.160.182]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.tpg.com.au", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B2CF42EA3 for ; Thu, 1 Jan 2015 11:29:14 +0000 (UTC) X-TPG-Junk-Status: Message not scanned X-TPG-Antivirus: Passed X-TPG-Abuse: host=[202.161.115.54]; ip=202.161.115.54; date=Thu, 1 Jan 2015 22:29:11 +1100 Received: from fish.ish.com.au (202-161-115-54.static.tpgi.com.au [202.161.115.54] (may be forged)) by mail14.tpgi.com.au (envelope-from ari@ish.com.au) (8.14.3/8.14.3) with ESMTP id t01BT99o015560 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 1 Jan 2015 22:29:11 +1100 Received: from ip-211.ish.com.au ([203.29.62.211]:62752 helo=ish.com.au) by fish.ish.com.au with esmtp (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1Y6dwA-0008AS-2h; Thu, 01 Jan 2015 22:29:07 +1100 Received: from [10.242.2.6] (HELO Aristedess-MacBook-Pro.local) by ish.com.au (CommuniGate Pro SMTP 6.1c1) with ESMTPS id 17945815; Thu, 01 Jan 2015 22:29:06 +1100 Message-ID: <54A52F81.4080409@ish.com.au> Date: Thu, 01 Jan 2015 22:29:05 +1100 From: Aristedes Maniatis User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Thunderbird/34.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: CARP vhid: across interfaces? References: <54A52966.9040407@ish.com.au> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: Freddie Cash X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jan 2015 11:29:15 -0000 On 1/01/2015 10:22pm, Freddie Cash wrote: > There's a sysctl specifically for this. Not at my computer right now, but the following should make it jump out at you: > > # sysctl -d | grep carp I'm guessing this one (from the openBSD docs)... net.inet.carp.preempt Allow hosts within a redundancy group that have a better advbase and advskew to preempt the master. In addition, this option also enables failing over a group of interfaces together in the event that one interface goes down. If one physical CARP-enabled interface goes down, CARP will increase the demotion counter, carpdemote, by 1 on interface groups that the carp(4) interface is a member of, in effect causing all group members to fail-over together. net.inet.carp.preempt is 0 (disabled) by default. But the FreeBSD man page doesn't talk about carpdemote net.inet.carp.preempt Allow virtual hosts to preempt each other. When enabled, a vhid in a backup state would preempt a master that is announcing itself with a lower advskew. Disabled by default. At any rate what does "interface groups that the carp(4) interface is a member of" mean? Freddie, thanks for pointing me to this setting. Maybe the answer is in the somewhere. Ari -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A From owner-freebsd-net@FreeBSD.ORG Thu Jan 1 11:32:21 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50DF6331 for ; Thu, 1 Jan 2015 11:32:21 +0000 (UTC) Received: from mail-ob0-x22e.google.com (mail-ob0-x22e.google.com [IPv6:2607:f8b0:4003:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F0502F9C for ; Thu, 1 Jan 2015 11:32:21 +0000 (UTC) Received: by mail-ob0-f174.google.com with SMTP id nt9so56177711obb.5 for ; Thu, 01 Jan 2015 03:32:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OXpp/XCR85rEJrH57FPw9GmsIdO2XEoflYc1e2gXVJA=; b=Vkz+FNnZVv1+aJcq+vQjfGna04y/4IoBCGLCE/A9eLDVHUaLgNXa3vbFKPipOeyst0 P606/WJZQPPcLk+k3RdVxYax6tHCWw24l8/ERkrLHyVydNp9Sv3E2F9mnX5TGvL+xQHp bbbxP+1JujcIsXsg3hM/80ZspgX3sK2eI0Wg/K2BppDZ18jEPz2Afvwdt/w0hfOYYosJ rwAnqJv1IyqCJbpH0H14IfIqSyrHQILo4zTjGMIUYjXxrJ7xIeWYIAGrVWIbocZQE2v/ rFFLk9YQX/wqkfrDVdvCsfVYLqUFER5Bq8qEcoP3d9no5c8b/GmbJWDc1h8j0UoPluEJ phOg== MIME-Version: 1.0 X-Received: by 10.202.185.69 with SMTP id j66mr39782370oif.86.1420111940437; Thu, 01 Jan 2015 03:32:20 -0800 (PST) Received: by 10.202.76.208 with HTTP; Thu, 1 Jan 2015 03:32:20 -0800 (PST) Received: by 10.202.76.208 with HTTP; Thu, 1 Jan 2015 03:32:20 -0800 (PST) In-Reply-To: <54A52F81.4080409@ish.com.au> References: <54A52966.9040407@ish.com.au> <54A52F81.4080409@ish.com.au> Date: Thu, 1 Jan 2015 03:32:20 -0800 Message-ID: Subject: Re: CARP vhid: across interfaces? From: Freddie Cash To: Aristedes Maniatis Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jan 2015 11:32:21 -0000 That is the correct sysctl. If any CARP-controlled interface goes down, then all CARP-interfaces ate failed. That's the setup you want. It's the one we use on two pairs of CARP-using routers, and it works wonderfully. Cheers, Freddie On Jan 1, 2015 3:29 AM, "Aristedes Maniatis" wrote: > On 1/01/2015 10:22pm, Freddie Cash wrote: > > There's a sysctl specifically for this. Not at my computer right now, > but the following should make it jump out at you: > > > > # sysctl -d | grep carp > > I'm guessing this one (from the openBSD docs)... > > net.inet.carp.preempt > Allow hosts within a redundancy group that have a better advbase and > advskew to preempt the master. In addition, this option also enables > failing over a group of interfaces together in the event that one interface > goes down. If one physical CARP-enabled interface goes down, CARP will > increase the demotion counter, carpdemote, by 1 on interface groups that > the carp(4) interface is a member of, in effect causing all group members > to fail-over together. net.inet.carp.preempt is 0 (disabled) by default. > > > But the FreeBSD man page doesn't talk about carpdemote > > net.inet.carp.preempt Allow virtual hosts to preempt each > other. When enabled, a vhid in > a > backup state would preempt a > master > that is announcing itself with a > lower advskew. Disabled by > default. > > > > At any rate what does "interface groups that the carp(4) interface is a > member of" mean? > > > > Freddie, thanks for pointing me to this setting. Maybe the answer is in > the somewhere. > > Ari > > > -- > --------------------------> > Aristedes Maniatis > ish > http://www.ish.com.au > Level 1, 30 Wilson Street Newtown 2042 Australia > phone +61 2 9550 5001 fax +61 2 9550 4001 > GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A > From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 13:39:01 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 153B7A07 for ; Fri, 2 Jan 2015 13:39:01 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 80ACB66FFF for ; Fri, 2 Jan 2015 13:39:00 +0000 (UTC) Received: by mail-wg0-f49.google.com with SMTP id n12so23971882wgh.8 for ; Fri, 02 Jan 2015 05:38:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=GioyB88Ot8w3nIxNVA+E1ZTsHG3rUGiRkBMZGZommRA=; b=XTdS+2QoHJ0y0iq8i4Fd24ty/9BjseF/VG2iR7X28kyeewNrLhbnzmGnEs6TFTaGBq agIC+H80D0kXacXJ68R+2yxecX2Wr9ob2lbmWcnk2BoMxjqnfAFnrvzSw9Y1R/OL5UeX FMc1mYwcXgJtW4wSoVAnvvDEHKp28hSSSF+0mWMBUN4eH4rzvONQgvW5IlA3Lp7vDTPW eQhOwq1hJHwsIjKjdTDY+9+ZInJEbzRsLWKwSdA3qYSvHU0wBFjtOqZ7Ob66+kzHjZfq nOcg9PupOhRZG5RaA0AHFymGSR7j5B5uBYxi1rt5DIzosqyYxvZqxyTIcAs9axg9QLhg SV1Q== X-Received: by 10.194.236.1 with SMTP id uq1mr132958452wjc.28.1420205937751; Fri, 02 Jan 2015 05:38:57 -0800 (PST) Received: from [10.99.0.3] ([213.188.107.182]) by mx.google.com with ESMTPSA id td6sm54163037wic.15.2015.01.02.05.38.56 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Jan 2015 05:38:57 -0800 (PST) Message-ID: <54A69F72.6060405@gmail.com> Date: Fri, 02 Jan 2015 14:38:58 +0100 From: Sascha User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: CARP Problem/Bug? on 10.1-RELEASE Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 13:39:01 -0000 Hi and a happy new year to everyone! I have problems with my carp setup between two Routers/Firewalls after upgrade from 10.0 RELEASE to 10.1 RELEASE. Before the upgrade my setup worked without any problems! After the upgrade I checked the carp status. Machine 2 (Backup machine) is in Master state for all Interfaces. Machine 1 remains in Backup state. Machine 1 is my primary Machine and should be the master! I restarted both machines several times and checked the rc.conf for errors. But Machine 2 gets every time the master. Machine 1 gets only the master when I turn off Machine 2. All interfaces remain in Master state even when I manually take a Interface for example igb4 with ifconfig on Machine 2 down. Then igb4 on Machine 1 goes into Master state.But the other interfaces remain in Backup state on Machine 1. preemt Option is set to 1 The machines are connected with Link Aggregation to the switch and they do VLAN Tagging. The CARP Interface is the Gateway for the clients. The address on the physical interface is used to bind demons for network services. PF Sync is done over a cross cable direct attached to both machines. PF is not blocking any carp traffic. I Think the problem has something to do with the net.inet.carp.demotion You can see in my output that machine 1 had this value net.inet.carp.demotion: 3840 Is this a bug or does someone see a configuration error in the config? Detailed config(Some output is cleared or omitted): rc.conf Machine 1: ################################################################################################################################################################################ ifconfig_igb0="up" ifconfig_igb1="up" ifconfig_igb2="up" ifconfig_igb3="up" ifconfig_igb4="up" ifconfig_igb5="up" ifconfig_igb4="inet xxx.xxx.0.253 netmask 255.255.255.0" ifconfig_igb4_alias0="vhid 1 pass secret xxx.xxx.0.1/32" # PF Sync Interface ifconfig_igb5="inet xxx.xxx.255.253/30" # Create Virutal Interfaces cloned_interfaces="lagg0 vlan10 vlan25 vlan35 vlan90 vlan95 vlan97 vlan98 vlan99 vlan100 vlan101 vlan102 vlan103 vlan106 vlan107 vlan108 vlan109" # lagg Interface ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 laggport igb2 laggport igb3" # VLAN Interfaces ifconfig_vlanxx0="inet xxx.xxxx.xxx.197/27" ifconfig_vlanxx0alias0="inet vhid 2 pass secret xxx.xxx.xxx.199/32 vlan xx0 vlandev lagg0" ifconfig_vlanxx5="inet xxx.xxx.25.253/24" ifconfig_vlanxx5_alias0="inet vhid 3 pass secret xxx.xxx.25.1/32 vlan xx5 vlandev lagg0" ifconfig_vlanxx5="inet xxx.xxx.35.253/24" ifconfig_vlanxx5_alias0="inet vhid 4 pass secret xxx.35.1/32 vlan xx5 vlandev lagg0" ifconfig_vlanxx0="inet xxx.xxx.7.253/21" ifconfig_vlanxx0_alias0="inet vhid 5 pass secret xxx.xxx.7.1/32 vlan xx0 vlandev lagg0" ifconfig_vlanxx5="inet xxx.xxx.95.253/24" ifconfig_vlanxx5_alias0="inet vhid 6 pass secret xxx.xxx.95.1/32 vlan xx5 vlandev lagg0" ifconfig_vlanxx7="inet xxx.xxx.90.253/16" ifconfig_vlanxx7_alias0="inet vhid 7 pass secret xxx.xxx.90.6/32 vlan xx7 vlandev lagg0" ifconfig_vlanxx8="inet xxx.xxx.239.253/21" ifconfig_vlanxx8_alias0="inet vhid 8 pass secret xxx.xxx.232.1/32 vlan xx8 vlandev lagg0" ifconfig_vlanxx9="inet xxx.xxx.0.29/27" ifconfig_vlanxx9_alias0="inet vhid 9 pass secret xxx.xxx.0.1/32 vlan xx9 vlandev lagg0" ifconfig_vlanxx0="inet xxx.xxx.100.253/24" ifconfig_vlanxx0_alias0="inet vhid 10 pass secret xxx.xxx.100.1/32 vlan xx0 vlandev lagg0" ifconfig_vlanxx1="inet xxx.xxx.1.253/24" ifconfig_vlanxx1_alias0="inet vhid 11 pass secret xxx.xxx.1.1/32 vlan xx1 vlandev lagg0" ifconfig_vlanxx2="inet xxx.xxx.116.253/24" ifconfig_vlanxx2_alias0="inet vhid 12 pass secret xxx.xxx.116.1/32 vlan xx2 vlandev lagg0" ifconfig_vlanxx3="inet xxx.xxx.10.253/24" ifconfig_vlanxx3_alias0="inet vhid 13 pass secret xxx.xxx.10.1/32 vlan xx3 vlandev lagg0" ifconfig_vlanxx6="inet xxx.xxx.255.252/16" ifconfig_vlanxx6_alias0="inet vhid 14 pass secret xxx.xxx.255.254/32 vlan xx6 vlandev lagg0" ifconfig_vlanxx7="inet xxx.xxx.90.253/16" ifconfig_vlanxx7_alias0="inet vhid 15 pass secret xxx.xxx.90.140/32 vlan xx7 vlandev lagg0" ifconfig_vlanxx8="inet xxx.xxx.89.12/29" ifconfig_vlanxx8_alias0="inet vhid 16 pass secret xxx.xxx.89.14/32 vlan xx8 vlandev lagg0" ifconfig_vlanxx9="inet xxx.xxx.110.253/24" ifconfig_vlanxx9_alias0="inet vhid 17 pass secret xxx.xxx.110.1/32 vlan xx9 vlandev lagg0" igb4: flags=8943 metric 0 mtu 1500 options=403bb ether 00:1b:21:96:65:78 inet xxx.xxx.0.253 netmask 0xffffff00 broadcast xxx.xxx.0.255 inet xxx.xxx.0.1 netmask 0xffffffff broadcast xxx.xxxx.0.1 vhid 1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active carp: BACKUP vhid 1 advbase 1 advskew 0 vlanxx0: flags=8943 metric 0 mtu 1500 options=303 ether 00:1b:21:96:c0:38 inet xxx.xxx.xxx.197 netmask 0xffffffe0 broadcast xxx.xxx.xxx.223 inet xxx.xxx.xxx.199 netmask 0xffffffff broadcast xxx.xxx.xxx.199 vhid 2 nd6 options=29 media: Ethernet autoselect status: active vlan: 10 parent interface: lagg0 carp: BACKUP vhid 2 advbase 1 advskew 0 root@xxx1:/# sysctl net.inet.carp net.inet.carp.allow: 1 net.inet.carp.preempt: 1 net.inet.carp.log: 2 net.inet.carp.demotion: 3840 net.inet.carp.senderr_demotion_factor: 240 net.inet.carp.ifdown_demotion_factor: 240 Log File when restarting machine 2 root@xxx1:/# less /var/log/messages Jan 2 12:29:11 xxx1 kernel: carp: VHID 15@vlanxx7: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 16@vlanxx8: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 17@vlanxx9: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 1@igb4: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 2@vlanxx0: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 3@vlanxx5: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 4@vlanxx5: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 5@vlanxx0: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 6@vlanxx5: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 7@vlanxx7: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 8@vlanxx8: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 10@vlanxx0: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 9@vlanxx9: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 11@vlanxx1: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 12@vlanxx2: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 13@vlanxx3: BACKUP -> MASTER (master down) Jan 2 12:29:11 xxx1 kernel: carp: VHID 14@vlanxx6: BACKUP -> MASTER (master down) Jan 2 12:29:12 xxx1 kernel: igb5: link state changed to DOWN Jan 2 12:29:14 xxx1 kernel: igb5: link state changed to UP Jan 2 12:29:14 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' Jan 2 12:29:20 xxx1 kernel: igb5: link state changed to DOWN Jan 2 12:29:22 xxx1 kernel: igb5: link state changed to UP Jan 2 12:29:22 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' Jan 2 12:29:23 xxx1 kernel: igb5: link state changed to DOWN Jan 2 12:29:24 xxx1 kernel: igb5: link state changed to UP Jan 2 12:29:24 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' Jan 2 12:29:35 xxx1 kernel: igb5: link state changed to DOWN Jan 2 12:29:37 xxx1 kernel: igb5: link state changed to UP Jan 2 12:29:37 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' Jan 2 12:31:06 xxx1 kernel: igb5: link state changed to DOWN Jan 2 12:31:08 xxx1 kernel: igb5: link state changed to UP Jan 2 12:31:08 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' Jan 2 12:31:17 xxx1 kernel: igb5: link state changed to DOWN Jan 2 12:31:19 xxx1 kernel: igb5: link state changed to UP Jan 2 12:31:19 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' Jan 2 12:31:21 xxx1 kernel: carp: VHID 1@igb4: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 2@vlanxx0: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 3@vlanxx5: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 5@vlanxx0: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 6@vlanxx5: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 9@vlanxx9: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 8@vlanxx8: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 7@vlanxx7: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 14@vlanxx6: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 10@vlanxx0: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 16@vlanxx8: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 11@vlanxx1: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 12@vlanxx2: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 15@vlanxx7: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 13@vlanxx3: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 17@vlanxx9: MASTER -> BACKUP (more frequent advertisement received) Jan 2 12:31:52 xxx1 kernel: carp: VHID 4@vlanxx5: MASTER -> BACKUP (more frequent advertisement received) TCP Dump on machine 1 during restarting machine 2 root@xxx1:/# sudo tcpdump -npi igb4 -T carp dst 224.0.0.18 13:47:55.455580 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167172 13:47:56.881428 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167173 13:47:59.883366 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167174 13:48:00.894534 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167175 . . 13:50:07.842383 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167240 13:50:09.782055 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167241 13:50:09.782914 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167242 13:50:11.206767 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167243 13:50:11.510203 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167244 13:50:12.450678 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167245 ################################################################################################################################################################################ rc.conf Machine 2: ################################################################################################################################################################################ ifconfig_igb0="up" ifconfig_igb1="up" ifconfig_igb2="up" ifconfig_igb3="up" ifconfig_igb4="up" ifconfig_igb5="up" ifconfig_igb4="inet xxx.xxx.0.254 netmask 255.255.255.0" ifconfig_igb4_alias0="vhid 1 advskew 100 pass secret xxx.xxx.0.1/32" # PF Sync Interface ifconfig_igb5="inet 192.168.255.254/30" # Create Virutal Interfaces cloned_interfaces="lagg0 vlan10 vlan25 vlan35 vlan90 vlan95 vlan97 vlan98 vlan99 vlan100 vlan101 vlan102 vlan103 vlan106 vlan107 vlan108 vlan109" # lagg Interface ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 laggport igb2 laggport igb3" # VLAN Interfaces ifconfig_vlanxx0="inet xxx.xxx.22.198/27" ifconfig_vlanxx0_alias0="inet vhid 2 advskew 100 pass secret xxx.xxx.22.199/32 vlan 10 vlandev lagg0" ifconfig_vlanxx5="inet xxx.xxx.25.254/24" ifconfig_vlanxx5_alias0="inet vhid 3 advskew 100 pass secret xxx.xxx.25.1/32 vlan 25 vlandev lagg0" ifconfig_vlanxx5="inet xxx.xxx.35.254/24" ifconfig_vlanxx5_alias0="inet vhid 4 advskew 100 pass secret xxx.xxx.35.1/32 vlan 35 vlandev lagg0" ifconfig_vlanxx0="inet xxx.xxx.7.254/21" ifconfig_vlanxx0_alias0="inet vhid 5 advskew 100 pass secret xxx.xxx.0.1/32 vlan 90 vlandev lagg0" ifconfig_vlanxx5="inet xxx.xxx.95.254/24" ifconfig_vlanxx5_alias0="inet vhid 6 advskew 100 pass secret xxx.xxx.95.1/32 vlan 95 vlandev lagg0" ifconfig_vlanxx7="inet xxx.xxx.90.254/16" ifconfig_vlanxx7_alias0="inet vhid 7 advskew 100 pass secret xxx.xxx.90.6/32 vlan 97 vlandev lagg0" ifconfig_vlanxx8="inet xxx.xxx.239.254/21" ifconfig_vlanxx8_alias0="inet vhid 8 advskew 100 pass secret xxx.xxx.232.1/32 vlan 98 vlandev lagg0" ifconfig_vlanxx9="inet xxx.xxx.0.30/27" ifconfig_vlanxx9_alias0="inet vhid 9 advskew 100 pass secret xxx.xxx.0.1/32 vlan 99 vlandev lagg0" ifconfig_vlanxx0="inet xxx.xxx.100.254/24" ifconfig_vlanxx0_alias0="inet vhid 10 advskew 100 pass secret xxx.xxx.100.1/32 vlan 100 vlandev lagg0" ifconfig_vlanxx1="inet xxx.xxx.1.254/24" ifconfig_vlanxx1_alias0="inet vhid 11 advskew 100 pass secret xxx.xxx.1.1/32 vlan 101 vlandev lagg0" ifconfig_vlanxx2="inet xxx.xxx.116.254/24" ifconfig_vlanxx2_alias0="inet vhid 12 advskew 100 pass secret xxx.xxx.116.1/32 vlan 102 vlandev lagg0" ifconfig_vlanxx3="inet xxx.xxx.10.254/24" ifconfig_vlanxx3_alias0="inet vhid 13 advskew 100 pass secret xxx.xxx.10.1/32 vlan 103 vlandev lagg0" ifconfig_vlanxx6="inet xxx.xxx.255.253/16" ifconfig_vlanxx6_alias0="inet vhid 14 advskew 100 pass secret xxx.xxx.255.254/32 vlan 106 vlandev lagg0" ifconfig_vlanxx7="inet xxx.xxx.90.254/16" ifconfig_vlanxx7_alias0="inet vhid 15 advskew 100 pass secret xxx.xxx.90.140/32 vlan 107 vlandev lagg0" ifconfig_vlanxx8="inet xxx.xxx.89.13/29" ifconfig_vlanxx8_alias0="inet vhid 16 advskew 100 pass secret xxx.xxx.89.14/32 vlan 108 vlandev lagg0" ifconfig_vlanxx9="inet xxx.xxx.110.254/24" ifconfig_vlanxx9_alias0="inet vhid 17 advskew 100 pass secret xxx.xxx.110.1/32 vlan 109 vlandev lagg0" igb4: flags=8943 metric 0 mtu 1500 options=403bb ether 00:1b:21:96:66:70 inet xxx.xxx.0.254 netmask 0xffffff00 broadcast xxx.xxx.0.255 inet xxx.xxx.0.1 netmask 0xffffffff broadcast xxx.xxx.0.1 vhid 1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active carp: MASTER vhid 1 advbase 1 advskew 100 vlanxx0: flags=8943 metric 0 mtu 1500 options=303 ether 00:1b:21:96:65:68 inet xxx.xxx.xxx.198 netmask 0xffffffe0 broadcast xxx.xxx.xxx.223 inet xxx.xxx.xxx.199 netmask 0xffffffff broadcast xxx.xxx.xxx.199 vhid 2 nd6 options=29 media: Ethernet autoselect status: active vlan: 10 parent interface: lagg0 carp: MASTER vhid 2 advbase 1 advskew 100 root@xxx2:/# sysctl net.inet.carp net.inet.carp.allow: 1 net.inet.carp.preempt: 1 net.inet.carp.log: 2 net.inet.carp.demotion: 720 net.inet.carp.senderr_demotion_factor: 240 net.inet.carp.ifdown_demotion_factor: 240 root@xxx2:/# ################################################################################################################################################################################ From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 15:15:01 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AD8BF3F for ; Fri, 2 Jan 2015 15:15:01 +0000 (UTC) Received: from mail-wi0-f173.google.com (mail-wi0-f173.google.com [209.85.212.173]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B254064F7D for ; Fri, 2 Jan 2015 15:15:00 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id r20so27888430wiv.6 for ; Fri, 02 Jan 2015 07:14:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=qStcWhnqMdtRSbmRT6SJ8sNhPQyga/HUrkZDqHCXh9s=; b=dqFmdPz4HktEsbEeTGPmTNaA/JKpwuf3TIeAyFSF7/x/br3zWLH6pWUjtJc1wp0AF2 tKLtfSDn3lpQpRvKixYb7Y3VrHKeboG/ruuoyMEVmWYXnj0uS+nHl10plmetKZCiCAfx YctWGFTXNZOaTMu0yTCO9FPCYnQ8hMeQiN50858dwv5TULxkF2zIUKUtrGZj7jLfaRQN gQe5M6f/5HWzDvqt0oWFTsA/lI8V1jlwTEXYxAgPFUJXRBUhFaXgzLdq8PcGdLnFMMQC 57dTTDtAZYxuC4R+wxF6vGG5WrrU2Szrf8PWIRsv1uxRkvBV1X4U7LIqv99L3H5HmBDd QTzA== X-Gm-Message-State: ALoCoQk7jMIsXHB67wuZzqWfoKx7NNKID+fMqd/nfS4fPn/0Z2GYEvN3qKTyflJYVwlan/1uyb4v X-Received: by 10.180.73.206 with SMTP id n14mr135989505wiv.60.1420211692084; Fri, 02 Jan 2015 07:14:52 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id b10sm54435905wiw.9.2015.01.02.07.14.50 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Jan 2015 07:14:51 -0800 (PST) Message-ID: <54A6B5E3.5090200@multiplay.co.uk> Date: Fri, 02 Jan 2015 15:14:43 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: CARP Problem/Bug? on 10.1-RELEASE References: <54A69F72.6060405@gmail.com> In-Reply-To: <54A69F72.6060405@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 15:15:01 -0000 Since 10 we're had to use the following settings in sysctl for carp to work as expected: net.inet.carp.preempt=1 net.inet.carp.senderr_demotion_factor=0 See if these help for your setup too? Regards Steve On 02/01/2015 13:38, Sascha wrote: > Hi and a happy new year to everyone! > > I have problems with my carp setup between two Routers/Firewalls after > upgrade from 10.0 RELEASE to 10.1 RELEASE. > Before the upgrade my setup worked without any problems! > > After the upgrade I checked the carp status. Machine 2 (Backup > machine) is in Master state for all Interfaces. Machine 1 remains in > Backup state. Machine 1 is my primary Machine and should be the master! > > I restarted both machines several times and checked the rc.conf for > errors. But Machine 2 gets every time the master. Machine 1 gets only > the master when I turn off Machine 2. > All interfaces remain in Master state even when I manually take a > Interface for example igb4 with ifconfig on Machine 2 down. Then igb4 > on Machine 1 goes into Master state.But the other interfaces remain in > Backup state on Machine 1. > > preemt Option is set to 1 > > The machines are connected with Link Aggregation to the switch and > they do VLAN Tagging. The CARP Interface is the Gateway for the clients. > The address on the physical interface is used to bind demons for > network services. > PF Sync is done over a cross cable direct attached to both machines. > PF is not blocking any carp traffic. > > > I Think the problem has something to do with the net.inet.carp.demotion > > You can see in my output that machine 1 had this value > net.inet.carp.demotion: 3840 > > Is this a bug or does someone see a configuration error in the config? > > > > Detailed config(Some output is cleared or omitted): > > rc.conf Machine 1: > > ################################################################################################################################################################################ > > ifconfig_igb0="up" > ifconfig_igb1="up" > ifconfig_igb2="up" > ifconfig_igb3="up" > ifconfig_igb4="up" > ifconfig_igb5="up" > > ifconfig_igb4="inet xxx.xxx.0.253 netmask 255.255.255.0" > ifconfig_igb4_alias0="vhid 1 pass secret xxx.xxx.0.1/32" > > # PF Sync Interface > ifconfig_igb5="inet xxx.xxx.255.253/30" > > # Create Virutal Interfaces > cloned_interfaces="lagg0 vlan10 vlan25 vlan35 vlan90 vlan95 vlan97 > vlan98 vlan99 vlan100 vlan101 vlan102 vlan103 vlan106 vlan107 vlan108 > vlan109" > # lagg Interface > ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 laggport > igb2 laggport igb3" > > # VLAN Interfaces > ifconfig_vlanxx0="inet xxx.xxxx.xxx.197/27" > ifconfig_vlanxx0alias0="inet vhid 2 pass secret xxx.xxx.xxx.199/32 > vlan xx0 vlandev lagg0" > ifconfig_vlanxx5="inet xxx.xxx.25.253/24" > ifconfig_vlanxx5_alias0="inet vhid 3 pass secret xxx.xxx.25.1/32 vlan > xx5 vlandev lagg0" > ifconfig_vlanxx5="inet xxx.xxx.35.253/24" > ifconfig_vlanxx5_alias0="inet vhid 4 pass secret xxx.35.1/32 vlan xx5 > vlandev lagg0" > ifconfig_vlanxx0="inet xxx.xxx.7.253/21" > ifconfig_vlanxx0_alias0="inet vhid 5 pass secret xxx.xxx.7.1/32 vlan > xx0 vlandev lagg0" > ifconfig_vlanxx5="inet xxx.xxx.95.253/24" > ifconfig_vlanxx5_alias0="inet vhid 6 pass secret xxx.xxx.95.1/32 vlan > xx5 vlandev lagg0" > ifconfig_vlanxx7="inet xxx.xxx.90.253/16" > ifconfig_vlanxx7_alias0="inet vhid 7 pass secret xxx.xxx.90.6/32 vlan > xx7 vlandev lagg0" > ifconfig_vlanxx8="inet xxx.xxx.239.253/21" > ifconfig_vlanxx8_alias0="inet vhid 8 pass secret xxx.xxx.232.1/32 vlan > xx8 vlandev lagg0" > ifconfig_vlanxx9="inet xxx.xxx.0.29/27" > ifconfig_vlanxx9_alias0="inet vhid 9 pass secret xxx.xxx.0.1/32 vlan > xx9 vlandev lagg0" > ifconfig_vlanxx0="inet xxx.xxx.100.253/24" > ifconfig_vlanxx0_alias0="inet vhid 10 pass secret xxx.xxx.100.1/32 > vlan xx0 vlandev lagg0" > ifconfig_vlanxx1="inet xxx.xxx.1.253/24" > ifconfig_vlanxx1_alias0="inet vhid 11 pass secret xxx.xxx.1.1/32 vlan > xx1 vlandev lagg0" > ifconfig_vlanxx2="inet xxx.xxx.116.253/24" > ifconfig_vlanxx2_alias0="inet vhid 12 pass secret xxx.xxx.116.1/32 > vlan xx2 vlandev lagg0" > ifconfig_vlanxx3="inet xxx.xxx.10.253/24" > ifconfig_vlanxx3_alias0="inet vhid 13 pass secret xxx.xxx.10.1/32 vlan > xx3 vlandev lagg0" > ifconfig_vlanxx6="inet xxx.xxx.255.252/16" > ifconfig_vlanxx6_alias0="inet vhid 14 pass secret xxx.xxx.255.254/32 > vlan xx6 vlandev lagg0" > ifconfig_vlanxx7="inet xxx.xxx.90.253/16" > ifconfig_vlanxx7_alias0="inet vhid 15 pass secret xxx.xxx.90.140/32 > vlan xx7 vlandev lagg0" > ifconfig_vlanxx8="inet xxx.xxx.89.12/29" > ifconfig_vlanxx8_alias0="inet vhid 16 pass secret xxx.xxx.89.14/32 > vlan xx8 vlandev lagg0" > ifconfig_vlanxx9="inet xxx.xxx.110.253/24" > ifconfig_vlanxx9_alias0="inet vhid 17 pass secret xxx.xxx.110.1/32 > vlan xx9 vlandev lagg0" > > igb4: flags=8943 > metric 0 mtu 1500 > options=403bb > > ether 00:1b:21:96:65:78 > inet xxx.xxx.0.253 netmask 0xffffff00 broadcast xxx.xxx.0.255 > inet xxx.xxx.0.1 netmask 0xffffffff broadcast xxx.xxxx.0.1 vhid 1 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > carp: BACKUP vhid 1 advbase 1 advskew 0 > > vlanxx0: flags=8943 > metric 0 mtu 1500 > options=303 > ether 00:1b:21:96:c0:38 > inet xxx.xxx.xxx.197 netmask 0xffffffe0 broadcast xxx.xxx.xxx.223 > inet xxx.xxx.xxx.199 netmask 0xffffffff broadcast > xxx.xxx.xxx.199 vhid 2 > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 10 parent interface: lagg0 > carp: BACKUP vhid 2 advbase 1 advskew 0 > > > > root@xxx1:/# sysctl net.inet.carp > net.inet.carp.allow: 1 > net.inet.carp.preempt: 1 > net.inet.carp.log: 2 > net.inet.carp.demotion: 3840 > net.inet.carp.senderr_demotion_factor: 240 > net.inet.carp.ifdown_demotion_factor: 240 > > > Log File when restarting machine 2 > > root@xxx1:/# less /var/log/messages > > Jan 2 12:29:11 xxx1 kernel: carp: VHID 15@vlanxx7: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 16@vlanxx8: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 17@vlanxx9: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 1@igb4: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 2@vlanxx0: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 3@vlanxx5: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 4@vlanxx5: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 5@vlanxx0: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 6@vlanxx5: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 7@vlanxx7: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 8@vlanxx8: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 10@vlanxx0: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 9@vlanxx9: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 11@vlanxx1: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 12@vlanxx2: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 13@vlanxx3: BACKUP -> MASTER > (master down) > Jan 2 12:29:11 xxx1 kernel: carp: VHID 14@vlanxx6: BACKUP -> MASTER > (master down) > Jan 2 12:29:12 xxx1 kernel: igb5: link state changed to DOWN > Jan 2 12:29:14 xxx1 kernel: igb5: link state changed to UP > Jan 2 12:29:14 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' > Jan 2 12:29:20 xxx1 kernel: igb5: link state changed to DOWN > Jan 2 12:29:22 xxx1 kernel: igb5: link state changed to UP > Jan 2 12:29:22 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' > Jan 2 12:29:23 xxx1 kernel: igb5: link state changed to DOWN > Jan 2 12:29:24 xxx1 kernel: igb5: link state changed to UP > Jan 2 12:29:24 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' > Jan 2 12:29:35 xxx1 kernel: igb5: link state changed to DOWN > Jan 2 12:29:37 xxx1 kernel: igb5: link state changed to UP > Jan 2 12:29:37 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' > Jan 2 12:31:06 xxx1 kernel: igb5: link state changed to DOWN > Jan 2 12:31:08 xxx1 kernel: igb5: link state changed to UP > Jan 2 12:31:08 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' > Jan 2 12:31:17 xxx1 kernel: igb5: link state changed to DOWN > Jan 2 12:31:19 xxx1 kernel: igb5: link state changed to UP > Jan 2 12:31:19 xxx1 devd: Executing '/etc/rc.d/dhclient quietstart igb5' > Jan 2 12:31:21 xxx1 kernel: carp: VHID 1@igb4: MASTER -> BACKUP (more > frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 2@vlanxx0: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 3@vlanxx5: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 5@vlanxx0: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 6@vlanxx5: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 9@vlanxx9: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 8@vlanxx8: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 7@vlanxx7: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 14@vlanxx6: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 10@vlanxx0: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 16@vlanxx8: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 11@vlanxx1: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 12@vlanxx2: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 15@vlanxx7: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 13@vlanxx3: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 17@vlanxx9: MASTER -> BACKUP > (more frequent advertisement received) > Jan 2 12:31:52 xxx1 kernel: carp: VHID 4@vlanxx5: MASTER -> BACKUP > (more frequent advertisement received) > > > TCP Dump on machine 1 during restarting machine 2 > > root@xxx1:/# sudo tcpdump -npi igb4 -T carp dst 224.0.0.18 > > 13:47:55.455580 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167172 > 13:47:56.881428 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167173 > 13:47:59.883366 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167174 > 13:48:00.894534 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167175 > . > . > 13:50:07.842383 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167240 > 13:50:09.782055 IP xxx.xxx.0.253 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167241 > 13:50:09.782914 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167242 > 13:50:11.206767 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167243 > 13:50:11.510203 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=240 authlen=7 counter=50029484353167244 > 13:50:12.450678 IP xxx.xxx.0.254 > 224.0.0.18: CARPv2-advertise 36: > vhid=1 advbase=1 advskew=100 authlen=7 counter=50029484353167245 > ################################################################################################################################################################################ > > > > rc.conf Machine 2: > > ################################################################################################################################################################################ > > ifconfig_igb0="up" > ifconfig_igb1="up" > ifconfig_igb2="up" > ifconfig_igb3="up" > ifconfig_igb4="up" > ifconfig_igb5="up" > > ifconfig_igb4="inet xxx.xxx.0.254 netmask 255.255.255.0" > ifconfig_igb4_alias0="vhid 1 advskew 100 pass secret xxx.xxx.0.1/32" > > # PF Sync Interface > ifconfig_igb5="inet 192.168.255.254/30" > > # Create Virutal Interfaces > cloned_interfaces="lagg0 vlan10 vlan25 vlan35 vlan90 vlan95 vlan97 > vlan98 vlan99 vlan100 vlan101 vlan102 vlan103 vlan106 vlan107 vlan108 > vlan109" > # lagg Interface > ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 laggport > igb2 laggport igb3" > > # VLAN Interfaces > ifconfig_vlanxx0="inet xxx.xxx.22.198/27" > ifconfig_vlanxx0_alias0="inet vhid 2 advskew 100 pass secret > xxx.xxx.22.199/32 vlan 10 vlandev lagg0" > ifconfig_vlanxx5="inet xxx.xxx.25.254/24" > ifconfig_vlanxx5_alias0="inet vhid 3 advskew 100 pass secret > xxx.xxx.25.1/32 vlan 25 vlandev lagg0" > ifconfig_vlanxx5="inet xxx.xxx.35.254/24" > ifconfig_vlanxx5_alias0="inet vhid 4 advskew 100 pass secret > xxx.xxx.35.1/32 vlan 35 vlandev lagg0" > ifconfig_vlanxx0="inet xxx.xxx.7.254/21" > ifconfig_vlanxx0_alias0="inet vhid 5 advskew 100 pass secret > xxx.xxx.0.1/32 vlan 90 vlandev lagg0" > ifconfig_vlanxx5="inet xxx.xxx.95.254/24" > ifconfig_vlanxx5_alias0="inet vhid 6 advskew 100 pass secret > xxx.xxx.95.1/32 vlan 95 vlandev lagg0" > ifconfig_vlanxx7="inet xxx.xxx.90.254/16" > ifconfig_vlanxx7_alias0="inet vhid 7 advskew 100 pass secret > xxx.xxx.90.6/32 vlan 97 vlandev lagg0" > ifconfig_vlanxx8="inet xxx.xxx.239.254/21" > ifconfig_vlanxx8_alias0="inet vhid 8 advskew 100 pass secret > xxx.xxx.232.1/32 vlan 98 vlandev lagg0" > ifconfig_vlanxx9="inet xxx.xxx.0.30/27" > ifconfig_vlanxx9_alias0="inet vhid 9 advskew 100 pass secret > xxx.xxx.0.1/32 vlan 99 vlandev lagg0" > ifconfig_vlanxx0="inet xxx.xxx.100.254/24" > ifconfig_vlanxx0_alias0="inet vhid 10 advskew 100 pass secret > xxx.xxx.100.1/32 vlan 100 vlandev lagg0" > ifconfig_vlanxx1="inet xxx.xxx.1.254/24" > ifconfig_vlanxx1_alias0="inet vhid 11 advskew 100 pass secret > xxx.xxx.1.1/32 vlan 101 vlandev lagg0" > ifconfig_vlanxx2="inet xxx.xxx.116.254/24" > ifconfig_vlanxx2_alias0="inet vhid 12 advskew 100 pass secret > xxx.xxx.116.1/32 vlan 102 vlandev lagg0" > ifconfig_vlanxx3="inet xxx.xxx.10.254/24" > ifconfig_vlanxx3_alias0="inet vhid 13 advskew 100 pass secret > xxx.xxx.10.1/32 vlan 103 vlandev lagg0" > ifconfig_vlanxx6="inet xxx.xxx.255.253/16" > ifconfig_vlanxx6_alias0="inet vhid 14 advskew 100 pass secret > xxx.xxx.255.254/32 vlan 106 vlandev lagg0" > ifconfig_vlanxx7="inet xxx.xxx.90.254/16" > ifconfig_vlanxx7_alias0="inet vhid 15 advskew 100 pass secret > xxx.xxx.90.140/32 vlan 107 vlandev lagg0" > ifconfig_vlanxx8="inet xxx.xxx.89.13/29" > ifconfig_vlanxx8_alias0="inet vhid 16 advskew 100 pass secret > xxx.xxx.89.14/32 vlan 108 vlandev lagg0" > ifconfig_vlanxx9="inet xxx.xxx.110.254/24" > ifconfig_vlanxx9_alias0="inet vhid 17 advskew 100 pass secret > xxx.xxx.110.1/32 vlan 109 vlandev lagg0" > > > > igb4: flags=8943 > metric 0 mtu 1500 > options=403bb > > ether 00:1b:21:96:66:70 > inet xxx.xxx.0.254 netmask 0xffffff00 broadcast xxx.xxx.0.255 > inet xxx.xxx.0.1 netmask 0xffffffff broadcast xxx.xxx.0.1 vhid 1 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > carp: MASTER vhid 1 advbase 1 advskew 100 > > vlanxx0: flags=8943 > metric 0 mtu 1500 > options=303 > ether 00:1b:21:96:65:68 > inet xxx.xxx.xxx.198 netmask 0xffffffe0 broadcast xxx.xxx.xxx.223 > inet xxx.xxx.xxx.199 netmask 0xffffffff broadcast > xxx.xxx.xxx.199 vhid 2 > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 10 parent interface: lagg0 > carp: MASTER vhid 2 advbase 1 advskew 100 > > > > > root@xxx2:/# sysctl net.inet.carp > net.inet.carp.allow: 1 > net.inet.carp.preempt: 1 > net.inet.carp.log: 2 > net.inet.carp.demotion: 720 > net.inet.carp.senderr_demotion_factor: 240 > net.inet.carp.ifdown_demotion_factor: 240 > root@xxx2:/# > ################################################################################################################################################################################ > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 15:37:14 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65F615DE for ; Fri, 2 Jan 2015 15:37:14 +0000 (UTC) Received: from forward-corp1m.cmail.yandex.net (forward-corp1m.cmail.yandex.net [IPv6:2a02:6b8:b030::69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15EDB662A6 for ; Fri, 2 Jan 2015 15:37:13 +0000 (UTC) Received: from smtpcorp1m.mail.yandex.net (smtpcorp1m.mail.yandex.net [77.88.61.150]) by forward-corp1m.cmail.yandex.net (Yandex) with ESMTP id 4BC726038E for ; Fri, 2 Jan 2015 18:36:59 +0300 (MSK) Received: from smtpcorp1m.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp1m.mail.yandex.net (Yandex) with ESMTP id 902652CA03DC for ; Fri, 2 Jan 2015 18:36:59 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp1m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id XsuR3qlBTU-axlCMapc; Fri, 2 Jan 2015 18:36:59 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1420213019; bh=VjQEz2vgS7q5uENInJ0wiAuVaPgHxhMCSduYboI2L78=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:Subject: Content-Type:Content-Transfer-Encoding; b=lAZKmF3nktdNxp8Sgcnq63H9EMsw0nuSNkdQfP7PP/lepfSBDbv9iqmSxTS6VxkMa DgpBSC8ZpPVst6O3fEIJNft9f7rwwEk0JozhOo+D4Ei44Lelr/wVVsCuBI3taS7YDV xxWoTzsbCZ4ip4ickH+PzhaX7XUI8rbSutjmMW1o= Authentication-Results: smtpcorp1m.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <54A6BAE0.9020404@yandex-team.ru> Date: Fri, 02 Jan 2015 18:36:00 +0300 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: host pipes and netmap 'emulated mode' Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 15:37:14 -0000 Hello list. It looks like it is impossible to use host pipes and emulated netmap mode in some cases. For example, if you're doing something like what traditional router do: packet processing, with kernel-visible logical interfaces, routing daemon running there, you can easily get a panic like this: #0 0xffffffff8094aa76 at kdb_backtrace+0x66 #1 0xffffffff809104ee at panic+0x1ce #2 0xffffffff80cf9660 at trap_fatal+0x290 #3 0xffffffff80cf99c1 at trap_pfault+0x211 #4 0xffffffff80cf9f89 at trap+0x329 #5 0xffffffff80ce30d3 at calltrap+0x8 #6 0xffffffff809d3b5f at ether_demux+0x6f #7 0xffffffff809d3f34 at ether_nh_input+0x204 #8 0xffffffff809dd6d8 at netisr_dispatch_src+0x218 #9 0xffffffff8061b2b5 at netmap_send_up+0x35 #10 0xffffffff8061b3d7 at netmap_txsync_to_host+0x97 #11 0xffffffff8061b400 at netmap_txsync_to_host_compat+0x10 #12 0xffffffff8061de8c at netmap_poll+0x2fc #13 0xffffffff807f2313 at devfs_poll_f+0x63 #14 0xffffffff8095ea3d at sys_poll+0x35d #15 0xffffffff80cf8e0a at amd64_syscall+0x5ea #16 0xffffffff80ce33b7 at Xfast_syscall+0xf7 Uptime: 4m21s The problem here is the following: netmap changes if_input() for the logical network interface and always assumes that generic_rx_handler() is called with netmap-enabled ifp (e.g. original inteface). Unfortunately, there are cases where we have different ifp passed to if_input handler. This particular case is triggered by (*ifp->if_input)(ifv->ifv_ifp, m); line, where "ifp" represents netmap-enabled NIC, and ifv->ifv_ifp represents vlan subinterface. Then, generic_rx_handler() tries to looking NA/GNA structure but fails since vlan subinterface is not netmap-enabled. So, it looks like that we need a way to call original if_input() but I can't imagine (good) one/ From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 15:59:02 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67699CBB; Fri, 2 Jan 2015 15:59:02 +0000 (UTC) Received: from forward-corp1f.mail.yandex.net (forward-corp1f.mail.yandex.net [IPv6:2a02:6b8:0:801::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 157D813A7; Fri, 2 Jan 2015 15:59:01 +0000 (UTC) Received: from smtpcorp1m.mail.yandex.net (smtpcorp1m.mail.yandex.net [77.88.61.150]) by forward-corp1f.mail.yandex.net (Yandex) with ESMTP id AA636242029C; Fri, 2 Jan 2015 18:58:49 +0300 (MSK) Received: from smtpcorp1m.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp1m.mail.yandex.net (Yandex) with ESMTP id 6CE452CA03DC; Fri, 2 Jan 2015 18:58:49 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp1m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tWK2hDgay6-wnl0L1AL; Fri, 2 Jan 2015 18:58:49 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1420214329; bh=g6MIVApD2dZCRAcj5TxNrANMDiH1ApDOZHQTqZVTnb4=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: Content-Type:Content-Transfer-Encoding; b=BKdsut6/mJmn8MY3W4HFXNOBpSVBO1smX4KcAxiq0kVinDISLSMZhBgYUca2xl9Hf PTwswJicun/CxlIW+HdrVq9o59BPvggoeQdmnugEQgG2NpmHrJULYFT77VABZlnBNy AWEloi5G9rGEzgb8OtPyEcn08b1WnQp2On7yCpUw= Authentication-Results: smtpcorp1m.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <54A6BFFE.5080103@yandex-team.ru> Date: Fri, 02 Jan 2015 18:57:50 +0300 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: cxgbe and netmap Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 15:59:02 -0000 Hello list! FreeBSD has netmap support for chelsio T5 cards, which is amazing. The great thing about implementation is that you can play with traffic-generating applications without affecting "main" OS interface, which has always been a problem for Intel cards. However, this approach (having additional netmap-only ifp) turns to be a bit problematic for netmap-based networking elements participating in routing. In Intel case you can configure all your interfaces, run routing daemon, run netmap application and punt all to-host traffic to kernel via host pipes. It looks like I can't do this using current implementation: mac addresses are different for main/netmap interfaces so I can't run routing daemon on main interface (or sub-interfaces). I also can't run routing daemon on top of ncxgbe* interface since it appears to ignore non-netmap-derived traffic.. Is it possible to make ncxgbe* interfaces behave more like ordinary ones? From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 16:41:25 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EF26185 for ; Fri, 2 Jan 2015 16:41:25 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 019901ACE for ; Fri, 2 Jan 2015 16:41:24 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 49A0A7300A; Fri, 2 Jan 2015 17:45:59 +0100 (CET) Date: Fri, 2 Jan 2015 17:45:59 +0100 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: host pipes and netmap 'emulated mode' Message-ID: <20150102164559.GA68836@onelab2.iet.unipi.it> References: <54A6BAE0.9020404@yandex-team.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54A6BAE0.9020404@yandex-team.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 16:41:25 -0000 On Fri, Jan 02, 2015 at 06:36:00PM +0300, Alexander V. Chernikov wrote: > Hello list. > > It looks like it is impossible to use host pipes and emulated netmap > mode in some cases. > > For example, if you're doing something like what traditional router do: > packet processing, with kernel-visible logical interfaces, routing > daemon running there, you can easily get a panic like this: > > #0 0xffffffff8094aa76 at kdb_backtrace+0x66 > #1 0xffffffff809104ee at panic+0x1ce > #2 0xffffffff80cf9660 at trap_fatal+0x290 > #3 0xffffffff80cf99c1 at trap_pfault+0x211 > #4 0xffffffff80cf9f89 at trap+0x329 > #5 0xffffffff80ce30d3 at calltrap+0x8 > #6 0xffffffff809d3b5f at ether_demux+0x6f > #7 0xffffffff809d3f34 at ether_nh_input+0x204 > #8 0xffffffff809dd6d8 at netisr_dispatch_src+0x218 > #9 0xffffffff8061b2b5 at netmap_send_up+0x35 > #10 0xffffffff8061b3d7 at netmap_txsync_to_host+0x97 > #11 0xffffffff8061b400 at netmap_txsync_to_host_compat+0x10 > #12 0xffffffff8061de8c at netmap_poll+0x2fc > #13 0xffffffff807f2313 at devfs_poll_f+0x63 > #14 0xffffffff8095ea3d at sys_poll+0x35d > #15 0xffffffff80cf8e0a at amd64_syscall+0x5ea > #16 0xffffffff80ce33b7 at Xfast_syscall+0xf7 > Uptime: 4m21s > > The problem here is the following: > netmap changes if_input() for the logical network interface and always > assumes that generic_rx_handler() is called with netmap-enabled ifp > (e.g. original inteface). > Unfortunately, there are cases where we have different ifp passed to > if_input handler. This particular case is triggered by > (*ifp->if_input)(ifv->ifv_ifp, m); > line, where "ifp" represents netmap-enabled NIC, and ifv->ifv_ifp > represents vlan subinterface. > > Then, generic_rx_handler() tries to looking NA/GNA structure but fails > since vlan subinterface is not netmap-enabled. > So, it looks like that we need a way to call original if_input() but I > can't imagine (good) one/ Surely we can put a check in generic_rx_handler() to make sure that NA(ifp) is NULL -- this is already a relatively expensive code path so the extra checks won't harm. But I am a bit unclear on how you trigger this error, can you give me more details ? The offending instruction (*ifp->if_input)(ifv->ifv_ifp, m) is in vlan_input(), so it looks like you are setting the parent interface in netmap mode, and (looking at the trace) sending packets to the host port. So i suppose the error path is when netmap_send_up() calls the original input handler, NA(ifp)->if_input [i should rename the field to something else] which is vlan_input(). I think a proper fix is to make vlan_input() netmap aware, and call NA(ifp)->if_input if the interface is in netmap mode. Otherwise, if vlan_input() is the only case where if_input() is called with a different ifp, _and_ the vlan (child) interface has a reference to the parent (ifp->parent, though i don't know where this is), we can tweak generic_rx_handler() so that it calls NA(ifp->parent)->if_input in case NA(ifp) is null. cheers luigi From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 16:48:53 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C80B4AB; Fri, 2 Jan 2015 16:48:53 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 003E31C36; Fri, 2 Jan 2015 16:48:52 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id DA6B27300A; Fri, 2 Jan 2015 17:53:34 +0100 (CET) Date: Fri, 2 Jan 2015 17:53:34 +0100 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: cxgbe and netmap Message-ID: <20150102165334.GB68836@onelab2.iet.unipi.it> References: <54A6BFFE.5080103@yandex-team.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54A6BFFE.5080103@yandex-team.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "freebsd-net@freebsd.org" , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 16:48:53 -0000 On Fri, Jan 02, 2015 at 06:57:50PM +0300, Alexander V. Chernikov wrote: > Hello list! > > FreeBSD has netmap support for chelsio T5 cards, which is amazing. > The great thing about implementation is that you can play with > traffic-generating applications without affecting "main" OS interface, > which has always been a problem for Intel cards. > However, this approach (having additional netmap-only ifp) turns to be a > bit problematic for netmap-based networking elements participating in > routing. > > In Intel case you can configure all your interfaces, run routing daemon, > run netmap application and punt all to-host traffic to kernel via host > pipes. for clarity, please call this "host netmap PORT", not pipe, as the latter can be confused with dummynet pipes. > It looks like I can't do this using current implementation: mac > addresses are different for main/netmap interfaces so I can't run > routing daemon on main interface (or sub-interfaces). > I also can't run routing daemon on top of ncxgbe* interface since it > appears to ignore non-netmap-derived traffic.. Maybe navdeep did not implement the host side for ncxgbe ? This should be a relatively trivial thing to do. Otherwise, for the time being, you could try the following hack: - create a tap interface, say tap*, and give it the same MAC as ncxgbe*. You will use only the host port for tap* - open both tap* and ncxgbe* in netmap mode, make sure to set the same flags, mode (promisc etc.), mtu on both; - run the routing daemon on top of tap* - use the tap*'s host netmap port to send up traffic coming from ncxgbe* directed to the host (and vice versa, inject netmap packets coming from tap*'s host netmap port into ncxgbe*'s netmap rings. cheers luigi > Is it possible to make ncxgbe* interfaces behave more like ordinary ones? > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 17:14:01 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6D72D96 for ; Fri, 2 Jan 2015 17:14:01 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A15CB660BD for ; Fri, 2 Jan 2015 17:14:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 1E4DCDFF7 for ; Fri, 2 Jan 2015 12:04:31 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -4.25 X-Spam-Level: X-Spam-Status: No, score=-4.25 required=5 tests=[ALL_TRUSTED=-1.8, AWL=0.149, BAYES_00=-2.599] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yCJopxi+oBcn for ; Fri, 2 Jan 2015 12:04:22 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 1FA3BE0BB for ; Fri, 2 Jan 2015 12:04:22 -0500 (EST) Message-ID: <54A6CFB6.7080103@astrodoggroup.com> Date: Fri, 02 Jan 2015 09:04:54 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: connect() returns EINVAL inappropriately Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 17:14:02 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 connect() returns EINVAL when called on TIMEWAIT or DROPPED sockets. Patch attached at: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196035 I'm not sure what exactly it *should* return, beyond "Not EINVAL". - --- Harrison Grundy From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 18:46:45 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34557325 for ; Fri, 2 Jan 2015 18:46:45 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0000B1DED for ; Fri, 2 Jan 2015 18:46:44 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id kq14so24132513pab.26 for ; Fri, 02 Jan 2015 10:46:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=Silk36QcxSx0j/twMK8Upjmt/s1Wd1ABKZJdlhxAg8E=; b=xvgG4WC7pnclWZKrHhw0h0qMRZz+RazlyALNYqJyQi9Gkn6O6y0krApiHfOsh0mQcj /MV8APc19ozlv89WDTHsB6TtBsMvQIxkxAFfl9Zb0YSZ4U0qpZIIbPx1PYLk6L4MLlxk MDkOvVpIsyLLmv+rRac6ZGGlR5I00VXUV8EOziZvHrIyvyJECC2G9FioCVEAJuQ/FIh+ G7v3A1DeMpJ0Q3DsYgk9bXGezACu/vwyPRC8CuoRB+44Pcg/sbHcgCMgdVBqo35Wn73c JNBvNc1Ah58qA5Y/FTCdhv+5uiuChc+iVE0GvFE7XI8j9Uiskh3NtIiykQ56hjnR1wy1 Cg4Q== X-Received: by 10.66.117.199 with SMTP id kg7mr85609792pab.92.1420224404458; Fri, 02 Jan 2015 10:46:44 -0800 (PST) Received: from ox ([24.6.44.228]) by mx.google.com with ESMTPSA id ig1sm47198495pbc.41.2015.01.02.10.46.42 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Fri, 02 Jan 2015 10:46:43 -0800 (PST) Sender: Navdeep Parhar Date: Fri, 2 Jan 2015 10:46:40 -0800 From: Navdeep Parhar To: "Alexander V. Chernikov" Subject: Re: cxgbe and netmap Message-ID: <20150102184640.GB28813@ox> Mail-Followup-To: "Alexander V. Chernikov" , "freebsd-net@freebsd.org" References: <54A6BFFE.5080103@yandex-team.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54A6BFFE.5080103@yandex-team.ru> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2015 18:46:45 -0000 On Fri, Jan 02, 2015 at 06:57:50PM +0300, Alexander V. Chernikov wrote: > Hello list! > > FreeBSD has netmap support for chelsio T5 cards, which is amazing. > The great thing about implementation is that you can play with > traffic-generating applications without affecting "main" OS interface, > which has always been a problem for Intel cards. > However, this approach (having additional netmap-only ifp) turns to be a > bit problematic for netmap-based networking elements participating in > routing. > > In Intel case you can configure all your interfaces, run routing daemon, > run netmap application and punt all to-host traffic to kernel via host > pipes. > It looks like I can't do this using current implementation: mac > addresses are different for main/netmap interfaces so I can't run > routing daemon on main interface (or sub-interfaces). > I also can't run routing daemon on top of ncxgbe* interface since it > appears to ignore non-netmap-derived traffic.. > > Is it possible to make ncxgbe* interfaces behave more like ordinary ones? > Yes, I need to write a simple transmit and receive handler for the non-netmap traffic on the ncxgbe/ncxl interfaces. This is a bit complicated because the normal rx runs in a mode where 1 rx buffer does not always equal 1 rx frame. Now that netmap is in GENERIC, it may be best to carve out a separate cxgbe_netmap module that can be loaded by those who want to use netmap on top of cxgbe/cxl hardware. So no more magic 'n' interfaces by default (some people were caught by surprise at the sudden appearance of the 'n' interfaces on HEAD), and fully functional 'n' interfaces as soon as you load the additional module. What do you and other netmap users think? I'm open to taking this driver's netmap support in whatever direction the users want it to go. Regards, Navdeep From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 12:02:04 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 440949B9; Sat, 3 Jan 2015 12:02:04 +0000 (UTC) Received: from forward-corp1f.mail.yandex.net (forward-corp1f.mail.yandex.net [IPv6:2a02:6b8:0:801::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E343919D1; Sat, 3 Jan 2015 12:02:03 +0000 (UTC) Received: from smtpcorp1m.mail.yandex.net (smtpcorp1m.mail.yandex.net [77.88.61.150]) by forward-corp1f.mail.yandex.net (Yandex) with ESMTP id 8D2A624202CE; Sat, 3 Jan 2015 15:01:59 +0300 (MSK) Received: from smtpcorp1m.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp1m.mail.yandex.net (Yandex) with ESMTP id 563EB2CA0302; Sat, 3 Jan 2015 15:01:59 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp1m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id UONxnCsg3v-1xlmDQb4; Sat, 3 Jan 2015 15:01:59 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1420286519; bh=OxbVapjiDQmM6Q3oAVUBfP5Pfwsr8/xK02PM0Y577qk=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=QCkfrhlpQIfjbVboKisNjsL3iKgbhduA076A1VNPiWEix9BSvP1+gdNat2yFsMOdT IUt2V0V0ClVc0dr4v3wXsKZWM01k+y9r01i3z/HsG3NO3VEzequJEspgF39asiKaaE icTCXtk9XGLfEJTOHLfqPoEWD9cPBqT4NgBQRCbc= Authentication-Results: smtpcorp1m.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <54A7D9F9.5080206@yandex-team.ru> Date: Sat, 03 Jan 2015 15:00:57 +0300 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: host pipes and netmap 'emulated mode' References: <54A6BAE0.9020404@yandex-team.ru> <20150102164559.GA68836@onelab2.iet.unipi.it> In-Reply-To: <20150102164559.GA68836@onelab2.iet.unipi.it> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 12:02:04 -0000 On 02.01.2015 19:45, Luigi Rizzo wrote: > On Fri, Jan 02, 2015 at 06:36:00PM +0300, Alexander V. Chernikov wrote: >> Hello list. >> >> It looks like it is impossible to use host pipes and emulated netmap >> mode in some cases. >> >> For example, if you're doing something like what traditional router do: >> packet processing, with kernel-visible logical interfaces, routing >> daemon running there, you can easily get a panic like this: >> >> #0 0xffffffff8094aa76 at kdb_backtrace+0x66 >> #1 0xffffffff809104ee at panic+0x1ce >> #2 0xffffffff80cf9660 at trap_fatal+0x290 >> #3 0xffffffff80cf99c1 at trap_pfault+0x211 >> #4 0xffffffff80cf9f89 at trap+0x329 >> #5 0xffffffff80ce30d3 at calltrap+0x8 >> #6 0xffffffff809d3b5f at ether_demux+0x6f >> #7 0xffffffff809d3f34 at ether_nh_input+0x204 >> #8 0xffffffff809dd6d8 at netisr_dispatch_src+0x218 >> #9 0xffffffff8061b2b5 at netmap_send_up+0x35 >> #10 0xffffffff8061b3d7 at netmap_txsync_to_host+0x97 >> #11 0xffffffff8061b400 at netmap_txsync_to_host_compat+0x10 >> #12 0xffffffff8061de8c at netmap_poll+0x2fc >> #13 0xffffffff807f2313 at devfs_poll_f+0x63 >> #14 0xffffffff8095ea3d at sys_poll+0x35d >> #15 0xffffffff80cf8e0a at amd64_syscall+0x5ea >> #16 0xffffffff80ce33b7 at Xfast_syscall+0xf7 >> Uptime: 4m21s >> >> The problem here is the following: >> netmap changes if_input() for the logical network interface and always >> assumes that generic_rx_handler() is called with netmap-enabled ifp >> (e.g. original inteface). >> Unfortunately, there are cases where we have different ifp passed to >> if_input handler. This particular case is triggered by >> (*ifp->if_input)(ifv->ifv_ifp, m); >> line, where "ifp" represents netmap-enabled NIC, and ifv->ifv_ifp >> represents vlan subinterface. >> >> Then, generic_rx_handler() tries to looking NA/GNA structure but fails >> since vlan subinterface is not netmap-enabled. >> So, it looks like that we need a way to call original if_input() but I >> can't imagine (good) one/ > Surely we can put a check in generic_rx_handler() to make sure that > NA(ifp) is NULL -- this is already a relatively expensive code > path so the extra checks won't harm. Yes, but it would be great if we can recover/call original input procedure instead of silently dropping frame > > But I am a bit unclear on how you trigger this error, > can you give me more details ? > > The offending instruction (*ifp->if_input)(ifv->ifv_ifp, m) > is in vlan_input(), so it looks like you are setting the parent > interface in netmap mode, and (looking at the trace) > sending packets to the host port. Yes. Basically, this is "netmap router case" - we configure vlan interface, bridge interfaces, etc using kernel interfaces, and propagate some (or all) configuration to netmap application. It somehow processes "fast path" traffic and sends control plane traffic to host via host pipe to handle routing daemon updates, icmp, fragments, etc.. > > So i suppose the error path is when netmap_send_up() calls > the original input handler, NA(ifp)->if_input [i should rename > the field to something else] which is vlan_input(). > > I think a proper fix is to make vlan_input() netmap aware, > and call NA(ifp)->if_input if the interface is in netmap mode. Well, than we will have to make bridge code netmap aware, netgraph and tunnel after that. It seems this is pretty invasive way of doing things. > > Otherwise, if vlan_input() is the only case where if_input() is called > with a different ifp, _and_ the vlan (child) interface has a reference > to the parent (ifp->parent, though i don't know where this is), > we can tweak generic_rx_handler() so that it calls > NA(ifp->parent)->if_input in case NA(ifp) is null. We will have to add different code for all type of virtual interfaces, which seems to be better approach. There is also an option like having per-vnet ifindex-based array with vlan (and other objects) tracking to get ifp quickly. What we really lacks here is set_if_input_func() method which can call some eventhandler so it can be done more generic way (so, adding glebius@ here) > > cheers > luigi From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 12:19:54 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB328C21; Sat, 3 Jan 2015 12:19:53 +0000 (UTC) Received: from forward-corp1m.cmail.yandex.net (forward-corp1m.cmail.yandex.net [5.255.216.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 651D71BBF; Sat, 3 Jan 2015 12:19:52 +0000 (UTC) Received: from smtpcorp1m.mail.yandex.net (smtpcorp1m.mail.yandex.net [77.88.61.150]) by forward-corp1m.cmail.yandex.net (Yandex) with ESMTP id DDB29603B4; Sat, 3 Jan 2015 15:19:47 +0300 (MSK) Received: from smtpcorp1m.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp1m.mail.yandex.net (Yandex) with ESMTP id 9ACFD2CA0302; Sat, 3 Jan 2015 15:19:47 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp1m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 2ebkHvFuso-JllO1SxE; Sat, 3 Jan 2015 15:19:47 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1420287587; bh=/mCKT7rNx+JrAPor6s1Jgk7r0EX400dcpyr0k1U8jR0=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=hcyDFzfn+sEo3qW8eGLr2glDcgOJ3fJw+F5MCRiMvq817u17lgAB/rcHms+P6PS4o vxbRYwi3ZSpVv613m3+bLerHGGaOHP7ff4kDJyab93gH34HHCXyNSTb9M/4O03REd+ dwQwvnHZuN5Cla0P5XYVMDnkJXHlH81MN0HqCV48= Authentication-Results: smtpcorp1m.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <54A7DE26.6020501@yandex-team.ru> Date: Sat, 03 Jan 2015 15:18:46 +0300 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: cxgbe and netmap References: <54A6BFFE.5080103@yandex-team.ru> <20150102165334.GB68836@onelab2.iet.unipi.it> In-Reply-To: <20150102165334.GB68836@onelab2.iet.unipi.it> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 12:19:54 -0000 On 02.01.2015 19:53, Luigi Rizzo wrote: > On Fri, Jan 02, 2015 at 06:57:50PM +0300, Alexander V. Chernikov wrote: >> Hello list! >> >> FreeBSD has netmap support for chelsio T5 cards, which is amazing. >> The great thing about implementation is that you can play with >> traffic-generating applications without affecting "main" OS interface, >> which has always been a problem for Intel cards. >> However, this approach (having additional netmap-only ifp) turns to be a >> bit problematic for netmap-based networking elements participating in >> routing. >> >> In Intel case you can configure all your interfaces, run routing daemon, >> run netmap application and punt all to-host traffic to kernel via host >> pipes. > for clarity, please call this "host netmap PORT", not pipe, as the > latter can be confused with dummynet pipes. Ok, understood. > >> It looks like I can't do this using current implementation: mac >> addresses are different for main/netmap interfaces so I can't run >> routing daemon on main interface (or sub-interfaces). >> I also can't run routing daemon on top of ncxgbe* interface since it >> appears to ignore non-netmap-derived traffic.. > Maybe navdeep did not implement the host side for ncxgbe ? > This should be a relatively trivial thing to do. > > Otherwise, for the time being, you could try the following hack: > > - create a tap interface, say tap*, and give it the same MAC as ncxgbe*. > You will use only the host port for tap* > > - open both tap* and ncxgbe* in netmap mode, make sure to set the same > flags, mode (promisc etc.), mtu on both; > > - run the routing daemon on top of tap* > > - use the tap*'s host netmap port to send up traffic coming from ncxgbe* > directed to the host (and vice versa, inject netmap packets coming > from tap*'s host netmap port into ncxgbe*'s netmap rings. Oh. vlans on top of tap inteface :) Ok, thanks for the suggestion, I'll try to do this. > > cheers > luigi > >> Is it possible to make ncxgbe* interfaces behave more like ordinary ones? >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 12:24:31 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02A0DE20 for ; Sat, 3 Jan 2015 12:24:31 +0000 (UTC) Received: from forward-corp1f.mail.yandex.net (forward-corp1f.mail.yandex.net [95.108.130.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0FCE64CCF for ; Sat, 3 Jan 2015 12:24:30 +0000 (UTC) Received: from smtpcorp1m.mail.yandex.net (smtpcorp1m.mail.yandex.net [77.88.61.150]) by forward-corp1f.mail.yandex.net (Yandex) with ESMTP id 62B3D24202D4 for ; Sat, 3 Jan 2015 15:24:26 +0300 (MSK) Received: from smtpcorp1m.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp1m.mail.yandex.net (Yandex) with ESMTP id 1DB392CA0302 for ; Sat, 3 Jan 2015 15:24:26 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp1m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id WgZbGMMq1k-OQlSEC5v; Sat, 3 Jan 2015 15:24:26 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1420287866; bh=BxXcP37dS0PGdTeu2fvVnF6bnwDi0WgE3fgkA0Ig5OM=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=udTt7OmURyvkgzbOumFACVm1FI4cuurjmNFgPPOfv2+UOuCIvG9FkXj3/oixx6bB6 sIgjzbA5rA2w1efPiWcUFzkJdpf/aV7TjSPv+mZLDvThh8MOWvVmrHYcXe4oYF/AP8 djwMP2sioTQlaxbJjoFgw4OpvMw/knu0G2U5NTq8= Authentication-Results: smtpcorp1m.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <54A7DF3C.7090905@yandex-team.ru> Date: Sat, 03 Jan 2015 15:23:24 +0300 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: Re: cxgbe and netmap References: <54A6BFFE.5080103@yandex-team.ru> <20150102184640.GB28813@ox> In-Reply-To: <20150102184640.GB28813@ox> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 12:24:31 -0000 On 02.01.2015 21:46, Navdeep Parhar wrote: > On Fri, Jan 02, 2015 at 06:57:50PM +0300, Alexander V. Chernikov wrote: >> Hello list! >> >> FreeBSD has netmap support for chelsio T5 cards, which is amazing. >> The great thing about implementation is that you can play with >> traffic-generating applications without affecting "main" OS interface, >> which has always been a problem for Intel cards. >> However, this approach (having additional netmap-only ifp) turns to be a >> bit problematic for netmap-based networking elements participating in >> routing. >> >> In Intel case you can configure all your interfaces, run routing daemon, >> run netmap application and punt all to-host traffic to kernel via host >> pipes. >> It looks like I can't do this using current implementation: mac >> addresses are different for main/netmap interfaces so I can't run >> routing daemon on main interface (or sub-interfaces). >> I also can't run routing daemon on top of ncxgbe* interface since it >> appears to ignore non-netmap-derived traffic.. >> >> Is it possible to make ncxgbe* interfaces behave more like ordinary ones? >> > Yes, I need to write a simple transmit and receive handler for the > non-netmap traffic on the ncxgbe/ncxl interfaces. This is a bit > complicated because the normal rx runs in a mode where 1 rx buffer does > not always equal 1 rx frame. > > Now that netmap is in GENERIC, it may be best to carve out a separate > cxgbe_netmap module that can be loaded by those who want to use netmap > on top of cxgbe/cxl hardware. So no more magic 'n' interfaces by > default (some people were caught by surprise at the sudden appearance of > the 'n' interfaces on HEAD), and fully functional 'n' interfaces as soon > as you load the additional module. > > What do you and other netmap users think? I'm open to taking this Having loadable netmap support would be great - this approach looks much more flexible. > driver's netmap support in whatever direction the users want it to go. > > Regards, > Navdeep From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 13:30:59 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BB4A1BC for ; Sat, 3 Jan 2015 13:30:59 +0000 (UTC) Received: from frv190.fwdcdn.com (frv190.fwdcdn.com [212.42.77.190]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DEB56667AC for ; Sat, 3 Jan 2015 13:30:58 +0000 (UTC) Received: from [10.10.2.23] (helo=frv198.fwdcdn.com) by frv190.fwdcdn.com with esmtp ID 1Y7OVc-0002f8-Il for freebsd-net@freebsd.org; Sat, 03 Jan 2015 15:12:48 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-Id:To:Subject:From:Date; bh=I5szegQDEAx3LQZz8HOPP1R0LVGaxVlBrH+WfD0ke9U=; b=odLekRYb0o7/0fZgQ6jvC46bhaPoskAp8J+avPel2SNJimgwqEDYJmThXglSgV+2Q0MUCgyA0/VDJS5mK55hEKfxRQZpezsAplb2AxLgg+C3hP8wlwiNP6cbNkbNvrYHEkxATWWADY0IbQzPdfJZJNcM8y5/NfJJNgaqb+wJx4E=; Received: from [10.10.10.34] (helo=frv34.fwdcdn.com) by frv198.fwdcdn.com with smtp ID 1Y7OVR-0002S4-4x for freebsd-net@freebsd.org; Sat, 03 Jan 2015 15:12:37 +0200 Date: Sat, 03 Jan 2015 15:12:36 +0200 From: wishmaster Subject: Issue with forwarding when creates new interface [was USB Tethering and forwarding] To: freebsd-net@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1420288398.485039365.so6mgquw@frv34.fwdcdn.com> References: <1419680989.938234917.k6otv1bh@frv34.fwdcdn.com> MIME-Version: 1.0 Received: from artemrts@ukr.net by frv34.fwdcdn.com; Sat, 03 Jan 2015 15:12:36 +0200 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 13:30:59 -0000 Hi, I have been seeing strange behavior of my system lately. After creating new interface the system variable net.inet.ip.forwarding becomes "0". E.g. manually load if_ral kernel module, then rel0 interface appears and net.inet.ip.forwarding becomes "0". Previously this happened when I attached smartphone with USB tethering is on. May be this is VIMAGE-related... Any ideas? Below my original first post. > Hi, list. > > Server works as router for small network and some services in the jails. When I connect Android-based smartphone and attempt to use USB Tethering, the net.inet.ip.forwarding becomes 0 and I must change it to 1 every time. > > Is this normal behavior? > > FreeBSD server.local 10.1-STABLE FreeBSD 10.1-STABLE #1 r275636: Mon Dec 22 11:05:33 EET 2014 wishmaster@server.local:/usr/obj/usr/src/sys/SMS i386 > > Kernel has been compiled with VIMAGE > > > Cheers, > Vitaliy > From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 14:18:29 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 691E4EE4 for ; Sat, 3 Jan 2015 14:18:29 +0000 (UTC) Received: from smtp2.mail.clearhost.co.uk (smtp2.mail.clearhost.co.uk [IPv6:2001:1420::25:102]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.mail.clearhost.co.uk", Issuer "RapidSSL CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F3091E0E for ; Sat, 3 Jan 2015 14:18:29 +0000 (UTC) Received: from [2001:1420:a:105:c62c:3ff:fe2f:bf] (port=60607 helo=parsnip.heronsbrook.org.uk) by smtp2.mail.clearhost.co.uk with esmtpa (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Y7PX8-000Mqj-2t for freebsd-net@freebsd.org; Sat, 03 Jan 2015 14:18:26 +0000 Message-ID: <54A7FA6C.8030603@prt.org> Date: Sat, 03 Jan 2015 14:19:24 +0000 From: Paul Thornton User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Issue with forwarding when creates new interface [was USB Tethering and forwarding] References: <1419680989.938234917.k6otv1bh@frv34.fwdcdn.com> <1420288398.485039365.so6mgquw@frv34.fwdcdn.com> In-Reply-To: <1420288398.485039365.so6mgquw@frv34.fwdcdn.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 14:18:29 -0000 Hi, I can also replicate this behaviour on 10.1-RELEASE by simply creating an additional vlan interface. It affects IPv4 and IPv6 forwarding. This is taken from a test setup of FreeBSD boxes running Quagga as BGP routers - but with a default GENERIC kernel. This machine has 2x ixgbe, 4x igb and 2x bce physical interfaces, with a cloned lo1 and vlan0. [root@test1 ~]# uname -a FreeBSD test1.prtsystems.ltd.uk 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 1 [root@test1 ~]# ifconfig vlan1 create [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 0 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 0 I haven't tried using 10.0 as a router, so don't know if this crept in between 10.0 and 10.1 or 9 and 10. Paul. On 03/01/2015 13:12, wishmaster wrote: > > Hi, > > I have been seeing strange behavior of my system lately. After creating new interface the system variable net.inet.ip.forwarding becomes "0". > > E.g. manually load if_ral kernel module, then rel0 interface appears and net.inet.ip.forwarding becomes "0". > > Previously this happened when I attached smartphone with USB tethering is on. > May be this is VIMAGE-related... Any ideas? > > Below my original first post. > >> Hi, list. >> >> Server works as router for small network and some services in the jails. When I connect Android-based smartphone and attempt to use USB Tethering, the net.inet.ip.forwarding becomes 0 and I must change it to 1 every time. >> >> Is this normal behavior? >> >> FreeBSD server.local 10.1-STABLE FreeBSD 10.1-STABLE #1 r275636: Mon Dec 22 11:05:33 EET 2014 wishmaster@server.local:/usr/obj/usr/src/sys/SMS i386 >> >> Kernel has been compiled with VIMAGE >> >> >> Cheers, >> Vitaliy >> From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 16:51:38 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E906E105 for ; Sat, 3 Jan 2015 16:51:38 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A6F641586 for ; Sat, 3 Jan 2015 16:51:38 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Y7Rkh-0006hO-Pi for freebsd-net@freebsd.org; Sat, 03 Jan 2015 20:40:35 +0400 Date: Sat, 3 Jan 2015 20:40:35 +0400 From: Slawa Olhovchenkov To: freebsd-net@freebsd.org Subject: netmap pipes Message-ID: <20150103164035.GC49169@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 16:51:39 -0000 Can anybody explain netmap pipes (more then netmap(4))? How use it? How it works? pipes works over existing network adapter in netmap mode? Or indepened (can I create netmap pipe named 'some_strange_name')? What purpose of master and slave? From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 17:17:02 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AC05472 for ; Sat, 3 Jan 2015 17:17:02 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id F28DF64972 for ; Sat, 3 Jan 2015 17:17:01 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 254D47300A; Sat, 3 Jan 2015 18:21:39 +0100 (CET) Date: Sat, 3 Jan 2015 18:21:39 +0100 From: Luigi Rizzo To: Slawa Olhovchenkov Subject: Re: netmap pipes Message-ID: <20150103172139.GB95134@onelab2.iet.unipi.it> References: <20150103164035.GC49169@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150103164035.GC49169@zxy.spb.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 17:17:02 -0000 On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote: > Can anybody explain netmap pipes (more then netmap(4))? > How use it? > How it works? > pipes works over existing network adapter in netmap mode? > Or indepened (can I create netmap pipe named 'some_strange_name')? > What purpose of master and slave? think of pipes as regular netmap ports connected back to back, (or as two ports on a VALE switch, if you like). They are unrelated to network devices (though the name indicates how they share memory, but forget that for the time being); are created using the "strange names" like valeX:Y{0 and valeX:Y}0 (which I realize are not yet in the manpage, sorry); and you have master and slave because need to name both endpoints. cheers luigi From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 17:32:58 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36419B27 for ; Sat, 3 Jan 2015 17:32:58 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E4F4C64C85 for ; Sat, 3 Jan 2015 17:32:57 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Y7SZL-0007bu-Qv; Sat, 03 Jan 2015 21:32:55 +0400 Date: Sat, 3 Jan 2015 21:32:55 +0400 From: Slawa Olhovchenkov To: Luigi Rizzo Subject: Re: netmap pipes Message-ID: <20150103173255.GD49169@zxy.spb.ru> References: <20150103164035.GC49169@zxy.spb.ru> <20150103172139.GB95134@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150103172139.GB95134@onelab2.iet.unipi.it> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 17:32:58 -0000 On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote: > On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote: > > Can anybody explain netmap pipes (more then netmap(4))? > > How use it? > > How it works? > > pipes works over existing network adapter in netmap mode? > > Or indepened (can I create netmap pipe named 'some_strange_name')? > > What purpose of master and slave? > > think of pipes as regular netmap ports connected back to back, > (or as two ports on a VALE switch, if you like). > They are unrelated to network devices (though the name indicates > how they share memory, but forget that for the time being); > are created using the "strange names" like valeX:Y{0 and valeX:Y}0 > (which I realize are not yet in the manpage, sorry); can I use names other then valeX:Y? for example 'inside0'? > and you have master and slave because need to name both endpoints. I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for example), how I can use this? Writing to master replicated to all slaves? Writing to any slaves reading from master? Or unidirected? From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 17:37:24 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABCB6CFF for ; Sat, 3 Jan 2015 17:37:24 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 6FCCC64D0A for ; Sat, 3 Jan 2015 17:37:24 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id B7DF37300A; Sat, 3 Jan 2015 18:42:07 +0100 (CET) Date: Sat, 3 Jan 2015 18:42:07 +0100 From: Luigi Rizzo To: Slawa Olhovchenkov Subject: Re: netmap pipes Message-ID: <20150103174207.GC95134@onelab2.iet.unipi.it> References: <20150103164035.GC49169@zxy.spb.ru> <20150103172139.GB95134@onelab2.iet.unipi.it> <20150103173255.GD49169@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150103173255.GD49169@zxy.spb.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 17:37:24 -0000 On Sat, Jan 03, 2015 at 09:32:55PM +0400, Slawa Olhovchenkov wrote: > On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote: > > > On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote: > > > Can anybody explain netmap pipes (more then netmap(4))? > > > How use it? > > > How it works? > > > pipes works over existing network adapter in netmap mode? > > > Or indepened (can I create netmap pipe named 'some_strange_name')? > > > What purpose of master and slave? > > > > think of pipes as regular netmap ports connected back to back, > > (or as two ports on a VALE switch, if you like). > > They are unrelated to network devices (though the name indicates > > how they share memory, but forget that for the time being); > > are created using the "strange names" like valeX:Y{0 and valeX:Y}0 > > (which I realize are not yet in the manpage, sorry); > > can I use names other then valeX:Y? for example 'inside0'? no, the basename has to be netmap:fooX (where foo is some existing ethernet device) or valeX:Y > > and you have master and slave because need to name both endpoints. > > I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for > example), how I can use this? Writing to master replicated to all > slaves? Writing to any slaves reading from master? Or unidirected? a pipe has only two endpoints. If you want a full n-port switch use a VALE switch. cheers luigi From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 17:57:24 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94B515EE for ; Sat, 3 Jan 2015 17:57:24 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E693208C for ; Sat, 3 Jan 2015 17:57:24 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Y7Swz-00083T-BS; Sat, 03 Jan 2015 21:57:21 +0400 Date: Sat, 3 Jan 2015 21:57:21 +0400 From: Slawa Olhovchenkov To: Luigi Rizzo Subject: Re: netmap pipes Message-ID: <20150103175721.GE49169@zxy.spb.ru> References: <20150103164035.GC49169@zxy.spb.ru> <20150103172139.GB95134@onelab2.iet.unipi.it> <20150103173255.GD49169@zxy.spb.ru> <20150103174207.GC95134@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150103174207.GC95134@onelab2.iet.unipi.it> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 17:57:24 -0000 On Sat, Jan 03, 2015 at 06:42:07PM +0100, Luigi Rizzo wrote: > On Sat, Jan 03, 2015 at 09:32:55PM +0400, Slawa Olhovchenkov wrote: > > On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote: > > > > > On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote: > > > > Can anybody explain netmap pipes (more then netmap(4))? > > > > How use it? > > > > How it works? > > > > pipes works over existing network adapter in netmap mode? > > > > Or indepened (can I create netmap pipe named 'some_strange_name')? > > > > What purpose of master and slave? > > > > > > think of pipes as regular netmap ports connected back to back, > > > (or as two ports on a VALE switch, if you like). > > > They are unrelated to network devices (though the name indicates > > > how they share memory, but forget that for the time being); > > > are created using the "strange names" like valeX:Y{0 and valeX:Y}0 > > > (which I realize are not yet in the manpage, sorry); > > > > can I use names other then valeX:Y? for example 'inside0'? > > no, the basename has to be netmap:fooX (where foo is some existing > ethernet device) or valeX:Y Can I transfer through netmap pipes packet with non-ethernet structure? > > > and you have master and slave because need to name both endpoints. > > > > I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for > > example), how I can use this? Writing to master replicated to all > > slaves? Writing to any slaves reading from master? Or unidirected? > > a pipe has only two endpoints. What differens between master and slave? Can I write to slave and read from master? Can I first create slave second master? > If you want a full n-port switch use a VALE switch. Hmm. NIOCREGIF binds the port named in nr_name to the file descriptor. For a phys- ical device this also switches it into netmap mode, disconnecting it from the host stack. Multiple file descriptors can be bound to the same port, with proper synchronization left to the user. NIOCREGIF can also bind a file descriptor to one endpoint of a netmap pipe, consisting of two netmap ports with a crossover con- nection. A netmap pipe share the same memory space of the parent port, and is meant to enable configuration where a master process acts as a dispatcher towards slave processes. To enable this function, the nr_arg1 field of the structure can be used as a hint to the kernel to indicate how many pipes we expect to use, and reserve extra space in the memory region. What this talk about? I can create multiple pipe pairs over existing netmap interface? What right way to use this? NIOCREGIF(ix0, NR_REG_ONE_NIC, nr_arg1=16, nr_ringid=0) NIOCREGIF(ix0, NR_REG_PIPE_MASTER, nr_ringid=0..15) NIOCREGIF(ix0, NR_REG_PIPE_SLAVE, nr_ringid=0..15) From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 18:07:10 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC871816 for ; Sat, 3 Jan 2015 18:07:10 +0000 (UTC) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A5A3B21D3 for ; Sat, 3 Jan 2015 18:07:10 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.9/8.14.9) with ESMTP id t03I76q2035438; Sat, 3 Jan 2015 13:07:07 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <54A82FA0.3090704@sentex.net> Date: Sat, 03 Jan 2015 13:06:24 -0500 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Paul Thornton , freebsd-net@freebsd.org Subject: Re: Issue with forwarding when creates new interface [was USB Tethering and forwarding] References: <1419680989.938234917.k6otv1bh@frv34.fwdcdn.com> <1420288398.485039365.so6mgquw@frv34.fwdcdn.com> <54A7FA6C.8030603@prt.org> In-Reply-To: <54A7FA6C.8030603@prt.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.75 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 18:07:10 -0000 On 1/3/2015 9:19 AM, Paul Thornton wrote: > Hi, > > I can also replicate this behaviour on 10.1-RELEASE by simply creating > an additional vlan interface. It affects IPv4 and IPv6 forwarding. Strange, I dont see that on RELENG_10 0{marble}# ifconfig em2 up 0{marble}# ifconfig em2.3 create 1.1.1.2/24 0{marble}# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 0 net.inet6.ip6.forwarding: 0 0{marble}# ifconfig vlan4 create 2.2.2.2 vlan 4 vlandev em2 0{marble}# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 0 net.inet6.ip6.forwarding: 0 0{marble}# do you set forwarding via just /etc/sysctl.conf or in /etc/rc.conf via ipv6_gateway_enable and gateway_enable. I seem to recall some discussion about there being a difference. Perhaps devd is calling something that then fiddles with the setting ignoring whats in sysctl.conf ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 18:08:21 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FF208A9 for ; Sat, 3 Jan 2015 18:08:21 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 07E4321F1 for ; Sat, 3 Jan 2015 18:08:21 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 386497300A; Sat, 3 Jan 2015 19:13:04 +0100 (CET) Date: Sat, 3 Jan 2015 19:13:04 +0100 From: Luigi Rizzo To: Slawa Olhovchenkov Subject: Re: netmap pipes Message-ID: <20150103181304.GE95134@onelab2.iet.unipi.it> References: <20150103164035.GC49169@zxy.spb.ru> <20150103172139.GB95134@onelab2.iet.unipi.it> <20150103173255.GD49169@zxy.spb.ru> <20150103174207.GC95134@onelab2.iet.unipi.it> <20150103175721.GE49169@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150103175721.GE49169@zxy.spb.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 18:08:21 -0000 On Sat, Jan 03, 2015 at 09:57:21PM +0400, Slawa Olhovchenkov wrote: > On Sat, Jan 03, 2015 at 06:42:07PM +0100, Luigi Rizzo wrote: > > > On Sat, Jan 03, 2015 at 09:32:55PM +0400, Slawa Olhovchenkov wrote: > > > On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote: > > > > > > > On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote: > > > > > Can anybody explain netmap pipes (more then netmap(4))? > > > > > How use it? > > > > > How it works? > > > > > pipes works over existing network adapter in netmap mode? > > > > > Or indepened (can I create netmap pipe named 'some_strange_name')? > > > > > What purpose of master and slave? > > > > > > > > think of pipes as regular netmap ports connected back to back, > > > > (or as two ports on a VALE switch, if you like). > > > > They are unrelated to network devices (though the name indicates > > > > how they share memory, but forget that for the time being); > > > > are created using the "strange names" like valeX:Y{0 and valeX:Y}0 > > > > (which I realize are not yet in the manpage, sorry); > > > > > > can I use names other then valeX:Y? for example 'inside0'? > > > > no, the basename has to be netmap:fooX (where foo is some existing > > ethernet device) or valeX:Y > > Can I transfer through netmap pipes packet with non-ethernet structure? yes. A pipe will not look at the packet's headers or content. > > > > and you have master and slave because need to name both endpoints. > > > > > > I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for > > > example), how I can use this? Writing to master replicated to all > > > slaves? Writing to any slaves reading from master? Or unidirected? > > > > a pipe has only two endpoints. > > What differens between master and slave? there is no difference between master and slave, you just need two names and a way to relate them. > Can I write to slave and read from master? pipes are bidirectional and blocking. So you can write on one and read from the other, in any order. > Can I first create slave second master? You can create the endpoints in any order (internally, they are both created at the same time). > > If you want a full n-port switch use a VALE switch. > > Hmm. > NIOCREGIF > binds the port named in nr_name to the file descriptor. For a phys- > ical device this also switches it into netmap mode, disconnecting > it from the host stack. Multiple file descriptors can be bound to > the same port, with proper synchronization left to the user. > > NIOCREGIF can also bind a file descriptor to one endpoint of a > netmap pipe, consisting of two netmap ports with a crossover con- > nection. A netmap pipe share the same memory space of the parent > port, and is meant to enable configuration where a master process > acts as a dispatcher towards slave processes. > > To enable this function, the nr_arg1 field of the structure can be > used as a hint to the kernel to indicate how many pipes we expect > to use, and reserve extra space in the memory region. > > What this talk about? This is a 'power user' feature which maybe is not what you need (and at the moment I don't have time to explain in more detail or update the manpage). Pipes share memory with the netmap port (VALE port or NIC) with the same basename, and since memory allocation occurs at once, on the first open you need to tell the OS how many pipes need to share memory with the same port -- that is the role of nr_arg1. cheers luigi From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 18:57:11 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1614ABD3 for ; Sat, 3 Jan 2015 18:57:11 +0000 (UTC) Received: from smtp2.mail.clearhost.co.uk (smtp2.mail.clearhost.co.uk [IPv6:2001:1420::25:102]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.mail.clearhost.co.uk", Issuer "RapidSSL CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D258338B2 for ; Sat, 3 Jan 2015 18:57:10 +0000 (UTC) Received: from [2001:1420:a:105:c62c:3ff:fe2f:bf] (port=62333 helo=parsnip.heronsbrook.org.uk) by smtp2.mail.clearhost.co.uk with esmtpa (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Y7Tso-0000PI-Dm; Sat, 03 Jan 2015 18:57:06 +0000 Message-ID: <54A83BBE.8000700@prt.org> Date: Sat, 03 Jan 2015 18:58:06 +0000 From: Paul Thornton User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Mike Tancsa , freebsd-net@freebsd.org Subject: Re: Issue with forwarding when creates new interface [was USB Tethering and forwarding] References: <1419680989.938234917.k6otv1bh@frv34.fwdcdn.com> <1420288398.485039365.so6mgquw@frv34.fwdcdn.com> <54A7FA6C.8030603@prt.org> <54A82FA0.3090704@sentex.net> In-Reply-To: <54A82FA0.3090704@sentex.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 18:57:11 -0000 Hi, On 03/01/2015 18:06, Mike Tancsa wrote: > do you set forwarding via just /etc/sysctl.conf or in /etc/rc.conf via > ipv6_gateway_enable and gateway_enable. I seem to recall some discussion > about there being a difference. Perhaps devd is calling something that > then fiddles with the setting ignoring whats in sysctl.conf ? That seems to be what is happening. In the earlier post, I was just setting the three sysctls in /etc/sysctl.conf - and observed that forwarding went away if an interface was added. Doing it by setting fastforwarding only in sysctl.conf, and setting both gateway_enables to yes in rc.conf fixes the problem: [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 1 [root@test1 ~]# ifconfig vlan1 create [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 1 That's quite ... odd, to sat the least. I can't see anything in devd.conf which would relate to a new interface being created, but that doesn't mean that there isn't some magic functionality in there. Paul.