From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 02:22:49 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 683AE6C; Sun, 10 Mar 2013 02:22:49 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 2A5CB70C; Sun, 10 Mar 2013 02:22:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2A2MnVL011103; Sun, 10 Mar 2013 02:22:49 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2A2MmNJ011102; Sun, 10 Mar 2013 02:22:48 GMT (envelope-from linimon) Date: Sun, 10 Mar 2013 02:22:48 GMT Message-Id: <201303100222.r2A2MmNJ011102@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/176771: [libnetgraph] [patch] user-mode netgraph node hangs when replying to control message X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 02:22:49 -0000 Old Synopsis: user-mode netgraph node hangs when replying to control message New Synopsis: [libnetgraph] [patch] user-mode netgraph node hangs when replying to control message Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 10 02:21:57 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=176771 From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 02:22:57 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8533CDD for ; Sun, 10 Mar 2013 02:22:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 2409970E for ; Sun, 10 Mar 2013 02:22:56 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAMTsO1GDaFvO/2dsb2JhbABCDogavCSBdHSCLAEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHASHbAYMqiKRboEjjCoKAQV9NAeCLYETA4hyiyWCPoEej1eCS10eMnwBCBce X-IronPort-AV: E=Sophos;i="4.84,816,1355115600"; d="scan'208";a="20300428" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 09 Mar 2013 21:22:49 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id BC572B3EEA; Sat, 9 Mar 2013 21:22:49 -0500 (EST) Date: Sat, 9 Mar 2013 21:22:49 -0500 (EST) From: Rick Macklem To: Garrett Wollman Message-ID: <1841214504.3736248.1362882169721.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201303091846.r29Ik9jX062596@hergotha.csail.mit.edu> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 02:22:57 -0000 Garett Wollman wrote: > In article <20795.29370.194678.963351@hergotha.csail.mit.edu>, I > wrote: > >< > said: > >> I've thought about this. My concern is that the separate thread > >> might > >> not keep up with the trimming demand. If that occurred, the cache > >> would > >> grow veryyy laarrggge, with effects like running out of mbuf > >> clusters. > > > >At a minimum, once one nfsd thread is committed to doing the cache > >trim, a flag should be set to discourage other threads from trying to > >do it. Having them all spinning their wheels punishes the clients > >much too much. > Yes, I think this is a good idea. The current code acquires the mutex before updating the once/sec variable. As such it would be easy to get multiple threads in there concurrently. This is easy to do. Just define a static variable in nfsrc_trimcache() initially 0. If not 0, return. set non-zero. do the trimming. set zero before returning. Since this is just a heuristic to avoid multiple threads doing the trim concurrently, I think it can be safely done outside of the mutex. If you need help coding this, just email and I can come up with a quick patch. > Also, it occurs to me that this strategy is subject to livelock. To > put backpressure on the clients, it is far better to get them to stop > sending (by advertising a small receive window) than to accept their > traffic but queue it for a long time. By the time the NFS code gets > an RPC, the system has already invested so much into it that it should > be processed as quickly as possible, and this strategy essentially > guarantees[1] that, once those 2 MB socket buffers start to fill up, > they > will stay filled, sending latency through the roof. If nfsd didn't > override the usual socket-buffer sizing mechanisms, then sysadmins > could limit the buffers to ensure a stable response time. > > The bandwidth-delay product in our network is somewhere between 12.5 > kB and 125 kB, depending on how the client is connected and what sort > of latency they experience. The usual theory would suggest that > socket buffers should be no more than twice that -- i.e., about 256 > kB. > Well, the code that uses sb_max_adj wasn't written by me (I just cloned it for the new server). In the author's defence, I believe SB_MAX was 256K when it was written. It was 256K in 2011. I think sb_max_adj was used because soreserve() fails for a larger value and the code doesn't check for such a failure. (Yea, it should be fixed so that it checks for a failure return from soreserve(). I did so for the client some time ago.;-) Just grep for sb_max_adj. You'll see it sets a variable called "siz". Make "siz" whatever you want (256K sounds like a good guess). Just make sure it isn't > sb_max_adj. The I/O sizes are limited to MAXBSIZE, which is currently 64Kb, although I'd like to increase that to 128Kb someday soon. (As you note below, the largest RPC is slightly bigger than that.) Btw, net.inet.tcp.{send/recv}buf_max are both 2Mbytes, just like sb_max, so those don't seem useful in this case? I'm no TCP guy, so suggestions w.r.t. how big soreserve() should be set are welcome. > I'd actually like to see something like WFQ in the NFS server to allow > me to limit the amount of damage one client or group of clients can > do without unnecessarily limiting other clients. > Sorry, I'll admit I have no idea what WFQ is? (I'll look it up on some web site someday soon, but obviously can't comment before then.) Since it is possible to receive RPC requests for a given client from multiple IP addresses, it is pretty hard for NFS to know what client a request has come from. rick > -GAWollman > > [1] The largest RPC is a bit more than 64 KiB (negotiated), so if the > server gets slow, the 2 MB receive queue will be refilled by the > client before the server manages to perform the RPC and send a > response. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 02:42:47 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 209E187A; Sun, 10 Mar 2013 02:42:47 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id B9EF47B5; Sun, 10 Mar 2013 02:42:46 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEACryO1GDaFvO/2dsb2JhbABCiCi8JIF0dIIsAQEBAwEBAQEgBCcgCxsYAgINGQIpAQkmBggHBAEcBIdsBgyqI5FvgSOMOn00B4ItgRMDiHKLJYI+gR6PV4MoHjKBBTU X-IronPort-AV: E=Sophos;i="4.84,816,1355115600"; d="scan'208";a="20301161" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 09 Mar 2013 21:42:45 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B9596B3F46; Sat, 9 Mar 2013 21:42:45 -0500 (EST) Date: Sat, 9 Mar 2013 21:42:45 -0500 (EST) From: Rick Macklem To: Garrett Wollman Message-ID: <663916089.3736429.1362883365710.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20795.30884.330015.123616@hergotha.csail.mit.edu> Subject: Re: NFS DRC size MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 02:42:47 -0000 Garrett Wollman wrote: > < said: > > > around the highwater mark basically indicates this is working. If it > > wasn't > > throwing away replies where the receipt has been ack'd at the TCP > > level, the cache would grow very large, since they would only be > > discarded after a loonnngg timeout (12hours unless you've changes > > NFSRVCACHE_TCPTIMEOUT in sys/fs/nfs/nfs.h). > > That seems unreasonably large. > I suppose. How long a network partitioning do you want the cache to deal with? (My original design was trying to achieve a high level of correctness by default.) The only time cache entries normally hang around this long is when a client has dismounted the volume(s) using the TCP connection. The cached replies for the last few replies will then hang around until the timeout. For a few clients this isn't an issue. For 2,000 clients, I can see that it might be, if the clients choose to dismount volumes (using something like amd). Feel free to make it smaller, based on the longest network partitioning that you anticipate might occur. > > Well, the DRC will try to cache replies until the client's TCP layer > > acknowledges receipt of the reply. It is hard to say how many > > replies > > that is for a given TCP connection, since it is a function of the > > level > > of concurrently (# of nfsiod threads in the FreeBSD client) > > in the client. I'd guess it's somewhere between 1<->20? > > Nearly all our clients are Linux, so it's likely to be whatever Debian > does by default. > > > Multiply that by the number of TCP connections from all clients and > > you have about how big the server's DRC will be. (Some clients use > > a single TCP connection for the client whereas others use a separate > > TCP connection for each mount point.) > > The Debian client appears to use a single TCP connection for > everything. > > So if I want to support 2,000 clients each with 20 requests in flight, > that would suggest that I need a DRC size of 40,000, which my > experience shows is not sufficient with even a much smaller number of > clients. > Well, especially since Debian is using one TCP connection for everything from a client, the guess of 20 could be way low. rick > -GAWollman > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 03:48:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C97C0B3E; Sun, 10 Mar 2013 03:48:04 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 524DCA0D; Sun, 10 Mar 2013 03:48:03 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAMkBPFGDaFvO/2dsb2JhbABDiCi8JIF0dIIsAQEBAwEBAQEgBCcgCwUWGAICDRkCKQEJJgYIBwQBHASHbAYMqjaRaYEjjCAKCwUKczQHgi2BEwOIcoslgj6BHo9XgygeMn0IFx4 X-IronPort-AV: E=Sophos;i="4.84,816,1355115600"; d="scan'208";a="17949507" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 09 Mar 2013 22:47:56 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id CDD83B3F41; Sat, 9 Mar 2013 22:47:56 -0500 (EST) Date: Sat, 9 Mar 2013 22:47:56 -0500 (EST) From: Rick Macklem To: Garrett Wollman Message-ID: <750902132.3736936.1362887276789.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20795.29370.194678.963351@hergotha.csail.mit.edu> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 03:48:04 -0000 Garrett Wollman wrote: > < said: > > > I suspect this indicates that it isn't mutex contention, since the > > threads would block waiting for the mutex for that case, I think? > > No, because our mutexes are adaptive, so each thread spins for a while > before blocking. With the current implementation, all of them end up > doing this in pretty close to lock-step. > > > (Bumping up NFSRVCACHE_HASHSIZE can't hurt if/when you get the > > chance.) > > I already have it set to 129 (up from 20); I could see putting it up > to, say, 1023. It would be nice to have a sysctl for maximum chain > length to see how bad it's getting (and if the hash function is > actually effective). > Yep, I'd bump it up to 1000 or so for a server the size you've built. > > I've thought about this. My concern is that the separate thread > > might > > not keep up with the trimming demand. If that occurred, the cache > > would > > grow veryyy laarrggge, with effects like running out of mbuf > > clusters. > > At a minimum, once one nfsd thread is committed to doing the cache > trim, a flag should be set to discourage other threads from trying to > do it. Having them all spinning their wheels punishes the clients > much too much. > Yes, this is a good idea, as I mentioned in another reply. > > By having the nfsd threads do it, they slow down, which provides > > feedback > > to the clients (slower RPC replies->generate fewer request->less to > > cache). > > (I think you are probably familiar with the generic concept that a > > system > > needs feedback to remain stable. An M/M/1 queue with open arrivals > > and > > no feedback to slow the arrival rate explodes when the arrival rate > > approaches the service rate, etc and so on...) > > Unfortunately, the feedback channel that I have is: one user starts > 500 virtual machines accessing a filesystem on the server -> other > users of this server see their goodput go to zero -> everyone sends in > angry trouble tickets -> I increase the DRC size manually. It would > be nice if, by the time I next want to take a vacation, I have this > figured out. > I probably shouldn't say this, but my response to complaints w.r.t. a slow NFS server was "Tell the boss to spend big bucks on a Netapp.", back when I was a sysadmin for a college.;-) Well, it would be easy to come up with a patch that disables the DRC for TCP. If you'd like a patch for that, just email. So long as your network fabric is solid, it isn't that big a risk to run that way. If 500 virtual machines start pounding on the NFS server, I'd be surprised if other parts of the server don't "hit the wall", but disabling the DRC will find that out. It would be nice if there was a way to guarantee that clients get a fair slice of the server pie, but I don't know of a way to do that. As I noted in another reply, a client may use multiple IP addresses for the requests. Also, since traffic from clients tends to be very bursty, putting a limit on traffic when there isn't a lot of load from other clients doesn't make sense, I think? Then there is the question of "How does the NFS server know the system is nearing its load limit so it should apply limits to clients sending a lot of RPC requests?". All the NFS server does is translate the RPC requests to VFS/VOP ops, so I don't see how it will know that the underlying file systems are nearing their load limit, as one example. When I ran (much smaller) NFS servers in production, I usually saw the disks hit their io ops limit. > I'm OK with throwing memory at the problem -- these servers have 96 GB > and can hold up to 144 GB -- so long as I can find a tuning that > provides stability and consistent, reasonable performance for the > users. > > > The nfs server does soreserve(so, sb_max_adj, sb_max_adj); I can't > > recall exactly why it is that way, except that it needs to be large > > enough to handle the largest RPC request a client might generate. > > > I should take another look at this, in case sb_max_adj is now > > too large? > > It probably shouldn't be larger than the > net.inet.tcp.{send,recv}buf_max, and the read and write sizes that are > negotiated should be chosen so that a whole RPC can fit in that > space. If that's too hard for whatever reason, nfsd should at least > log a message saying "hey, your socket buffer limits are too small, > I'm going to ignore them". > As I mentioned in another reply, *buf_max is 2Mbytes these days. I think I agree that 2Mbytes is larger than you need for your server, given your LAN environment. The problem is, I can't think of how an NFS server will know that a new client connection is on a LAN and not a long-fat WAN connection. The latter may need to but 2Mbytes on the wire to fill the pipe. My TCP is *very rusty*, but I think that a sb_hiwat of 256Kbytes is going to make the send windows shrink so that neither end can send 2Mbytes of unacknowledged data segments to fill the pipe? Also, the intent is to apply the "feedback" in cases where the server is overloaded and I think doing so "late" might be sufficient, if not ideal. The server has "agreed" to do the RPC once it has allowed it to be received into the TCP receive queue for the connection. How fast the RPC is sent will always vary dramatically, based on server load and type of RPC. (A big write will normally take many times what a Getattr does.) rick > -GAWollman > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 04:53:25 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2ED52E37; Sun, 10 Mar 2013 04:53:25 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id E4FA3BEA; Sun, 10 Mar 2013 04:53:24 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2A4rOon040509; Sun, 10 Mar 2013 04:53:24 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2A4rOfi040508; Sun, 10 Mar 2013 04:53:24 GMT (envelope-from linimon) Date: Sun, 10 Mar 2013 04:53:24 GMT Message-Id: <201303100453.r2A4rOfi040508@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/176484: [ipsec] [enc] [patch] panic: IPsec + enc(4); device name clash with CAM X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 04:53:25 -0000 Old Synopsis: panic: IPsec + enc(4); device name clash with CAM New Synopsis: [ipsec] [enc] [patch] panic: IPsec + enc(4); device name clash with CAM Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 10 04:52:34 UTC 2013 Responsible-Changed-Why: Appears to be a missing MFC. http://www.freebsd.org/cgi/query-pr.cgi?pr=176484 From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 05:22:38 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4E8C977A; Sun, 10 Mar 2013 05:22:38 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 0BB3AD59; Sun, 10 Mar 2013 05:22:38 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2A5Mbic047953; Sun, 10 Mar 2013 05:22:37 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2A5MbWv047952; Sun, 10 Mar 2013 05:22:37 GMT (envelope-from linimon) Date: Sun, 10 Mar 2013 05:22:37 GMT Message-Id: <201303100522.r2A5MbWv047952@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/173871: [gif] process of 'ifconfig gif0 create hangs' when if_gif_load exists in /etc/loader.conf and 'device gif' exists in kernel config. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 05:22:38 -0000 Old Synopsis: process of 'ifconfig gif0 create hangs' when if_gif_load exists in /etc/loader.conf and 'device gif' exists in kernel config. New Synopsis: [gif] process of 'ifconfig gif0 create hangs' when if_gif_load exists in /etc/loader.conf and 'device gif' exists in kernel config. Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 10 05:22:24 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=173871 From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 06:05:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3A012E3A for ; Sun, 10 Mar 2013 06:05:01 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id CE01AE02 for ; Sun, 10 Mar 2013 06:04:59 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2A64wWI069227; Sun, 10 Mar 2013 01:04:58 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2A64w19069224; Sun, 10 Mar 2013 01:04:58 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20796.8842.691923.899276@hergotha.csail.mit.edu> Date: Sun, 10 Mar 2013 01:04:58 -0500 From: Garrett Wollman To: Jack Vogel Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: References: <20793.36593.774795.720959@hergotha.csail.mit.edu> <51399926.6020201@freebsd.org> <20794.6692.191898.682241@hergotha.csail.mit.edu> <513A2887.2010408@freebsd.org> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Sun, 10 Mar 2013 01:04:59 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 06:05:01 -0000 < said: > Yes, in the past the code was in this form, it should work fine Garrett, > just make sure > the 4K pool is large enough. [Andre Oppermann's patch:] >> if (adapter->max_frame_size <= 2048) adapter-> rx_mbuf_sz = MCLBYTES; >> - else if (adapter->max_frame_size <= 4096) >> + else adapter-> rx_mbuf_sz = MJUMPAGESIZE; >> - else if (adapter->max_frame_size <= 9216) >> - adapter->rx_mbuf_sz = MJUM9BYTES; >> - else >> - adapter->rx_mbuf_sz = MJUM16BYTES; So I tried exactly this, and it certainly worked insofar as only 4k clusters were allocated, but NFS performance went down precipitously (to fewer than 100 ops/s where normally it would be doing 2000 ops/s). I took a tcpdump while it was in this state, which I will try to make some sense of when I get back to the office. (It wasn't livelocked; in fact, the server was mostly idle, but responses would take seconds rather than milliseconds -- assuming the client could even successfully mount the server at all, which the Debian automounter frequently refused to do.) I ended up reverting back to the old kernel (which I managed to lose the sources for), and once I get my second server up, I will try to do some more testing to see if I can identify the source of the problem. -GAWollman From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 16:50:00 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A6A12E80 for ; Sun, 10 Mar 2013 16:50:00 +0000 (UTC) (envelope-from lastewart@swin.edu.au) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 6FE2F703 for ; Sun, 10 Mar 2013 16:50:00 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 8772F7E88D; Mon, 11 Mar 2013 03:49:51 +1100 (EST) Message-ID: <513CB9AF.3090409@swin.edu.au> Date: Mon, 11 Mar 2013 03:49:51 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120613 Thunderbird/13.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: Multipath TCP for FreeBSD v0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: Nigel Williams , grenville armitage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 16:50:00 -0000 Hi all, The CAIA MPTCP team is pleased to announce the initial release of our multipath TCP implementation for FreeBSD 10-CURRENT which is available from [1]. This release contains wire-related protocol code and a lot of core stack infrastructure. It is capable of running regular TCP flows and single or multi-subflow MPTCP flows (with some caveats as documented in the readme [2]). We consider this code to be of alpha quality and plan to release frequent updates going forward as we continue to flesh out additional features and fix the rough edges. That being said, we welcome everyone to start playing with the code and provide feedback, bug reports, fixes, praise and/or abuse ;) The "Multipath TCP for FreeBSD" project team consists of: Nigel Williams: lead R&D engineer Lawrence Stewart: supporting R&D engineer Grenville Armitage: principal investigator & overall project lead Many thanks go to the Cisco University Research Program Fund at Community Foundation Silicon Valley for their support of this work. Have fun with it! Cheers, Lawrence, Nigel & Grenville http://caia.swin.edu.au [1] http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html [2] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-readme-v0.1.txt From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 22:06:55 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DC119280 for ; Sun, 10 Mar 2013 22:06:55 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5352B283 for ; Sun, 10 Mar 2013 22:06:54 +0000 (UTC) Received: (qmail 97275 invoked from network); 10 Mar 2013 23:19:40 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 10 Mar 2013 23:19:40 -0000 Message-ID: <513D03F7.6090206@freebsd.org> Date: Sun, 10 Mar 2013 23:06:47 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Rick Macklem Subject: Re: Limits on jumbo mbuf cluster allocation References: <2050712270.3721724.1362790033662.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <2050712270.3721724.1362790033662.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: jfv@freebsd.org, freebsd-net@freebsd.org, Garrett Wollman , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 22:06:55 -0000 On 09.03.2013 01:47, Rick Macklem wrote: > Garrett Wollman wrote: >> <> said: >> >>> [stuff I wrote deleted] >>> You have an amd64 kernel running HEAD or 9.x? >> >> Yes, these are 9.1 with some patches to reduce mutex contention on the >> NFS server's replay "cache". >> > The cached replies are copies of the mbuf list done via m_copym(). > As such, the clusters in these replies won't be free'd (ref cnt -> 0) > until the cache is trimmed (nfsrv_trimcache() gets called after the > TCP layer has received an ACK for receipt of the reply from the client). If these are not received mbufs but locally generated with m_getm2() or so they won't be jumbo mbufs > PAGE_SIZE. > If reducing the size to 4K doesn't fix the problem, you might want to > consider shrinking the tunable vfs.nfsd.tcphighwater and suffering > the increased CPU overhead (and some increased mutex contention) of > calling nfsrv_trimcache() more frequently. > (I'm assuming that you are using drc2.patch + drc3.patch. If you are > using one of ivoras@'s variants of the patch, I'm not sure if the > tunable is called the same thing, although it should have basically > the same effect.) > > Good luck with it and thanks for running on the "bleeding edge" so > these issues get identified, rick -- Andre From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 22:22:30 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 110E7906 for ; Sun, 10 Mar 2013 22:22:30 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 60E86331 for ; Sun, 10 Mar 2013 22:22:29 +0000 (UTC) Received: (qmail 97356 invoked from network); 10 Mar 2013 23:35:20 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 10 Mar 2013 23:35:20 -0000 Message-ID: <513D07A4.4010503@freebsd.org> Date: Sun, 10 Mar 2013 23:22:28 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Garrett Wollman Subject: Re: Limits on jumbo mbuf cluster allocation References: <20793.36593.774795.720959@hergotha.csail.mit.edu> <51399926.6020201@freebsd.org> <20794.6692.191898.682241@hergotha.csail.mit.edu> <513A2887.2010408@freebsd.org> <20796.8842.691923.899276@hergotha.csail.mit.edu> In-Reply-To: <20796.8842.691923.899276@hergotha.csail.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 22:22:30 -0000 On 10.03.2013 07:04, Garrett Wollman wrote: > < said: > >> Yes, in the past the code was in this form, it should work fine Garrett, >> just make sure >> the 4K pool is large enough. > > [Andre Oppermann's patch:] >>> if (adapter->max_frame_size <= 2048) > adapter-> rx_mbuf_sz = MCLBYTES; >>> - else if (adapter->max_frame_size <= 4096) >>> + else > adapter-> rx_mbuf_sz = MJUMPAGESIZE; >>> - else if (adapter->max_frame_size <= 9216) >>> - adapter->rx_mbuf_sz = MJUM9BYTES; >>> - else >>> - adapter->rx_mbuf_sz = MJUM16BYTES; > > So I tried exactly this, and it certainly worked insofar as only 4k > clusters were allocated, but NFS performance went down precipitously > (to fewer than 100 ops/s where normally it would be doing 2000 > ops/s). I took a tcpdump while it was in this state, which I will try > to make some sense of when I get back to the office. (It wasn't > livelocked; in fact, the server was mostly idle, but responses would > take seconds rather than milliseconds -- assuming the client could > even successfully mount the server at all, which the Debian > automounter frequently refused to do.) This is very weird and unlikely to come from the 4k mbufs by itself considering they are in heavy use in the write() path. Such a high delay smells like an issue in either the driver dealing with multiple mbufs per packet or nfs having a problem with it. > I ended up reverting back to the old kernel (which I managed to lose > the sources for), and once I get my second server up, I will try to do > some more testing to see if I can identify the source of the problem. -- Andre From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 22:29:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3DCA1C0B for ; Sun, 10 Mar 2013 22:29:56 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 904E537C for ; Sun, 10 Mar 2013 22:29:55 +0000 (UTC) Received: (qmail 97388 invoked from network); 10 Mar 2013 23:42:46 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 10 Mar 2013 23:42:46 -0000 Message-ID: <513D0962.5080606@freebsd.org> Date: Sun, 10 Mar 2013 23:29:54 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Rick Macklem Subject: Re: Limits on jumbo mbuf cluster allocation References: <1841214504.3736248.1362882169721.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1841214504.3736248.1362882169721.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 22:29:56 -0000 On 10.03.2013 03:22, Rick Macklem wrote: > Garett Wollman wrote: >> Also, it occurs to me that this strategy is subject to livelock. To >> put backpressure on the clients, it is far better to get them to stop >> sending (by advertising a small receive window) than to accept their >> traffic but queue it for a long time. By the time the NFS code gets >> an RPC, the system has already invested so much into it that it should >> be processed as quickly as possible, and this strategy essentially >> guarantees[1] that, once those 2 MB socket buffers start to fill up, >> they >> will stay filled, sending latency through the roof. If nfsd didn't >> override the usual socket-buffer sizing mechanisms, then sysadmins >> could limit the buffers to ensure a stable response time. >> >> The bandwidth-delay product in our network is somewhere between 12.5 >> kB and 125 kB, depending on how the client is connected and what sort >> of latency they experience. The usual theory would suggest that >> socket buffers should be no more than twice that -- i.e., about 256 >> kB. >> > Well, the code that uses sb_max_adj wasn't written by me (I just cloned > it for the new server). In the author's defence, I believe SB_MAX was 256K when > it was written. It was 256K in 2011. I think sb_max_adj was used because > soreserve() fails for a larger value and the code doesn't check for such a failure. > (Yea, it should be fixed so that it checks for a failure return from soreserve(). > I did so for the client some time ago.;-) We have TCP sockbuf size autotuning for some time now. So explicitly setting the size shouldn't be necessary anymore. > Just grep for sb_max_adj. You'll see it sets a variable called "siz". > Make "siz" whatever you want (256K sounds like a good guess). Just make > sure it isn't > sb_max_adj. > > The I/O sizes are limited to MAXBSIZE, which is currently 64Kb, although > I'd like to increase that to 128Kb someday soon. (As you note below, the > largest RPC is slightly bigger than that.) > > Btw, net.inet.tcp.{send/recv}buf_max are both 2Mbytes, just like sb_max, > so those don't seem useful in this case? These are just the limits for auto-tuning. > I'm no TCP guy, so suggestions w.r.t. how big soreserve() should be set > are welcome. I'd have to look more at the NFS code to see what exactly is going on and what the most likely settings are going to be. Won't promise any ETA though. -- Andre From owner-freebsd-net@FreeBSD.ORG Sun Mar 10 23:46:18 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2F68AB19; Sun, 10 Mar 2013 23:46:18 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id D74097B3; Sun, 10 Mar 2013 23:46:17 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEANcaPVGDaFvO/2dsb2JhbABDiBq8K4FidIImAQEBAwEBAQEgBCcgCxsYAgINGQIpAQkmBggHBAEcBIdsBgyqCJFdgSOMNAEFfTQHgi2BEwOUF4I+gR6PV4MmIDJ8AQgXHg X-IronPort-AV: E=Sophos;i="4.84,819,1355115600"; d="scan'208";a="20365195" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 10 Mar 2013 19:46:05 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 6B829B4031; Sun, 10 Mar 2013 19:46:05 -0400 (EDT) Date: Sun, 10 Mar 2013 19:46:05 -0400 (EDT) From: Rick Macklem To: Andre Oppermann Message-ID: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <513D0962.5080606@freebsd.org> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 23:46:18 -0000 Andre Oppermann wrote: > On 10.03.2013 03:22, Rick Macklem wrote: > > Garett Wollman wrote: > >> Also, it occurs to me that this strategy is subject to livelock. To > >> put backpressure on the clients, it is far better to get them to > >> stop > >> sending (by advertising a small receive window) than to accept > >> their > >> traffic but queue it for a long time. By the time the NFS code gets > >> an RPC, the system has already invested so much into it that it > >> should > >> be processed as quickly as possible, and this strategy essentially > >> guarantees[1] that, once those 2 MB socket buffers start to fill > >> up, > >> they > >> will stay filled, sending latency through the roof. If nfsd didn't > >> override the usual socket-buffer sizing mechanisms, then sysadmins > >> could limit the buffers to ensure a stable response time. > >> > >> The bandwidth-delay product in our network is somewhere between > >> 12.5 > >> kB and 125 kB, depending on how the client is connected and what > >> sort > >> of latency they experience. The usual theory would suggest that > >> socket buffers should be no more than twice that -- i.e., about 256 > >> kB. > >> > > Well, the code that uses sb_max_adj wasn't written by me (I just > > cloned > > it for the new server). In the author's defence, I believe SB_MAX > > was 256K when > > it was written. It was 256K in 2011. I think sb_max_adj was used > > because > > soreserve() fails for a larger value and the code doesn't check for > > such a failure. > > (Yea, it should be fixed so that it checks for a failure return from > > soreserve(). > > I did so for the client some time ago.;-) > > We have TCP sockbuf size autotuning for some time now. So explicitly > setting the size shouldn't be necessary anymore. > Ok. Is it possible for the size to drop below the size of the largest RPC? (Currently a little over 64K and hopefully a little over 128K soon.) I'm thinking of the restriction in sosend_generic() where it won't allow a request greater than sb_hiwat to be added to the send queue. (It is passed in as an mbuf list via the "top" argument, which makes "atomic" true, I think?) The soreserve() calls were done in the old days to make sure sb_hiwat was big enough that sosend() wouldn't return EMSGSIZE. (I'll take a look at the code and try to see if/when sb_hiwat gets autotuned.) > > Just grep for sb_max_adj. You'll see it sets a variable called > > "siz". > > Make "siz" whatever you want (256K sounds like a good guess). Just > > make > > sure it isn't > sb_max_adj. > > > > The I/O sizes are limited to MAXBSIZE, which is currently 64Kb, > > although > > I'd like to increase that to 128Kb someday soon. (As you note below, > > the > > largest RPC is slightly bigger than that.) > > > > Btw, net.inet.tcp.{send/recv}buf_max are both 2Mbytes, just like > > sb_max, > > so those don't seem useful in this case? > > These are just the limits for auto-tuning. > > > I'm no TCP guy, so suggestions w.r.t. how big soreserve() should be > > set > > are welcome. > > I'd have to look more at the NFS code to see what exactly is going on > and what the most likely settings are going to be. Won't promise any > ETA though. > Basically an RPC request/reply is an mbuf list where its size can be up to MAXBSIZE + a hundred bytes or so. (64Kb+ --> 128Kb+ soon) These need to be queued for sending without getting EMSGSIZE back. Then, if the mount is for a high bandwidth WAN, it would be nice if the send window allows several of these to be "in flight" (not yet acknowledged) so that the "bit pipe" can be kept full (use the available bandwidth). These could be read-aheads/write-behinds or requests for other processes/threads in the client. For example: - with a 128Kbyte MAXBSIZE and a read-ahead of 15, it would be possible to have 128 * 1024 * 16 bytes on the wire, if the TCP window allows that. (This would fill a 1Gbps network with a 20msec rtt, if I got my rusty math correct. It is rtt and not the time for a packet to go in one direction, since the RPC replies need to get back to the client before it will do any more reads.) This sounds like the upper bound of the current setup, given the 2Mbyte setting for net.inet.tcp.sendbuf_max, I think? (Yes, I know most use NFS over a LAN, but it would be nice if it can work well enough over a WAN to be useful.) - for a fast LAN, obviously the rtt is much lower, so the limit can be a lot lower. However, I'm not sure that there is an advantage w.r.t. NFS for this. So long as the client sees that it can't send more RPCs once several are queued for the server, it won't cause a "congestion collapse" for the server. I think the large window might make Garrett's case worse, since it seems that he is running out of mbuf clusters (actually the ability to allocate more jumbo ones), but that seems to be a fairly specific resource issue for his case caused in part by the fact the network interface is using so many of them. In other words, I'm not sure a generic NFS fix would make sense for this specific case. Hope this helps and didn't just muddy the waters, rick > -- > Andre > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 00:04:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5962CE7C; Mon, 11 Mar 2013 00:04:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E51E885B; Mon, 11 Mar 2013 00:04:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAEIfPVGDaFvO/2dsb2JhbAA5CogavCuBYnSCJgEBAQMBAQEBIAQnIAsbGAICDRkCIwYBCSYGCAcEARwEh2ADCQYMqgyHdg2JW4EjiyOBDQp9NAeCLYETA5QXXoFggR6KPoUZgyYgMoEFNQ X-IronPort-AV: E=Sophos;i="4.84,819,1355115600"; d="scan'208";a="18016060" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 10 Mar 2013 20:04:06 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5E611B4047; Sun, 10 Mar 2013 20:04:06 -0400 (EDT) Date: Sun, 10 Mar 2013 20:04:06 -0400 (EDT) From: Rick Macklem To: Andre Oppermann Message-ID: <564543211.3749080.1362960246372.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <513D07A4.4010503@freebsd.org> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, Jack Vogel , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 00:04:08 -0000 Andre Oppermann wrote: > On 10.03.2013 07:04, Garrett Wollman wrote: > > < > > said: > > > >> Yes, in the past the code was in this form, it should work fine > >> Garrett, > >> just make sure > >> the 4K pool is large enough. > > > > [Andre Oppermann's patch:] > >>> if (adapter->max_frame_size <= 2048) > > adapter-> rx_mbuf_sz = MCLBYTES; > >>> - else if (adapter->max_frame_size <= 4096) > >>> + else > > adapter-> rx_mbuf_sz = MJUMPAGESIZE; > >>> - else if (adapter->max_frame_size <= 9216) > >>> - adapter->rx_mbuf_sz = MJUM9BYTES; > >>> - else > >>> - adapter->rx_mbuf_sz = MJUM16BYTES; > > > > So I tried exactly this, and it certainly worked insofar as only 4k > > clusters were allocated, but NFS performance went down precipitously > > (to fewer than 100 ops/s where normally it would be doing 2000 > > ops/s). I took a tcpdump while it was in this state, which I will > > try > > to make some sense of when I get back to the office. (It wasn't > > livelocked; in fact, the server was mostly idle, but responses would > > take seconds rather than milliseconds -- assuming the client could > > even successfully mount the server at all, which the Debian > > automounter frequently refused to do.) > > This is very weird and unlikely to come from the 4k mbufs by itself > considering they are in heavy use in the write() path. Such a high > delay smells like an issue in either the driver dealing with multiple > mbufs per packet or nfs having a problem with it. > I am not aware of anything within the NFS server that would care. The code simply believes the m_len field. --> However, this is a good way to reduce server load. At 100ops/sec I'd think you shouldn't have any server resource exhaustion issues. --> Problem solved ;-);-) rick > > I ended up reverting back to the old kernel (which I managed to lose > > the sources for), and once I get my second server up, I will try to > > do > > some more testing to see if I can identify the source of the > > problem. > > -- > Andre > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 00:12:52 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 947ECFC0; Mon, 11 Mar 2013 00:12:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 27E8D8B3; Mon, 11 Mar 2013 00:12:51 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAJogPVGDaFvO/2dsb2JhbABDiBy8K4FjdIImAQEBAwEBAQEgBCcgCxsYAgINGQIpAQkmBggHBAEcBIdsBgyqDpFegSOMOn00B4ItgRMDlBeCPoEej1eDJiAygQU1 X-IronPort-AV: E=Sophos;i="4.84,819,1355115600"; d="scan'208";a="18016685" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 10 Mar 2013 20:12:50 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id F3552B403D; Sun, 10 Mar 2013 20:12:50 -0400 (EDT) Date: Sun, 10 Mar 2013 20:12:50 -0400 (EDT) From: Rick Macklem To: Andre Oppermann Message-ID: <595542882.3749278.1362960770982.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <513D03F7.6090206@freebsd.org> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: jfv@freebsd.org, freebsd-net@freebsd.org, Garrett Wollman , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 00:12:52 -0000 Andre Oppermann wrote: > On 09.03.2013 01:47, Rick Macklem wrote: > > Garrett Wollman wrote: > >> < >> said: > >> > >>> [stuff I wrote deleted] > >>> You have an amd64 kernel running HEAD or 9.x? > >> > >> Yes, these are 9.1 with some patches to reduce mutex contention on > >> the > >> NFS server's replay "cache". > >> > > The cached replies are copies of the mbuf list done via m_copym(). > > As such, the clusters in these replies won't be free'd (ref cnt -> > > 0) > > until the cache is trimmed (nfsrv_trimcache() gets called after the > > TCP layer has received an ACK for receipt of the reply from the > > client). > > If these are not received mbufs but locally generated with m_getm2() > or so they won't be jumbo mbufs > PAGE_SIZE. > Yes, you are correct. Since the DRC caches replies, they shouldn't have jumbo clusters in them. (For his case of 60,000 cached entries, there could be 100,000 or more regular clusters held. I'd think that could make finding the space for the 3 page jumbo clusters harder, wouldn't it?) rick > > If reducing the size to 4K doesn't fix the problem, you might want > > to > > consider shrinking the tunable vfs.nfsd.tcphighwater and suffering > > the increased CPU overhead (and some increased mutex contention) of > > calling nfsrv_trimcache() more frequently. > > (I'm assuming that you are using drc2.patch + drc3.patch. If you are > > using one of ivoras@'s variants of the patch, I'm not sure if the > > tunable is called the same thing, although it should have > > basically > > the same effect.) > > > > Good luck with it and thanks for running on the "bleeding edge" so > > these issues get identified, rick > > -- > Andre > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 07:54:15 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6EB1146D for ; Mon, 11 Mar 2013 07:54:15 +0000 (UTC) (envelope-from ihsan@grep.my) Received: from svc02-kul.b.n3labs.my (svc02-kul.b.n3labs.my [IPv6:2400:3700:10::61]) by mx1.freebsd.org (Postfix) with ESMTP id 3398BB4D for ; Mon, 11 Mar 2013 07:54:15 +0000 (UTC) Received: from [192.168.2.178] (unknown [175.142.133.189]) by svc02-kul.b.n3labs.my (Postfix) with ESMTPSA id A8E0AC60237 for ; Mon, 11 Mar 2013 15:54:03 +0800 (MYT) From: Ihsan Junaidi Ibrahim Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: increasing 'requests for jumbo clusters denied' Message-Id: <1713E286-B096-44B6-846D-81854D6F809B@grep.my> Date: Mon, 11 Mar 2013 15:52:40 +0800 To: "freebsd-net@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 07:54:15 -0000 Hi, I'm on 9.0-RELEASE-p3 and have had a number of instances where my igb0 = network connectivity locked up under heavy load. I've had another em1 device which I used for side-band access which does = not have as much load as my primary igb0 device. I've read previous cases of this but can't seem to find any fitting = solutions. I've tried increasing mbufs and jumbo mbufs: nsv01-kul:ihsan $ sysctl -a | grep nmb kern.ipc.nmbjumbo16: 3200 kern.ipc.nmbjumbo9: 25600 kern.ipc.nmbjumbop: 12800 kern.ipc.nmbclusters: 262144 but I'm still receiving a lot of 'requests for jumbo clusters denied' = stats on netstat -m output. nsv01-kul:ihsan $ netstat -m 7347/3063/10410 mbufs in use (current/cache/total) 334/2658/2992/262144 mbuf clusters in use (current/cache/total/max) 334/1330 mbuf+clusters out of packet secondary zone in use = (current/cache) 1749/1310/3059/12800 4k (page size) jumbo clusters in use = (current/cache/total/max) 5116/563/5679/25600 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 55544K/16388K/71933K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/25154247/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines The server has not locked up yet after increasing the mbufs (been less = than 8 hours since increasing this) but can someone enlighten me what = could be the causes for jumbo clusters requests being denied although = I've increased the 9k jumbo clusters to 25600? Thanks. ihsan= From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 08:18:39 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DD0C288B for ; Mon, 11 Mar 2013 08:18:39 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 30178D14 for ; Mon, 11 Mar 2013 08:18:38 +0000 (UTC) Received: (qmail 99803 invoked from network); 11 Mar 2013 09:31:25 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 11 Mar 2013 09:31:25 -0000 Message-ID: <513D935E.2080202@freebsd.org> Date: Mon, 11 Mar 2013 09:18:38 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Ihsan Junaidi Ibrahim Subject: Re: increasing 'requests for jumbo clusters denied' References: <1713E286-B096-44B6-846D-81854D6F809B@grep.my> In-Reply-To: <1713E286-B096-44B6-846D-81854D6F809B@grep.my> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 08:18:39 -0000 On 11.03.2013 08:52, Ihsan Junaidi Ibrahim wrote: > Hi, > > I'm on 9.0-RELEASE-p3 and have had a number of instances where my igb0 network connectivity locked up under heavy load. This problem is also known on CURRENT and we are under active investigation on how to solve it properly. > I've had another em1 device which I used for side-band access which does not have as much load as my primary igb0 device. Jumbo mbufs are only used if you increase the MTU on your interface. As long as you stay on the standard ethernet MTU of 1500 no jumbo mbufs will be used. > I've read previous cases of this but can't seem to find any fitting solutions. > > I've tried increasing mbufs and jumbo mbufs: > nsv01-kul:ihsan $ sysctl -a | grep nmb > kern.ipc.nmbjumbo16: 3200 > kern.ipc.nmbjumbo9: 25600 > kern.ipc.nmbjumbop: 12800 > kern.ipc.nmbclusters: 262144 > > but I'm still receiving a lot of 'requests for jumbo clusters denied' stats on netstat -m output. > nsv01-kul:ihsan $ netstat -m > 7347/3063/10410 mbufs in use (current/cache/total) > 334/2658/2992/262144 mbuf clusters in use (current/cache/total/max) > 334/1330 mbuf+clusters out of packet secondary zone in use (current/cache) > 1749/1310/3059/12800 4k (page size) jumbo clusters in use (current/cache/total/max) > 5116/563/5679/25600 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 55544K/16388K/71933K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/25154247/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > The server has not locked up yet after increasing the mbufs (been less than 8 hours since increasing this) but can someone enlighten me what could be the causes for jumbo clusters requests being denied although I've increased the 9k jumbo clusters to 25600? The problem is not the limits but memory fragmentation. Jumbo mbufs larger than 4k (PAGE_SIZE) have to be created with adjacent and contiguous VM pages in physical and virtual memory. As soon as your server becomes busy the physical memory becomes 'fragmented' in the sense that such jumbo mbuf requests can't be fulfilled anymore at allocation time. Most modern high speed network cards can do scatter DMA and could simply split up a received jumbo frame across as many normal mbuf clusters as necessary. For a temporary fix you could forgo using a MTU larger than 4096. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 10:00:00 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1787895; Mon, 11 Mar 2013 10:00:00 +0000 (UTC) (envelope-from ihsan@grep.my) Received: from svc02-kul.b.n3labs.my (svc02-kul.b.n3labs.my [IPv6:2400:3700:10::61]) by mx1.freebsd.org (Postfix) with ESMTP id CF148210; Mon, 11 Mar 2013 09:59:57 +0000 (UTC) Received: from [192.168.2.178] (unknown [175.142.133.189]) by svc02-kul.b.n3labs.my (Postfix) with ESMTPSA id A3DEDC60237; Mon, 11 Mar 2013 17:59:55 +0800 (MYT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: increasing 'requests for jumbo clusters denied' From: Ihsan Junaidi Ibrahim In-Reply-To: <513D935E.2080202@freebsd.org> Date: Mon, 11 Mar 2013 17:59:54 +0800 Content-Transfer-Encoding: quoted-printable Message-Id: <8CBEE72C-81FB-47B2-98A9-C3C2A7FEA082@grep.my> References: <1713E286-B096-44B6-846D-81854D6F809B@grep.my> <513D935E.2080202@freebsd.org> To: Andre Oppermann X-Mailer: Apple Mail (2.1499) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 10:00:00 -0000 On Mar 11, 2013, at 4:18 PM, Andre Oppermann wrote: > On 11.03.2013 08:52, Ihsan Junaidi Ibrahim wrote: >> Hi, >>=20 >> I'm on 9.0-RELEASE-p3 and have had a number of instances where my = igb0 network connectivity locked up under heavy load. >=20 > This problem is also known on CURRENT and we are under active = investigation > on how to solve it properly. >=20 >> I've had another em1 device which I used for side-band access which = does not have as much load as my primary igb0 device. >=20 > Jumbo mbufs are only used if you increase the MTU on your interface. > As long as you stay on the standard ethernet MTU of 1500 no jumbo > mbufs will be used. I'm actually using 9k jumbo on igb0 since it's a requirement of my = upstream customer. The server is functioning as some form of internal = route/tunnel server. The issue is manageable for now on my end and since this is being worked = on, I assume I can let it rest for now. Thank you.= From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 10:43:31 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2E89AC63 for ; Mon, 11 Mar 2013 10:43:31 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 653C5644 for ; Mon, 11 Mar 2013 10:43:29 +0000 (UTC) Received: (qmail 1165 invoked from network); 11 Mar 2013 11:56:15 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 11 Mar 2013 11:56:15 -0000 Message-ID: <513DB550.5010004@freebsd.org> Date: Mon, 11 Mar 2013 11:43:28 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Rick Macklem Subject: Re: Limits on jumbo mbuf cluster allocation References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 10:43:31 -0000 On 11.03.2013 00:46, Rick Macklem wrote: > Andre Oppermann wrote: >> On 10.03.2013 03:22, Rick Macklem wrote: >>> Garett Wollman wrote: >>>> Also, it occurs to me that this strategy is subject to livelock. To >>>> put backpressure on the clients, it is far better to get them to >>>> stop >>>> sending (by advertising a small receive window) than to accept >>>> their >>>> traffic but queue it for a long time. By the time the NFS code gets >>>> an RPC, the system has already invested so much into it that it >>>> should >>>> be processed as quickly as possible, and this strategy essentially >>>> guarantees[1] that, once those 2 MB socket buffers start to fill >>>> up, >>>> they >>>> will stay filled, sending latency through the roof. If nfsd didn't >>>> override the usual socket-buffer sizing mechanisms, then sysadmins >>>> could limit the buffers to ensure a stable response time. >>>> >>>> The bandwidth-delay product in our network is somewhere between >>>> 12.5 >>>> kB and 125 kB, depending on how the client is connected and what >>>> sort >>>> of latency they experience. The usual theory would suggest that >>>> socket buffers should be no more than twice that -- i.e., about 256 >>>> kB. >>>> >>> Well, the code that uses sb_max_adj wasn't written by me (I just >>> cloned >>> it for the new server). In the author's defence, I believe SB_MAX >>> was 256K when >>> it was written. It was 256K in 2011. I think sb_max_adj was used >>> because >>> soreserve() fails for a larger value and the code doesn't check for >>> such a failure. >>> (Yea, it should be fixed so that it checks for a failure return from >>> soreserve(). >>> I did so for the client some time ago.;-) >> >> We have TCP sockbuf size autotuning for some time now. So explicitly >> setting the size shouldn't be necessary anymore. >> > Ok. Is it possible for the size to drop below the size of the largest RPC? > (Currently a little over 64K and hopefully a little over 128K soon.) Auto-tuning only goes up. The start values are 32k for TX and 64k for RX. > I'm thinking of the restriction in sosend_generic() where it won't allow a > request greater than sb_hiwat to be added to the send queue. (It is passed > in as an mbuf list via the "top" argument, which makes "atomic" true, I think?) IIRC we actually do enforce the limit when you pass in an mbuf chain through "top". Have to check again though what the current situation is. Officially a send on a socket is only atomic when it is marked as atomic. Due to the non-enforced limit in the "top" case the append becomes atomic in practice. Though the API does not guarantee it. We have to analyze the interaction socket/NFS further to make sure no assumptions are violated. > The soreserve() calls were done in the old days to make sure sb_hiwat was > big enough that sosend() wouldn't return EMSGSIZE. > (I'll take a look at the code and try to see if/when sb_hiwat gets autotuned.) Having a socket buffer large enough for the largest RPC doesn't guarantee anything for TCP as there may still be unacknowledged data sitting in the send buffer. This also counts against the limit. The soreserve() call probably comes from UDP where the socket buffer must be as large as the largest packet you're trying to send (64K is the limit). UDP sockets do not buffer on send but go straight down to the interface queue. >>> Just grep for sb_max_adj. You'll see it sets a variable called >>> "siz". >>> Make "siz" whatever you want (256K sounds like a good guess). Just >>> make >>> sure it isn't > sb_max_adj. >>> >>> The I/O sizes are limited to MAXBSIZE, which is currently 64Kb, >>> although >>> I'd like to increase that to 128Kb someday soon. (As you note below, >>> the >>> largest RPC is slightly bigger than that.) >>> >>> Btw, net.inet.tcp.{send/recv}buf_max are both 2Mbytes, just like >>> sb_max, >>> so those don't seem useful in this case? >> >> These are just the limits for auto-tuning. >> >>> I'm no TCP guy, so suggestions w.r.t. how big soreserve() should be >>> set >>> are welcome. >> >> I'd have to look more at the NFS code to see what exactly is going on >> and what the most likely settings are going to be. Won't promise any >> ETA though. >> > Basically an RPC request/reply is an mbuf list where its size can be > up to MAXBSIZE + a hundred bytes or so. (64Kb+ --> 128Kb+ soon) OK. > These need to be queued for sending without getting EMSGSIZE back. If the socket buffer doesn't have this "bug" there isn't any guarantee. We really have to look into that NFS socket interaction to get this right. > Then, if the mount is for a high bandwidth WAN, it would be nice if > the send window allows several of these to be "in flight" (not yet > acknowledged) so that the "bit pipe" can be kept full (use the > available bandwidth). These could be read-aheads/write-behinds or > requests for other processes/threads in the client. > For example: > - with a 128Kbyte MAXBSIZE and a read-ahead of 15, it would be possible > to have 128 * 1024 * 16 bytes on the wire, if the TCP window allows > that. (This would fill a 1Gbps network with a 20msec rtt, if I got > my rusty math correct. It is rtt and not the time for a packet to > go in one direction, since the RPC replies need to get back to the > client before it will do any more reads.) This sounds like the > upper bound of the current setup, given the 2Mbyte setting for > net.inet.tcp.sendbuf_max, I think? > (Yes, I know most use NFS over a LAN, but it would be nice if it > can work well enough over a WAN to be useful.) > - for a fast LAN, obviously the rtt is much lower, so the limit can > be a lot lower. However, I'm not sure that there is an advantage > w.r.t. NFS for this. So long as the client sees that it can't send > more RPCs once several are queued for the server, it won't cause a > "congestion collapse" for the server. Thanks for the explanation. > I think the large window might make Garrett's case worse, since it seems that > he is running out of mbuf clusters (actually the ability to allocate > more jumbo ones), but that seems to be a fairly specific resource > issue for his case caused in part by the fact the network interface > is using so many of them. > In other words, I'm not sure a generic NFS fix would make sense for > this specific case. The send side doesn't use jumbo mbufs unless you explicitly allocate them while creating the packet. Garrett's problem is receive side specific and NFS can't do much about it. Unless, of course, NFS is holding on to received mbufs for a longer time. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 11:06:46 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9916DB27 for ; Mon, 11 Mar 2013 11:06:46 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 892407D2 for ; Mon, 11 Mar 2013 11:06:46 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2BB6kmZ089071 for ; Mon, 11 Mar 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2BB6k78089069 for freebsd-net@FreeBSD.org; Mon, 11 Mar 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 11 Mar 2013 11:06:46 GMT Message-Id: <201303111106.r2BB6k78089069@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 11:06:46 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/176771 net [libnetgraph] [patch] user-mode netgraph node hangs wh o kern/176764 net [net] [if_bridge] [patch] use-after-free in if_bridge o kern/176671 net [epair] MAC address for epair device not unique o kern/176667 net [libalias] [patch] libalias locks on uninitalized data o kern/176596 net [firewire] [ip6] Crash with IPv6 and Firewire o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176446 net [netinet] [patch] Concurrency in ixgbe driving out-of- o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176097 net [lagg] [patch] lagg/lacp broken when aggregated interf o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o bin/175974 net ppp(8): logic issue o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172985 net [patch] [ip6] lltable leak when adding and removing IP o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos o kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169664 net [bgp] Wrongful replacement of interface connected net o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/169438 net [ipsec] ipv4-in-ipv6 tunnel mode IPsec does not work p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167947 net [setfib] [patch] arpresolve checks only the default FI o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/167059 net [tcp] [panic] System does panic in in_pcbbind() and ha o kern/166940 net [ipfilter] [panic] Double fault in kern 8.2 o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166372 net [patch] ipfilter drops UDP packets with zero checksum o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165963 net [panic] [ipf] ipfilter/nat NULL pointer deference o kern/165903 net mbuf leak o kern/165643 net [net] [patch] Missing vnet restores in net/if_ethersub o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162926 net [ipfilter] Infinite loop in ipfilter with fragmented I o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157209 net [ip6] [patch] locking error in rip6_input() (sys/netin o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel p kern/155030 net [igb] igb(4) DEVICE_POLLING does not work with carp(4) o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl o kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o kern/131601 net [ipfilter] [panic] 7-STABLE panic in nat_finalise (tcp o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ p kern/85320 net [gre] [patch] possible depletion of kernel stack in ip o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 455 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 12:07:17 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4BC829E2; Mon, 11 Mar 2013 12:07:17 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 24D73D22; Mon, 11 Mar 2013 12:07:17 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2BC7HoR003548; Mon, 11 Mar 2013 12:07:17 GMT (envelope-from glebius@freefall.freebsd.org) Received: (from glebius@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2BC7GsL003547; Mon, 11 Mar 2013 12:07:16 GMT (envelope-from glebius) Date: Mon, 11 Mar 2013 12:07:16 GMT Message-Id: <201303111207.r2BC7GsL003547@freefall.freebsd.org> To: adrian@FreeBSD.org, glebius@FreeBSD.org, freebsd-net@FreeBSD.org, glebius@FreeBSD.org From: glebius@FreeBSD.org Subject: Re: kern/176764: [net] [if_bridge] [patch] use-after-free in if_bridge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 12:07:17 -0000 Synopsis: [net] [if_bridge] [patch] use-after-free in if_bridge State-Changed-From-To: open->patched State-Changed-By: glebius State-Changed-When: Mon Mar 11 12:00:29 UTC 2013 State-Changed-Why: Committed, thanks. Responsible-Changed-From-To: freebsd-net->glebius Responsible-Changed-By: glebius Responsible-Changed-When: Mon Mar 11 12:00:29 UTC 2013 Responsible-Changed-Why: Committed, thanks. http://www.freebsd.org/cgi/query-pr.cgi?pr=176764 From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 12:30:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 95B7B6D6 for ; Mon, 11 Mar 2013 12:30:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 703B3E12 for ; Mon, 11 Mar 2013 12:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2BCU1P0008114 for ; Mon, 11 Mar 2013 12:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2BCU1cw008113; Mon, 11 Mar 2013 12:30:01 GMT (envelope-from gnats) Date: Mon, 11 Mar 2013 12:30:01 GMT Message-Id: <201303111230.r2BCU1cw008113@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: dfilter@FreeBSD.ORG (dfilter service) Subject: Re: kern/176667: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 12:30:01 -0000 The following reply was made to PR kern/176667; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/176667: commit references a PR Date: Mon, 11 Mar 2013 12:22:52 +0000 (UTC) Author: glebius Date: Mon Mar 11 12:22:44 2013 New Revision: 248158 URL: http://svnweb.freebsd.org/changeset/base/248158 Log: Remove LIBALIAS_LOCK_ASSERT(), including a couple with an uninitialzed argument, in code that isn't compiled in kernel. PR: kern/176667 Sponsored by: Nginx, Inc. Modified: head/sys/netinet/libalias/alias_db.c Modified: head/sys/netinet/libalias/alias_db.c ============================================================================== --- head/sys/netinet/libalias/alias_db.c Mon Mar 11 12:14:20 2013 (r248157) +++ head/sys/netinet/libalias/alias_db.c Mon Mar 11 12:22:44 2013 (r248158) @@ -2729,7 +2729,6 @@ static void InitPunchFW(struct libalias *la) { - LIBALIAS_LOCK_ASSERT(la); la->fireWallField = malloc(la->fireWallNumNums); if (la->fireWallField) { memset(la->fireWallField, 0, la->fireWallNumNums); @@ -2745,7 +2744,6 @@ static void UninitPunchFW(struct libalias *la) { - LIBALIAS_LOCK_ASSERT(la); ClearAllFWHoles(la); if (la->fireWallFD >= 0) close(la->fireWallFD); @@ -2765,7 +2763,6 @@ PunchFWHole(struct alias_link *lnk) struct ip_fw rule; /* On-the-fly built rule */ int fwhole; /* Where to punch hole */ - LIBALIAS_LOCK_ASSERT(la); la = lnk->la; /* Don't do anything unless we are asked to */ @@ -2839,7 +2836,6 @@ ClearFWHole(struct alias_link *lnk) { struct libalias *la; - LIBALIAS_LOCK_ASSERT(la); la = lnk->la; if (lnk->link_type == LINK_TCP) { int fwhole = lnk->data.tcp->fwhole; /* Where is the firewall @@ -2864,7 +2860,6 @@ ClearAllFWHoles(struct libalias *la) struct ip_fw rule; /* On-the-fly built rule */ int i; - LIBALIAS_LOCK_ASSERT(la); if (la->fireWallFD < 0) return; @@ -2878,7 +2873,7 @@ ClearAllFWHoles(struct libalias *la) memset(la->fireWallField, 0, la->fireWallNumNums); } -#endif +#endif /* !NO_FW_PUNCH */ void LibAliasSetFWBase(struct libalias *la, unsigned int base, unsigned int num) _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 13:05:33 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4900FB57; Mon, 11 Mar 2013 13:05:33 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 23C25FF1; Mon, 11 Mar 2013 13:05:33 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2BD5WEC015229; Mon, 11 Mar 2013 13:05:32 GMT (envelope-from glebius@freefall.freebsd.org) Received: (from glebius@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2BD5Wlo015228; Mon, 11 Mar 2013 13:05:32 GMT (envelope-from glebius) Date: Mon, 11 Mar 2013 13:05:32 GMT Message-Id: <201303111305.r2BD5Wlo015228@freefall.freebsd.org> To: keith.reynolds@tidalscale.com, glebius@FreeBSD.org, freebsd-net@FreeBSD.org, glebius@FreeBSD.org From: glebius@FreeBSD.org Subject: Re: kern/176771: [libnetgraph] [patch] user-mode netgraph node hangs when replying to control message X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 13:05:33 -0000 Synopsis: [libnetgraph] [patch] user-mode netgraph node hangs when replying to control message State-Changed-From-To: open->patched State-Changed-By: glebius State-Changed-When: Mon Mar 11 12:59:12 UTC 2013 State-Changed-Why: Committed. Thanks for submission! Responsible-Changed-From-To: freebsd-net->glebius Responsible-Changed-By: glebius Responsible-Changed-When: Mon Mar 11 12:59:12 UTC 2013 Responsible-Changed-Why: Committed. Thanks for submission! http://www.freebsd.org/cgi/query-pr.cgi?pr=176771 From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 15:05:23 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3CD1F457 for ; Mon, 11 Mar 2013 15:05:23 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x22c.google.com (mail-bk0-x22c.google.com [IPv6:2a00:1450:4008:c01::22c]) by mx1.freebsd.org (Postfix) with ESMTP id C5687967 for ; Mon, 11 Mar 2013 15:05:22 +0000 (UTC) Received: by mail-bk0-f44.google.com with SMTP id j4so1739723bkw.17 for ; Mon, 11 Mar 2013 08:05:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=yAHTqpZ6wezuRuGmMIXuJGzSYgIfGrRrmf5SQT3CVVY=; b=MGH+VsnY3yLjAGVj58DzimE32O54MRlcYW9kT5bItY+mSUnZozfclPMH74CiFpNyIn g+cCinh4oFPWnSVlqyoZVOXl9Jvk9GMJ0bcXp/KQKb9J+Vo0vRU1khKIMKutvrUOteiI 2Cu059h/Cpb9RJAWOXyrHymn5rQ5r6fhZOBZfRqwg/bSHsUOQcbS3J6+36A15EiXvkuC LvI7HUUzab7+v/iY8Txg5BHFg/W8gODwJ09YJ1x9VoigNwCcGeUMgnDBXA0X2+ri4mwF SBnCSQTIcqeRy1qx+Y/VI/77wcKCmUnwFneDbDPZ907BvQElps2Q1b2DaopBwcg/TK70 3Udg== X-Received: by 10.204.195.133 with SMTP id ec5mr4731790bkb.32.1363014321478; Mon, 11 Mar 2013 08:05:21 -0700 (PDT) Received: from zvezda.localnet ([212.48.107.10]) by mx.google.com with ESMTPS id g28sm4198353bkv.17.2013.03.11.08.05.20 (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 11 Mar 2013 08:05:20 -0700 (PDT) From: Kajetan Staszkiewicz To: Ermal =?iso-8859-1?q?Lu=E7i?= Subject: Re: [patch] Source entries removing is awfully slow. Date: Mon, 11 Mar 2013 16:05:19 +0100 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <201303081419.17743.vegeta@tuxpowered.net> <201303091437.51945.vegeta@tuxpowered.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201303111605.19518.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQnMp82eb/yeF9je4m/e2vIPyd4SY/SRGHwdYNAGNv+olDcI25/v/v7adYO0gQ93x0ThOOQo Cc: "freebsd-net@freebsd.org" , "freebsd-pf@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 15:05:23 -0000 There are some things I find flawed in your patch: 1. +#if 0 if (killed > 0) pf_purge_expired_src_nodes(1); +#endif This means that after using `pfctl -K` the src nodes are still around until purged and any new states created will still use them and bump their expire timer. This also changes behavior from DIOCCLRSRCNODES, which also performs the purge immediately. You also moved s->src_node=s->nat_src_node=NULL code to inside of pf_purge_expired_src_nodes, therefore I believe it should be called immediately. If detaching state from source is done in pf_purge_expired_src_nodes, DIOCCLRSRCNODES does not have to traverse the state table anymore, so we achieve another performance improvement. 2. /* Handle state to src_node linkage */ +#ifndef __FreeBSD__ if (sn->states != 0) { RB_FOREACH(s, pf_state_tree_id, #ifdef __FreeBSD__ &V_tree_id) { #else &tree_id) { #endif if (s->src_node == sn) s->src_node = NULL; if (s->nat_src_node == sn) s->nat_src_node = NULL; } sn->states = 0; } +#endif sn->expire = 1; killed++; This removes a bit too much code, that is zeroing of source's state counter. Please find the next version of the patch here: http://vegeta.tuxpowered.net/download/link-states-to-src_node-3.patch This one also takes care of removing states linked to found sources if pfctl is given extra -c parameter (that can stand for "clear", I could not find any other free pfctl parameter better matching). Thanks to this parameter, the default behavior is not changed. -- | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 15:27:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A9DC7A5D; Mon, 11 Mar 2013 15:27:36 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-qe0-f51.google.com (mail-qe0-f51.google.com [209.85.128.51]) by mx1.freebsd.org (Postfix) with ESMTP id 468D4A47; Mon, 11 Mar 2013 15:27:35 +0000 (UTC) Received: by mail-qe0-f51.google.com with SMTP id nd7so2325310qeb.24 for ; Mon, 11 Mar 2013 08:27:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=mJaK4Fg2x2PZ9Tq04oPGZOgMA3o+ot3/feS5TJ+sB0M=; b=EFOvUgDYK27zjJJHQhsKNuvEYDY4bUymODKafmtN01FlvtQnlNK+LogfbJBOhe61Ic z6rUy708j0aWy+LLpF6FZLTazgDGqVFtHAx5wKI7cQgqxTi2uZb2bDuL0PiK4qB7d4+H aUKQ/HmrUmv4coS+/n2hwVLqzbENWNRI1Kgczi/qT6qcaqejtjgzZpmZTcdGJZwnSg/e C3/GEAxoWdlVn/EWL2g6MWw/+/pv0AM72luqvtWB4TOFE9uVdUKJdP+nhTu1wh3tVzqf WuZquHcAYIZ5caecS0t7DAzgHCjAYaATwvya/FUmAn7UC6vAhhS7QKup4ZI/FTDdexhD 2PcA== MIME-Version: 1.0 X-Received: by 10.229.69.24 with SMTP id x24mr4030045qci.16.1363015653965; Mon, 11 Mar 2013 08:27:33 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.49.27.197 with HTTP; Mon, 11 Mar 2013 08:27:33 -0700 (PDT) In-Reply-To: <201303111605.19518.vegeta@tuxpowered.net> References: <201303081419.17743.vegeta@tuxpowered.net> <201303091437.51945.vegeta@tuxpowered.net> <201303111605.19518.vegeta@tuxpowered.net> Date: Mon, 11 Mar 2013 16:27:33 +0100 X-Google-Sender-Auth: u_ysQLfDXzv2yicH4zsprTPBp9c Message-ID: Subject: Re: [patch] Source entries removing is awfully slow. From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Kajetan Staszkiewicz Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , "freebsd-pf@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 15:27:36 -0000 On Mon, Mar 11, 2013 at 4:05 PM, Kajetan Staszkiewicz wrote: > There are some things I find flawed in your patch: > > 1. > > +#if 0 > if (killed > 0) > pf_purge_expired_src_nodes(1); > +#endif > > This means that after using `pfctl -K` the src nodes are still around until > purged and any new states created will still use them and bump their expire > timer. This also changes behavior from DIOCCLRSRCNODES, which also > performs the > purge immediately. You also moved s->src_node=s->nat_src_node=NULL code to > inside of pf_purge_expired_src_nodes, therefore I believe it should be > called > immediately. If detaching state from source is done in > pf_purge_expired_src_nodes, DIOCCLRSRCNODES does not have to traverse the > state > table anymore, so we achieve another performance improvement. > Well probably just need to clear the rtaddr member of a source node. Its an optimization to keep the source node there until next purge since you would avoid a memory allocation. It seems to me a better behaviour rather than having to loop N amount of states during an ioctl which is supposed to be fast. You need to take into consideration that an ioctl freezes the whole operation of network in this case because you are modifying the core of pf(4). I would rather be keen to mark a src_node state as expired and let the purge_thread collect that? > > 2. > /* Handle state to src_node linkage */ > +#ifndef __FreeBSD__ > if (sn->states != 0) { > RB_FOREACH(s, pf_state_tree_id, > #ifdef __FreeBSD__ > &V_tree_id) { > #else > &tree_id) { > #endif > if (s->src_node == sn) > s->src_node = NULL; > if (s->nat_src_node == sn) > s->nat_src_node = NULL; > } > sn->states = 0; > } > +#endif > sn->expire = 1; > killed++; > > This removes a bit too much code, that is zeroing of source's state > counter. > > Please find the next version of the patch here: > http://vegeta.tuxpowered.net/download/link-states-to-src_node-3.patch > > This one also takes care of removing states linked to found sources if > pfctl is > given extra -c parameter (that can stand for "clear", I could not find any > other free pfctl parameter better matching). Thanks to this parameter, the > default behavior is not changed. > > Its ok for the extra -c, there is even -S free just to be consistent with the flush and vieweing options probably better use that?! The addition of 2 extra members to pf_state structure i am reluctant a bit since its just for reporting the number of states killed and it really is not a useful information compared to the increased size of the structure that consumes RAM memory. Also pf_src_node structure if the 2 options for display are not added can use the member that is used to report the number of states killed to request killing linked states. What do you think? > -- > | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | > | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | > | Vegeta | www: http://vegeta.tuxpowered.net | > `------------------------^---------------------------------------' > -- Ermal From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 16:05:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E1AD5336; Mon, 11 Mar 2013 16:05:19 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 7E400C31; Mon, 11 Mar 2013 16:05:19 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2BG5Ixp073053; Mon, 11 Mar 2013 12:05:18 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2BG5I6v073052; Mon, 11 Mar 2013 12:05:18 -0400 (EDT) (envelope-from wollman) Date: Mon, 11 Mar 2013 12:05:18 -0400 (EDT) From: Garrett Wollman Message-Id: <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> To: andre@freebsd.org Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: <513DB550.5010004@freebsd.org> References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Mon, 11 Mar 2013 12:05:18 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org, rmacklem@uoguelph.ca X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 16:05:19 -0000 In article <513DB550.5010004@freebsd.org>, andre@freebsd.org writes: >Garrett's problem is receive side specific and NFS can't do much about it. >Unless, of course, NFS is holding on to received mbufs for a longer time. Well, I have two problems: one is running out of mbufs (caused, we think, by ixgbe requiring 9k clusters when it doesn't actually need them), and one is livelock. Allowing potentially hundreds of clients to queue 2 MB of requests before TCP pushes back on them helps to sustain the livelock once it gets started, and of course those packets will be of the 9k jumbo variety, which makes the first problem worse as well. -GAWollman From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 16:38:13 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6A6D31C2; Mon, 11 Mar 2013 16:38:13 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vb0-x230.google.com (mail-vb0-x230.google.com [IPv6:2607:f8b0:400c:c02::230]) by mx1.freebsd.org (Postfix) with ESMTP id 1B690ED4; Mon, 11 Mar 2013 16:38:13 +0000 (UTC) Received: by mail-vb0-f48.google.com with SMTP id fc21so1716058vbb.7 for ; Mon, 11 Mar 2013 09:38:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=z0J2x4ZETnmxWUX3NiOEpEHzEGLJFLx2qjig+GFlnZA=; b=Bq3a3Lm+dG9/COGiRiCbr9a0sEv+aPiRpxnatFgpDoQ0ZEwCURoNdxPMvLOiSlgIZ6 PQNONncsl17pG5VuZUfe6s7unN2VYaEwwurhrbjYtFb3lfUWxwe6yh2Rxm3tNbivJFdV PTfAKoeeezZ0yzrjXTQ9NKLODCMf1CkfFxxDBWiXs1xg533wUjWpDZAXlg/RhFrL58j7 a/hhIaWm37Gni7ZYVd7sdSosYXwAjbzRq8Tw70XjFRs5VGn27/IvEXeWmBovWLYxm8iu YEcDLKYU4X7Lp5l7Lri3JBR4Kaxwg4+AMlr2TW7SasQBKlfKCeeTQ7Zc+OKDV0ENCcPz OaHA== MIME-Version: 1.0 X-Received: by 10.52.30.48 with SMTP id p16mr4566607vdh.118.1363019892571; Mon, 11 Mar 2013 09:38:12 -0700 (PDT) Received: by 10.220.191.132 with HTTP; Mon, 11 Mar 2013 09:38:12 -0700 (PDT) In-Reply-To: <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> <513DB550.5010004@freebsd.org> <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> Date: Mon, 11 Mar 2013 09:38:12 -0700 Message-ID: Subject: Re: Limits on jumbo mbuf cluster allocation From: Jack Vogel To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, rmacklem@uoguelph.ca, andre@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 16:38:13 -0000 How large are you configuring your rings Garrett? Maybe if you tried reducing them? Jack On Mon, Mar 11, 2013 at 9:05 AM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article <513DB550.5010004@freebsd.org>, andre@freebsd.org writes: > > >Garrett's problem is receive side specific and NFS can't do much about it. > >Unless, of course, NFS is holding on to received mbufs for a longer time. > > Well, I have two problems: one is running out of mbufs (caused, we > think, by ixgbe requiring 9k clusters when it doesn't actually need > them), and one is livelock. Allowing potentially hundreds of clients > to queue 2 MB of requests before TCP pushes back on them helps to > sustain the livelock once it gets started, and of course those packets > will be of the 9k jumbo variety, which makes the first problem worse > as well. > > -GAWollman > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 16:51:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9A5D487C for ; Mon, 11 Mar 2013 16:51:21 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x232.google.com (mail-bk0-x232.google.com [IPv6:2a00:1450:4008:c01::232]) by mx1.freebsd.org (Postfix) with ESMTP id 24721F85 for ; Mon, 11 Mar 2013 16:51:20 +0000 (UTC) Received: by mail-bk0-f50.google.com with SMTP id jg9so1828326bkc.37 for ; Mon, 11 Mar 2013 09:51:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=WXjJkv9LY0P3kftCOg/sWVzJR7yEN9eks/TjFAt1hsA=; b=GiDmSpGpptAXL48jcZxjmTfCyiAgT8R2EfK9v02Psl9DaVqnIsRWwu/UoQcUOf+Ja/ L1QNxdblkmopMci4cnYTq7r01OelyqBDWoloeXZrGv+IAOvEXpGTrQe0vyfoQ7OCmfXY e+VPzXdsClsV6cg9twG7PaiBBndzYwF3PmX0ZwXkKGex0UbooNBSWdgrDlBKSL7O+PDB 4Tw1OTt4UMjcD6b4kbgZnS3gO/KZdIsxocuMrnNUYa0v5uYAwP0OWon5tGiphRs1GLqN UmQcoR3WzqbsyGGMIOf5C91P0BV3WcZfnU2/VmU0fOhcSOAQ2lbrmWIJ5bVPSLU50HAG u/kQ== X-Received: by 10.205.122.80 with SMTP id gf16mr4973231bkc.130.1363020680024; Mon, 11 Mar 2013 09:51:20 -0700 (PDT) Received: from zvezda.localnet ([212.48.107.10]) by mx.google.com with ESMTPS id v2sm4339726bkw.5.2013.03.11.09.51.19 (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 11 Mar 2013 09:51:19 -0700 (PDT) From: Kajetan Staszkiewicz To: Ermal =?utf-8?q?Lu=C3=A7i?= Subject: Re: [patch] Source entries removing is awfully slow. Date: Mon, 11 Mar 2013 17:51:18 +0100 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <201303081419.17743.vegeta@tuxpowered.net> <201303111605.19518.vegeta@tuxpowered.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201303111751.18274.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQmBqLLWVVDz/mYELYnWKm0NYS5PEwX8cKjH618J7fwyOEwuo9jr4GlBS+4eFoL4YO5O45NU Cc: "freebsd-net@freebsd.org" , "freebsd-pf@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 16:51:21 -0000 Dnia poniedzia=C5=82ek, 11 marca 2013 o 16:27:33 Ermal Lu=C3=A7i napisa=C5= =82(a): > On Mon, Mar 11, 2013 at 4:05 PM, Kajetan Staszkiewicz > =20 > > wrote: > >=20 > > There are some things I find flawed in your patch: > >=20 > > 1. > >=20 > > +#if 0 > >=20 > > if (killed > 0) > > =20 > > pf_purge_expired_src_nodes(1); > >=20 > > +#endif > >=20 > > This means that after using `pfctl -K` the src nodes are still around > > until purged and any new states created will still use them and bump > > their expire timer. This also changes behavior from DIOCCLRSRCNODES, > > which also performs the > > purge immediately. You also moved s->src_node=3Ds->nat_src_node=3DNULL = code > > to inside of pf_purge_expired_src_nodes, therefore I believe it should > > be called > > immediately. If detaching state from source is done in > > pf_purge_expired_src_nodes, DIOCCLRSRCNODES does not have to traverse t= he > > state > > table anymore, so we achieve another performance improvement. >=20 > Well probably just need to clear the rtaddr member of a source node. rt_addr is a property of pf_state (or did you mean raddr of pf_src_node?), = I do=20 not know if the rest of pf code allows it to be just cleared, and even if, = it=20 would only create broken State entries not forwarding data to the server be= hind=20 the loadbalancer (I *might* want to leave existing connections alive, depen= ding=20 on usage of further mentioned pfctl's "-c" switch, but I always want to rem= ove=20 Sources ASAP so no new connections use them). And of course I prefer to kee= p=20 the default behavior unchanged. > Its an optimization to keep the source node there until next purge since > you would avoid a memory allocation. > It seems to me a better behaviour rather than having to loop N amount of > states during an ioctl which is supposed to be fast. This was the original behavior of the code, I only reduced N to reasonable= =20 values (and therefore time of operation). > You need to take into consideration that an ioctl freezes the whole > operation of network in this case because you are modifying > the core of pf(4). Yes, this is why I started working on the issue in the first place. My appr= oach=20 introduces no additional overhead, only redurces the existing one. > I would rather be keen to mark a src_node state as expired and let the > purge_thread collect that? What I observed was that without calling pf_purge_expired_src_nodes (your=20 patch), Sources were still displayed with 0 expiration time and if any clie= nt=20 connected before they got purged, expiration time was bumped up and the Sou= rce=20 entry was alive again and never purged anymore. This is definetely not the= =20 behavior I'd expect. >=20 > > 2. > >=20 > > /* Handle state to src_node linkage */ > >=20 > > +#ifndef __FreeBSD__ > >=20 > > if (sn->states !=3D 0) { > > =20 > > RB_FOREACH(s, pf_state_tree_id, > >=20 > > #ifdef __FreeBSD__ > >=20 > > &V_tree_id) { > >=20 > > #else > >=20 > > &tree_id) { > >=20 > > #endif > >=20 > > if (s->src_node =3D=3D sn) > > =20 > > s->src_node =3D NULL; > > =20 > > if (s->nat_src_node =3D=3D sn) > > =20 > > s->nat_src_node =3D NULL; > > =20 > > } > > sn->states =3D 0; > > =20 > > } > >=20 > > +#endif > >=20 > > sn->expire =3D 1; > > killed++; > >=20 > > This removes a bit too much code, that is zeroing of source's state > > counter. > >=20 > > Please find the next version of the patch here: > > http://vegeta.tuxpowered.net/download/link-states-to-src_node-3.patch > >=20 > > This one also takes care of removing states linked to found sources if > > pfctl is > > given extra -c parameter (that can stand for "clear", I could not find > > any other free pfctl parameter better matching). Thanks to this > > parameter, the default behavior is not changed. >=20 > Its ok for the extra -c, there is even -S free just to be consistent with > the flush and vieweing options probably better use that?! > The addition of 2 extra members to pf_state structure i am reluctant a bit > since its just for reporting the number of states killed > and it really is not a useful information compared to the increased size = of > the structure that consumes RAM memory. Variable for counting terminated states was added to pfioc_src_node_kill, n= ot=20 to pf_state. > Also pf_src_node structure if the 2 options for display are not added can > use the member that is used to report > the number of states killed to request killing linked states. > What do you think? Again, the counter is in pfioc_src_node_kill, not pf_src_node. =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 17:09:11 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 12D20FC6 for ; Mon, 11 Mar 2013 17:09:11 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4EEA8116 for ; Mon, 11 Mar 2013 17:09:10 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2BH99jq073668; Mon, 11 Mar 2013 13:09:09 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2BH999Z073667; Mon, 11 Mar 2013 13:09:09 -0400 (EDT) (envelope-from wollman) Date: Mon, 11 Mar 2013 13:09:09 -0400 (EDT) From: Garrett Wollman Message-Id: <201303111709.r2BH999Z073667@hergotha.csail.mit.edu> To: jfvogel@gmail.com Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> <513DB550.5010004@freebsd.org> <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Mon, 11 Mar 2013 13:09:09 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 17:09:11 -0000 In article , jfvogel@gmail.com writes: >How large are you configuring your rings Garrett? Maybe if you tried >reducing them? I'm not configuring them at all. (Well, hmmm, I did limit the number of queues to 6 (per interface, it appears, so that's 12 in all).) There's a limit to how much experimentation my users will tolerate on a production file server, so I can't realistically try anything out until next weekend at the very earliest. I'm bringing up a second server soon, but it won't have anything remotely like a real load on it for some time. -GAWollman From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 17:36:57 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EB8DAEF8 for ; Mon, 11 Mar 2013 17:36:57 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ve0-f171.google.com (mail-ve0-f171.google.com [209.85.128.171]) by mx1.freebsd.org (Postfix) with ESMTP id AF61328A for ; Mon, 11 Mar 2013 17:36:57 +0000 (UTC) Received: by mail-ve0-f171.google.com with SMTP id b10so2924339vea.30 for ; Mon, 11 Mar 2013 10:36:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=qoKs05O2a9Nmas0iaSlsk2rv1v9d2qg4mf2wVTpug9Q=; b=zsdlPP01JfWFkfgkBpdWBsXZnjloo5CA4PbcaJzSjRjBtTMqqMRuvAbcmluQkhGmPN bhIIXiUhg3SdA2Za22Yj3RNMjz3cH9bF3ijF03Diwi+AD4+a4gCVgnRYWY3Fz7JP9Cj/ wjM0kVR2qtwWEWy0JjPU3q4SkVGxRF3v1RjppCF0M5C22JchAltIlgXggdbKerhuiLXd hnJA7Xpo5Xiedb3UkFcQcXx0kj4UOxXixyXlbwrgRK0jzdoB55c5sBsHttEvwBVNnUzv FWt8dGcbRoCpF0hTcCjq2ToGAk9922KGjSBmMkZT2jp+vfNYYLCpIWqrCeHLBhbLSNzx NIpA== MIME-Version: 1.0 X-Received: by 10.58.214.231 with SMTP id od7mr5356611vec.44.1363023411087; Mon, 11 Mar 2013 10:36:51 -0700 (PDT) Received: by 10.220.191.132 with HTTP; Mon, 11 Mar 2013 10:36:50 -0700 (PDT) In-Reply-To: <201303111709.r2BH999Z073667@hergotha.csail.mit.edu> References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> <513DB550.5010004@freebsd.org> <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> <201303111709.r2BH999Z073667@hergotha.csail.mit.edu> Date: Mon, 11 Mar 2013 10:36:50 -0700 Message-ID: Subject: Re: Limits on jumbo mbuf cluster allocation From: Jack Vogel To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 17:36:58 -0000 Then you are using the default ring size, which is 2K descriptors, you might try reducing to 1K and see how that works. Jack On Mon, Mar 11, 2013 at 10:09 AM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article > , > jfvogel@gmail.com writes: > > >How large are you configuring your rings Garrett? Maybe if you tried > >reducing them? > > I'm not configuring them at all. (Well, hmmm, I did limit the number > of queues to 6 (per interface, it appears, so that's 12 in all).) > > There's a limit to how much experimentation my users will tolerate on > a production file server, so I can't realistically try anything out > until next weekend at the very earliest. I'm bringing up a second > server soon, but it won't have anything remotely like a real load on > it for some time. > > -GAWollman > > From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 20:24:25 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1190D316 for ; Mon, 11 Mar 2013 20:24:25 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 7C196E90 for ; Mon, 11 Mar 2013 20:24:24 +0000 (UTC) Received: (qmail 5619 invoked from network); 11 Mar 2013 21:37:04 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 11 Mar 2013 21:37:04 -0000 Message-ID: <513E3D75.7010803@freebsd.org> Date: Mon, 11 Mar 2013 21:24:21 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Garrett Wollman Subject: Re: Limits on jumbo mbuf cluster allocation References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> In-Reply-To: <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, rmacklem@uoguelph.ca X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 20:24:25 -0000 On 11.03.2013 17:05, Garrett Wollman wrote: > In article <513DB550.5010004@freebsd.org>, andre@freebsd.org writes: > >> Garrett's problem is receive side specific and NFS can't do much about it. >> Unless, of course, NFS is holding on to received mbufs for a longer time. > > Well, I have two problems: one is running out of mbufs (caused, we > think, by ixgbe requiring 9k clusters when it doesn't actually need > them), and one is livelock. Allowing potentially hundreds of clients > to queue 2 MB of requests before TCP pushes back on them helps to > sustain the livelock once it gets started, and of course those packets > will be of the 9k jumbo variety, which makes the first problem worse > as well. I think that TCP, or rather the send socket buffer, currently doesn't push back at all but simply accepts everything that gets thrown at it. This obviously is a problem and the NFS server seems to depend somewhat on that by requiring atomicity on a RPC send. I have to trace the mbuf path through NFS to the socket to be sure. The code is slightly opaque though. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Mar 11 22:04:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CBC8EA9C; Mon, 11 Mar 2013 22:04:50 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 69381698; Mon, 11 Mar 2013 22:04:50 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2BM4nao077096; Mon, 11 Mar 2013 18:04:49 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2BM4ngW077095; Mon, 11 Mar 2013 18:04:49 -0400 (EDT) (envelope-from wollman) Date: Mon, 11 Mar 2013 18:04:49 -0400 (EDT) From: Garrett Wollman Message-Id: <201303112204.r2BM4ngW077095@hergotha.csail.mit.edu> To: andre@freebsd.org Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: <513E3D75.7010803@freebsd.org> References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca> <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Mon, 11 Mar 2013 18:04:49 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 22:04:50 -0000 In article <513E3D75.7010803@freebsd.org>, andre@freebsd.org writes: >On 11.03.2013 17:05, Garrett Wollman wrote: >> Well, I have two problems: one is running out of mbufs (caused, we >> think, by ixgbe requiring 9k clusters when it doesn't actually need >> them), and one is livelock. Allowing potentially hundreds of clients >> to queue 2 MB of requests before TCP pushes back on them helps to >> sustain the livelock once it gets started, and of course those packets >> will be of the 9k jumbo variety, which makes the first problem worse >> as well. > >I think that TCP, or rather the send socket buffer, currently doesn't >push back at all but simply accepts everything that gets thrown at it. I'm talking about the receive side here -- the send side doesn't really matter (and for NFS you probably want to have larger send buffers than receive buffers anyway -- once you've invested all the resources required to complete a request, you don't want it to fail at the very last moment). The normal TCP window mechanism provides all the backpressure that's needed, if the buffers were not artificially inflated. It's very frustrating to configure a reasonable buffer size for the application and then have the NFS code stomp all over the configured values. -GAWollman From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 00:39:34 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8C7B1C9F; Tue, 12 Mar 2013 00:39:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 3BB48CC3; Tue, 12 Mar 2013 00:39:33 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAFJ4PlGDaFvO/2dsb2JhbABDiCm8O4FzdIIpAQEBBAEBASArIAsbGAICDRkCKQEJJgYIBwQBHASHcgytE5JegSOMOn00B4ItgRMDlBeCPoEej1eDJiAygQU1 X-IronPort-AV: E=Sophos;i="4.84,827,1355115600"; d="scan'208";a="18199196" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 11 Mar 2013 20:39:26 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9D5B1B3F14; Mon, 11 Mar 2013 20:39:26 -0400 (EDT) Date: Mon, 11 Mar 2013 20:39:26 -0400 (EDT) From: Rick Macklem To: Andre Oppermann Message-ID: <1475929068.3794602.1363048766615.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <513E3D75.7010803@freebsd.org> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 00:39:34 -0000 Andre Oppermann wrote: > On 11.03.2013 17:05, Garrett Wollman wrote: > > In article <513DB550.5010004@freebsd.org>, andre@freebsd.org writes: > > > >> Garrett's problem is receive side specific and NFS can't do much > >> about it. > >> Unless, of course, NFS is holding on to received mbufs for a longer > >> time. > > > > Well, I have two problems: one is running out of mbufs (caused, we > > think, by ixgbe requiring 9k clusters when it doesn't actually need > > them), and one is livelock. Allowing potentially hundreds of clients > > to queue 2 MB of requests before TCP pushes back on them helps to > > sustain the livelock once it gets started, and of course those > > packets > > will be of the 9k jumbo variety, which makes the first problem worse > > as well. > > I think that TCP, or rather the send socket buffer, currently doesn't > push back at all but simply accepts everything that gets thrown at it. > This obviously is a problem and the NFS server seems to depend > somewhat > on that by requiring atomicity on a RPC send. I have to trace the mbuf > path through NFS to the socket to be sure. The code is slightly opaque > though. > Yes, I think you are correct that when NFS sends RPC messages over TCP, they just get queued via sbappendstream(). For some reason I thought the krpc used sosend_generic(), whereas I just looked and it just uses sosend(), which does nothing except call tcp_user_send() { for TCP sockets, of course }. I tend to agree with Garrett that this is ok for the NFS server, since once it has done the work of generating a reply, why should the nfsd thread get stuck trying to queue the reply for the client. For the NFS client, it isn't quite so obvious, but after queuing a request to be sent, it will sit waiting for the reply, so it will "see" the NFS server's slow response. (Anyhow, Garrett couldn't care less about the FreeBSD NFS client, since he isn't using it.;-) rick > -- > Andre > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 01:25:46 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9D6E83FB; Tue, 12 Mar 2013 01:25:46 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 55F3FE52; Tue, 12 Mar 2013 01:25:46 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAByDPlGDaFvO/2dsb2JhbABDDogdvDuBdXSCKQEBAQMBAQEBIAQnIAsbGAICDRkCKQEJJgYIBwQBGgIEh2wGDK0PkmGBI4wqCwV9NAeCLYETA5QXgj6BHo9XgktbIDJ8AQQEFx4 X-IronPort-AV: E=Sophos;i="4.84,827,1355115600"; d="scan'208";a="18203533" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 11 Mar 2013 21:25:45 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 70F08B3F0B; Mon, 11 Mar 2013 21:25:45 -0400 (EDT) Date: Mon, 11 Mar 2013 21:25:45 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <22122027.3796089.1363051545440.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, andre@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 01:25:46 -0000 Garrett Wollman wrote: > In article <513DB550.5010004@freebsd.org>, andre@freebsd.org writes: > > >Garrett's problem is receive side specific and NFS can't do much > >about it. > >Unless, of course, NFS is holding on to received mbufs for a longer > >time. The NFS server only holds onto receive mbufs until it performs the RPC requested. Of course, if the server hits its load limit, there will then be a backlog of RPC requests --> the received mbufs for these requests will be held for a longer time. To be honest, I'd consider seeing a lot of non-empty receive queues for TCP connections to the NFS server to be an indication that it is near/at its load limit. (Sure, if you do netstat a lot, you will occasionally see a non-empty queue here or there, but I would not expect to see a lot of them non-empty a lot of the time.) If that is the case, then the question becomes "what is the bottleneck?". Below I suggest getting rid of the DRC in case it is the bottleneck for your server. > > Well, I have two problems: one is running out of mbufs (caused, we > think, by ixgbe requiring 9k clusters when it doesn't actually need > them), and one is livelock. Allowing potentially hundreds of clients > to queue 2 MB of requests before TCP pushes back on them helps to > sustain the livelock once it gets started, and of course those packets > will be of the 9k jumbo variety, which makes the first problem worse > as well. > The problem for the receive side is "how small should you make it?". Suppose we have the following situation: - only one client is active and it is flushing writes for a large file written into that client's buffer cache. --> If you set the receive size so that it is just big enough for one write, then the client will end up doing: - send one write, wait a long while for the NFS_OK reply - send the next write, wait a long while for the NFS_OK reply and so on --> the write back will take a long time, even though no other client is generating load on the server. --> the user for this client won't be happy If you make the receive side large enough to handle several Write requests, then the above works much faster, however... - the receive size is now large enough to accept many many other RPC requests (a Write request is 64Kbytes+ however Read requests are typically less than 100bytes) Even if you set the receive size to the minimum that will handle one Write request, that will allow the client to issue something like 650 Read requests. Since NFS clients wait for replies to the RPC requests they send, they will only queue so many requests before sending no more of them until they receive some replies. This does delay the "feedback" somewhat, but I'd argue that buffering of requests in the server's receive queue helps when clients generate bursts of requests on a server that is well below its load limit. Now, I'm not sure I understand what you mean by "livelock"? A - Do you mean that the server becomes unresponsive and is generating almost no RPC replies, with all the clients are reporting "NFS server not responding"? or B - Do you mean that the server keeps responding to RPCs at a steady rate, but that rate is slower than what the clients (and their users) would like to see? If it is B, I'd just consider that as hitting the server's load limit. For either A or B, I'd suggest that you disable the DRC for TCP connections (email if you need a patch for that), which will have a couple of effects: 1 - It will avoid the DRC from defining the server's load limit. (If the DRC is the server's bottleneck, this will increase the server's load limit to whatever else is the next bottleneck.) 2 - If the mbuf clusters held by the DRC are somehow contributing to the mbuf cluster allocation problem for the receive side of the network interface, this would alleviate that. (I'm not saying it fixes the problem, but might allow the server to avoid it under the driver guys come up with a good solution for it.) rick > -GAWollman > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 04:30:02 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 325EBD1; Tue, 12 Mar 2013 04:30:02 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id DE9EE758; Tue, 12 Mar 2013 04:30:01 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2C4TxJk080797; Tue, 12 Mar 2013 00:29:59 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2C4Tx1A080794; Tue, 12 Mar 2013 00:29:59 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20798.44871.601547.24628@hergotha.csail.mit.edu> Date: Tue, 12 Mar 2013 00:29:59 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: <22122027.3796089.1363051545440.JavaMail.root@erie.cs.uoguelph.ca> References: <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu> <22122027.3796089.1363051545440.JavaMail.root@erie.cs.uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Tue, 12 Mar 2013 00:29:59 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org, andre@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 04:30:02 -0000 < said: > To be honest, I'd consider seeing a lot of non-empty receive queues > for TCP connections to the NFS server to be an indication that it is > near/at its load limit. (Sure, if you do netstat a lot, you will occasionally > see a non-empty queue here or there, but I would not expect to see a lot > of them non-empty a lot of the time.) If that is the case, then the > question becomes "what is the bottleneck?". Below I suggest getting rid > of the DRC in case it is the bottleneck for your server. The problem is not the DRC in "normal" operation, but the DRC when it gets into the livelocked state. I think we've talked about a number of solutions to the livelock problem, but I haven't managed to implement or test these ideas yet. I have a duplicate server up now, so I hope to do some testing this week. In normal operation, the server is mostly idle, and the nfsd threads that aren't themselves idle are sleeping deep in ZFS waiting for something to happen on disk. When the arrival rate exceeds the rate at which requests are cleared from the DRC, *all* of the nfsd threads will spin, either waiting for the DRC mutex or walking the DRC finding that there is nothing that can be released yet. *That* is the livelock condition -- the spinning that takes over all nfsd threads is what causes the receive buffers to build up, and the large queues then maintain the livelocked condition -- and that is why it clears *immediately* when the DRC size is increased. (It's possible to reproduce this condition on a loaded server by simply reducing the tcphighwater to less than the current size.) Unfortunately, I'm at the NFSRC_FLOODSIZE limit right now (64k), so there is no room for further increases until I recompile the kernel. It's probably a bug that the sysctl definition in drc3.patch doesn't check the new value against this limit. Note that I'm currently running 64 nfsd threads on a 12-core (24-thread) system. In the livelocked condition, as you would expect, the system goes to 100% CPU utilization and the load average peaks out at 64, while goodput goes to nearly nil. > For either A or B, I'd suggest that you disable the DRC for TCP connections > (email if you need a patch for that), which will have a couple of effects: I would like to see your patch, since it's more likely to be correct than one I might dream up. The alternative solution is twofold: first, nfsrv_trimcache() needs to do something to ensure forward progress, even when that means dropping something that hasn't timed out yet, and second, the server code needs to ensure that nfsrv_trimcache() is only executing on one thread at a time. An easy way to do the first part would be to maintain an LRU queue for TCP in addition to the UDP LRU, and just blow away the first N (>NCPU) entries on the queue if, after checking all the TCP replies, the DRC is still larger than the limit. The second part is just an atomic_cmpset_int(). -GAWollman From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 17:20:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8E12B43F for ; Tue, 12 Mar 2013 17:20:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 7E470223 for ; Tue, 12 Mar 2013 17:20:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2CHK18r038250 for ; Tue, 12 Mar 2013 17:20:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2CHK1vp038249; Tue, 12 Mar 2013 17:20:01 GMT (envelope-from gnats) Date: Tue, 12 Mar 2013 17:20:01 GMT Message-Id: <201303121720.r2CHK1vp038249@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Charbon, Julien" Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: "Charbon, Julien" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 17:20:01 -0000 The following reply was made to PR kern/176446; it has been noted by GNATS. From: "Charbon, Julien" To: Cc: bug-followup@freebsd.org, "De La Gueronniere, Marc" Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST Date: Tue, 12 Mar 2013 14:59:11 +0100 This is a multi-part message in MIME format. --------------050609010000050001050509 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 3/7/13 11:11 AM, Charbon, Julien wrote: > On 2/28/13 8:10 PM, Charbon, Julien wrote: >> On 2/28/13 4:57 PM, John Baldwin wrote: >>> Can you try the fixes from http://svnweb.freebsd.org/base?view=revision&revision=240968? >> >> Actually, Marc (I CC'ed him) did find the r240968 fix for concurrency >> between ixgbe_msix_que() and ixgbe_handle_que(), and made a backport for >> release-8.3.0 (see patch [1] below). However, the issue was still >> reproducible, then Marc found another place for concurrency from >> ixgbe_local_timer() and fix it (see patch [2]). But it was still not >> enough, and he found a last place for concurrency due to >> ixgbe_rearm_queues() call (see patch [3]). We all these patches >> applied, we were not able to reproduce this issue. > > Just for the record: As expected this issue is reproducible on > 9.1-RELEASE: Just for the record: As expected this issue is reproducible also on 10.0-CURRENT: # uname -a FreeBSD atlas 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r248173M: Tue Mar 12 07:52:58 UTC 2013 jcharbon@atlas:/usr/obj/app/jcharbon/head/sys/GENERIC amd64 1. Enable TCP debug log: # sysctl net.inet.tcp.log_debug=1 net.inet.tcp.log_debug: 1 -> 1 2. Load a TCP service with numerous small requests/responses: 3. Look in /var/log/debug.log for the pattern: Mar 12 10:31:22 atlas kernel: TCP: [192.168.100.35]:4698 to [192.168.100.152]:8080; syncache_socket: in_pcbconnect failed with error 48 Mar 12 10:31:22 atlas-dl360-4 kernel: TCP: [192.168.100.35]:4698 to [192.168.100.152]:8080 tcpflags 0x10; tcp_input: Listen socket: Socket allocation failed due to limits or memory shortage, sending RST Mar 12 10:31:22 atlas-dl360-4 kernel: TCP: [192.168.100.35]:4698 to [192.168.100.152]:8080 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Joined the patch we use to fix this issue in 10-CURRENT. -- Julien --------------050609010000050001050509 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="ixgbe.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ixgbe.c.patch" Index: sys/dev/ixgbe/ixgbe.c =================================================================== --- sys/dev/ixgbe/ixgbe.c (revision 248173) +++ sys/dev/ixgbe/ixgbe.c (working copy) @@ -2038,14 +2038,14 @@ (paused == 0)) ++hung; else if (txr->queue_status == IXGBE_QUEUE_WORKING) - taskqueue_enqueue(que->tq, &que->que_task); + taskqueue_enqueue(que->tq, &txr->txq_task); } /* Only truely watchdog if all queues show hung */ if (hung == adapter->num_queues) goto watchdog; out: - ixgbe_rearm_queues(adapter, adapter->que_mask); + // ixgbe_rearm_queues(adapter, adapter->que_mask); callout_reset(&adapter->timer, hz, ixgbe_local_timer, adapter); return; @@ -4575,7 +4575,7 @@ ** Schedule another interrupt if so. */ if ((staterr & IXGBE_RXD_STAT_DD) != 0) { - ixgbe_rearm_queues(adapter, (u64)(1 << que->msix)); + // ixgbe_rearm_queues(adapter, (u64)(1 << que->msix)); return (TRUE); } --------------050609010000050001050509-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 20:06:42 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EE5B4A49 for ; Tue, 12 Mar 2013 20:06:42 +0000 (UTC) (envelope-from universite@ukr.net) Received: from ffe17.ukr.net (ffe17.ukr.net [195.214.192.83]) by mx1.freebsd.org (Postfix) with ESMTP id A6E61347 for ; Tue, 12 Mar 2013 20:06:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:Subject:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=5o/DMcNvfXGwCLjaSIHybBfL/BosqkWDeIo5pTt2bSg=; b=VW+1UZ5SfRrIow9Y51h6lhAo3W96wnRJ5+6aVV3Kr3wYD8wQUX9Xkg2ZxAmeV5qG+ysRePHZYA/SNFjLmv0hZNjk//loKS7+cTfqpCEW/Y3wZxDFqD5+4mIvS7IcYiNtrPPQyFl36WPyAOFOUBoUYrwy8oBODDoLsBgFnHQO5BA=; Received: from mail by ffe17.ukr.net with local ID 1UFVT1-000ImE-1V for net@freebsd.org; Tue, 12 Mar 2013 22:06:35 +0200 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" Subject: Problems of using ipv6 addresses on loc0 To: net@freebsd.org From: "Vladislav Prodan" X-Mailer: freemail.ukr.net 4.0 Message-Id: <71891.1363118795.16472820492257460224@ffe17.ukr.net> Date: Tue, 12 Mar 2013 22:06:35 +0200 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 20:06:43 -0000 I have a FreeBSD 8.2-STABLE amd64. Network: vlan300: flags=8843 metric 0 mtu 1500 description: Uplink_1 inet inet6 vlan400: flags=8843 metric 0 mtu 1500 description: Uplink_2 inet inet6 vlan100: flags=8843 metric 0 mtu 1500 description: Internal_LAN inet inet6 2001:67c:21f0::ffff prefixlen 64 lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9 inet6 2001:67c:21f0:f::1 prefixlen 64 nd6 options=3 Some services, such as named, refused to use ipv6 with lo0. Option query-source-v6 address 2001:67c:21f0::1; makes a mistake and named service will not start. Problems with the ipv6 address are on vlan100 - no. P.S. BIND 9.9.2-P1 -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 21:19:02 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 487A4EAF for ; Tue, 12 Mar 2013 21:19:02 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [64.147.113.26]) by mx1.freebsd.org (Postfix) with ESMTP id E1DAF8C6 for ; Tue, 12 Mar 2013 21:19:01 +0000 (UTC) Received: (qmail 60843 invoked from network); 12 Mar 2013 21:12:19 -0000 Received: from unknown (HELO ?10.50.50.200?) (spawk@64.147.100.2) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 12 Mar 2013 21:12:19 -0000 Message-ID: <513F9A2D.8050507@acm.poly.edu> Date: Tue, 12 Mar 2013 17:12:13 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:16.0) Gecko/20121110 Thunderbird/16.0.2 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: mbuf tuning on 9.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 21:19:02 -0000 Hi. I have a FreeBSD 9.1/amd64 machine. It runs HAProxy and runs out of various things frequently. Sometimes it's mbufs. netstat -m: 68202/1698/69900 mbufs in use (current/cache/total) 41449/1229/42678/2622144 mbuf clusters in use (current/cache/total/max) 41449/1175 mbuf+clusters out of packet secondary zone in use (current/cache) 26405/538/26943/3276800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 205568K/5034K/210603K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines How can I increase "mbufs," as they appear above, and "mbuf clusters," as they appear above? Thank you. -Boris From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 21:24:34 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A479A218 for ; Tue, 12 Mar 2013 21:24:34 +0000 (UTC) (envelope-from pprocacci@datapipe.com) Received: from EXFESMQ03.datapipe-corp.net (exfesmq03.datapipe.com [64.27.120.67]) by mx1.freebsd.org (Postfix) with ESMTP id 6FA3B919 for ; Tue, 12 Mar 2013 21:24:32 +0000 (UTC) Received: from nat.myhome (192.168.128.103) by EXFESMQ03.datapipe-corp.net (192.168.128.28) with Microsoft SMTP Server (TLS) id 14.2.318.4; Tue, 12 Mar 2013 17:23:21 -0400 Date: Tue, 12 Mar 2013 16:23:44 -0500 From: "Paul A. Procacci" To: Boris Kochergin Subject: Re: mbuf tuning on 9.1 Message-ID: <20130312212344.GN63316@nat.myhome> References: <513F9A2D.8050507@acm.poly.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <513F9A2D.8050507@acm.poly.edu> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [192.168.128.103] Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 21:24:34 -0000 > How can I increase "mbufs," as they appear above, and "mbuf clusters," > as they appear above? You can modify the sysctl's associated with mbufs to suit your needs. https://wiki.freebsd.org/NetworkPerformanceTuning The following link describes what mbufs are and sysctl's governing their operation. ~Paul ________________________________ This message may contain confidential or privileged information. If you are= not the intended recipient, please advise us immediately and delete this m= essage. See http://www.datapipe.com/legal/email_disclaimer/ for further inf= ormation on confidentiality and the risks of non-secure electronic communic= ation. If you cannot access these links, please notify us by reply message = and we will send the contents to you. From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 21:31:54 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 73047389 for ; Tue, 12 Mar 2013 21:31:54 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [64.147.113.26]) by mx1.freebsd.org (Postfix) with ESMTP id 16990960 for ; Tue, 12 Mar 2013 21:31:53 +0000 (UTC) Received: (qmail 60956 invoked from network); 12 Mar 2013 21:31:53 -0000 Received: from unknown (HELO ?10.50.50.200?) (spawk@64.147.100.2) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 12 Mar 2013 21:31:53 -0000 Message-ID: <513F9EC4.8090303@acm.poly.edu> Date: Tue, 12 Mar 2013 17:31:48 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:16.0) Gecko/20121110 Thunderbird/16.0.2 MIME-Version: 1.0 To: "Paul A. Procacci" Subject: Re: mbuf tuning on 9.1 References: <513F9A2D.8050507@acm.poly.edu> <20130312212344.GN63316@nat.myhome> In-Reply-To: <20130312212344.GN63316@nat.myhome> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 21:31:54 -0000 Additionally, can someone clarify the meaning of "total" vs. "max" for these values? -Boris On 03/12/13 17:23, Paul A. Procacci wrote: >> How can I increase "mbufs," as they appear above, and "mbuf clusters," >> as they appear above? > You can modify the sysctl's associated with mbufs to suit your needs. > > https://wiki.freebsd.org/NetworkPerformanceTuning > > The following link describes what mbufs are and sysctl's governing > their operation. > > ~Paul > > ________________________________ > > This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 21:52:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5317CC51 for ; Tue, 12 Mar 2013 21:52:50 +0000 (UTC) (envelope-from m-freebsd@fuglos.org) Received: from m.fuglos.org (m.fuglos.org [217.11.61.114]) by mx1.freebsd.org (Postfix) with ESMTP id 1E152A50 for ; Tue, 12 Mar 2013 21:52:49 +0000 (UTC) Received: by m.fuglos.org (Postfix, from userid 1001) id 4BA3635C456; Tue, 12 Mar 2013 22:42:51 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by m.fuglos.org (Postfix) with ESMTP id 441AC35C44F for ; Tue, 12 Mar 2013 22:42:51 +0100 (CET) Date: Tue, 12 Mar 2013 22:42:51 +0100 (CET) From: "M. Schulte" X-X-Sender: mel@m.fuglos.org To: freebsd-net@freebsd.org Subject: ntpd bind() failure: Can't assign requested address Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 21:52:50 -0000 Hi! [First of all, I have posted this question already on the FreeBSD forum -- so far without replies -- and now my hope is that the set of subscribers here and those of the forum do not completely coincide.] I have installed FreeBSD 9.1 on my server (it's a virtualized qemu/kvm environment, in case that matters) and during boot the following messages appear: ntpd[766]: ntpd 4.2.4p5-a (1) ntpd[767]: bind() fd 23, family AF_INET6, port 123, scope 2, addr fe80::216:36ff:fe74:2076, mcast=0 flags=0x11 fails: Can't assign requested address ntpd[767]: unable to create socket on re0 (3) for fe80::216:36ff:fe74:2076#123 This happens with a GENERIC kernel. I have not touched any IPv6 related configuration after install, so everything should be in the default state. Neither have I changed the ntp/ntpd configuration. Although I could find some threads where people were discussing the same problem, none of the mentioned suggestions fixed this for me. In particular I tried setting ipv6_activate_all_interfaces="YES" in rc.conf -- didn't fix the problem. According to ifconfig, my interfaces, re0 in particular, have IPv6 addresses associated to them. So why would it fail that a process tries to bind to it? Note that I'm rather unexperienced wrt IPv6. Would be great if somebody could give me a hint into the right direction. Anybody else experiencing this? Thank you very much! ~ melanie From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 22:44:39 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 845A11AE for ; Tue, 12 Mar 2013 22:44:39 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 20A3ADF0 for ; Tue, 12 Mar 2013 22:44:38 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hn17so2294796wib.4 for ; Tue, 12 Mar 2013 15:44:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to:x-gm-message-state; bh=Zuj13vEDBC3+jE16L+P1+8prRMwBE7MaqKjWoCfxN2U=; b=JV9ypCbz40L5KCK1g8OQ4SA5AalU8n8av8YeJrUfUVLGu5n4mwPj3uO9aeIhap0oxj J5JKL62CX3VmhtBNKzkaPNHtaVd9A+c8/o1nOgZQ6Wf9iD7vtxBp9Jbx+dlXV7Azlco+ Po8TVc0lz0EJZPB1jMjdrXIZHVHcq4x/Z9R9HNdoa95swbiO40gK6jRDrt4OHou+j6Gd tM71xZPFwZz3waxN5Ad8SKaj55JdN/2uKlgSjFN7EoG5yh9xWalyA1MYdz62Ar5HnBaH gpBzz6QBMnPOJrwBuPEcRYbTZly9URDxzK8gFlTfF/4+usrTbcOL9kzWkOpdlgTsB4mD 2erg== X-Received: by 10.194.109.35 with SMTP id hp3mr30224129wjb.15.1363128278238; Tue, 12 Mar 2013 15:44:38 -0700 (PDT) Received: from [192.168.0.14] (did75-17-88-165-130-96.fbx.proxad.net. [88.165.130.96]) by mx.google.com with ESMTPS id o8sm24911205wix.7.2013.03.12.15.44.36 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 12 Mar 2013 15:44:37 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <76E2EE0D-B6D3-428E-9112-AF0E04E98DDE@my.gd> X-Mailer: iPhone Mail (10B144) From: Damien Fleuriot Subject: Re: ntpd bind() failure: Can't assign requested address Date: Tue, 12 Mar 2013 23:44:18 +0100 To: "M. Schulte" X-Gm-Message-State: ALoCoQk99FwkBSQBjk7c+tBQYOwrLzQV0uH9G7EAedqMPXT/x2R9lVgV0St1i4o/wkqQe01myMf8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 22:44:39 -0000 On 12 Mar 2013, at 22:42, "M. Schulte" wrote: > Hi! >=20 > [First of all, I have posted this question already on the FreeBSD > forum -- so far without replies -- and now my hope is that the set of > subscribers here and those of the forum do not completely coincide.] >=20 > I have installed FreeBSD 9.1 on my server (it's a virtualized qemu/kvm > environment, in case that matters) and during boot the following > messages appear: >=20 > ntpd[766]: ntpd 4.2.4p5-a (1) > ntpd[767]: bind() fd 23, family AF_INET6, port 123, scope 2, addr fe80::= 216:36ff:fe74:2076, mcast=3D0 flags=3D0x11 fails: Can't assign requested add= ress > ntpd[767]: unable to create socket on re0 (3) for fe80::216:36ff:fe74:20= 76#123 >=20 > This happens with a GENERIC kernel. I have not touched any IPv6 > related configuration after install, so everything should be in the > default state. Neither have I changed the ntp/ntpd configuration. >=20 > Although I could find some threads where people were discussing the > same problem, none of the mentioned suggestions fixed this for me. >=20 > In particular I tried setting >=20 > ipv6_activate_all_interfaces=3D"YES" >=20 > in rc.conf -- didn't fix the problem. >=20 > According to ifconfig, my interfaces, re0 in particular, have IPv6 > addresses associated to them. So why would it fail that a process > tries to bind to it? Note that I'm rather unexperienced wrt IPv6. >=20 > Would be great if somebody could give me a hint into the right > direction. Anybody else experiencing this? Thank you very much! >=20 > ~ melanie >=20 I'll check tomorrow if we have the same issue at work, although it doesn't l= ook familiar. We've got a 9.1 with a custom kernel (although nothing related to ipv6 chang= ed) running in KVM.= From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 22:48:50 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 93A323AE for ; Tue, 12 Mar 2013 22:48:50 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 1596DE29 for ; Tue, 12 Mar 2013 22:48:49 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hn17so2296806wib.4 for ; Tue, 12 Mar 2013 15:48:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to:x-gm-message-state; bh=LVP96pu9j/5jTTLARSzICk4VQPzG8V7n+CjWL7AveiA=; b=ecEhERdhrTGOqa1yDQzQZOx61mNvdxQl1fKKMzODQSug/qmEeag9mhr7t+Zsr1huAh +ICyc0jO4U4rlaN4yWlGJybv3mzdj0O9DTcMw6YWRmn+GoLgJ8dConOAwZsCRd9mdIyO K6kW63G/3tjOuVGmq+WVqlFupkCJAygTO2oRHaER3u0SMfMDwqH+kwSYWN9WXJQ1ODRc 4KoXZ/WqOkeAgtPoWiBhfJmFH+gWLCCDViz1tIIdp7HXwjiOyqY2/PGmTVX8mKlet1HW r4qLvPvM09MywavnKcA06dD5zFQ1+bU7D01YQr48X7Z1wmFtJajCddtdAjh7biTjTRXL RJMg== X-Received: by 10.181.11.164 with SMTP id ej4mr23274743wid.29.1363128529265; Tue, 12 Mar 2013 15:48:49 -0700 (PDT) Received: from ?IPv6:2a01:e35:8a58:2600:44f:9d86:7662:2487? ([2a01:e35:8a58:2600:44f:9d86:7662:2487]) by mx.google.com with ESMTPS id ed6sm27252525wib.9.2013.03.12.15.48.47 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 12 Mar 2013 15:48:48 -0700 (PDT) References: <71891.1363118795.16472820492257460224@ffe17.ukr.net> Mime-Version: 1.0 (1.0) In-Reply-To: <71891.1363118795.16472820492257460224@ffe17.ukr.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <2F8A1662-CE4F-44A8-81B9-B84970A2E4FD@my.gd> X-Mailer: iPhone Mail (10B144) From: Damien Fleuriot Subject: Re: Problems of using ipv6 addresses on loc0 Date: Tue, 12 Mar 2013 23:48:29 +0100 To: Vladislav Prodan X-Gm-Message-State: ALoCoQnIZph8oxvqccmehrdBFqCm9JFvVm1wdCJexCADPQr1ver7BwTW3DrhBfNZ1pWk0vXdAupo Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 22:48:50 -0000 On 12 Mar 2013, at 21:06, "Vladislav Prodan" wrote: >=20 > I have a FreeBSD 8.2-STABLE amd64. >=20 > Network: > vlan300: flags=3D8843 metric 0 mtu= 1500 > description: Uplink_1 > inet > inet6 =20 >=20 > vlan400: flags=3D8843 metric 0 mtu= 1500 > description: Uplink_2 > inet > inet6 =20 >=20 > vlan100: flags=3D8843 metric 0 mtu= 1500 > description: Internal_LAN > inet > inet6 2001:67c:21f0::ffff prefixlen 64 >=20 > lo0: flags=3D8049 metric 0 mtu 16384 > options=3D3 > inet 127.0.0.1 netmask 0xff000000 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9 > inet6 2001:67c:21f0:f::1 prefixlen 64 > nd6 options=3D3 >=20 > Some services, such as named, refused to use ipv6 with lo0. > Option > query-source-v6 address 2001:67c:21f0::1; > makes a mistake and named service will not start. > Problems with the ipv6 address are on vlan100 - no. >=20 > P.S. BIND 9.9.2-P1 >=20 > --=20 > Vladislav V. Prodan =20 > System & Network Administrator=20 > http://support.od.ua =20 > +380 67 4584408, +380 99 4060508 > VVP88-RIPE >=20 The addresses for lo0 and BIND's query-source differ (extraneous :f), so unl= ess this is a typo, that may be your problem.= From owner-freebsd-net@FreeBSD.ORG Tue Mar 12 22:50:30 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8A7454BE for ; Tue, 12 Mar 2013 22:50:30 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [IPv6:2001:41d0:1:c74c::1]) by mx1.freebsd.org (Postfix) with ESMTP id 12F88E46 for ; Tue, 12 Mar 2013 22:50:29 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id 5218C1CD7F2 for ; Tue, 12 Mar 2013 22:50:21 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kmm2trbFo9w7 for ; Tue, 12 Mar 2013 22:50:21 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 973921CD7F1 for ; Tue, 12 Mar 2013 22:50:20 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2CMoJCN014548 for ; Tue, 12 Mar 2013 22:50:19 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2CMoINT014547 for freebsd-net@freebsd.org; Tue, 12 Mar 2013 22:50:18 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Tue, 12 Mar 2013 22:50:18 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: ipv6 default router Operation not permitted Message-ID: <20130312225018.GA13589@defiant.konundrum.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qDbXVdCdHGoSgWSk" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 22:50:30 -0000 --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I have a problem reaching my ipv6 default router. # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff ping6: sendmsg: Operation not permitted ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 0 packets received, 100.0% packet loss It is the same issue as posted here : http://lists.freebsd.org/pipermail/freebsd-net/2012-February/031518.html However, I believe the problem Mr Dandy was experiencing was he should have changed his default gateway from: ipv6_defaultrouter=3D"2a01:4f8:61:50c0::1" To: ipv6_defaultrouter=3D"fe80::1%re0" As per Hertzner documentation. http://wiki.hetzner.de/index.php/FreeBSD_installieren/en#Network_Configurat= ion I am not hosted with Hertzner but I have the same broken network configuration and am uncertain of the Correct Fix. I have: ::: /etc/rc.conf ::: ipv6_activate_all_interfaces=3D"YES" ipv6_default_interface=3D"re0" ipv6_network_interfaces=3D"auto" ip6addrctl_policy=3D"ipv6_prefer" ifconfig_re0_ipv6=3D"inet6 2001:41D0:2:E7c4::1 prefixlen 64" ipv6_static_routes=3D"ovhgw default" ipv6_route_ovhgw=3D"2001:41d0:2:e7ff:ff:ff:ff:ff -prefixlen 128 -interface = re0" ipv6_route_default=3D"default 2001:41d0:2:e7ff:ff:ff:ff:ff" The default IPv6 router is _outside_ of my assigned /64 prefix, this is why an interface route is set and is configured this way to ensure the interface route is applied before the default IPv6 router is added to the routing table. If I do not do this I will receive an error on boot to say that the default gateway cannot be reached. There is also no firewall in place on the host OS. However this configuration does not work. # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff ping6: sendmsg: Operation not permitted ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 0 packets received, 100.0% packet loss If I do=20 # ifconfig re0 inet6 accept_rtadv It works : # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff 16 bytes from 2001:41d0:2:e7ff:ff:ff:ff:ff, icmp_seq=3D0 hlim=3D64 time=3D4= =2E935 ms --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev =3D 4.935/4.935/4.935/0.000 ms Alternatively, knowing the Linklayer Address of my default gateway, I could set a static ndp pair : # ifconfig re0 inet6 -accept_rtadv # ndp -s 2001:41d0:2:e7ff:ff:ff:ff:ff 00:05:73:a0:00:00 And it works: # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff 16 bytes from 2001:41d0:2:e7ff:ff:ff:ff:ff, icmp_seq=3D0 hlim=3D64 time=3D3= =2E582 ms --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev =3D 3.582/3.582/3.582/0.000 ms If I do : # ndp -d 2001:41d0:2:e7ff:ff:ff:ff:ff It stops working again : # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff ping6: sendmsg: Operation not permitted ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 0 packets received, 100.0% packet loss It seems the simple solution is to have re0 ACCEPT_RTADV but I am told that my default gateway does not perform router advertisements.=20 In my own networks I have a static default route for inet6 and I am not required to set any interfaces ACCEPT_RTADV. Can anybody shed some light on the correct configuration? Who is at fault here ? - Me - Hosting - FreeBSD - The universe Cheers, C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --qDbXVdCdHGoSgWSk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRP7EpAAoJEBBi7cjNKnTjLS4QAK5XpcmF463QWWvzPp8pH48t 6QXEwBE2FK0BL2LkWykd32Bu6LRCu25rahGP99RDsUCy1pm5rD+j69ubll0YNn6I WcGjfVYokx2MYX7lZbMslIkOk+Vh3KnA1GQdvRSulS/tk9UAM2YAhXKA5TbOmZgM uppGPc1FHcFV1UZDj4BaLXDWWrQMgftedv0uUBeL9pfkBtBoCmMAnPnVt6gpLxWZ 0dtuxz4wk2MxzCT/gpXolaTZZtoI/6Jnu2Y6+aTE4sK8jQmg4/UTAGipZjBDRdJf oI7/x7kEqNl/HbnWrlgurWPSxIFgYdN4u+AljYrq5MKcReVcyQirEt0gTBejr99L fWj/+6XtnW5AYYBtQHHP1fLb5NTz+El3RLUrtCduQcmZq6czoaWXUaMnKP5QCTv0 h7KT95nQeGJnLtiXZVewnoCezllJg8RNPwi8XSkV+AsYFLKw45IEReGVJMWk4SER 6+MEs7iOgap0YcDShitnxcaLauFMp+MKDLWh0g3DSAgjwbSttpshtHAnD1QDxPpG pNZA7QHgg25/++OjbmLp96zuC4zbVJXtvMYRPgiv0p8fMVJR2f83k7QZOb/MwKWc nU+LRs+AMEgQUXLi0YY9RRlwOjVBGLVlE7aetGCZacQT0ruMcFJEDZ/NQlM4O/WO AvnLOpThE9B0mkuvwQFF =RWxT -----END PGP SIGNATURE----- --qDbXVdCdHGoSgWSk-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 00:18:08 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0ACBDD10 for ; Wed, 13 Mar 2013 00:18:08 +0000 (UTC) (envelope-from universite@ukr.net) Received: from ffe17.ukr.net (ffe17.ukr.net [195.214.192.83]) by mx1.freebsd.org (Postfix) with ESMTP id B40932EA for ; Wed, 13 Mar 2013 00:18:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=xOXDDLaJP7KgZbOcYVKtOpfH8txbRIP9tiRqv+IIIgU=; b=K6vLDWNlowPYSOIiqNYBseyVxdQ7BkwGTcs2XG6NTbI6kFUA26wrHdTT3Pb+NT1XSZuoDFRn8Qo2Mgan2+bTkD+cRRIO3YdZ8rN6TG1o6leQ8NTsWzb3sPkuIxNoXwNSd3AVeBQwNcMiHEJfkE520/BxU/b+s6rKMx47a2+AK10=; Received: from mail by ffe17.ukr.net with local ID 1UFZOQ-000IGu-5I ; Wed, 13 Mar 2013 02:18:06 +0200 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" Subject: Re[2]: Problems of using ipv6 addresses on loc0 In-Reply-To: <2F8A1662-CE4F-44A8-81B9-B84970A2E4FD@my.gd> References: <71891.1363118795.16472820492257460224@ffe17.ukr.net> <2F8A1662-CE4F-44A8-81B9-B84970A2E4FD@my.gd> To: "Damien Fleuriot" From: "Vladislav Prodan" X-Mailer: freemail.ukr.net 4.0 Message-Id: <63422.1363133886.17200904183672733696@ffe17.ukr.net> Date: Wed, 13 Mar 2013 02:18:06 +0200 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 00:18:08 -0000 > On 12 Mar 2013, at 21:06, "Vladislav Prodan" wrote: > > > lo0: flags=8049 metric 0 mtu 16384 > > options=3 > > inet 127.0.0.1 netmask 0xff000000 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9 > > inet6 2001:67c:21f0:f::1 prefixlen 64 > > nd6 options=3 > > > > Some services, such as named, refused to use ipv6 with lo0. > > Option > > query-source-v6 address 2001:67c:21f0::1; > > makes a mistake and named service will not start. > > Problems with the ipv6 address are on vlan100 - no. > > > > P.S. BIND 9.9.2-P1 > > > > The addresses for lo0 and BIND's query-source differ (extraneous :f), so unless this is a typo, that may be your problem. Thanks. It's my typo! -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 00:28:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A2633E6D for ; Wed, 13 Mar 2013 00:28:28 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-oa0-f53.google.com (mail-oa0-f53.google.com [209.85.219.53]) by mx1.freebsd.org (Postfix) with ESMTP id 53033343 for ; Wed, 13 Mar 2013 00:28:27 +0000 (UTC) Received: by mail-oa0-f53.google.com with SMTP id m1so490252oag.40 for ; Tue, 12 Mar 2013 17:28:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=fWLnu0+ggQ+Ui8BTNQFy2Pk7RTmpPRTPQIC2F83ppxg=; b=Nx9J3T7Yxkey/alR1upN5mov6PjDHzLRfzsLJ7x38HkAd332DaYediUxmr0+zDnWj4 wuLfdmxNkWuwA0n/azUCz7cAJZwsbPbGkTnRNvbVEGD0oRwktqgT2TBaZEoBPJF6ecPC VXncO+3mxjaFf7U0ht38oYUwr0aVyq4c4YUhgMlJm/DwArzONHehOGvTlT3+pAbp4Q/H J7R8GZXfzFc1b3qHJ4tl4S6FJ1nZrxIuRVfsbVFrVEc1OxcAHxMpoe9ol3sQKdS/Ilv6 rzEyJCjJv1STj9fioHIb9emQhGI/y/WmSaUnb7FErIg6VgphrePaSQ31Onw4oLtZP20X 2jWw== MIME-Version: 1.0 X-Received: by 10.182.139.37 with SMTP id qv5mr13553346obb.92.1363134507300; Tue, 12 Mar 2013 17:28:27 -0700 (PDT) Received: by 10.76.109.236 with HTTP; Tue, 12 Mar 2013 17:28:27 -0700 (PDT) In-Reply-To: <513F9EC4.8090303@acm.poly.edu> References: <513F9A2D.8050507@acm.poly.edu> <20130312212344.GN63316@nat.myhome> <513F9EC4.8090303@acm.poly.edu> Date: Tue, 12 Mar 2013 20:28:27 -0400 Message-ID: Subject: Re: mbuf tuning on 9.1 From: Ryan Stone To: Boris Kochergin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net , "Paul A. Procacci" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 00:28:28 -0000 On Tue, Mar 12, 2013 at 5:31 PM, Boris Kochergin wrote: > Additionally, can someone clarify the meaning of "total" vs. "max" for > these values? > > -Boris > It measures the amount of memory being used as that type of data structure. This is the total of the number of allocated items (current) and the number of free items that are being cached by the allocater for faster allocations in the future (cached). From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 00:44:38 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 72365429 for ; Wed, 13 Mar 2013 00:44:38 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ob0-x22f.google.com (mail-ob0-x22f.google.com [IPv6:2607:f8b0:4003:c01::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 41EAD3FE for ; Wed, 13 Mar 2013 00:44:38 +0000 (UTC) Received: by mail-ob0-f175.google.com with SMTP id uz6so486531obc.34 for ; Tue, 12 Mar 2013 17:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=dvwQy+SCsNX019dkJB90/VPhGc2ElhDb0W+es+IES2g=; b=zroc1Ob41x7hGrVALjNQTbeyvcEN6aCDk5RGgE/9Jf6odOsTUojW0uiycfxuV+FS6h R95z4oHZyNu32GaA/k2Ji9oWVpBK8YaLdnvs88s+9ZDP8ohWAEcFg2gMfZyXUokI2Qtk laBCGPIaC4XbfQ+zJ1vnBBRKKp0e85A8Uk9cSNCRS1Osnk+slg1KSx5IeLgiIwSFbNWC 2B0J9S0M4ZCexMugz0sPcav5xYmZCkb0rTrkd77wWo+9g2KWqseUBoG5aKLIwDYfiDnZ 7z/CrAz8uuudL0n5Vwc0eg+ZysaGetnQIR7fk166PfQ1FAzDgKX2ByEVqQowI2oeITxE Wslw== MIME-Version: 1.0 X-Received: by 10.60.30.231 with SMTP id v7mr14123133oeh.118.1363135477887; Tue, 12 Mar 2013 17:44:37 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.76.11.165 with HTTP; Tue, 12 Mar 2013 17:44:37 -0700 (PDT) In-Reply-To: References: Date: Tue, 12 Mar 2013 17:44:37 -0700 X-Google-Sender-Auth: bBrWzog-Di7MnCp4QqZjwjXbaO4 Message-ID: Subject: Re: ntpd bind() failure: Can't assign requested address From: Kevin Oberman To: "M. Schulte" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 00:44:38 -0000 On Tue, Mar 12, 2013 at 2:42 PM, M. Schulte wrote: > Hi! > > [First of all, I have posted this question already on the FreeBSD > forum -- so far without replies -- and now my hope is that the set of > subscribers here and those of the forum do not completely coincide.] > > I have installed FreeBSD 9.1 on my server (it's a virtualized qemu/kvm > environment, in case that matters) and during boot the following > messages appear: > > ntpd[766]: ntpd 4.2.4p5-a (1) > ntpd[767]: bind() fd 23, family AF_INET6, port 123, scope 2, addr > fe80::216:36ff:fe74:2076, mcast=0 flags=0x11 fails: Can't assign requested > address > ntpd[767]: unable to create socket on re0 (3) for > fe80::216:36ff:fe74:2076#123 > > This happens with a GENERIC kernel. I have not touched any IPv6 > related configuration after install, so everything should be in the > default state. Neither have I changed the ntp/ntpd configuration. > > Although I could find some threads where people were discussing the > same problem, none of the mentioned suggestions fixed this for me. > > In particular I tried setting > > ipv6_activate_all_interfaces="**YES" > > in rc.conf -- didn't fix the problem. > > According to ifconfig, my interfaces, re0 in particular, have IPv6 > addresses associated to them. So why would it fail that a process > tries to bind to it? Note that I'm rather unexperienced wrt IPv6. > > Would be great if somebody could give me a hint into the right > direction. Anybody else experiencing this? Thank you very much! > > ~ melanie > What is the address space is the address you are seeing on the interface? Is it is a link-local address or one assigned by the router, a static from rc.conf or DHCPv6? You should have two inet6 addresses, one is link-local (starts with "fe") and another should start with "2". There may be more than two. -- R. Kevin Oberman, Network Engineer E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 01:25:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 51F52E2A for ; Wed, 13 Mar 2013 01:25:45 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wg0-f44.google.com (mail-wg0-f44.google.com [74.125.82.44]) by mx1.freebsd.org (Postfix) with ESMTP id DF31176A for ; Wed, 13 Mar 2013 01:25:44 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id dr12so457042wgb.11 for ; Tue, 12 Mar 2013 18:25:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to:x-gm-message-state; bh=1qR9nZzHZFvYbdEboVwOYS12xqLmRKKcDf09YYx0JU4=; b=I23JnYXMpq1Vh2L/dFQSHQvzfF8XSOiEbuk9AcXDycUbBMCoo0RQhKfRyE+Y8QEwcm XQmrsg2uOcJLfgK8/jo8f4ZjPABOdGQrt/Kijps0hP0+HAHUwX8TXvFVlZlQspZe0YS4 KK7auIty/K6ZbP24eyvcvqBd+AkA5901jmswfoAJ0lSdPWuRc4bX54NDRV2uyRAm6XXi SebOG9oC7C+q2OXHUzR6CHy/oStSW9FNmtU7jdsCEg2GISHVGcQlYhG+iuns+dEAiHLY l73hXCJxp5JR6/MRVZQs71SbxtLA0M1zEPjZD+i7iTG+TFXRqmUkkiiLVEIQUmGkZ+ZW 2KTQ== X-Received: by 10.194.109.136 with SMTP id hs8mr30825106wjb.8.1363137943694; Tue, 12 Mar 2013 18:25:43 -0700 (PDT) Received: from ?IPv6:2a01:e35:8a58:2600:dd05:db5b:e25e:ec7c? ([2a01:e35:8a58:2600:dd05:db5b:e25e:ec7c]) by mx.google.com with ESMTPS id q2sm27765027wiz.8.2013.03.12.18.25.42 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 12 Mar 2013 18:25:42 -0700 (PDT) References: <20130312225018.GA13589@defiant.konundrum.org> Mime-Version: 1.0 (1.0) In-Reply-To: <20130312225018.GA13589@defiant.konundrum.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> X-Mailer: iPhone Mail (10B144) From: Damien Fleuriot Subject: Re: ipv6 default router Operation not permitted Date: Wed, 13 Mar 2013 02:25:23 +0100 To: Schrodinger X-Gm-Message-State: ALoCoQk389GBeANtFpF39tsbr0e64Gs6lj7zyl/qb3phxdMjCDJOJzJnmDCcvqRQw+3wN1+5pCBH Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 01:25:45 -0000 On 12 Mar 2013, at 23:50, Schrodinger wrote: > Hi, >=20 > I have a problem reaching my ipv6 default router. >=20 > # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff > PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff= :ff > ping6: sendmsg: Operation not permitted > ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 >=20 > --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss >=20 > It is the same issue as posted here : >=20 > http://lists.freebsd.org/pipermail/freebsd-net/2012-February/031518.html >=20 > However, I believe the problem Mr Dandy was experiencing was he should > have changed his default gateway from: >=20 > ipv6_defaultrouter=3D"2a01:4f8:61:50c0::1" >=20 > To: >=20 > ipv6_defaultrouter=3D"fe80::1%re0" >=20 > As per Hertzner documentation. >=20 > http://wiki.hetzner.de/index.php/FreeBSD_installieren/en#Network_Configura= tion >=20 > I am not hosted with Hertzner but I have the same broken network > configuration and am uncertain of the Correct Fix. >=20 > I have: >=20 > ::: /etc/rc.conf ::: >=20 > ipv6_activate_all_interfaces=3D"YES" > ipv6_default_interface=3D"re0" > ipv6_network_interfaces=3D"auto" > ip6addrctl_policy=3D"ipv6_prefer" >=20 > ifconfig_re0_ipv6=3D"inet6 2001:41D0:2:E7c4::1 prefixlen 64" >=20 > ipv6_static_routes=3D"ovhgw default" > ipv6_route_ovhgw=3D"2001:41d0:2:e7ff:ff:ff:ff:ff -prefixlen 128 -interface= re0" > ipv6_route_default=3D"default 2001:41d0:2:e7ff:ff:ff:ff:ff" >=20 > The default IPv6 router is _outside_ of my assigned /64 prefix, this is > why an interface route is set and is configured this way to ensure the > interface route is applied before the default IPv6 router is added to > the routing table. If I do not do this I will receive an error on boot > to say that the default gateway cannot be reached. There is also no > firewall in place on the host OS. >=20 > However this configuration does not work. >=20 > # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff > PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff= :ff > ping6: sendmsg: Operation not permitted > ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 >=20 > --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss >=20 > If I do=20 >=20 > # ifconfig re0 inet6 accept_rtadv >=20 > It works : >=20 > # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff > PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff= :ff > 16 bytes from 2001:41d0:2:e7ff:ff:ff:ff:ff, icmp_seq=3D0 hlim=3D64 time=3D= 4.935 ms >=20 > --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/std-dev =3D 4.935/4.935/4.935/0.000 ms >=20 >=20 > Alternatively, knowing the Linklayer Address of my default gateway, I > could set a static ndp pair : >=20 > # ifconfig re0 inet6 -accept_rtadv > # ndp -s 2001:41d0:2:e7ff:ff:ff:ff:ff 00:05:73:a0:00:00 >=20 > And it works: >=20 > # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff > PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff= :ff > 16 bytes from 2001:41d0:2:e7ff:ff:ff:ff:ff, icmp_seq=3D0 hlim=3D64 time=3D= 3.582 ms >=20 > --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/std-dev =3D 3.582/3.582/3.582/0.000 ms >=20 > If I do : >=20 > # ndp -d 2001:41d0:2:e7ff:ff:ff:ff:ff >=20 > It stops working again : >=20 > # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff > PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff= :ff > ping6: sendmsg: Operation not permitted > ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 >=20 > --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss >=20 > It seems the simple solution is to have re0 ACCEPT_RTADV but I am told > that my default gateway does not perform router advertisements.=20 >=20 > In my own networks I have a static default route for inet6 and I am not > required to set any interfaces ACCEPT_RTADV. >=20 > Can anybody shed some light on the correct configuration? >=20 > Who is at fault here ? >=20 > - Me > - Hosting > - FreeBSD > - The universe >=20 > Cheers, > C. >=20 The network is actually /48 and you get assigned a /64 inside it. Set your interface to use the /48 prefix and voodoo will happen (I can assur= e you with a 97% certainty that your default GW is inside the /48). Of course, using the /48 prefix doesn't mean you can/may use IPs from outsid= e the /64 that was given you. Also, remove all the junk from rc.conf to only set the ipv6 default gw, none= of these static routes anymore. Kindly reply with topic "SOLVED" if that fixed you up, that googlers in the f= uture may find the solution easily.= From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 01:46:55 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7D9FE7CD for ; Wed, 13 Mar 2013 01:46:55 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pb0-f45.google.com (mail-pb0-f45.google.com [209.85.160.45]) by mx1.freebsd.org (Postfix) with ESMTP id 5803C82E for ; Wed, 13 Mar 2013 01:46:55 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id ro8so468110pbb.18 for ; Tue, 12 Mar 2013 18:46:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:date:to:cc:subject:message-id:reply-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=5KkwsPTK7ZEtM/ACXo2ohJSv1Q/uQ4Ly4+ZTK1K5+cM=; b=ZPr9xFmRINI61hA/IOANs/K08ORnxpxuGmR9fevgS/1vUVRVrAn2NiKHjmcgqJ/bmR 3kD4yRqH7YrsKHo8V2EbXZTTUI6Cg8bVsOWCqpUOeSZ7j5zj58qkjUmUrcz5KgqMWVzM cC4yAANYMg62blieeUiUE3cyGUEt1U6sZ37P9m+/DdK3Y7QnEpPp9gM+ZRHvhm6etCU7 Mcnyn4uUE+IKgNqkhlxhCr1SdgrwwOX++Z73N0Al7udLitl+kqqzLsXF4ocRHxXlp6b+ ZJe2SAHRpQEPEVPiZP98Q7dJ24HBG2M73ChjtCb0AXD7MN5CZ+VUk8AqBIp+4K2DScJA X+6w== X-Received: by 10.68.248.74 with SMTP id yk10mr33937627pbc.38.1363139209026; Tue, 12 Mar 2013 18:46:49 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPS id ix6sm27328560pbc.17.2013.03.12.18.46.45 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 12 Mar 2013 18:46:47 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 13 Mar 2013 10:46:41 +0900 From: YongHyeon PYUN Date: Wed, 13 Mar 2013 10:46:41 +0900 To: Miroslav Lachman <000.fbsd@quip.cz> Subject: Re: FreeBSD 9.1-RELEASE + bge0 == watchdog timeout Message-ID: <20130313014641.GC3062@michelle.cdnetworks.com> References: <201302241106.42477.vegeta@tuxpowered.net> <20130225082042.GB1426@michelle.cdnetworks.com> <512CF97B.8030805@norma.perm.ru> <20130227020123.GA3581@michelle.cdnetworks.com> <512DE968.4020409@quip.cz> <20130228053558.GA1474@michelle.cdnetworks.com> <512F415F.3040807@quip.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <512F415F.3040807@quip.cz> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 01:46:55 -0000 On Thu, Feb 28, 2013 at 12:37:03PM +0100, Miroslav Lachman wrote: > YongHyeon PYUN wrote: > >On Wed, Feb 27, 2013 at 12:09:28PM +0100, Miroslav Lachman wrote: > > [...] > > >>I can provide you full access to this machine (if you want) or let me > >>know, what version I should check. Older versions (6.x - 8.3) are > >>working fine with hw.bge.allow_asf="1" in loader.conf. I didn't test > >>newer releases on these old machines. > > > >The reporter said the machine was Sun Fire X2200 M2 so I guess you > >may see the same issue on both stable/9 and stable/8. Ideally the > >loader tunable hw.bge.allow_asf should not be there and driver > >should take care of it by checking the existence of ASF/IPMI > >firmware. > > > >Can you setup a remote debugging environments(+ IPMI access) like > >the following URL? > >http://people.freebsd.org/~yongari/remote_debugging.txt > > The one Sun Fire X2100 M2 is idling in datacenter and connected to > internet, so I can remotely reinstall it to stable/9 withing day or two > and give you full access to it (ssh user, root, BMC / IPMI admin account > with remote KVM + remote media). But as I understand, you need another > machine connected to it with serial and another ethernet. It will take > me some more time, as I will need to go to the datacenter, find some > serial cable etc. > > Let me know if ssh + ipmi access to X2100 alone is useful for you to > start, or only full remote debugging setup is needed. > > Can you point me to the original problem report with X2200 M2? > I had been working on fixing the IPMI regression with the help of Miroslav. It was fixed in r248226. Many thanks to Miroslav for providing full remote debugging environments. From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 01:58:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A2A3BC08 for ; Wed, 13 Mar 2013 01:58:07 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pb0-f51.google.com (mail-pb0-f51.google.com [209.85.160.51]) by mx1.freebsd.org (Postfix) with ESMTP id 770958C2 for ; Wed, 13 Mar 2013 01:58:07 +0000 (UTC) Received: by mail-pb0-f51.google.com with SMTP id un15so470631pbc.24 for ; Tue, 12 Mar 2013 18:58:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:date:to:cc:subject:message-id:reply-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=CZuvzbXbskLafvGOtKnjX2wKg3IPLXLMvsroM1+S0VY=; b=ia3ThWGJKehX0rJ2IhhpO4b/4LVQWQVj251qFRBkj4S2Iz8Kmh5oi2cTyzDwKjMzAX sGZ7eIcQOBIL2kobZSIJlq7pQrCDm9qQ9g0F3k+LTpzgTBskvPOH8WhVOeSHdZLjRJ7S bIuEc+HAXIGiH8mtsNpz0sO+F9Pncae/2Dx98ltqbP1Hfte1fdfChyyAa5Q42O/Zjwer p7icj697OLSIlyTcyZ+ykHC5AVFIwODdOGpSQtq6IgcejXlELCzuIJTh19mV1/2bBc+D 0GkkehRXtu1W1HXcZeP6Z8oMQso0+53ckGseBQs2hcKNaiT4ioRpwP9muZP66CjaIwmU kOBA== X-Received: by 10.68.75.72 with SMTP id a8mr41486379pbw.143.1363139880726; Tue, 12 Mar 2013 18:58:00 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPS id z1sm4577102pbw.19.2013.03.12.18.57.57 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 12 Mar 2013 18:57:59 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 13 Mar 2013 10:57:53 +0900 From: YongHyeon PYUN Date: Wed, 13 Mar 2013 10:57:53 +0900 To: "Eugene M. Zheganin" Subject: Re: FreeBSD 9.1-RELEASE + bge0 == watchdog timeout Message-ID: <20130313015753.GD3062@michelle.cdnetworks.com> References: <512DE968.4020409@quip.cz> <20130228053558.GA1474@michelle.cdnetworks.com> <5136D89D.4000902@norma.perm.ru> <20130306062658.GC1483@michelle.cdnetworks.com> <513713C2.1000007@norma.perm.ru> <20130307022446.GB3108@michelle.cdnetworks.com> <513820E2.806@norma.perm.ru> <20130307062335.GB1478@michelle.cdnetworks.com> <51383E3B.5030007@norma.perm.ru> <20130307081548.GD1478@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130307081548.GD1478@michelle.cdnetworks.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 01:58:07 -0000 On Thu, Mar 07, 2013 at 05:15:48PM +0900, YongHyeon PYUN wrote: > On Thu, Mar 07, 2013 at 01:14:03PM +0600, Eugene M. Zheganin wrote: > > Hi. > > > > On 07.03.2013 12:23, YongHyeon PYUN wrote: > > > On Thu, Mar 07, 2013 at 11:08:50AM +0600, Eugene M. Zheganin wrote: > > >> It was definitely older than "months". It was running something similar > > >> to "FreeBSD 8.2-STABLE #0: Mon Sep 19 08:10:00 YEKST 2011", this is the > > >> uname from a neighbor machine. > > >> > > >> I have, as I said, identical servers running FreeBSD. Here are some of > > >> the unames that I don't see timeouts on: > > >> > > >> 8.3-STABLE #2: Wed Aug 29 13:00:02 YEKT 2012 (up 187 days) > > >> 8.3-PRERELEASE #1: Thu Mar 29 16:14:11 MSK 2012 (up 15 days, previous > > >> uptime around 180 days) > > > These servers do not have 5718/5719/5720 changes. > > > > > >> 8.2-STABLE #0: Wed Dec 14 16:56:11 YEKT 2011 (up 99 days) > > > This server has the bge(4) change but it didn't trigger watchdog > > > timeouts. Does this server use the same controller? If yes, the > > > issue didn't come from bge(4) change. > > > > > How's that ? It's running even older version than previous two. I guess > > you misread the year. > > Oops, you're right. If your controller supports ASF/IPMI access please apply r248226 to stable/8 and let me know whether that makes any difference. I believe ignoring ASF/IPMI firmware is not good idea since the ASF/IPMI firmware will run regardless of hw.bge.allow_asf loader tunable configuration. You may have to set hw.bge.allow_asf=1 since it's off by default on stable/8. From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 03:48:13 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E4D224AF; Wed, 13 Mar 2013 03:48:13 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id F04E7E34; Wed, 13 Mar 2013 03:48:12 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEABn2P1GDaFvO/2dsb2JhbAA7CIgZvGCBYnSCKQEBAQMBAQEBFwkEJyALGxgRGQIEJQEJJgYIBwQBGgIEh20GDK80gkCPaReNTAsBBH0ZGweCLYETA480hGSCPoEfj1eDJiAyfQgXHg X-IronPort-AV: E=Sophos;i="4.84,833,1355115600"; d="scan'208";a="20887967" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 12 Mar 2013 23:48:00 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A11D2B3F15; Tue, 12 Mar 2013 23:48:00 -0400 (EDT) Date: Tue, 12 Mar 2013 23:48:00 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <75232221.3844453.1363146480616.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20798.44871.601547.24628@hergotha.csail.mit.edu> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_3844452_1526038691.1363146480611" X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, andre@freebsd.org, Ivan Voras X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 03:48:14 -0000 ------=_Part_3844452_1526038691.1363146480611 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Garrett Wollman wrote: > < said: > > > To be honest, I'd consider seeing a lot of non-empty receive queues > > for TCP connections to the NFS server to be an indication that it is > > near/at its load limit. (Sure, if you do netstat a lot, you will > > occasionally > > see a non-empty queue here or there, but I would not expect to see a > > lot > > of them non-empty a lot of the time.) If that is the case, then the > > question becomes "what is the bottleneck?". Below I suggest getting > > rid > > of the DRC in case it is the bottleneck for your server. > > The problem is not the DRC in "normal" operation, but the DRC when it > gets into the livelocked state. I think we've talked about a number > of solutions to the livelock problem, but I haven't managed to > implement or test these ideas yet. I have a duplicate server up now, > so I hope to do some testing this week. > > In normal operation, the server is mostly idle, and the nfsd threads > that aren't themselves idle are sleeping deep in ZFS waiting for > something to happen on disk. When the arrival rate exceeds the rate > at which requests are cleared from the DRC, *all* of the nfsd threads > will spin, either waiting for the DRC mutex or walking the DRC finding > that there is nothing that can be released yet. *That* is the > livelock condition -- the spinning that takes over all nfsd threads is > what causes the receive buffers to build up, and the large queues then > maintain the livelocked condition -- and that is why it clears > *immediately* when the DRC size is increased. (It's possible to > reproduce this condition on a loaded server by simply reducing the > tcphighwater to less than the current size.) Unfortunately, I'm at > the NFSRC_FLOODSIZE limit right now (64k), so there is no room for > further increases until I recompile the kernel. It's probably a bug > that the sysctl definition in drc3.patch doesn't check the new value > against this limit. > > Note that I'm currently running 64 nfsd threads on a 12-core > (24-thread) system. In the livelocked condition, as you would expect, > the system goes to 100% CPU utilization and the load average peaks out > at 64, while goodput goes to nearly nil. > Ok, I think I finally understand what you are referring to by your livelock. Basically, you are at the tcphighwater mark and the nfsd threads don't succeed in freeing up many cache entries so each nfsd thread tries to trim the cache for each RPC and that slows the server right down. I suspect it is the cached entries from dismounted clients that are filling up the cache (you did mention clients using amd at some point in the discussion, which implies frequent mounts/dismounts). I'm guessing that the tcp cache timeout needs to be made a lot smaller for your case. > > For either A or B, I'd suggest that you disable the DRC for TCP > > connections > > (email if you need a patch for that), which will have a couple of > > effects: > > I would like to see your patch, since it's more likely to be correct > than one I might dream up. > > The alternative solution is twofold: first, nfsrv_trimcache() needs to > do something to ensure forward progress, even when that means dropping > something that hasn't timed out yet, and second, the server code needs > to ensure that nfsrv_trimcache() is only executing on one thread at a > time. An easy way to do the first part would be to maintain an LRU > queue for TCP in addition to the UDP LRU, and just blow away the first > N (>NCPU) entries on the queue if, after checking all the TCP replies, > the DRC is still larger than the limit. The second part is just an > atomic_cmpset_int(). > I've attached a patch that has assorted changes. I didn't use an LRU list, since that results in a single mutex to contend on, but I added a second pass to the nfsrc_trimcache() function that frees old entries. (Approximate LRU, using a histogram of timeout values to select a timeout value that frees enough of the oldest ones.) Basically, this patch: - allows setting of the tcp timeout via vfs.nfsd.tcpcachetimeo (I'd suggest you go down to a few minutes instead of 12hrs) - allows TCP caching to be disabled by setting vfs.nfsd.cachetcp=0 - does the above 2 things you describe to try and avoid the livelock, although not quite using an lru list - increases the hash table size to 500 (still a compile time setting) (feel free to make it even bigger) - sets nfsrc_floodlevel to at least nfsrc_tcphighwater, so you can grow vfs.nfsd.tcphighwater as big as you dare The patch includes a lot of drc2.patch and drc3.patch, so don't try and apply it to a patched kernel. Hopefully it will apply cleanly to vanilla sources. Tha patch has been minimally tested. If you'd rather not apply the patch, you can change NFSRVCACHE_TCPTIMEOUT and set the variable nfsrc_tcpidempotent == 0 to get a couple of the changes. (You'll have to recompile the kernel for these changes to take effect.) Good luck with it, rick > -GAWollman > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" ------=_Part_3844452_1526038691.1363146480611 Content-Type: text/x-patch; name=drc4.patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=drc4.patch LS0tIGZzL25mc3NlcnZlci9uZnNfbmZzZGNhY2hlLmMub3JpZwkyMDEzLTAxLTA3IDA5OjA0OjEz LjAwMDAwMDAwMCAtMDUwMAorKysgZnMvbmZzc2VydmVyL25mc19uZnNkY2FjaGUuYwkyMDEzLTAz LTEyIDIyOjQyOjA1LjAwMDAwMDAwMCAtMDQwMApAQCAtMTYwLDEyICsxNjAsMzEgQEAgX19GQlNE SUQoIiRGcmVlQlNEOiBwcm9qZWN0cy9uZnN2NC1wYWNrcgogI2luY2x1ZGUgPGZzL25mcy9uZnNw b3J0Lmg+CiAKIGV4dGVybiBzdHJ1Y3QgbmZzc3RhdHMgbmV3bmZzc3RhdHM7Ci1ORlNDQUNIRU1V VEVYOworZXh0ZXJuIHN0cnVjdCBtdHggbmZzcmNfdGNwbXR4W05GU1JWQ0FDSEVfSEFTSFNJWkVd OworZXh0ZXJuIHN0cnVjdCBtdHggbmZzcmNfdWRwbXR4OwogaW50IG5mc3JjX2Zsb29kbGV2ZWwg PSBORlNSVkNBQ0hFX0ZMT09ETEVWRUwsIG5mc3JjX3RjcHNhdmVkcmVwbGllcyA9IDA7CiAjZW5k aWYJLyogIUFQUExFS0VYVCAqLwogCi1zdGF0aWMgaW50IG5mc3JjX3RjcG5vbmlkZW1wb3RlbnQg PSAxOwotc3RhdGljIGludCBuZnNyY191ZHBoaWdod2F0ZXIgPSBORlNSVkNBQ0hFX1VEUEhJR0hX QVRFUiwgbmZzcmNfdWRwY2FjaGVzaXplID0gMDsKK1NZU0NUTF9ERUNMKF92ZnNfbmZzZCk7CisK K3N0YXRpYyB1X2ludAluZnNyY190Y3BoaWdod2F0ZXIgPSAwOworU1lTQ1RMX1VJTlQoX3Zmc19u ZnNkLCBPSURfQVVUTywgdGNwaGlnaHdhdGVyLCBDVExGTEFHX1JXLAorICAgICZuZnNyY190Y3Bo aWdod2F0ZXIsIDAsCisgICAgIkhpZ2ggd2F0ZXIgbWFyayBmb3IgVENQIGNhY2hlIGVudHJpZXMi KTsKK3N0YXRpYyB1X2ludAluZnNyY191ZHBoaWdod2F0ZXIgPSBORlNSVkNBQ0hFX1VEUEhJR0hX QVRFUjsKK1NZU0NUTF9VSU5UKF92ZnNfbmZzZCwgT0lEX0FVVE8sIHVkcGhpZ2h3YXRlciwgQ1RM RkxBR19SVywKKyAgICAmbmZzcmNfdWRwaGlnaHdhdGVyLCAwLAorICAgICJIaWdoIHdhdGVyIG1h cmsgZm9yIFVEUCBjYWNoZSBlbnRyaWVzIik7CitzdGF0aWMgdV9pbnQJbmZzcmNfdGNwdGltZW91 dCA9IE5GU1JWQ0FDSEVfVENQVElNRU9VVDsKK1NZU0NUTF9VSU5UKF92ZnNfbmZzZCwgT0lEX0FV VE8sIHRjcGNhY2hldGltZW8sIENUTEZMQUdfUlcsCisgICAgJm5mc3JjX3RjcHRpbWVvdXQsIDAs CisgICAgIlRpbWVvdXQgZm9yIFRDUCBlbnRyaWVzIGluIHRoZSBEUkMiKTsKK3N0YXRpYyB1X2lu dCBuZnNyY190Y3Bub25pZGVtcG90ZW50ID0gMTsKK1NZU0NUTF9VSU5UKF92ZnNfbmZzZCwgT0lE X0FVVE8sIGNhY2hldGNwLCBDVExGTEFHX1JXLAorICAgICZuZnNyY190Y3Bub25pZGVtcG90ZW50 LCAwLAorICAgICJFbmFibGUgdGhlIERSQyBmb3IgTkZTIG92ZXIgVENQIik7CisKK3N0YXRpYyBp bnQgbmZzcmNfdWRwY2FjaGVzaXplID0gMDsKIHN0YXRpYyBUQUlMUV9IRUFEKCwgbmZzcnZjYWNo ZSkgbmZzcnZ1ZHBscnU7CiBzdGF0aWMgc3RydWN0IG5mc3J2aGFzaGhlYWQgbmZzcnZoYXNodGJs W05GU1JWQ0FDSEVfSEFTSFNJWkVdLAogICAgIG5mc3J2dWRwaGFzaHRibFtORlNSVkNBQ0hFX0hB U0hTSVpFXTsKQEAgLTE5NywxMCArMjE2LDExIEBAIHN0YXRpYyBpbnQgbmV3bmZzdjJfcHJvY2lk W05GU19WM05QUk9DU10KIAlORlNWMlBST0NfTk9PUCwKIH07CiAKKyNkZWZpbmUJbmZzcmNfaGFz aCh4aWQpCSgoKHhpZCkgKyAoKHhpZCkgPj4gMjQpKSAlIE5GU1JWQ0FDSEVfSEFTSFNJWkUpCiAj ZGVmaW5lCU5GU1JDVURQSEFTSCh4aWQpIFwKLQkoJm5mc3J2dWRwaGFzaHRibFsoKHhpZCkgKyAo KHhpZCkgPj4gMjQpKSAlIE5GU1JWQ0FDSEVfSEFTSFNJWkVdKQorCSgmbmZzcnZ1ZHBoYXNodGJs W25mc3JjX2hhc2goeGlkKV0pCiAjZGVmaW5lCU5GU1JDSEFTSCh4aWQpIFwKLQkoJm5mc3J2aGFz aHRibFsoKHhpZCkgKyAoKHhpZCkgPj4gMjQpKSAlIE5GU1JWQ0FDSEVfSEFTSFNJWkVdKQorCSgm bmZzcnZoYXNodGJsW25mc3JjX2hhc2goeGlkKV0pCiAjZGVmaW5lCVRSVUUJMQogI2RlZmluZQlG QUxTRQkwCiAjZGVmaW5lCU5GU1JWQ0FDSEVfQ0hFQ0tMRU4JMTAwCkBAIC0yNTEsNiArMjcxLDE4 IEBAIHN0YXRpYyBpbnQgbmZzcmNfZ2V0bGVuYW5kY2tzdW0obWJ1Zl90IG0KIHN0YXRpYyB2b2lk IG5mc3JjX21hcmtzYW1ldGNwY29ubih1X2ludDY0X3QpOwogCiAvKgorICogUmV0dXJuIHRoZSBj b3JyZWN0IG11dGV4IGZvciB0aGlzIGNhY2hlIGVudHJ5LgorICovCitzdGF0aWMgX19pbmxpbmUg c3RydWN0IG10eCAqCituZnNyY19jYWNoZW11dGV4KHN0cnVjdCBuZnNydmNhY2hlICpycCkKK3sK KworCWlmICgocnAtPnJjX2ZsYWcgJiBSQ19VRFApICE9IDApCisJCXJldHVybiAoJm5mc3JjX3Vk cG10eCk7CisJcmV0dXJuICgmbmZzcmNfdGNwbXR4W25mc3JjX2hhc2gocnAtPnJjX3hpZCldKTsK K30KKworLyoKICAqIEluaXRpYWxpemUgdGhlIHNlcnZlciByZXF1ZXN0IGNhY2hlIGxpc3QKICAq LwogQVBQTEVTVEFUSUMgdm9pZApAQCAtMzI1LDEwICszNTcsMTIgQEAgbmZzcmNfZ2V0dWRwKHN0 cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIAogCXN0cnVjdCBzb2NrYWRkcl9pbjYgKnNhZGRyNjsK IAlzdHJ1Y3QgbmZzcnZoYXNoaGVhZCAqaHA7CiAJaW50IHJldCA9IDA7CisJc3RydWN0IG10eCAq bXV0ZXg7CiAKKwltdXRleCA9IG5mc3JjX2NhY2hlbXV0ZXgobmV3cnApOwogCWhwID0gTkZTUkNV RFBIQVNIKG5ld3JwLT5yY194aWQpOwogbG9vcDoKLQlORlNMT0NLQ0FDSEUoKTsKKwltdHhfbG9j ayhtdXRleCk7CiAJTElTVF9GT1JFQUNIKHJwLCBocCwgcmNfaGFzaCkgewogCSAgICBpZiAobmV3 cnAtPnJjX3hpZCA9PSBycC0+cmNfeGlkICYmCiAJCW5ld3JwLT5yY19wcm9jID09IHJwLT5yY19w cm9jICYmCkBAIC0zMzYsOCArMzcwLDggQEAgbG9vcDoKIAkJbmZzYWRkcl9tYXRjaChORVRGQU1J TFkocnApLCAmcnAtPnJjX2hhZGRyLCBuZC0+bmRfbmFtKSkgewogCQkJaWYgKChycC0+cmNfZmxh ZyAmIFJDX0xPQ0tFRCkgIT0gMCkgewogCQkJCXJwLT5yY19mbGFnIHw9IFJDX1dBTlRFRDsKLQkJ CQkodm9pZCltdHhfc2xlZXAocnAsIE5GU0NBQ0hFTVVURVhQVFIsCi0JCQkJICAgIChQWkVSTyAt IDEpIHwgUERST1AsICJuZnNyYyIsIDEwICogaHopOworCQkJCSh2b2lkKW10eF9zbGVlcChycCwg bXV0ZXgsIChQWkVSTyAtIDEpIHwgUERST1AsCisJCQkJICAgICJuZnNyYyIsIDEwICogaHopOwog CQkJCWdvdG8gbG9vcDsKIAkJCX0KIAkJCWlmIChycC0+cmNfZmxhZyA9PSAwKQpAQCAtMzQ3LDE0 ICszODEsMTQgQEAgbG9vcDoKIAkJCVRBSUxRX0lOU0VSVF9UQUlMKCZuZnNydnVkcGxydSwgcnAs IHJjX2xydSk7CiAJCQlpZiAocnAtPnJjX2ZsYWcgJiBSQ19JTlBST0cpIHsKIAkJCQluZXduZnNz dGF0cy5zcnZjYWNoZV9pbnByb2doaXRzKys7Ci0JCQkJTkZTVU5MT0NLQ0FDSEUoKTsKKwkJCQlt dHhfdW5sb2NrKG11dGV4KTsKIAkJCQlyZXQgPSBSQ19EUk9QSVQ7CiAJCQl9IGVsc2UgaWYgKHJw LT5yY19mbGFnICYgUkNfUkVQU1RBVFVTKSB7CiAJCQkJLyoKIAkJCQkgKiBWMiBvbmx5LgogCQkJ CSAqLwogCQkJCW5ld25mc3N0YXRzLnNydmNhY2hlX25vbmlkZW1kb25laGl0cysrOwotCQkJCU5G U1VOTE9DS0NBQ0hFKCk7CisJCQkJbXR4X3VubG9jayhtdXRleCk7CiAJCQkJbmZzcnZkX3JlcGhl YWQobmQpOwogCQkJCSoobmQtPm5kX2VycnApID0gcnAtPnJjX3N0YXR1czsKIAkJCQlyZXQgPSBS Q19SRVBMWTsKQEAgLTM2Miw3ICszOTYsNyBAQCBsb29wOgogCQkJCQlORlNSVkNBQ0hFX1VEUFRJ TUVPVVQ7CiAJCQl9IGVsc2UgaWYgKHJwLT5yY19mbGFnICYgUkNfUkVQTUJVRikgewogCQkJCW5l d25mc3N0YXRzLnNydmNhY2hlX25vbmlkZW1kb25laGl0cysrOwotCQkJCU5GU1VOTE9DS0NBQ0hF KCk7CisJCQkJbXR4X3VubG9jayhtdXRleCk7CiAJCQkJbmQtPm5kX21yZXEgPSBtX2NvcHltKHJw LT5yY19yZXBseSwgMCwKIAkJCQkJTV9DT1BZQUxMLCBNX1dBSVRPSyk7CiAJCQkJcmV0ID0gUkNf UkVQTFk7CkBAIC0zOTIsNyArNDI2LDcgQEAgbG9vcDoKIAl9CiAJTElTVF9JTlNFUlRfSEVBRCho cCwgbmV3cnAsIHJjX2hhc2gpOwogCVRBSUxRX0lOU0VSVF9UQUlMKCZuZnNydnVkcGxydSwgbmV3 cnAsIHJjX2xydSk7Ci0JTkZTVU5MT0NLQ0FDSEUoKTsKKwltdHhfdW5sb2NrKG11dGV4KTsKIAlu ZC0+bmRfcnAgPSBuZXdycDsKIAlyZXQgPSBSQ19ET0lUOwogCkBAIC00MTAsMTIgKzQ0NCwxNiBA QCBuZnNydmRfdXBkYXRlY2FjaGUoc3RydWN0IG5mc3J2X2Rlc2NyaXB0CiAJc3RydWN0IG5mc3J2 Y2FjaGUgKnJwOwogCXN0cnVjdCBuZnNydmNhY2hlICpyZXRycCA9IE5VTEw7CiAJbWJ1Zl90IG07 CisJc3RydWN0IG10eCAqbXV0ZXg7CiAKKwlpZiAobmZzcmNfdGNwaGlnaHdhdGVyID4gbmZzcmNf Zmxvb2RsZXZlbCkKKwkJbmZzcmNfZmxvb2RsZXZlbCA9IG5mc3JjX3RjcGhpZ2h3YXRlcjsKIAly cCA9IG5kLT5uZF9ycDsKIAlpZiAoIXJwKQogCQlwYW5pYygibmZzcnZkX3VwZGF0ZWNhY2hlIG51 bGwgcnAiKTsKIAluZC0+bmRfcnAgPSBOVUxMOwotCU5GU0xPQ0tDQUNIRSgpOworCW11dGV4ID0g bmZzcmNfY2FjaGVtdXRleChycCk7CisJbXR4X2xvY2sobXV0ZXgpOwogCW5mc3JjX2xvY2socnAp OwogCWlmICghKHJwLT5yY19mbGFnICYgUkNfSU5QUk9HKSkKIAkJcGFuaWMoIm5mc3J2ZF91cGRh dGVjYWNoZSBub3QgaW5wcm9nIik7CkBAIC00MzAsNyArNDY4LDcgQEAgbmZzcnZkX3VwZGF0ZWNh Y2hlKHN0cnVjdCBuZnNydl9kZXNjcmlwdAogCSAqLwogCWlmIChuZC0+bmRfcmVwc3RhdCA9PSBO RlNFUlJfUkVQTFlGUk9NQ0FDSEUpIHsKIAkJbmV3bmZzc3RhdHMuc3J2Y2FjaGVfbm9uaWRlbWRv bmVoaXRzKys7Ci0JCU5GU1VOTE9DS0NBQ0hFKCk7CisJCW10eF91bmxvY2sobXV0ZXgpOwogCQlu ZC0+bmRfcmVwc3RhdCA9IDA7CiAJCWlmIChuZC0+bmRfbXJlcSkKIAkJCW1idWZfZnJlZW0obmQt Pm5kX21yZXEpOwpAQCAtNDM4LDcgKzQ3Niw3IEBAIG5mc3J2ZF91cGRhdGVjYWNoZShzdHJ1Y3Qg bmZzcnZfZGVzY3JpcHQKIAkJCXBhbmljKCJyZXBseSBmcm9tIGNhY2hlIik7CiAJCW5kLT5uZF9t cmVxID0gbV9jb3B5bShycC0+cmNfcmVwbHksIDAsCiAJCSAgICBNX0NPUFlBTEwsIE1fV0FJVE9L KTsKLQkJcnAtPnJjX3RpbWVzdGFtcCA9IE5GU0RfTU9OT1NFQyArIE5GU1JWQ0FDSEVfVENQVElN RU9VVDsKKwkJcnAtPnJjX3RpbWVzdGFtcCA9IE5GU0RfTU9OT1NFQyArIG5mc3JjX3RjcHRpbWVv dXQ7CiAJCW5mc3JjX3VubG9jayhycCk7CiAJCWdvdG8gb3V0OwogCX0KQEAgLTQ2MywyMSArNTAx LDIxIEBAIG5mc3J2ZF91cGRhdGVjYWNoZShzdHJ1Y3QgbmZzcnZfZGVzY3JpcHQKIAkJICAgIG5m c3YyX3JlcHN0YXRbbmV3bmZzdjJfcHJvY2lkW25kLT5uZF9wcm9jbnVtXV0pIHsKIAkJCXJwLT5y Y19zdGF0dXMgPSBuZC0+bmRfcmVwc3RhdDsKIAkJCXJwLT5yY19mbGFnIHw9IFJDX1JFUFNUQVRV UzsKLQkJCU5GU1VOTE9DS0NBQ0hFKCk7CisJCQltdHhfdW5sb2NrKG11dGV4KTsKIAkJfSBlbHNl IHsKIAkJCWlmICghKHJwLT5yY19mbGFnICYgUkNfVURQKSkgewotCQkJICAgIG5mc3JjX3RjcHNh dmVkcmVwbGllcysrOworCQkJICAgIGF0b21pY19hZGRfaW50KCZuZnNyY190Y3BzYXZlZHJlcGxp ZXMsIDEpOwogCQkJICAgIGlmIChuZnNyY190Y3BzYXZlZHJlcGxpZXMgPgogCQkJCW5ld25mc3N0 YXRzLnNydmNhY2hlX3RjcHBlYWspCiAJCQkJbmV3bmZzc3RhdHMuc3J2Y2FjaGVfdGNwcGVhayA9 CiAJCQkJICAgIG5mc3JjX3RjcHNhdmVkcmVwbGllczsKIAkJCX0KLQkJCU5GU1VOTE9DS0NBQ0hF KCk7Ci0JCQltID0gbV9jb3B5bShuZC0+bmRfbXJlcSwgMCwgTV9DT1BZQUxMLCBNX1dBSVRPSyk7 Ci0JCQlORlNMT0NLQ0FDSEUoKTsKKwkJCW10eF91bmxvY2sobXV0ZXgpOworCQkJbSA9IG1fY29w eW0obmQtPm5kX21yZXEsIDAsIE1fQ09QWUFMTCwgTV9XQUlUKTsKKwkJCW10eF9sb2NrKG11dGV4 KTsKIAkJCXJwLT5yY19yZXBseSA9IG07CiAJCQlycC0+cmNfZmxhZyB8PSBSQ19SRVBNQlVGOwot CQkJTkZTVU5MT0NLQ0FDSEUoKTsKKwkJCW10eF91bmxvY2sobXV0ZXgpOwogCQl9CiAJCWlmIChy cC0+cmNfZmxhZyAmIFJDX1VEUCkgewogCQkJcnAtPnJjX3RpbWVzdGFtcCA9IE5GU0RfTU9OT1NF QyArCkBAIC00ODUsNyArNTIzLDcgQEAgbmZzcnZkX3VwZGF0ZWNhY2hlKHN0cnVjdCBuZnNydl9k ZXNjcmlwdAogCQkJbmZzcmNfdW5sb2NrKHJwKTsKIAkJfSBlbHNlIHsKIAkJCXJwLT5yY190aW1l c3RhbXAgPSBORlNEX01PTk9TRUMgKwotCQkJICAgIE5GU1JWQ0FDSEVfVENQVElNRU9VVDsKKwkJ CSAgICBuZnNyY190Y3B0aW1lb3V0OwogCQkJaWYgKHJwLT5yY19yZWZjbnQgPiAwKQogCQkJCW5m c3JjX3VubG9jayhycCk7CiAJCQllbHNlCkBAIC00OTMsNyArNTMxLDcgQEAgbmZzcnZkX3VwZGF0 ZWNhY2hlKHN0cnVjdCBuZnNydl9kZXNjcmlwdAogCQl9CiAJfSBlbHNlIHsKIAkJbmZzcmNfZnJl ZWNhY2hlKHJwKTsKLQkJTkZTVU5MT0NLQ0FDSEUoKTsKKwkJbXR4X3VubG9jayhtdXRleCk7CiAJ fQogCiBvdXQ6CkBAIC01MDksMTQgKzU0NywxNiBAQCBvdXQ6CiBBUFBMRVNUQVRJQyB2b2lkCiBu ZnNydmRfZGVsY2FjaGUoc3RydWN0IG5mc3J2Y2FjaGUgKnJwKQogeworCXN0cnVjdCBtdHggKm11 dGV4OwogCisJbXV0ZXggPSBuZnNyY19jYWNoZW11dGV4KHJwKTsKIAlpZiAoIShycC0+cmNfZmxh ZyAmIFJDX0lOUFJPRykpCiAJCXBhbmljKCJuZnNydmRfZGVsY2FjaGUgbm90IGluIHByb2ciKTsK LQlORlNMT0NLQ0FDSEUoKTsKKwltdHhfbG9jayhtdXRleCk7CiAJcnAtPnJjX2ZsYWcgJj0gflJD X0lOUFJPRzsKIAlpZiAocnAtPnJjX3JlZmNudCA9PSAwICYmICEocnAtPnJjX2ZsYWcgJiBSQ19M T0NLRUQpKQogCQluZnNyY19mcmVlY2FjaGUocnApOwotCU5GU1VOTE9DS0NBQ0hFKCk7CisJbXR4 X3VubG9jayhtdXRleCk7CiB9CiAKIC8qCkBAIC01MjgsNyArNTY4LDkgQEAgQVBQTEVTVEFUSUMg dm9pZAogbmZzcnZkX3NlbnRjYWNoZShzdHJ1Y3QgbmZzcnZjYWNoZSAqcnAsIHN0cnVjdCBzb2Nr ZXQgKnNvLCBpbnQgZXJyKQogewogCXRjcF9zZXEgdG1wX3NlcTsKKwlzdHJ1Y3QgbXR4ICptdXRl eDsKIAorCW11dGV4ID0gbmZzcmNfY2FjaGVtdXRleChycCk7CiAJaWYgKCEocnAtPnJjX2ZsYWcg JiBSQ19MT0NLRUQpKQogCQlwYW5pYygibmZzcnZkX3NlbnRjYWNoZSBub3QgbG9ja2VkIik7CiAJ aWYgKCFlcnIpIHsKQEAgLTUzNywxMCArNTc5LDEwIEBAIG5mc3J2ZF9zZW50Y2FjaGUoc3RydWN0 IG5mc3J2Y2FjaGUgKnJwLCAKIAkJICAgICBzby0+c29fcHJvdG8tPnByX3Byb3RvY29sICE9IElQ UFJPVE9fVENQKQogCQkJcGFuaWMoIm5mcyBzZW50IGNhY2hlIik7CiAJCWlmIChuZnNydl9nZXRz b2Nrc2VxbnVtKHNvLCAmdG1wX3NlcSkpIHsKLQkJCU5GU0xPQ0tDQUNIRSgpOworCQkJbXR4X2xv Y2sobXV0ZXgpOwogCQkJcnAtPnJjX3RjcHNlcSA9IHRtcF9zZXE7CiAJCQlycC0+cmNfZmxhZyB8 PSBSQ19UQ1BTRVE7Ci0JCQlORlNVTkxPQ0tDQUNIRSgpOworCQkJbXR4X3VubG9jayhtdXRleCk7 CiAJCX0KIAl9CiAJbmZzcmNfdW5sb2NrKHJwKTsKQEAgLTU1OSwxMSArNjAxLDEzIEBAIG5mc3Jj X2dldHRjcChzdHJ1Y3QgbmZzcnZfZGVzY3JpcHQgKm5kLCAKIAlzdHJ1Y3QgbmZzcnZjYWNoZSAq aGl0cnA7CiAJc3RydWN0IG5mc3J2aGFzaGhlYWQgKmhwLCBuZnNyY190ZW1wbGlzdDsKIAlpbnQg aGl0LCByZXQgPSAwOworCXN0cnVjdCBtdHggKm11dGV4OwogCisJbXV0ZXggPSBuZnNyY19jYWNo ZW11dGV4KG5ld3JwKTsKIAlocCA9IE5GU1JDSEFTSChuZXdycC0+cmNfeGlkKTsKIAluZXdycC0+ cmNfcmVxbGVuID0gbmZzcmNfZ2V0bGVuYW5kY2tzdW0obmQtPm5kX21yZXAsICZuZXdycC0+cmNf Y2tzdW0pOwogdHJ5YWdhaW46Ci0JTkZTTE9DS0NBQ0hFKCk7CisJbXR4X2xvY2sobXV0ZXgpOwog CWhpdCA9IDE7CiAJTElTVF9JTklUKCZuZnNyY190ZW1wbGlzdCk7CiAJLyoKQEAgLTYyMSw4ICs2 NjUsOCBAQCB0cnlhZ2FpbjoKIAkJcnAgPSBoaXRycDsKIAkJaWYgKChycC0+cmNfZmxhZyAmIFJD X0xPQ0tFRCkgIT0gMCkgewogCQkJcnAtPnJjX2ZsYWcgfD0gUkNfV0FOVEVEOwotCQkJKHZvaWQp bXR4X3NsZWVwKHJwLCBORlNDQUNIRU1VVEVYUFRSLAotCQkJICAgIChQWkVSTyAtIDEpIHwgUERS T1AsICJuZnNyYyIsIDEwICogaHopOworCQkJKHZvaWQpbXR4X3NsZWVwKHJwLCBtdXRleCwgKFBa RVJPIC0gMSkgfCBQRFJPUCwKKwkJCSAgICAibmZzcmMiLCAxMCAqIGh6KTsKIAkJCWdvdG8gdHJ5 YWdhaW47CiAJCX0KIAkJaWYgKHJwLT5yY19mbGFnID09IDApCkBAIC02MzAsNyArNjc0LDcgQEAg dHJ5YWdhaW46CiAJCXJwLT5yY19mbGFnIHw9IFJDX0xPQ0tFRDsKIAkJaWYgKHJwLT5yY19mbGFn ICYgUkNfSU5QUk9HKSB7CiAJCQluZXduZnNzdGF0cy5zcnZjYWNoZV9pbnByb2doaXRzKys7Ci0J CQlORlNVTkxPQ0tDQUNIRSgpOworCQkJbXR4X3VubG9jayhtdXRleCk7CiAJCQlpZiAobmV3cnAt PnJjX3NvY2tyZWYgPT0gcnAtPnJjX3NvY2tyZWYpCiAJCQkJbmZzcmNfbWFya3NhbWV0Y3Bjb25u KHJwLT5yY19zb2NrcmVmKTsKIAkJCXJldCA9IFJDX0RST1BJVDsKQEAgLTYzOSwyNCArNjgzLDI0 IEBAIHRyeWFnYWluOgogCQkJICogVjIgb25seS4KIAkJCSAqLwogCQkJbmV3bmZzc3RhdHMuc3J2 Y2FjaGVfbm9uaWRlbWRvbmVoaXRzKys7Ci0JCQlORlNVTkxPQ0tDQUNIRSgpOworCQkJbXR4X3Vu bG9jayhtdXRleCk7CiAJCQlpZiAobmV3cnAtPnJjX3NvY2tyZWYgPT0gcnAtPnJjX3NvY2tyZWYp CiAJCQkJbmZzcmNfbWFya3NhbWV0Y3Bjb25uKHJwLT5yY19zb2NrcmVmKTsKIAkJCXJldCA9IFJD X1JFUExZOwogCQkJbmZzcnZkX3JlcGhlYWQobmQpOwogCQkJKihuZC0+bmRfZXJycCkgPSBycC0+ cmNfc3RhdHVzOwogCQkJcnAtPnJjX3RpbWVzdGFtcCA9IE5GU0RfTU9OT1NFQyArCi0JCQkJTkZT UlZDQUNIRV9UQ1BUSU1FT1VUOworCQkJCW5mc3JjX3RjcHRpbWVvdXQ7CiAJCX0gZWxzZSBpZiAo cnAtPnJjX2ZsYWcgJiBSQ19SRVBNQlVGKSB7CiAJCQluZXduZnNzdGF0cy5zcnZjYWNoZV9ub25p ZGVtZG9uZWhpdHMrKzsKLQkJCU5GU1VOTE9DS0NBQ0hFKCk7CisJCQltdHhfdW5sb2NrKG11dGV4 KTsKIAkJCWlmIChuZXdycC0+cmNfc29ja3JlZiA9PSBycC0+cmNfc29ja3JlZikKIAkJCQluZnNy Y19tYXJrc2FtZXRjcGNvbm4ocnAtPnJjX3NvY2tyZWYpOwogCQkJcmV0ID0gUkNfUkVQTFk7CiAJ CQluZC0+bmRfbXJlcSA9IG1fY29weW0ocnAtPnJjX3JlcGx5LCAwLAogCQkJCU1fQ09QWUFMTCwg TV9XQUlUT0spOwogCQkJcnAtPnJjX3RpbWVzdGFtcCA9IE5GU0RfTU9OT1NFQyArCi0JCQkJTkZT UlZDQUNIRV9UQ1BUSU1FT1VUOworCQkJCW5mc3JjX3RjcHRpbWVvdXQ7CiAJCX0gZWxzZSB7CiAJ CQlwYW5pYygibmZzIHRjcCBjYWNoZTEiKTsKIAkJfQpAQCAtNjc0LDcgKzcxOCw3IEBAIHRyeWFn YWluOgogCW5ld3JwLT5yY19jYWNoZXRpbWUgPSBORlNEX01PTk9TRUM7CiAJbmV3cnAtPnJjX2Zs YWcgfD0gUkNfSU5QUk9HOwogCUxJU1RfSU5TRVJUX0hFQUQoaHAsIG5ld3JwLCByY19oYXNoKTsK LQlORlNVTkxPQ0tDQUNIRSgpOworCW10eF91bmxvY2sobXV0ZXgpOwogCW5kLT5uZF9ycCA9IG5l d3JwOwogCXJldCA9IFJDX0RPSVQ7CiAKQEAgLTY4NSwxNiArNzI5LDE3IEBAIG91dDoKIAogLyoK ICAqIExvY2sgYSBjYWNoZSBlbnRyeS4KLSAqIEFsc28gcHV0cyBhIG11dGV4IGxvY2sgb24gdGhl IGNhY2hlIGxpc3QuCiAgKi8KIHN0YXRpYyB2b2lkCiBuZnNyY19sb2NrKHN0cnVjdCBuZnNydmNh Y2hlICpycCkKIHsKLQlORlNDQUNIRUxPQ0tSRVFVSVJFRCgpOworCXN0cnVjdCBtdHggKm11dGV4 OworCisJbXV0ZXggPSBuZnNyY19jYWNoZW11dGV4KHJwKTsKKwltdHhfYXNzZXJ0KG11dGV4LCBN QV9PV05FRCk7CiAJd2hpbGUgKChycC0+cmNfZmxhZyAmIFJDX0xPQ0tFRCkgIT0gMCkgewogCQly cC0+cmNfZmxhZyB8PSBSQ19XQU5URUQ7Ci0JCSh2b2lkKW10eF9zbGVlcChycCwgTkZTQ0FDSEVN VVRFWFBUUiwgUFpFUk8gLSAxLAotCQkgICAgIm5mc3JjIiwgMCk7CisJCSh2b2lkKW10eF9zbGVl cChycCwgbXV0ZXgsIFBaRVJPIC0gMSwgIm5mc3JjIiwgMCk7CiAJfQogCXJwLT5yY19mbGFnIHw9 IFJDX0xPQ0tFRDsKIH0KQEAgLTcwNSwxMSArNzUwLDEzIEBAIG5mc3JjX2xvY2soc3RydWN0IG5m c3J2Y2FjaGUgKnJwKQogc3RhdGljIHZvaWQKIG5mc3JjX3VubG9jayhzdHJ1Y3QgbmZzcnZjYWNo ZSAqcnApCiB7CisJc3RydWN0IG10eCAqbXV0ZXg7CiAKLQlORlNMT0NLQ0FDSEUoKTsKKwltdXRl eCA9IG5mc3JjX2NhY2hlbXV0ZXgocnApOworCW10eF9sb2NrKG11dGV4KTsKIAlycC0+cmNfZmxh ZyAmPSB+UkNfTE9DS0VEOwogCW5mc3JjX3dhbnRlZChycCk7Ci0JTkZTVU5MT0NLQ0FDSEUoKTsK KwltdHhfdW5sb2NrKG11dGV4KTsKIH0KIAogLyoKQEAgLTczMiw3ICs3NzksNiBAQCBzdGF0aWMg dm9pZAogbmZzcmNfZnJlZWNhY2hlKHN0cnVjdCBuZnNydmNhY2hlICpycCkKIHsKIAotCU5GU0NB Q0hFTE9DS1JFUVVJUkVEKCk7CiAJTElTVF9SRU1PVkUocnAsIHJjX2hhc2gpOwogCWlmIChycC0+ cmNfZmxhZyAmIFJDX1VEUCkgewogCQlUQUlMUV9SRU1PVkUoJm5mc3J2dWRwbHJ1LCBycCwgcmNf bHJ1KTsKQEAgLTc0Miw3ICs3ODgsNyBAQCBuZnNyY19mcmVlY2FjaGUoc3RydWN0IG5mc3J2Y2Fj aGUgKnJwKQogCWlmIChycC0+cmNfZmxhZyAmIFJDX1JFUE1CVUYpIHsKIAkJbWJ1Zl9mcmVlbShy cC0+cmNfcmVwbHkpOwogCQlpZiAoIShycC0+cmNfZmxhZyAmIFJDX1VEUCkpCi0JCQluZnNyY190 Y3BzYXZlZHJlcGxpZXMtLTsKKwkJCWF0b21pY19hZGRfaW50KCZuZnNyY190Y3BzYXZlZHJlcGxp ZXMsIC0xKTsKIAl9CiAJRlJFRSgoY2FkZHJfdClycCwgTV9ORlNSVkNBQ0hFKTsKIAluZXduZnNz dGF0cy5zcnZjYWNoZV9zaXplLS07CkBAIC03NTcsMjAgKzgwMywyMiBAQCBuZnNydmRfY2xlYW5j YWNoZSh2b2lkKQogCXN0cnVjdCBuZnNydmNhY2hlICpycCwgKm5leHRycDsKIAlpbnQgaTsKIAot CU5GU0xPQ0tDQUNIRSgpOwogCWZvciAoaSA9IDA7IGkgPCBORlNSVkNBQ0hFX0hBU0hTSVpFOyBp KyspIHsKKwkJbXR4X2xvY2soJm5mc3JjX3RjcG10eFtpXSk7CiAJCUxJU1RfRk9SRUFDSF9TQUZF KHJwLCAmbmZzcnZoYXNodGJsW2ldLCByY19oYXNoLCBuZXh0cnApIHsKIAkJCW5mc3JjX2ZyZWVj YWNoZShycCk7CiAJCX0KKwkJbXR4X3VubG9jaygmbmZzcmNfdGNwbXR4W2ldKTsKIAl9CisJbXR4 X2xvY2soJm5mc3JjX3VkcG10eCk7CiAJZm9yIChpID0gMDsgaSA8IE5GU1JWQ0FDSEVfSEFTSFNJ WkU7IGkrKykgewogCQlMSVNUX0ZPUkVBQ0hfU0FGRShycCwgJm5mc3J2dWRwaGFzaHRibFtpXSwg cmNfaGFzaCwgbmV4dHJwKSB7CiAJCQluZnNyY19mcmVlY2FjaGUocnApOwogCQl9CiAJfQogCW5l d25mc3N0YXRzLnNydmNhY2hlX3NpemUgPSAwOworCW10eF91bmxvY2soJm5mc3JjX3VkcG10eCk7 CiAJbmZzcmNfdGNwc2F2ZWRyZXBsaWVzID0gMDsKLQlORlNVTkxPQ0tDQUNIRSgpOwogfQogCiAv KgpAQCAtNzgwLDI4ICs4MjgsOTcgQEAgc3RhdGljIHZvaWQKIG5mc3JjX3RyaW1jYWNoZSh1X2lu dDY0X3Qgc29ja3JlZiwgc3RydWN0IHNvY2tldCAqc28pCiB7CiAJc3RydWN0IG5mc3J2Y2FjaGUg KnJwLCAqbmV4dHJwOwotCWludCBpOworCWludCBpLCBqLCBrLCB0aW1lX2hpc3RvWzEwXTsKKwl0 aW1lX3QgdGhpc3N0YW1wOworCXN0YXRpYyB0aW1lX3QgdWRwX2xhc3R0cmltID0gMCwgdGNwX2xh c3R0cmltID0gMDsKKwlzdGF0aWMgaW50IG9uZXRocmVhZCA9IDA7CiAKLQlORlNMT0NLQ0FDSEUo KTsKLQlUQUlMUV9GT1JFQUNIX1NBRkUocnAsICZuZnNydnVkcGxydSwgcmNfbHJ1LCBuZXh0cnAp IHsKLQkJaWYgKCEocnAtPnJjX2ZsYWcgJiAoUkNfSU5QUk9HfFJDX0xPQ0tFRHxSQ19XQU5URUQp KQotCQkgICAgICYmIHJwLT5yY19yZWZjbnQgPT0gMAotCQkgICAgICYmICgocnAtPnJjX2ZsYWcg JiBSQ19SRUZDTlQpIHx8Ci0JCQkgTkZTRF9NT05PU0VDID4gcnAtPnJjX3RpbWVzdGFtcCB8fAot CQkJIG5mc3JjX3VkcGNhY2hlc2l6ZSA+IG5mc3JjX3VkcGhpZ2h3YXRlcikpCi0JCQluZnNyY19m cmVlY2FjaGUocnApOwotCX0KLQlmb3IgKGkgPSAwOyBpIDwgTkZTUlZDQUNIRV9IQVNIU0laRTsg aSsrKSB7Ci0JCUxJU1RfRk9SRUFDSF9TQUZFKHJwLCAmbmZzcnZoYXNodGJsW2ldLCByY19oYXNo LCBuZXh0cnApIHsKKwlpZiAoYXRvbWljX2NtcHNldF9hY3FfaW50KCZvbmV0aHJlYWQsIDAsIDEp ID09IDApCisJCXJldHVybjsKKwlpZiAoTkZTRF9NT05PU0VDICE9IHVkcF9sYXN0dHJpbSB8fAor CSAgICBuZnNyY191ZHBjYWNoZXNpemUgPj0gKG5mc3JjX3VkcGhpZ2h3YXRlciArCisJICAgIG5m c3JjX3VkcGhpZ2h3YXRlciAvIDIpKSB7CisJCW10eF9sb2NrKCZuZnNyY191ZHBtdHgpOworCQl1 ZHBfbGFzdHRyaW0gPSBORlNEX01PTk9TRUM7CisJCVRBSUxRX0ZPUkVBQ0hfU0FGRShycCwgJm5m c3J2dWRwbHJ1LCByY19scnUsIG5leHRycCkgewogCQkJaWYgKCEocnAtPnJjX2ZsYWcgJiAoUkNf SU5QUk9HfFJDX0xPQ0tFRHxSQ19XQU5URUQpKQogCQkJICAgICAmJiBycC0+cmNfcmVmY250ID09 IDAKIAkJCSAgICAgJiYgKChycC0+cmNfZmxhZyAmIFJDX1JFRkNOVCkgfHwKLQkJCQkgTkZTRF9N T05PU0VDID4gcnAtPnJjX3RpbWVzdGFtcCB8fAotCQkJCSBuZnNyY19hY3RpdmVzb2NrZXQocnAs IHNvY2tyZWYsIHNvKSkpCisJCQkJIHVkcF9sYXN0dHJpbSA+IHJwLT5yY190aW1lc3RhbXAgfHwK KwkJCQkgbmZzcmNfdWRwY2FjaGVzaXplID4gbmZzcmNfdWRwaGlnaHdhdGVyKSkKIAkJCQluZnNy Y19mcmVlY2FjaGUocnApOwogCQl9CisJCW10eF91bmxvY2soJm5mc3JjX3VkcG10eCk7CisJfQor CWlmIChORlNEX01PTk9TRUMgIT0gdGNwX2xhc3R0cmltIHx8CisJICAgIG5mc3JjX3RjcHNhdmVk cmVwbGllcyA+PSBuZnNyY190Y3BoaWdod2F0ZXIpIHsKKwkJZm9yIChpID0gMDsgaSA8IDEwOyBp KyspCisJCQl0aW1lX2hpc3RvW2ldID0gMDsKKwkJZm9yIChpID0gMDsgaSA8IE5GU1JWQ0FDSEVf SEFTSFNJWkU7IGkrKykgeworCQkJbXR4X2xvY2soJm5mc3JjX3RjcG10eFtpXSk7CisJCQlpZiAo aSA9PSAwKQorCQkJCXRjcF9sYXN0dHJpbSA9IE5GU0RfTU9OT1NFQzsKKwkJCUxJU1RfRk9SRUFD SF9TQUZFKHJwLCAmbmZzcnZoYXNodGJsW2ldLCByY19oYXNoLAorCQkJICAgIG5leHRycCkgewor CQkJCWlmICghKHJwLT5yY19mbGFnICYKKwkJCQkgICAgIChSQ19JTlBST0d8UkNfTE9DS0VEfFJD X1dBTlRFRCkpCisJCQkJICAgICAmJiBycC0+cmNfcmVmY250ID09IDApIHsKKwkJCQkJLyoKKwkJ CQkJICogVGhlIHRpbWVzdGFtcHMgcmFuZ2UgZnJvbSByb3VnaGx5IHRoZQorCQkJCQkgKiBwcmVz ZW50ICh0Y3BfbGFzdHRyaW0pIHRvIHRoZSBwcmVzZW50CisJCQkJCSAqICsgbmZzcmNfdGNwdGlt ZW91dC4gR2VuZXJhdGUgYSBzaW1wbGUKKwkJCQkJICogaGlzdG9ncmFtIG9mIHdoZXJlIHRoZSB0 aW1lb3V0cyBmYWxsLgorCQkJCQkgKi8KKwkJCQkJaiA9IHJwLT5yY190aW1lc3RhbXAgLSB0Y3Bf bGFzdHRyaW07CisJCQkJCWlmIChqID49IG5mc3JjX3RjcHRpbWVvdXQpCisJCQkJCQlqID0gbmZz cmNfdGNwdGltZW91dCAtIDE7CisJCQkJCWlmIChqIDwgMCkKKwkJCQkJCWogPSAwOworCQkJCQlq ID0gKGogKiAxMCAvIG5mc3JjX3RjcHRpbWVvdXQpICUgMTA7CisJCQkJCXRpbWVfaGlzdG9bal0r KzsKKwkJCQkJaWYgKChycC0+cmNfZmxhZyAmIFJDX1JFRkNOVCkgfHwKKwkJCQkJICAgIHRjcF9s YXN0dHJpbSA+IHJwLT5yY190aW1lc3RhbXAgfHwKKwkJCQkJICAgIG5mc3JjX2FjdGl2ZXNvY2tl dChycCwgc29ja3JlZiwgc28pKQorCQkJCQkJbmZzcmNfZnJlZWNhY2hlKHJwKTsKKwkJCQl9CisJ CQl9CisJCQltdHhfdW5sb2NrKCZuZnNyY190Y3BtdHhbaV0pOworCQl9CisJCWogPSBuZnNyY190 Y3BoaWdod2F0ZXIgLyA1OwkvKiAyMCUgb2YgaXQgKi8KKwkJaWYgKGogPiAwICYmIChuZnNyY190 Y3BzYXZlZHJlcGxpZXMgKyBqKSA+IG5mc3JjX3RjcGhpZ2h3YXRlcikgeworCQkJLyoKKwkJCSAq IFRyaW0gc29tZSBtb3JlIHdpdGggYSBzbWFsbGVyIHRpbWVvdXQgb2YgYXMgbGl0dGxlCisJCQkg KiBhcyAyMCUgb2YgbmZzcmNfdGNwdGltZW91dCB0byB0cnkgYW5kIGdldCBiZWxvdworCQkJICog ODAlIG9mIHRoZSBuZnNyY190Y3BoaWdod2F0ZXIuCisJCQkgKi8KKwkJCWsgPSAwOworCQkJZm9y IChpID0gMDsgaSA8IDg7IGkrKykgeworCQkJCWsgKz0gdGltZV9oaXN0b1tpXTsKKwkJCQlpZiAo ayA+IGopCisJCQkJCWJyZWFrOworCQkJfQorCQkJayA9IG5mc3JjX3RjcHRpbWVvdXQgKiAoaSAr IDEpIC8gMTA7CisJCQlpZiAoayA8IDEpCisJCQkJayA9IDE7CisJCQl0aGlzc3RhbXAgPSB0Y3Bf bGFzdHRyaW0gKyBrOworCQkJZm9yIChpID0gMDsgaSA8IE5GU1JWQ0FDSEVfSEFTSFNJWkU7IGkr KykgeworCQkJCW10eF9sb2NrKCZuZnNyY190Y3BtdHhbaV0pOworCQkJCUxJU1RfRk9SRUFDSF9T QUZFKHJwLCAmbmZzcnZoYXNodGJsW2ldLCByY19oYXNoLAorCQkJCSAgICBuZXh0cnApIHsKKwkJ CQkJaWYgKCEocnAtPnJjX2ZsYWcgJgorCQkJCQkgICAgIChSQ19JTlBST0d8UkNfTE9DS0VEfFJD X1dBTlRFRCkpCisJCQkJCSAgICAgJiYgcnAtPnJjX3JlZmNudCA9PSAwCisJCQkJCSAgICAgJiYg KChycC0+cmNfZmxhZyAmIFJDX1JFRkNOVCkgfHwKKwkJCQkJCSB0aGlzc3RhbXAgPiBycC0+cmNf dGltZXN0YW1wIHx8CisJCQkJCQkgbmZzcmNfYWN0aXZlc29ja2V0KHJwLCBzb2NrcmVmLAorCQkJ CQkJICAgIHNvKSkpCisJCQkJCQluZnNyY19mcmVlY2FjaGUocnApOworCQkJCX0KKwkJCQltdHhf dW5sb2NrKCZuZnNyY190Y3BtdHhbaV0pOworCQkJfQorCQl9CiAJfQotCU5GU1VOTE9DS0NBQ0hF KCk7CisJYXRvbWljX3N0b3JlX3JlbF9pbnQoJm9uZXRocmVhZCwgMCk7CiB9CiAKIC8qCkBAIC04 MTAsMTIgKzkyNywxNCBAQCBuZnNyY190cmltY2FjaGUodV9pbnQ2NF90IHNvY2tyZWYsIHN0cnVj CiBBUFBMRVNUQVRJQyB2b2lkCiBuZnNydmRfcmVmY2FjaGUoc3RydWN0IG5mc3J2Y2FjaGUgKnJw KQogeworCXN0cnVjdCBtdHggKm11dGV4OwogCi0JTkZTTE9DS0NBQ0hFKCk7CisJbXV0ZXggPSBu ZnNyY19jYWNoZW11dGV4KHJwKTsKKwltdHhfbG9jayhtdXRleCk7CiAJaWYgKHJwLT5yY19yZWZj bnQgPCAwKQogCQlwYW5pYygibmZzIGNhY2hlIHJlZmNudCIpOwogCXJwLT5yY19yZWZjbnQrKzsK LQlORlNVTkxPQ0tDQUNIRSgpOworCW10eF91bmxvY2sobXV0ZXgpOwogfQogCiAvKgpAQCAtODI0 LDE0ICs5NDMsMTYgQEAgbmZzcnZkX3JlZmNhY2hlKHN0cnVjdCBuZnNydmNhY2hlICpycCkKIEFQ UExFU1RBVElDIHZvaWQKIG5mc3J2ZF9kZXJlZmNhY2hlKHN0cnVjdCBuZnNydmNhY2hlICpycCkK IHsKKwlzdHJ1Y3QgbXR4ICptdXRleDsKIAotCU5GU0xPQ0tDQUNIRSgpOworCW11dGV4ID0gbmZz cmNfY2FjaGVtdXRleChycCk7CisJbXR4X2xvY2sobXV0ZXgpOwogCWlmIChycC0+cmNfcmVmY250 IDw9IDApCiAJCXBhbmljKCJuZnMgY2FjaGUgZGVyZWZjbnQiKTsKIAlycC0+cmNfcmVmY250LS07 CiAJaWYgKHJwLT5yY19yZWZjbnQgPT0gMCAmJiAhKHJwLT5yY19mbGFnICYgKFJDX0xPQ0tFRCB8 IFJDX0lOUFJPRykpKQogCQluZnNyY19mcmVlY2FjaGUocnApOwotCU5GU1VOTE9DS0NBQ0hFKCk7 CisJbXR4X3VubG9jayhtdXRleCk7CiB9CiAKIC8qCi0tLSBmcy9uZnNzZXJ2ZXIvbmZzX25mc2Rw b3J0LmMub3JpZwkyMDEzLTAzLTAyIDE4OjE5OjM0LjAwMDAwMDAwMCAtMDUwMAorKysgZnMvbmZz c2VydmVyL25mc19uZnNkcG9ydC5jCTIwMTMtMDMtMTIgMTc6NTE6MzEuMDAwMDAwMDAwIC0wNDAw CkBAIC02MSw3ICs2MSw4IEBAIGV4dGVybiBzdHJ1Y3QgbmZzdjRsb2NrIG5mc2Rfc3VzcGVuZF9s b2MKIGV4dGVybiBzdHJ1Y3QgbmZzc2Vzc2lvbmhhc2ggbmZzc2Vzc2lvbmhhc2hbTkZTU0VTU0lP TkhBU0hTSVpFXTsKIHN0cnVjdCB2ZnNvcHRsaXN0IG5mc3Y0cm9vdF9vcHQsIG5mc3Y0cm9vdF9u ZXdvcHQ7CiBORlNETE9DS01VVEVYOwotc3RydWN0IG10eCBuZnNfY2FjaGVfbXV0ZXg7CitzdHJ1 Y3QgbXR4IG5mc3JjX3RjcG10eFtORlNSVkNBQ0hFX0hBU0hTSVpFXTsKK3N0cnVjdCBtdHggbmZz cmNfdWRwbXR4Owogc3RydWN0IG10eCBuZnNfdjRyb290X211dGV4Owogc3RydWN0IG5mc3J2Zmgg bmZzX3Jvb3RmaCwgbmZzX3B1YmZoOwogaW50IG5mc19wdWJmaHNldCA9IDAsIG5mc19yb290Zmhz ZXQgPSAwOwpAQCAtMzMwNSw3ICszMzA2LDEwIEBAIG5mc2RfbW9kZXZlbnQobW9kdWxlX3QgbW9k LCBpbnQgdHlwZSwgdm8KIAkJaWYgKGxvYWRlZCkKIAkJCWdvdG8gb3V0OwogCQluZXduZnNfcG9y dGluaXQoKTsKLQkJbXR4X2luaXQoJm5mc19jYWNoZV9tdXRleCwgIm5mc19jYWNoZV9tdXRleCIs IE5VTEwsIE1UWF9ERUYpOworCQlmb3IgKGkgPSAwOyBpIDwgTkZTUlZDQUNIRV9IQVNIU0laRTsg aSsrKQorCQkJbXR4X2luaXQoJm5mc3JjX3RjcG10eFtpXSwgIm5mc190Y3BjYWNoZV9tdXRleCIs IE5VTEwsCisJCQkgICAgTVRYX0RFRik7CisJCW10eF9pbml0KCZuZnNyY191ZHBtdHgsICJuZnNf dWRwY2FjaGVfbXV0ZXgiLCBOVUxMLCBNVFhfREVGKTsKIAkJbXR4X2luaXQoJm5mc192NHJvb3Rf bXV0ZXgsICJuZnNfdjRyb290X211dGV4IiwgTlVMTCwgTVRYX0RFRik7CiAJCW10eF9pbml0KCZu ZnN2NHJvb3RfbW50Lm1udF9tdHgsICJzdHJ1Y3QgbW91bnQgbXR4IiwgTlVMTCwKIAkJICAgIE1U WF9ERUYpOwpAQCAtMzM1Miw3ICszMzU2LDkgQEAgbmZzZF9tb2RldmVudChtb2R1bGVfdCBtb2Qs IGludCB0eXBlLCB2bwogCQkJc3ZjcG9vbF9kZXN0cm95KG5mc3J2ZF9wb29sKTsKIAogCQkvKiBh bmQgZ2V0IHJpZCBvZiB0aGUgbG9ja3MgKi8KLQkJbXR4X2Rlc3Ryb3koJm5mc19jYWNoZV9tdXRl eCk7CisJCWZvciAoaSA9IDA7IGkgPCBORlNSVkNBQ0hFX0hBU0hTSVpFOyBpKyspCisJCQltdHhf ZGVzdHJveSgmbmZzcmNfdGNwbXR4W2ldKTsKKwkJbXR4X2Rlc3Ryb3koJm5mc3JjX3VkcG10eCk7 CiAJCW10eF9kZXN0cm95KCZuZnNfdjRyb290X211dGV4KTsKIAkJbXR4X2Rlc3Ryb3koJm5mc3Y0 cm9vdF9tbnQubW50X210eCk7CiAJCWZvciAoaSA9IDA7IGkgPCBORlNTRVNTSU9OSEFTSFNJWkU7 IGkrKykKLS0tIGZzL25mcy9uZnNwb3J0Lmgub3JpZwkyMDEzLTAzLTAyIDE4OjM1OjEzLjAwMDAw MDAwMCAtMDUwMAorKysgZnMvbmZzL25mc3BvcnQuaAkyMDEzLTAzLTEyIDE3OjUxOjMxLjAwMDAw MDAwMCAtMDQwMApAQCAtNjA5LDExICs2MDksNiBAQCB2b2lkIG5mc3J2ZF9yY3Yoc3RydWN0IHNv Y2tldCAqLCB2b2lkICosCiAjZGVmaW5lCU5GU1JFUVNQSU5MT0NLCQlleHRlcm4gc3RydWN0IG10 eCBuZnNfcmVxX211dGV4CiAjZGVmaW5lCU5GU0xPQ0tSRVEoKQkJbXR4X2xvY2soJm5mc19yZXFf bXV0ZXgpCiAjZGVmaW5lCU5GU1VOTE9DS1JFUSgpCQltdHhfdW5sb2NrKCZuZnNfcmVxX211dGV4 KQotI2RlZmluZQlORlNDQUNIRU1VVEVYCQlleHRlcm4gc3RydWN0IG10eCBuZnNfY2FjaGVfbXV0 ZXgKLSNkZWZpbmUJTkZTQ0FDSEVNVVRFWFBUUgkoJm5mc19jYWNoZV9tdXRleCkKLSNkZWZpbmUJ TkZTTE9DS0NBQ0hFKCkJCW10eF9sb2NrKCZuZnNfY2FjaGVfbXV0ZXgpCi0jZGVmaW5lCU5GU1VO TE9DS0NBQ0hFKCkJbXR4X3VubG9jaygmbmZzX2NhY2hlX211dGV4KQotI2RlZmluZQlORlNDQUNI RUxPQ0tSRVFVSVJFRCgpCW10eF9hc3NlcnQoJm5mc19jYWNoZV9tdXRleCwgTUFfT1dORUQpCiAj ZGVmaW5lCU5GU1NPQ0tNVVRFWAkJZXh0ZXJuIHN0cnVjdCBtdHggbmZzX3Nsb2NrX211dGV4CiAj ZGVmaW5lCU5GU1NPQ0tNVVRFWFBUUgkJKCZuZnNfc2xvY2tfbXV0ZXgpCiAjZGVmaW5lCU5GU0xP Q0tTT0NLKCkJCW10eF9sb2NrKCZuZnNfc2xvY2tfbXV0ZXgpCi0tLSBmcy9uZnMvbmZzcnZjYWNo ZS5oLm9yaWcJMjAxMy0wMS0wNyAwOTowNDoxNS4wMDAwMDAwMDAgLTA1MDAKKysrIGZzL25mcy9u ZnNydmNhY2hlLmgJMjAxMy0wMy0xMiAxODowMjo0Mi4wMDAwMDAwMDAgLTA0MDAKQEAgLTQxLDcg KzQxLDcgQEAKICNkZWZpbmUJTkZTUlZDQUNIRV9NQVhfU0laRQkyMDQ4CiAjZGVmaW5lCU5GU1JW Q0FDSEVfTUlOX1NJWkUJICA2NAogCi0jZGVmaW5lCU5GU1JWQ0FDSEVfSEFTSFNJWkUJMjAKKyNk ZWZpbmUJTkZTUlZDQUNIRV9IQVNIU0laRQk1MDAKIAogc3RydWN0IG5mc3J2Y2FjaGUgewogCUxJ U1RfRU5UUlkobmZzcnZjYWNoZSkgcmNfaGFzaDsJCS8qIEhhc2ggY2hhaW4gKi8K ------=_Part_3844452_1526038691.1363146480611-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 05:15:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C4F5621F; Wed, 13 Mar 2013 05:15:01 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 525571E5; Wed, 13 Mar 2013 05:15:00 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2D5ExCU037978; Wed, 13 Mar 2013 01:14:59 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2D5EwWC037975; Wed, 13 Mar 2013 01:14:58 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20800.2898.798984.157469@hergotha.csail.mit.edu> Date: Wed, 13 Mar 2013 01:14:58 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: <75232221.3844453.1363146480616.JavaMail.root@erie.cs.uoguelph.ca> References: <20798.44871.601547.24628@hergotha.csail.mit.edu> <75232221.3844453.1363146480616.JavaMail.root@erie.cs.uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 13 Mar 2013 01:14:59 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org, andre@freebsd.org, Ivan Voras X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 05:15:01 -0000 < said: > Basically, this patch: > - allows setting of the tcp timeout via vfs.nfsd.tcpcachetimeo > (I'd suggest you go down to a few minutes instead of 12hrs) > - allows TCP caching to be disabled by setting vfs.nfsd.cachetcp=0 > - does the above 2 things you describe to try and avoid the livelock, > although not quite using an lru list > - increases the hash table size to 500 (still a compile time setting) > (feel free to make it even bigger) > - sets nfsrc_floodlevel to at least nfsrc_tcphighwater, so you can > grow vfs.nfsd.tcphighwater as big as you dare Thanks, this looks very good. One quibble with the last bit: I'd do that in a sysctl() handler rather than checking it every time through. If somebody uses a debugger rather than sysctl to change tcphighwater, they deserve what's coming to them. Also, I might suggest adding a counter for how many times we had to go through the "try harder" phase, so that the sysadmin has some indication that the defaults need adjustment. I will test this out later this week and see how it performs. I have a user who has been able to reproducibly clobber servers before, so if he has time and cycles available it should be pretty easy to tell whether it's working or not. -GAWollman From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 05:30:29 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3DB5D4CB; Wed, 13 Mar 2013 05:30:29 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 1A4EC25E; Wed, 13 Mar 2013 05:30:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2D5USq3012112; Wed, 13 Mar 2013 05:30:28 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2D5USVG012111; Wed, 13 Mar 2013 05:30:28 GMT (envelope-from linimon) Date: Wed, 13 Mar 2013 05:30:28 GMT Message-Id: <201303130530.r2D5USVG012111@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/176884: [re] re0 flapping up/down X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 05:30:29 -0000 Old Synopsis: re0 flapping up/down New Synopsis: [re] re0 flapping up/down Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Mar 13 05:30:13 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=176884 From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 05:55:11 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5B18B964; Wed, 13 Mar 2013 05:55:11 +0000 (UTC) (envelope-from dkandula@gmail.com) Received: from mail-wg0-f45.google.com (mail-wg0-f45.google.com [74.125.82.45]) by mx1.freebsd.org (Postfix) with ESMTP id B5436327; Wed, 13 Mar 2013 05:55:10 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id dq12so616688wgb.12 for ; Tue, 12 Mar 2013 22:55:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to:cc :content-type; bh=KAHcTKgZaxqw2p8Kb3Ee1YTw1+Y+O7/p4/HTdBX3JLs=; b=QPPxoLOq0KTMMyAUPbYyc6acIbb4bCnrjaDTMWcxFtKdyl6gfJD0KqKmQAGxeOHLjc 4wLgxsR1S8N/0CLiWDIncAEbxQVBiJ3DuLFCX5ZblQ0Yr9C53/FV3kt30qPgTT2Ar/BK /hmiO5p2mbJgRuZcVuinR7R36tN5kTL0KXyyRSY5mfgvvtFuYbVBVBD/OByI/P3Hnws6 NwtZkIvrkSQpWw/O1BpvDaqp+vyC29T65s1JivGshE0OOzsaWFvOF8CRN1hqNQ9N3D1t vdkQFuvU15BFHcoceojFMEUkiSn2C8PAdo/dqfxbJ+ciLNrlpk29R0KsRhFlLTAIQbEi AVCQ== MIME-Version: 1.0 X-Received: by 10.194.92.65 with SMTP id ck1mr31526305wjb.54.1363154109531; Tue, 12 Mar 2013 22:55:09 -0700 (PDT) Received: by 10.194.37.167 with HTTP; Tue, 12 Mar 2013 22:55:09 -0700 (PDT) Date: Wed, 13 Mar 2013 11:25:09 +0530 Message-ID: Subject: NDP prefix list locking From: Dheeraj Kandula To: markj@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 05:55:11 -0000 Hi Mark, In the patch that you provided to freebsd-net, http://lists.freebsd.org/pipermail/freebsd-net/2013-February/034695.html, I have a question. nd6_timer() acquires the write lock using ND_DEFRTR_WLOCK. However, defrouter_select again tries to acquire a read lock using ND_DEFRTR_RLOCK. This will lead to a deadlock as the read lock can't be acquired as the write lock is already acquired. The solution is to check if the write lock is already acquired. If not the read lock can be acquired, else there is no need to acquire the read lock. Can you share your thoughts on the same. Dheeraj From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 07:02:26 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6CF4A5B9 for ; Wed, 13 Mar 2013 07:02:26 +0000 (UTC) (envelope-from az0859@gmail.com) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) by mx1.freebsd.org (Postfix) with ESMTP id F352E755 for ; Wed, 13 Mar 2013 07:02:25 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id ez12so501901wid.12 for ; Wed, 13 Mar 2013 00:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=Qug4+b3ZtgSIBLGcSNp75RcHNufEJ1utJJlvV4jWvdo=; b=po+itDJL52uyx0nuSh6T26W2/+iPn/bviIW04tToYQMK6QcJxmJ/k68tgU9Fi4+2cM mXjEgWqYEaxrUY7GgHjvwI458390gATByEPp5A2R+IFTh0M5KK5RWTCJh3TazQP8igU0 sugvazIpHxZec4LA0GwAGVRB/W0z6/3cfD+WUSNOwtacaL7S6k/l4Q5oymY8Te1nulC3 L13x8NLHMxk8HvfJtzTg/4bO4fubHotOMit345NvOYgHGdL8symrfehHBcV3Lp2l1XZB RcG6Lj/5qEueEPaIfrJa+dIbwUFTeUnxe2WwaevphHvNP4iPXfogpQ1nasJsnydF+yw3 yxoQ== MIME-Version: 1.0 X-Received: by 10.194.88.138 with SMTP id bg10mr31936767wjb.13.1363158145013; Wed, 13 Mar 2013 00:02:25 -0700 (PDT) Received: by 10.217.106.137 with HTTP; Wed, 13 Mar 2013 00:02:24 -0700 (PDT) Date: Wed, 13 Mar 2013 12:32:24 +0530 Message-ID: Subject: request From: rathish mudaliar To: net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 07:02:26 -0000 Hi, I request you to kindly add me to your mailing list. Regards Rathish.R From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 07:14:08 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 58C5F828 for ; Wed, 13 Mar 2013 07:14:08 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-f49.google.com (mail-ee0-f49.google.com [74.125.83.49]) by mx1.freebsd.org (Postfix) with ESMTP id D80127D3 for ; Wed, 13 Mar 2013 07:14:07 +0000 (UTC) Received: by mail-ee0-f49.google.com with SMTP id d41so268141eek.36 for ; Wed, 13 Mar 2013 00:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=9/dO9djOPF+DkbkFWQqXbc3Hy2IA9XCilEXDAUE+4dI=; b=zdcDW3f0Ob1MxF0zCHgBieDjbgV4mBO65NtZMR6I6GJA1QI0taUL36cW/dn8CE3gb8 i9W1vwfX56fd2F7skyRagUZ5VwPra8tF2fKyprbFEWi8opwS+roJLYa4aPp/5QGSMJvT AHKGvR8D5I82r1Ng3aghejGw96iUZ5gFdqMYiwYSEs8epd10F/fLP5d5O10xuc3RTtte TSrQEan4NxHmYiX+m+JhVpR+O4Jgj9y28MqkGJ63Gp142AIUjGhUztTwUlB2T+UzcH5G v6gG8ZYpd5hzfILJ0LYBgDpm/xXhXQuBJYwPfmgpS6JhQxcXO63vn4r7vmaF/swhp7G5 XIrw== MIME-Version: 1.0 X-Received: by 10.14.184.68 with SMTP id r44mr22777262eem.40.1363158841227; Wed, 13 Mar 2013 00:14:01 -0700 (PDT) Received: by 10.14.133.204 with HTTP; Wed, 13 Mar 2013 00:14:01 -0700 (PDT) In-Reply-To: References: Date: Wed, 13 Mar 2013 00:14:01 -0700 Message-ID: Subject: Re: request From: hiren panchasara To: rathish mudaliar Content-Type: text/plain; charset=UTF-8 Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 07:14:08 -0000 On Wed, Mar 13, 2013 at 12:02 AM, rathish mudaliar wrote: > Hi, > > I request you to kindly add me to your mailing list. http://lists.freebsd.org/mailman/listinfo/freebsd-net This is where you can go and subscribe. cheers, Hiren > > > Regards > Rathish.R > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 09:17:33 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 11E145FA for ; Wed, 13 Mar 2013 09:17:33 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id A0D76F04 for ; Wed, 13 Mar 2013 09:17:31 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id EE3E31CD7F3 for ; Wed, 13 Mar 2013 09:17:29 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tehaksT+L2z0 for ; Wed, 13 Mar 2013 09:17:29 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 3F1891CD7F2 for ; Wed, 13 Mar 2013 09:17:29 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2D9HRCQ017974 for ; Wed, 13 Mar 2013 09:17:27 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2D9HR9C017973 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 09:17:27 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 09:17:27 +0000 From: Schrodinger To: "freebsd-net@freebsd.org" Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313091727.GA17859@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="X1bOJ3K7DJ5YkBrT" Content-Disposition: inline In-Reply-To: <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 09:17:33 -0000 --X1bOJ3K7DJ5YkBrT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 02:25, Damien Fleuriot wrote: [...] >=20 >=20 > The network is actually /48 and you get assigned a /64 inside it. >=20 > Set your interface to use the /48 prefix and voodoo will happen (I can as= sure you with a 97% certainty that your default GW is inside the /48). > Of course, using the /48 prefix doesn't mean you can/may use IPs from out= side the /64 that was given you. Voodoo, indeed... I'm sure there's a /48 used somewhere but to be more specific, or rather obvious, my default gateway resides at the boundary of a /56 - 2001:41D0:2:E700::/56 If you pay close attention you will notice that the default gateway is the last usable address from that range. I had already tried this btw, I spent some time confirming what I am was seeing; what was the actual case and ways I could perhaps change the configuration to get it Just Working. However, I would rather it worked correctly and not contain a configuration option that I either do not understand it's necessity nor do I see it as necessary. I don't claim to know IPv6 inside and out but one consideration I had was that because of the host route for the default gateway FreeBSD does not solicit for the "on-link gateway" because the interface is not set=20 to ACCEPT_RTADV. But that doesn't make immediate sense. Corrections and education welcome. >=20 >=20 > Kindly reply with topic "SOLVED" if that fixed you up, that googlers in t= he future may find the solution easily. I would, but IMO this isn't the most optimal solution; changing my prefix length so that I can reach the gateway... Can this kind of host routing just not be done ? The way I see this issue is that without ACCEPT_RTADV on my interface FreeBSD won't attempt Neighbour Solicitation for the default gateway but I am uncertain why this is the case. Bug or policy or That's How It Works. C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --X1bOJ3K7DJ5YkBrT Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQEQlAAoJEBBi7cjNKnTjKMUP/2YmdQ0buk+D/aF6Gzi5IWjh BQpu/UOAZLRf15dwiQVQhL13uC/mZb+JftwTVHoUtMNtyd/hQ+CLeIOZSzj9LqsX nC8kwFhJvStQMcESo0F9mZ+ejcae8PIK6n6V4t/hicLjQsuiJt+ujekdUL55CKlW ABnh/sb2Bnz5IXKoVQ/Bp/0WG68KVlo2YtNmInOBd8+HxXUil4d8yunU8cxqdHgC kuXNjmVx+EtcAgeTpmcx73fOW1n/Y2KfjvdZiSyQIbaVs0yBp2W8AJftQQrPXlXn U6RScWpUyssFlASapNjUkoKHHfDElzz2WHDDtEy5CSaCWT/5ltesh8z+tETgBY6G YyfNMykMYbmbHsDdy62JXksrqLyjmwrCEp0gmedVmQiCqIsakCUl7st71RSSlqe5 jbDxBj5DK7AVWSv/kixAI3o7dzp54AmvaVOJsr5tnx5xPLblI3Zt7FItyzUCIQ0T N09smD1OTLEIFcJz6PT7Nv/epSllNVs+r3vHFrKxxCqa09T+rhrc/A7WHG7l8Oxt pWg+cCwtqw4YX2BaP9koY+cgwEnHNTQq7/UFqxsK98E9LOUkns7NDBnpooQlNmEO UTkpW7Bi/y/ouqlbcKCU5fOQeOj2XZYkNSsCnwHOuXglp8LJ0M1guyICobhbTGq8 UBrwsZPOKyTJ3p+JF/oK =C0Wj -----END PGP SIGNATURE----- --X1bOJ3K7DJ5YkBrT-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 11:16:55 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0F1AFF0B for ; Wed, 13 Mar 2013 11:16:55 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) by mx1.freebsd.org (Postfix) with ESMTP id 96FB38E0 for ; Wed, 13 Mar 2013 11:16:54 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id ez12so653244wid.6 for ; Wed, 13 Mar 2013 04:16:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to:x-gm-message-state; bh=ZgK761DN4zS0P87RVDmSd1HUQ3H6Glfe6aozFVOF0OM=; b=B/0HhMWWprOF0PKEg1mVuPUdm/p/Yl0DdRaypR8LOkqUZkUQroWjMJo4ksb3dKTmUe QpZjnDrT2Iw67lc0AzlIqC+aDy3yccradzSc8uEy4ehkJgLnhh38o+0+lNpiwWXynLsi ex+JkAIWwJWRWGm+/W1lZR6h8fJuAKdkigeVWPvcKM3U1p6F+UgB2S7FyLvR0UPBKQyJ 0Dg3kkAZ1J9tLcOfVZ1zkg8P/Ko2rhoqGvw9ZLRo8HjGTCCtVM0juVUry4VI/VkRbzFA uuzwy6mBWVsLBsCuFYbWdWyEw4yznH++sEPMkn8it8MTTiSNhAOqWH6NRxTQoZDXAvy+ EaTA== X-Received: by 10.194.242.163 with SMTP id wr3mr33048983wjc.35.1363173413808; Wed, 13 Mar 2013 04:16:53 -0700 (PDT) Received: from [10.41.32.4] ([92.90.16.94]) by mx.google.com with ESMTPS id q2sm2271781wiz.8.2013.03.13.04.16.50 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 13 Mar 2013 04:16:53 -0700 (PDT) References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> Mime-Version: 1.0 (1.0) In-Reply-To: <20130313091727.GA17859@defiant.konundrum.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (10B144) From: Damien Fleuriot Subject: Re: ipv6 default router Operation not permitted Date: Wed, 13 Mar 2013 12:16:27 +0100 To: Schrodinger X-Gm-Message-State: ALoCoQmEGEkHBbsDA8k6/uFoIcIIaEJ8EPihttvRhk8BswFv9RVLDXiWuHL0DB1jG5bBEqJYAQzY Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 11:16:55 -0000 On 13 Mar 2013, at 10:17, Schrodinger wrote: > On 2013/03/13 02:25, Damien Fleuriot wrote: >=20 > [...] >=20 >>=20 >>=20 >> The network is actually /48 and you get assigned a /64 inside it. >>=20 >> Set your interface to use the /48 prefix and voodoo will happen (I can as= sure you with a 97% certainty that your default GW is inside the /48). >> Of course, using the /48 prefix doesn't mean you can/may use IPs from out= side the /64 that was given you. >=20 > Voodoo, indeed... I'm sure there's a /48 used somewhere but to be more > specific, or rather obvious, my default gateway resides at the boundary > of a /56 - 2001:41D0:2:E700::/56 If you pay close attention you will > notice that the default gateway is the last usable address from that > range. I had already tried this btw, I spent some time confirming what I > am was seeing; what was the actual case and ways I could perhaps change > the configuration to get it Just Working. However, I would rather it > worked correctly and not contain a configuration option that I either do > not understand it's necessity nor do I see it as necessary. >=20 > I don't claim to know IPv6 inside and out but one consideration I had > was that because of the host route for the default gateway FreeBSD does > not solicit for the "on-link gateway" because the interface is not set=20 > to ACCEPT_RTADV. But that doesn't make immediate sense. >=20 > Corrections and education welcome. >=20 >>=20 >>=20 >> Kindly reply with topic "SOLVED" if that fixed you up, that googlers in t= he future may find the solution easily. >=20 > I would, but IMO this isn't the most optimal solution; changing my > prefix length so that I can reach the gateway... Can this kind of host > routing just not be done ? The way I see this issue is that without > ACCEPT_RTADV on my interface FreeBSD won't attempt Neighbour > Solicitation for the default gateway but I am uncertain why this is the > case. Bug or policy or That's How It Works. >=20 > C. > --=20 The thing is, your gateway sitting there in the middle of a /48 or /56 has t= o do with how OVH distributes IPv6 prefixes and how they configure their rou= ters. FreeBSD's hardly responsible here. I've had the very same problem with the very same host, hence why I immediat= ely pointed out using a smaller prefix. This is the same use case as getting A /29 assignment inside a /24, but havi= ng to use /24 anyway because the host: - doesn't want to bother configuring a gateway for each customer within its /= 24 (likely the case with ipv6) - doesn't want to waste IPs (likely the case with ipv4) Using a smaller prefix of /56 effectively puts you in the correct network an= d gives you the ability to reach the GW, just keep to your /64 for assignmen= ts ;)= From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 11:36:27 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0EAD6A84 for ; Wed, 13 Mar 2013 11:36:27 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (amavis-proxy-ori.ijs.si [193.2.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id BBC0EA0D for ; Wed, 13 Mar 2013 11:36:26 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3ZQrQJ5pk3zGMmP for ; Wed, 13 Mar 2013 12:28:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:organization:in-reply-to:references:user-agent :date:date:subject:subject:from:from:received:received:received :vbr-info; s=jakla2; t=1363174078; x=1365766079; bh=9ywMUXCeIfiD EPvoH4Yr13DHfmj3C9qSurByNB1Qktk=; b=MLmXoNRecAKZHz/XS36/lg1uDBTZ LD6uqjAs3eZtZ2PpKkv6HBKDnhns0RZmb9F+dkctVAyDDZps4JLT70U/YfJmonPj P7XnAvNW+73Fth98djc+Cqy6Orq/76LqeG0NFxe3LaeCiGo3vLLphiYoxTZ7/8Uz ERR5qPX8K8JRA38= VBR-Info: md=ijs.si; mc=all; mv=dwl.spamhaus.org; X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id 3dX3fuRu5u8i for ; Wed, 13 Mar 2013 12:27:58 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP for ; Wed, 13 Mar 2013 12:27:58 +0100 (CET) Received: from sleepy.ijs.si (sleepy.ijs.si [IPv6:2001:1470:ff80:e001::1:1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id 40D96C25 for ; Wed, 13 Mar 2013 12:27:58 +0100 (CET) From: Mark Martinec To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Date: Wed, 13 Mar 2013 12:27:57 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.1-STABLE; KDE/4.9.5; amd64; ; ) References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> In-Reply-To: <20130313091727.GA17859@defiant.konundrum.org> Organization: J. Stefan Institute MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201303131227.57751.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 11:36:27 -0000 On Wednesday March 13 2013 10:17:27 Schrodinger wrote: > ifconfig_re0_ipv6="inet6 2001:41D0:2:E7c4::1 prefixlen 64" > [...] > Voodoo, indeed... I'm sure there's a /48 used somewhere but to be more > specific, or rather obvious, my default gateway resides at the boundary > of a /56 - 2001:41D0:2:E700::/56 Having multiple IPv6 subnets on the same wire is asking for trouble. For example, I believe an ICMP redirect still (in 9.1) does not create a temporary route: http://www.freebsd.org/cgi/query-pr.cgi?pr=152791 which beat us hard time (random unreachability between hosts), having to rearrange that legacy segment which happened to have two subnets on the same wire. The static routes destinations must be directly reachable (on-link). Either use a single /56 for the whole LAN, adjusting the prefix length on each interface, or provide a router within each subnet. Doing otherwise (like using static ndp/arp entries) is just a hack. Mark From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 12:52:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 85F8E350 for ; Wed, 13 Mar 2013 12:52:28 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id 218CBFA2 for ; Wed, 13 Mar 2013 12:52:27 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id C385E1CD7F4 for ; Wed, 13 Mar 2013 12:52:26 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hqrZ2qMsvAlq for ; Wed, 13 Mar 2013 12:52:26 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 51DFE1CD7F3 for ; Wed, 13 Mar 2013 12:52:26 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DCqNrk018892 for ; Wed, 13 Mar 2013 12:52:23 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DCqMHm018891 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 12:52:22 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 12:52:21 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313125221.GD17859@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="EY/WZ/HvNxOox07X" Content-Disposition: inline In-Reply-To: <201303131227.57751.Mark.Martinec+freebsd@ijs.si> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 12:52:28 -0000 --EY/WZ/HvNxOox07X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 12:27, Mark Martinec wrote: Hi Mark, > On Wednesday March 13 2013 10:17:27 Schrodinger wrote: > > ifconfig_re0_ipv6=3D"inet6 2001:41D0:2:E7c4::1 prefixlen 64" > > [...] > > Voodoo, indeed... I'm sure there's a /48 used somewhere but to be more > > specific, or rather obvious, my default gateway resides at the boundary > > of a /56 - 2001:41D0:2:E700::/56 >=20 > Having multiple IPv6 subnets on the same wire is asking for trouble. >=20 This isn't my network so I don't have any input into the matter. This is the OVH configuration for their dedicated servers, at least in my product range. > For example, I believe an ICMP redirect still (in 9.1) does not create > a temporary route: > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D152791 > which beat us hard time (random unreachability between hosts), > having to rearrange that legacy segment which happened to have > two subnets on the same wire. >=20 > The static routes destinations must be directly reachable (on-link). >=20 Does adding the interface route not put the default gateway on-link though ? > Either use a single /56 for the whole LAN, adjusting the prefix > length on each interface, or provide a router within each subnet. >=20 If I am to change my prefix length to /56 this means that anyone else in that /56 who is configured with a prefix length of 64 will be routing to me and I will be swicthing to them.... This could cause problems. > Doing otherwise (like using static ndp/arp entries) is just a hack. >=20 Agreed, I had no intention of fixing the problem this way. C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --EY/WZ/HvNxOox07X Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQHaEAAoJEBBi7cjNKnTjCPcP/3lK8deK5uDWtc4ruDKs8k2e Jaxk1IibNiU4WjuwxGqkoBQ0NpShLw6WdGHsnzXnicCigEmokunGZZnfYHzWt7wZ BGRhZgw2aSyRZi2mjzhiFq+7ZVGn928x+/h1F+csYJ/ynkVjpLK4BLQiPwJWQCZa LeQfUtw4pnW479fzZq0WVAuPGNVxntqPvgyVtA3FHN+PQmrlxQI9sJkKROopIvGq Kqf2+8R1KV64ZtffVRMGKFqGVPdR/otPjarBa1lstFk9lVeGxKDY+y7GKHfva8d8 JQMRCgyb86T08LpTRXSO2UfsDWd4VfKbZx+Un+QDvLmP1FHg9ZX7zAaK+lqNvAly Cw/tSLQcd5103OOCbvKsX1VUz6wmqQd+bTmMqwgTg/hM2durNvC4g/103E6SLs7q qTMxrlMfQS3Reimp3ZvlSMXsV+6EhSD8h05hohx9TE1Zm7wIMV1urFQYkra/a0JD GDyZfaR511fq7DPYLzMTmX6UkUa1YbUDt7Fr7Q71cXLeo0QSB7TNpGmJR1jdopN5 IMtBt5bDBb4NeUSQXbVE7BGPXUEJJMSwGUyEGeqOUIO73hTyCgcvaZLnTNtTg/0X ma/gzVMm6AMwwKOODyIY3Xww4ddCp3WeSE84RfgZRltdVwUVfvDOp5RuXxw4frAm sNrV3fSlQQmSZ4g6rdYg =gec9 -----END PGP SIGNATURE----- --EY/WZ/HvNxOox07X-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 13:02:30 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7641B945 for ; Wed, 13 Mar 2013 13:02:30 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) by mx1.freebsd.org (Postfix) with ESMTP id 0E54F10B for ; Wed, 13 Mar 2013 13:02:29 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hi8so734862wib.13 for ; Wed, 13 Mar 2013 06:02:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=KPpG3FGeluLrhOrvvjxJEfnQyQUJgjD5xDUXVdNGG54=; b=TgBKdqSEyNg0dvjmI8MpiBvwXTzEyXO/IC8PLLvyvEYH49/HLo971a1UVBrYNKo1ho pt8WDtdZJBlXM1e0nr+RtZSCQnG6BFGrrJxsi3jtQXJNfjh8hwhM2Alw0xBfwlUa6j6s qCWweL74KcRnsu0reFi9mLm6/JBHgO5TS9vCgTPaP2VKD5F6zHAw2922SevYH2IuvsVW 9d3BLC8Sb1YOv61+pUuu5aopKI6d8ms2MEX71z0CaLQwY+MLJM3bgmlz6pT2fqWxi42Z aKvdJW2IiyfhXHRORM9jNqS3VwlZdYDo6A+ucDLfjmloPKe9P1DJnt1pscGdRsO44yC6 TDuQ== X-Received: by 10.195.12.133 with SMTP id eq5mr33723078wjd.52.1363179749158; Wed, 13 Mar 2013 06:02:29 -0700 (PDT) Received: from [10.75.0.66] ([83.167.62.196]) by mx.google.com with ESMTPS id ej8sm2607751wib.9.2013.03.13.06.02.27 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 13 Mar 2013 06:02:27 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ipv6 default router Operation not permitted From: Fleuriot Damien In-Reply-To: <20130313125221.GD17859@defiant.konundrum.org> Date: Wed, 13 Mar 2013 14:02:26 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> To: Schrodinger X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQlhA9YORG4Fyl39PwHwQ+Ylw3lhILINK1N7E000duWEuqA/DGAGPuuPNvVJ6JCUe1MUeNRL Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 13:02:30 -0000 On Mar 13, 2013, at 1:52 PM, Schrodinger = wrote: > On 2013/03/13 12:27, Mark Martinec wrote: >=20 > Hi Mark, >=20 >> On Wednesday March 13 2013 10:17:27 Schrodinger wrote: >>> ifconfig_re0_ipv6=3D"inet6 2001:41D0:2:E7c4::1 prefixlen 64" >>> [...] >>> Voodoo, indeed... I'm sure there's a /48 used somewhere but to be = more >>> specific, or rather obvious, my default gateway resides at the = boundary >>> of a /56 - 2001:41D0:2:E700::/56 >>=20 >> Having multiple IPv6 subnets on the same wire is asking for trouble. >>=20 >=20 > This isn't my network so I don't have any input into the matter. This > is the OVH configuration for their dedicated servers, at least in my > product range. >=20 >> For example, I believe an ICMP redirect still (in 9.1) does not = create >> a temporary route: >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D152791 >> which beat us hard time (random unreachability between hosts), >> having to rearrange that legacy segment which happened to have >> two subnets on the same wire. >>=20 >> The static routes destinations must be directly reachable (on-link). >>=20 >=20 > Does adding the interface route not put the default gateway on-link > though ? >=20 >> Either use a single /56 for the whole LAN, adjusting the prefix >> length on each interface, or provide a router within each subnet. >>=20 >=20 > If I am to change my prefix length to /56 this means that anyone else = in > that /56 who is configured with a prefix length of 64 will be routing = to > me and I will be swicthing to them.... This could cause problems. I fail to see how they would be routing to you and you would be = switching to them. OVH allocates a /64 per customer. To avoid having to setup 1 gateway per customer, they set up a single = one within a /56 , allowing for 256 /64s This mimics the situation where your host gives you a /32 ipv4 withing a = /24 network and uses a single gateway, again for 250ish customers. Whenever an IPv6 packet arrives on OVH's router for your /64, it is = routed to your server. I don't see how this qualifies as "another customer routing to you" ? From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 13:10:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 684D1BBA for ; Wed, 13 Mar 2013 13:10:21 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [IPv6:2001:41d0:1:c74c::1]) by mx1.freebsd.org (Postfix) with ESMTP id C9AD918A for ; Wed, 13 Mar 2013 13:10:20 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id 347A11CD7F4 for ; Wed, 13 Mar 2013 13:10:19 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Be5J22jIs5mB for ; Wed, 13 Mar 2013 13:10:18 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 946931CD7F3 for ; Wed, 13 Mar 2013 13:10:18 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DDAH50018989 for ; Wed, 13 Mar 2013 13:10:17 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DDAHwU018988 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 13:10:17 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 13:10:17 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313131016.GE17859@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="zjcmjzIkjQU2rmur" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 13:10:21 -0000 --zjcmjzIkjQU2rmur Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 14:02, Fleuriot Damien wrote: >=20 > On Mar 13, 2013, at 1:52 PM, Schrodinger wrot= e: >=20 > > On 2013/03/13 12:27, Mark Martinec wrote: > >=20 > > Hi Mark, > >=20 > >> On Wednesday March 13 2013 10:17:27 Schrodinger wrote: > >>> ifconfig_re0_ipv6=3D"inet6 2001:41D0:2:E7c4::1 prefixlen 64" > >>> [...] > >>> Voodoo, indeed... I'm sure there's a /48 used somewhere but to be more > >>> specific, or rather obvious, my default gateway resides at the bounda= ry > >>> of a /56 - 2001:41D0:2:E700::/56 > >>=20 > >> Having multiple IPv6 subnets on the same wire is asking for trouble. > >>=20 > >=20 > > This isn't my network so I don't have any input into the matter. This > > is the OVH configuration for their dedicated servers, at least in my > > product range. > >=20 > >> For example, I believe an ICMP redirect still (in 9.1) does not create > >> a temporary route: > >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D152791 > >> which beat us hard time (random unreachability between hosts), > >> having to rearrange that legacy segment which happened to have > >> two subnets on the same wire. > >>=20 > >> The static routes destinations must be directly reachable (on-link). > >>=20 > >=20 > > Does adding the interface route not put the default gateway on-link > > though ? > >=20 > >> Either use a single /56 for the whole LAN, adjusting the prefix > >> length on each interface, or provide a router within each subnet. > >>=20 > >=20 > > If I am to change my prefix length to /56 this means that anyone else in > > that /56 who is configured with a prefix length of 64 will be routing to > > me and I will be swicthing to them.... This could cause problems. >=20 >=20 > I fail to see how they would be routing to you and you would be switching= to them. >=20 >=20 > OVH allocates a /64 per customer. > To avoid having to setup 1 gateway per customer, they set up a single one= within a /56 , allowing for 256 /64s > This mimics the situation where your host gives you a /32 ipv4 withing a = /24 network and uses a single gateway, again for 250ish customers. >=20 > Whenever an IPv6 packet arrives on OVH's router for your /64, it is route= d to your server. > I don't see how this qualifies as "another customer routing to you" ? >=20 I am informed that I must configure my interface to /64 by OVH. The same as everyone else. So if everyone was on a /64 then we will send packets to each other via our shared default gateway. E.g.: I am 2001:41d0:2:e7c4::1/64 My default gateway is 2001:41d0:2:e7ff:ff:ff:ff:ff If I wanted to communicate with a host in 2001:41d0:2:e7c5::/64 and his default gateway is also 2001:41d0:2:e7ff:ff:ff:ff:ff then we will route packets to each other. Correct? If I were to change my interface prefix length to /56 my host would no longer consider the need to send packets to the default gateway for any host within this /56. I would simply perform Neighbour Solicitation on my link. E.g.: I am 2001:41d0:2:e7c4::1/56 My default gateway is 2001:41d0:2:e7ff:ff:ff:ff:ff If I wanted to communicate with a host in 2001:41d0:2:e7c5::/64 and his default gateway is also 2001:41d0:2:e7ff:ff:ff:ff:ff then I would switch to him because the /56 is "on-link" to me but to the recipient he must route to me via his default gateway. Correct? C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --zjcmjzIkjQU2rmur Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQHq3AAoJEBBi7cjNKnTj9acP/0xpWygTCVqAi5YIrSX/kidR doQyF9GfQyjFzuhtIPKEQiP38hNlA6pHr5k5ZGCsc7smou+CNWBeAplR32nHt2mm NBu+yJbGpE3owpFnt+VDaix0LVdk4YYrohaaGZ/qpsjuPbj+hVW25PKLGmwDBsHp R54lXeKMFIbzziop/LT1amZ2NVUhADhvO2mZJovVXa9NpCNOAczDCa6qX938zmTg C9BxeBLoRrxksdy0ex66hGIz5QCyj8oLu1zX21pl2Kzs9cGSb45RmG87s4mOxmdv ad3WOTpjqDv3byYoyNsteoQzjVpV3BH/o3gtYkSzzQgWv/FQD05EadOZo8mY95Zu tLb43U6kpvd18we/+YmxErDUG2l87OiCBfBz9IXStyNKOJ5WuXXkOOw1pLUPTui6 vXKGexNhQigUg6tJuub+kiQAGk8Tfi1OOjSBgsrq2EGJ/X8C/Nbz7XhNP82OgfJ0 kmWWv4keyH6xhy3ZLTiBb+J4wubfTpttlxo4W+U3Ou0dimknqjLe0TfiLNYirmxy SB8JfSWKpKlEuEZWBJ3GC3qAWHrLrqA2dUQfCiQP1SCr5C6JY4/PJSg/cP3M6IzC 4VdHGaW3u86zvEf4Xpq83IcTGWxlkANIjb05jjJr7iMitqYAmZdj8Akgfwm1DbLW XK6QeNThglbqNcefGOxi =+JyH -----END PGP SIGNATURE----- --zjcmjzIkjQU2rmur-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 13:19:29 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D5930EB6 for ; Wed, 13 Mar 2013 13:19:29 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) by mx1.freebsd.org (Postfix) with ESMTP id 64A1C23B for ; Wed, 13 Mar 2013 13:19:29 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id hm6so752107wib.14 for ; Wed, 13 Mar 2013 06:19:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=msuD12+L+tt7OfzLbKihTyF8D9DLn8Upv1czu/egfaU=; b=Uk42JKZ9X9Hl23Xy1EU7jIOEXKfVQaxqqzXTG/kwzcql+pJTRXPENXqPmbvcnqurh5 fBmKtrcaMtMyMoTT+tOEuFIkT5AWCrL9TH0CVZkt+fsk9DQV0MlO6nlrvSZx+jLBmaRG 9z65rYYLNR/X/8VDZpic2PRTX5g9YXX2+OCr8VTOw9I510a4YZshwSdMWeD4s6ihCxo+ otH+wwtsnelnKDsJedpm1mzJEsZNM/oDLKc3cfWUrhJkXpju/9zbb2nb1Ylwn1Pgee50 WY5792L0y/GLphO0Kw5eumE78SzWwRw/PdEPb43QPBpco50NysRz+RdHa5xoIk5jF0rq yMzw== X-Received: by 10.180.109.82 with SMTP id hq18mr9719005wib.0.1363180768366; Wed, 13 Mar 2013 06:19:28 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id n2sm2703799wiy.6.2013.03.13.06.19.26 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 13 Mar 2013 06:19:27 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ipv6 default router Operation not permitted From: Fleuriot Damien In-Reply-To: <20130313131016.GE17859@defiant.konundrum.org> Date: Wed, 13 Mar 2013 14:19:25 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> <20130313131016.GE17859@defiant.konundrum.org> To: Schrodinger X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQn22wQTyoAa6CoG7IpcclQOaEWuSivYcXjrfphDYlbGh5tcaMjoLU+9x+cWHOGRqQuHANhO Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 13:19:29 -0000 On Mar 13, 2013, at 2:10 PM, Schrodinger = wrote: > On 2013/03/13 14:02, Fleuriot Damien wrote: >>=20 >> On Mar 13, 2013, at 1:52 PM, Schrodinger = wrote: >>=20 >>> On 2013/03/13 12:27, Mark Martinec wrote: >>>=20 >>> Hi Mark, >>>=20 >>>> On Wednesday March 13 2013 10:17:27 Schrodinger wrote: >>>>> ifconfig_re0_ipv6=3D"inet6 2001:41D0:2:E7c4::1 prefixlen 64" >>>>> [...] >>>>> Voodoo, indeed... I'm sure there's a /48 used somewhere but to be = more >>>>> specific, or rather obvious, my default gateway resides at the = boundary >>>>> of a /56 - 2001:41D0:2:E700::/56 >>>>=20 >>>> Having multiple IPv6 subnets on the same wire is asking for = trouble. >>>>=20 >>>=20 >>> This isn't my network so I don't have any input into the matter. = This >>> is the OVH configuration for their dedicated servers, at least in my >>> product range. >>>=20 >>>> For example, I believe an ICMP redirect still (in 9.1) does not = create >>>> a temporary route: >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D152791 >>>> which beat us hard time (random unreachability between hosts), >>>> having to rearrange that legacy segment which happened to have >>>> two subnets on the same wire. >>>>=20 >>>> The static routes destinations must be directly reachable = (on-link). >>>>=20 >>>=20 >>> Does adding the interface route not put the default gateway on-link >>> though ? >>>=20 >>>> Either use a single /56 for the whole LAN, adjusting the prefix >>>> length on each interface, or provide a router within each subnet. >>>>=20 >>>=20 >>> If I am to change my prefix length to /56 this means that anyone = else in >>> that /56 who is configured with a prefix length of 64 will be = routing to >>> me and I will be swicthing to them.... This could cause problems. >>=20 >>=20 >> I fail to see how they would be routing to you and you would be = switching to them. >>=20 >>=20 >> OVH allocates a /64 per customer. >> To avoid having to setup 1 gateway per customer, they set up a single = one within a /56 , allowing for 256 /64s >> This mimics the situation where your host gives you a /32 ipv4 = withing a /24 network and uses a single gateway, again for 250ish = customers. >>=20 >> Whenever an IPv6 packet arrives on OVH's router for your /64, it is = routed to your server. >> I don't see how this qualifies as "another customer routing to you" ? >>=20 >=20 > I am informed that I must configure my interface to /64 by OVH. The = same > as everyone else. So if everyone was on a /64 then we will send = packets > to each other via our shared default gateway. >=20 > E.g.: >=20 > I am 2001:41d0:2:e7c4::1/64 My default gateway is > 2001:41d0:2:e7ff:ff:ff:ff:ff >=20 > If I wanted to communicate with a host in 2001:41d0:2:e7c5::/64 and = his > default gateway is also 2001:41d0:2:e7ff:ff:ff:ff:ff then we will = route > packets to each other. >=20 > Correct? >=20 > If I were to change my interface prefix length to /56 my host would no > longer consider the need to send packets to the default gateway for = any > host within this /56. I would simply perform Neighbour Solicitation on > my link. >=20 > E.g.: >=20 > I am 2001:41d0:2:e7c4::1/56 My default gateway is > 2001:41d0:2:e7ff:ff:ff:ff:ff >=20 > If I wanted to communicate with a host in 2001:41d0:2:e7c5::/64 and = his > default gateway is also 2001:41d0:2:e7ff:ff:ff:ff:ff then I would = switch > to him because the /56 is "on-link" to me but to the recipient he must > route to me via his default gateway. >=20 > Correct? >=20 > C. > --=20 These are indeed correct, thanks for clarifying. Find below the config I'm using on an old OVH box. Said config might be outdated now (as per OVH's guide on setting up IPv6 = [1]) , however that was at the time the only way to get things working = properly. rc.conf =3D=3D=3D #Range IPv6: 2001:41D0:2:613b::/64 ipv6_enable=3D"YES" ipv6_ifconfig_re0=3D"fe80::21c:c0ff:fef3:31fa/64 scopeid 0x1" ipv6_ifconfig_re0_alias0=3D"2001:41d0:2:613b::dead:beef/56" ipv6_defaultrouter=3D"2001:41d0:2:61ff:ff:ff:ff:ff" =3D=3D=3D routing table =3D=3D=3D $ netstat -f inet6 -rn Routing tables Internet6: Destination Gateway Flags = Netif Expire ::/96 ::1 UGRS = lo0 =3D> default 2001:41d0:2:61ff:ff:ff:ff:ff UGS = re0 ::1 ::1 UH = lo0 ::ffff:0.0.0.0/96 ::1 UGRS = lo0 2001:41d0:2:6100::/56 link#1 U = re0 2001:41d0:2:613b::dead:beef link#1 UHS = lo0 fe80::/10 ::1 UGRS = lo0 fe80::%re0/64 link#1 U = re0 fe80::21c:c0ff:fef3:31fa%re0 link#1 UHS = lo0 fe80::%lo0/64 link#2 U = lo0 fe80::1%lo0 link#2 UHS = lo0 ff01:1::/32 fe80::21c:c0ff:fef3:31fa%re0 U = re0 ff01:2::/32 ::1 U = lo0 ff02::/16 ::1 UGRS = lo0 ff02::%re0/32 fe80::21c:c0ff:fef3:31fa%re0 U = re0 ff02::%lo0/32 ::1 U = lo0 =3D=3D=3D Notice that said config actually works: =3D=3D=3D $ ping6 www.google.com PING6(56=3D40+8+8 bytes) 2001:41d0:2:613b::dead:beef --> = 2a00:1450:4007:804::1014 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D0 hlim=3D57 = time=3D4.461 ms 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D1 hlim=3D57 = time=3D4.462 ms 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D2 hlim=3D57 = time=3D4.405 ms ^C --- www.google.com ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev =3D 4.405/4.443/4.462/0.027 ms =3D=3D=3D Either way, you might want to have a look at OVH's guide [1] but in my = own case, using a /56 was, at the time, the only way to get things = working in a clean way. [1] http://help.ovh.com/Ipv4Ipv6#link10 From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 13:38:18 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A3DF855A for ; Wed, 13 Mar 2013 13:38:18 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 3F90C39C for ; Wed, 13 Mar 2013 13:38:18 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id u54so1007514wey.16 for ; Wed, 13 Mar 2013 06:38:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=CEwJGXZbrnIV+8vmR6ObVc8fi2sk2FxukgeLGSPciCM=; b=M5a8YkMRtylV4L4jwmI6tmXHpuQU45O/67SocvxdOw2aj/nLBCEYV9CWv182wEYH17 lDIQwOPakRQQeEBkDAzDcv4o6ZliXVy7hMBXtQA/M7e2iAO6ZUCPcC3Yiu/O0ktLiF5Q oabpIqnGBcw4ld2Kp43iQy2BEsimfBDB3blY/Hz0gx4OOmA4M29pyhTgR7v2OYuA7Gw+ saWMpKurP5T7gyXtE7ZAFEGbBci3fWQZ7VOL22YOQduQK9kUHSQrDTwI5D+m2/KgYZEg 7UgW56xvlsUK7VSfcTaZ/JLU38hOhP2qQYRGTkxEdvGTsKuUSJ7ksYx/PAqhBdz0SyWI gFuQ== X-Received: by 10.180.104.225 with SMTP id gh1mr26749657wib.27.1363181897122; Wed, 13 Mar 2013 06:38:17 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id j4sm2800228wiz.10.2013.03.13.06.38.14 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 13 Mar 2013 06:38:15 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ntpd bind() failure: Can't assign requested address From: Fleuriot Damien In-Reply-To: <76E2EE0D-B6D3-428E-9112-AF0E04E98DDE@my.gd> Date: Wed, 13 Mar 2013 14:38:12 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <76E2EE0D-B6D3-428E-9112-AF0E04E98DDE@my.gd> To: "M. Schulte" X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQktQXxlf50UCZ/4u0lLr/VzbI4/ZCntMKvIQaU2o3moiUlVlEWQ2X3SoZa6wcxNkOZsill8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 13:38:18 -0000 On Mar 12, 2013, at 11:44 PM, Damien Fleuriot wrote: >=20 > On 12 Mar 2013, at 22:42, "M. Schulte" wrote: >=20 >> Hi! >>=20 >> [First of all, I have posted this question already on the FreeBSD >> forum -- so far without replies -- and now my hope is that the set of >> subscribers here and those of the forum do not completely coincide.] >>=20 >> I have installed FreeBSD 9.1 on my server (it's a virtualized = qemu/kvm >> environment, in case that matters) and during boot the following >> messages appear: >>=20 >> ntpd[766]: ntpd 4.2.4p5-a (1) >> ntpd[767]: bind() fd 23, family AF_INET6, port 123, scope 2, addr = fe80::216:36ff:fe74:2076, mcast=3D0 flags=3D0x11 fails: Can't assign = requested address >> ntpd[767]: unable to create socket on re0 (3) for = fe80::216:36ff:fe74:2076#123 >>=20 >> This happens with a GENERIC kernel. I have not touched any IPv6 >> related configuration after install, so everything should be in the >> default state. Neither have I changed the ntp/ntpd configuration. >>=20 >> Although I could find some threads where people were discussing the >> same problem, none of the mentioned suggestions fixed this for me. >>=20 >> In particular I tried setting >>=20 >> ipv6_activate_all_interfaces=3D"YES" >>=20 >> in rc.conf -- didn't fix the problem. >>=20 >> According to ifconfig, my interfaces, re0 in particular, have IPv6 >> addresses associated to them. So why would it fail that a process >> tries to bind to it? Note that I'm rather unexperienced wrt IPv6. >>=20 >> Would be great if somebody could give me a hint into the right >> direction. Anybody else experiencing this? Thank you very much! >>=20 >> ~ melanie >>=20 >=20 > I'll check tomorrow if we have the same issue at work, although it = doesn't look familiar. >=20 > We've got a 9.1 with a custom kernel (although nothing related to ipv6 = changed) running in KVM. I confirm I have the same issue on 9.1 r247912 , as below: Clearing /tmp (X related). Updating motd:. Starting ntpd. Mar 8 14:57:23 pf1-drt ntpd[938]: bind() fd 23, family AF_INET6, port = 123, scope 2, addr fe80::90bd:54ff:fe55:2d21, mcast=3D0 flags=3D0x11 = fails: Can't assign requested address Mar 8 14:57:23 pf1-drt ntpd[938]: unable to create socket on vlan12 (3) = for fe80::90bd:54ff:fe55:2d21#123 However, ntpd runs fine and is bound to the following addresses: root@pf1-drt:/var/run # sockstat | grep ntp root ntpd 938 3 dgram -> /var/run/logpriv root ntpd 938 20 udp4 *:123 *:* root ntpd 938 21 udp6 *:123 *:* root ntpd 938 22 udp4 195.158.240.37:123 *:* root ntpd 938 23 udp4 192.168.32.253:123 *:* root ntpd 938 24 udp6 fe80:3::a8b0:eeff:fef9:dbb:123 *:* root ntpd 938 25 udp4 192.168.22.254:123 *:* root ntpd 938 26 udp6 fe80:4::6820:e3ff:fe0b:76dc:123 *:* root ntpd 938 27 udp6 ::1:123 *:* root ntpd 938 28 udp6 fe80:7::1:123 *:* root ntpd 938 29 udp4 127.0.0.1:123 *:* root ntpd 938 31 udp6 fe80:2::90bd:54ff:fe55:2d21:123 *:* root ntpd 938 32 udp4 192.168.22.253:123 *:* From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 13:52:57 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EC96680 for ; Wed, 13 Mar 2013 13:52:57 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [IPv6:2001:41d0:1:c74c::1]) by mx1.freebsd.org (Postfix) with ESMTP id 9948F6F8 for ; Wed, 13 Mar 2013 13:52:57 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id BB81D1CD7F4 for ; Wed, 13 Mar 2013 13:52:56 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AQz4suJw4ZC2 for ; Wed, 13 Mar 2013 13:52:56 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 06F8E1CD7F3 for ; Wed, 13 Mar 2013 13:52:55 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DDqtqL019163 for ; Wed, 13 Mar 2013 13:52:55 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DDqsYc019162 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 13:52:54 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 13:52:54 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313135253.GA18992@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> <20130313131016.GE17859@defiant.konundrum.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="7JfCtLOvnd9MIVvH" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 13:52:58 -0000 --7JfCtLOvnd9MIVvH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Damien,=20 I appreciate your replies very much, but I'm a subscriber so just reply to the mailing list. Thanks. On 2013/03/13 14:19, Fleuriot Damien wrote: [SNARF] >=20 >=20 > These are indeed correct, thanks for clarifying. >=20 I thought that's what I said in my first email ;) Sorry for any confusion. >=20 > Find below the config I'm using on an old OVH box. > Said config might be outdated now (as per OVH's guide on setting up IPv6 = [1]) , however that was at the time the only way to get things working prop= erly. >=20 > rc.conf > =3D=3D=3D > #Range IPv6: 2001:41D0:2:613b::/64 > ipv6_enable=3D"YES" > ipv6_ifconfig_re0=3D"fe80::21c:c0ff:fef3:31fa/64 scopeid 0x1" > ipv6_ifconfig_re0_alias0=3D"2001:41d0:2:613b::dead:beef/56" > ipv6_defaultrouter=3D"2001:41d0:2:61ff:ff:ff:ff:ff" > =3D=3D=3D >=20 You have /56 and this is what I believe to be the incorrect way to get this to Just Work. I think this assume that anyone else in this /56 is in the same layer two segment as you....=20 >=20 > routing table > =3D=3D=3D [SNARF] > =3D=3D=3D >=20 >=20 >=20 > Notice that said config actually works: > =3D=3D=3D > $ ping6 www.google.com > PING6(56=3D40+8+8 bytes) 2001:41d0:2:613b::dead:beef --> 2a00:1450:4007:8= 04::1014 > 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D0 hlim=3D57 time=3D4.4= 61 ms > 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D1 hlim=3D57 time=3D4.4= 62 ms > 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D2 hlim=3D57 time=3D4.4= 05 ms > ^C > --- www.google.com ping6 statistics --- > 3 packets transmitted, 3 packets received, 0.0% packet loss > round-trip min/avg/max/std-dev =3D 4.405/4.443/4.462/0.027 ms > =3D=3D=3D >=20 > Either way, you might want to have a look at OVH's guide [1] but in my ow= n case, using a /56 was, at the time, the only way to get things working in= a clean way. >=20 > [1] http://help.ovh.com/Ipv4Ipv6#link10 >=20 I read this, I made sure to read this and then I read it a second time. No where does it indicate the use of a /56. I am in the process of a migration from an old OVH server to a new OVH server. My old box uses the /56 prefix length "fix" but based on the documentation this is incorrect and IMO this assumes that anyone else in the /56 is in the=20 same segment as me and if they are using /64 - well, There Be Dragons. Also from the information I have received, router advertisements may be turned off in the future, my host should simply Neighbour Solicit for the global scope unicast address of my default gateway. And as pointed out in previous emails without ACCEPT_RTADV for re0 - FreeBSD does not perform this action. So again, what is the correct way ? I think this is a debate of IPv6 Protocol vs. IPv6 Policy vs. Network architecture. I'll go and get Tina Turner. You get Masterblaster and we'll meet in Thunderdome. C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --7JfCtLOvnd9MIVvH Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQIS0AAoJEBBi7cjNKnTj/EIP/1qF1J+UxUpbVEwv4kzA7hcR 6JNL2wR+k9XrB7LhtgtO+PXknhKS2gY5R0T4VzM1+rGAVhi7K6eVcqmV5zWnIntZ 49Zs2t8g6bU2hpZ8gwIDeG+5f7EJhwOs0IKjOSZjuENRzWXk8QYP4sI96DUtsUHA vn3Du/NDehGH+Y/Bb0QGGdqssEoeLWpE7qXLgytonpM6E3L9PV2XqbQzgFcC2iSv jc7bWO8KwehctUPqMaJoHp8RKkMKA+TNCkXpPLVU7UthFoLt/iSVgE83y+G6o/YA jHVVeUG1mTnz/aNHcquJvXJMpzXd9PA51+m+gz+mkprcFxr7gGeaQSJls6KHwXO5 t1GrkQSzNfTg6wu0XbrzEJ5G6j6LiiE1JhXtBBI6+p6cMDQ/dedpanDGUV8fIoug NY2N4LqjAFSj/KOZUuSUohv3quUf98W3IWikgvRJ9dVT+JbtyO9ByFQXxin3VGYc AZ35/TQUd6ssnv/Yj/Jot5fRyMIiNSwFEnFjucvEJI0CeRlydZgWQ9tyhRWUt1AN wMuI0oV3a1DT1faJVsbB8z7dKn5/CqRADQoYJld86YuZaApa1dCc8sib0ImVl2oG iDwfOvX19sdLpc1MtR8sABd8PgEReYcTkpYYlmF/u3C2ZadvfwkiRYH8plrlOjNw d2V3Seo9Kno07WbqgDpU =CN0T -----END PGP SIGNATURE----- --7JfCtLOvnd9MIVvH-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 14:17:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 99CCBE07 for ; Wed, 13 Mar 2013 14:17:21 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id 12C1391A for ; Wed, 13 Mar 2013 14:17:19 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id 1386A1CD7F4 for ; Wed, 13 Mar 2013 14:17:19 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NqUZFlNq8dWY for ; Wed, 13 Mar 2013 14:17:18 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 9BA341CD7F3 for ; Wed, 13 Mar 2013 14:17:18 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DEHH7A019290 for ; Wed, 13 Mar 2013 14:17:17 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DEHHGA019289 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 14:17:17 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 14:17:16 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ntpd bind() failure: Can't assign requested address Message-ID: <20130313141716.GB18992@defiant.konundrum.org> References: <76E2EE0D-B6D3-428E-9112-AF0E04E98DDE@my.gd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="QKdGvSO+nmPlgiQ/" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 14:17:21 -0000 --QKdGvSO+nmPlgiQ/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 14:38, Fleuriot Damien wrote: >=20 > On Mar 12, 2013, at 11:44 PM, Damien Fleuriot wrote: >=20 > >=20 > > On 12 Mar 2013, at 22:42, "M. Schulte" wrote: > >=20 > >> Hi! > >>=20 > >> [First of all, I have posted this question already on the FreeBSD > >> forum -- so far without replies -- and now my hope is that the set of > >> subscribers here and those of the forum do not completely coincide.] > >>=20 > >> I have installed FreeBSD 9.1 on my server (it's a virtualized qemu/kvm > >> environment, in case that matters) and during boot the following > >> messages appear: > >>=20 > >> ntpd[766]: ntpd 4.2.4p5-a (1) > >> ntpd[767]: bind() fd 23, family AF_INET6, port 123, scope 2, addr fe8= 0::216:36ff:fe74:2076, mcast=3D0 flags=3D0x11 fails: Can't assign requested= address > >> ntpd[767]: unable to create socket on re0 (3) for fe80::216:36ff:fe74= :2076#123 > >>=20 > >> This happens with a GENERIC kernel. I have not touched any IPv6 > >> related configuration after install, so everything should be in the > >> default state. Neither have I changed the ntp/ntpd configuration. > >>=20 > >> Although I could find some threads where people were discussing the > >> same problem, none of the mentioned suggestions fixed this for me. > >>=20 > >> In particular I tried setting > >>=20 > >> ipv6_activate_all_interfaces=3D"YES" > >>=20 > >> in rc.conf -- didn't fix the problem. > >>=20 > >> According to ifconfig, my interfaces, re0 in particular, have IPv6 > >> addresses associated to them. So why would it fail that a process > >> tries to bind to it? Note that I'm rather unexperienced wrt IPv6. > >>=20 > >> Would be great if somebody could give me a hint into the right > >> direction. Anybody else experiencing this? Thank you very much! > >>=20 > >> ~ melanie > >>=20 > >=20 > > I'll check tomorrow if we have the same issue at work, although it does= n't look familiar. > >=20 > > We've got a 9.1 with a custom kernel (although nothing related to ipv6 = changed) running in KVM. >=20 >=20 > I confirm I have the same issue on 9.1 r247912 , as below: >=20 > Clearing /tmp (X related). > Updating motd:. > Starting ntpd. > Mar 8 14:57:23 pf1-drt ntpd[938]: bind() fd 23, family AF_INET6, port 12= 3, scope 2, addr fe80::90bd:54ff:fe55:2d21, mcast=3D0 flags=3D0x11 fails: C= an't assign requested address > Mar 8 14:57:23 pf1-drt ntpd[938]: unable to create socket on vlan12 (3) = for fe80::90bd:54ff:fe55:2d21#123 >=20 >=20 >=20 > However, ntpd runs fine and is bound to the following addresses: >=20 > root@pf1-drt:/var/run # sockstat | grep ntp > root ntpd 938 3 dgram -> /var/run/logpriv > root ntpd 938 20 udp4 *:123 *:* > root ntpd 938 21 udp6 *:123 *:* > root ntpd 938 22 udp4 195.158.240.37:123 *:* > root ntpd 938 23 udp4 192.168.32.253:123 *:* > root ntpd 938 24 udp6 fe80:3::a8b0:eeff:fef9:dbb:123 *:* > root ntpd 938 25 udp4 192.168.22.254:123 *:* > root ntpd 938 26 udp6 fe80:4::6820:e3ff:fe0b:76dc:123 *:* > root ntpd 938 27 udp6 ::1:123 *:* > root ntpd 938 28 udp6 fe80:7::1:123 *:* > root ntpd 938 29 udp4 127.0.0.1:123 *:* > root ntpd 938 31 udp6 fe80:2::90bd:54ff:fe55:2d21:123 *:* > root ntpd 938 32 udp4 192.168.22.253:123 *:* >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" I have observed the same behaviour : Mar 13 14:10:27 howard ntpd[1603]: bind() fd 32, family AF_INET6, port 123,= scope 9, addr fe80::20a:faff:fe20:3a1, mcast=3D0 flags=3D0x13 fails: Can't= assign requested address Mar 13 14:10:27 howard ntpd[1603]: unable to create socket on gif6 (3994) f= or fe80::20a:faff:fe20:3a1#123 FreeBSD howard.tog.space 9.1-STABLE FreeBSD 9.1-STABLE #1: Mon Feb 25 18:37= :43 GMT 2013 root@howard.tog.space:/usr/obj/usr/src/sys/HOWARD amd64 # sockstat -lp123 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS = =20 root ntpd 1603 20 udp4 *:123 *:* root ntpd 1603 21 udp6 *:123 *:* root ntpd 1603 22 udp4 87.198.63.241:123 *:* root ntpd 1603 23 udp4 192.168.1.2:123 *:* root ntpd 1603 24 udp4 172.16.0.1:123 *:* root ntpd 1603 25 udp6 fe80:2::219:b9ff:fef8:9700:123 *:* root ntpd 1603 26 udp6 2001:770:100:81da::1:123 *:* root ntpd 1603 27 udp6 ::1:123 *:* root ntpd 1603 28 udp6 fe80:8::1:123 *:* root ntpd 1603 29 udp4 127.0.0.1:123 *:* root ntpd 1603 31 udp6 2001:770:100:1da::2:123 *:* gif6: flags=3D8051 metric 0 mtu 1280 tunnel inet XXX.XXX.XXX.XXX --> XXX.XXX.XXX.XXX inet6 fe80::20a:faff:fe20:3a1%gif6 prefixlen 64 tentative scopeid 0= x9=20 inet6 removed --> removed prefixlen 128=20 nd6 options=3D21 options=3D1 C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --QKdGvSO+nmPlgiQ/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQIprAAoJEBBi7cjNKnTjZegP/2ahlTW7Wxv/+HCW6bKUabzx oPAyxaO80ieDDsT0FAYJi2M8f0FSXeTsJePZjkpWBLmPO+aGhrExeJgRCF8kFw8h Y1pJW3c3bqCxF8kWLvziJe3FNTFGQ6XdMcXN1kJJ8TfHPDVOaNt5ZYzle6uTTpJM 0M9BPTqRFDfM1zplivapLE/9uU4qlEMVtDyZcV89unQ8ybkgq+WjjUx9VeJLUZkx NkAo7/7xwttzElqsOVC4ad1e6rqtyYQFEZP5uWnxG3yChn2W6Xf45TjS7lzQMyPQ e0umVYXpms4TyD/XmZg9R1NlouaKi7tmrJ8RD/E2BNuX92vCrH70m3yeK4/ZY+Ot CgjMKXXGCFFFCgoEGfvNzb4DUoG6wkGjei4lQoyKnVOUwpDtSNPnRLpCuOdWpDxF rqh9IwqBcvgW+FglOFg5M2kTKTYJ5bmN5I4yDRDzZAdgpp/HufVK3Y3B4gIwcAjZ YzmAIwVJeGO8pwmTD2bHZyQxGyXVDRyfDxebgpSo8E1BijzA+614qY44zeMLMop+ dt79tCXBLqCygYxp3+zXKAD8Ah2Wep4D6hOLgkBoe/UOj97hhDptB9JGS3Lat7RN VbcKiZueKuxx/Mi+Ia4csWiJIPb/fAsMoSK1f3aQVySRDL8JxaDDWxfBmhfFovLw 6IApEkYgUHwS7hThNhlv =tD7N -----END PGP SIGNATURE----- --QKdGvSO+nmPlgiQ/-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 15:22:03 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9852A359 for ; Wed, 13 Mar 2013 15:22:03 +0000 (UTC) (envelope-from mattmiller1@gmail.com) Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) by mx1.freebsd.org (Postfix) with ESMTP id 638A4E2A for ; Wed, 13 Mar 2013 15:22:03 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id o13so2288499qaj.19 for ; Wed, 13 Mar 2013 08:21:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:from:date:x-google-sender-auth :message-id:subject:to:content-type; bh=YHWL/eYZcWbSm1b0J/HUDArAE3XzVLXOUrNobR32tis=; b=jZ2yvGduif1w7aQtFiQ9IVb+hU5ZozzoJGsHwiNFyg/PIMvMNx9ln7HIfTifaO8y97 dPUCrDNHnzKsFipcGiBZbOK9wIgyp/vkC/yGkgvzkkmIEo4/HIQy/f5jR+vgMvZ8J6I5 77nyDy/h25sxybIxtHDGow4xY+FnazyPfaY6sMk/tf9BGJqmcTmvNkA8Nc04CVwx+2UH ETqTs+xs0LRrUmKoCXtp3x2WjOq0UmsQ2sIgRN89Sm/JuqncIFMHsSqYLmMHNUres9ul mm6n1q+ShQBdUzomyH21ZXpbYv5qZ7rm9bL+PNC6QC8vAxRaYR/Ys22gUDsVEEbPkWvm AjRw== X-Received: by 10.49.116.52 with SMTP id jt20mr9065010qeb.32.1363188116870; Wed, 13 Mar 2013 08:21:56 -0700 (PDT) MIME-Version: 1.0 Sender: mattmiller1@gmail.com Received: by 10.49.24.166 with HTTP; Wed, 13 Mar 2013 08:21:15 -0700 (PDT) From: Matt Miller Date: Wed, 13 Mar 2013 11:21:15 -0400 X-Google-Sender-Auth: w_5p5nKFHtIzM2WsqARnhCghKOc Message-ID: Subject: ip_output() Error Handling in tcp_output() To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 15:22:03 -0000 If we have a connection that has received a SYN and ip_output() returns, say, EHOSTUNREACH, is there anything that guarantees the connection would always eventually be dropped if the condition persists? E.g., similar to this case for ENOBUFS: http://svnweb.freebsd.org/base?view=revision&revision=61179 Should we ensure that a timer is set for the EHOSTUNREACH, et al. cases too in addition to setting t_softerror? error = ip_output(m, tp->t_inpcb->inp_options, &ro, ((so->so_options & SO_DONTROUTE) ? IP_ROUTETOIF : 0), 0, tp->t_inpcb); ... switch (error) { case EPERM: tp->t_softerror = error; return (error); case ENOBUFS: if (!tcp_timer_active(tp, TT_REXMT) && !tcp_timer_active(tp, TT_PERSIST)) tcp_timer_activate(tp, TT_REXMT, tp->t_rxtcur); tp->snd_cwnd = tp->t_maxseg; return (0); ... case EHOSTDOWN: case EHOSTUNREACH: case ENETDOWN: case ENETUNREACH: if (TCPS_HAVERCVDSYN(tp->t_state)) { tp->t_softerror = error; return (0); } Thanks, Matt From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 15:51:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 09F60442 for ; Wed, 13 Mar 2013 15:51:07 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x235.google.com (mail-bk0-x235.google.com [IPv6:2a00:1450:4008:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 88EB1E8 for ; Wed, 13 Mar 2013 15:51:06 +0000 (UTC) Received: by mail-bk0-f53.google.com with SMTP id j10so532272bkw.40 for ; Wed, 13 Mar 2013 08:51:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=F6yqG1/38K14avVxMvRqadOiK8R1IAZXFddoi4E4v7Q=; b=CwlO/tgHOzFwhklRW1oFYytWTM2wtMJpmiMLbFmooDpYzze43usPTFArjiut2jr0Et iE+KhGO4Ups8xp67aT1+pkxvmlwUV217ft4wcMW1O/6WEKr/90lSkwYaG5MnNEtx0DO2 ZJy5P35bbiTVswhnJoYgtGGAxnS8eBfltY5Je8sXnYy5Nrx4gzM3e2n8BAWghCg23+aA gWVaIeCXlErycqphhuWEiGQa8ESbSRU1f66nlJBWuTb/DuxgiiXTHGXQcjTE4zX0sukm YOja7uShktZWR9XSaub8dMNjyHJUF3plc6SeuwKTVXVH4fRIYegfj0O0ty7k+x27P+pZ wPEw== X-Received: by 10.204.238.133 with SMTP id ks5mr7923229bkb.101.1363189865108; Wed, 13 Mar 2013 08:51:05 -0700 (PDT) Received: from zvezda.localnet ([212.48.107.10]) by mx.google.com with ESMTPS id o2sm6185337bkv.3.2013.03.13.08.51.03 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 13 Mar 2013 08:51:04 -0700 (PDT) From: Kajetan Staszkiewicz To: Ermal =?iso-8859-1?q?Lu=E7i?= Subject: Re: [patch] Source entries removing is awfully slow. Date: Wed, 13 Mar 2013 16:51:03 +0100 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <201303081419.17743.vegeta@tuxpowered.net> <201303111751.18274.vegeta@tuxpowered.net> In-Reply-To: <201303111751.18274.vegeta@tuxpowered.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201303131651.03250.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQl9pwidQrSm9w+xZfu5dlLQGIC32tX+/5X9dwgKigRXp08e6Cj0aSCFt0vccVUNkocR9bnX Cc: "freebsd-net@freebsd.org" , "freebsd-pf@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 15:51:07 -0000 I managed to run your patch on production machine and it produced panic in a few seconds after network traffic was directed through it. The difference I spotted is that you insert the State entry to state_list near the end of pf_create_state: @@ -3950,8 +3970,18 @@ pf_create_state(struct pf_rule *r, struc pool_put(&pf_state_pl, s); #endif return (PF_DROP); +#ifdef __FreeBSD__ + } else { + if (sn != NULL) + TAILQ_INSERT_HEAD(&sn->state_list, s, srcnode_link); + if (nsn != NULL) + TAILQ_INSERT_HEAD(&nsn->state_list, s, srcnode_link); + *sm = s; + } +#else } else *sm = s; +#endif while my original aproach was to do it much earlier. The working version is: @@ -3895,12 +3915,18 @@ pf_create_state(struct pf_rule *r, struc if (sn != NULL) { s->src_node = sn; s->src_node->states++; +#ifdef __FreeBSD__ + TAILQ_INSERT_HEAD(&sn->state_list, s, srcnode_link); +#endif } if (nsn != NULL) { /* XXX We only modify one side for now. */ PF_ACPY(&nsn->raddr, &nk->addr[1], pd->af); s->nat_src_node = nsn; s->nat_src_node->states++; +#ifdef __FreeBSD__ + TAILQ_INSERT_HEAD(&nsn->state_list, s, srcnode_link); +#endif } if (pd->proto == IPPROTO_TCP) { if ((pd->flags & PFDESC_TCP_NORM) && pf_normalize_tcp_init(m, backtrace: #7 0xffffffff810d7d2a in pf_src_tree_remove_state (s=0xfffffe007bfd55f0) at /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:1564 #8 0xffffffff810e01c2 in pf_test_rule (rm=0xffffff862f6ec848, sm=0xffffff862f6ec840, direction=2, kif=0xfffffe000ab67400, m=0xfffffe0070be9500, off=20, h=0xfffffe0070be9580, pd=0xffffff862f6ec780, am=0xffffff862f6ec850, rsm=0xffffff862f6ec838, ifq=0x0, inp=0x0) at /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:3965 #9 0xffffffff810e1d26 in pf_test (dir=2, ifp=Variable "ifp" is not available. ) at /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:6964 #10 0xffffffff810e8cc1 in pf_check_out (arg=Variable "arg" is not available. ) at /usr/src/sys/modules/pf/../../contrib/pf/net/pf_ioctl.c:4187 #11 0xffffffff8078816e in pfil_run_hooks (ph=Variable "ph" is not available. ) at /usr/src/sys/net/pfil.c:82 #12 0xffffffff807a20f5 in ip_fastforward (m=0xfffffe0070be9500) at /usr/src/sys/netinet/ip_fastfwd.c:445 contrib/pf/net/pf.c:3965 3961 if (pf_state_insert(BOUND_IFACE(r, kif), skw, sks, s)) { 3962 if (pd->proto == IPPROTO_TCP) 3963 pf_normalize_tcp_cleanup(s); 3964 REASON_SET(&reason, PFRES_STATEINS); 3965 pf_src_tree_remove_state(s); 3966 STATE_DEC_COUNTERS(s); 3967 #ifdef __FreeBSD__ 3968 pool_put(&V_pf_state_pl, s); 3969 #else 3970 pool_put(&pf_state_pl, s); 3971 #endif 3972 return (PF_DROP); contrib/pf/net/pf.c:1564 1561 } 1562 #ifdef __FreeBSD__ 1563 if (!TAILQ_EMPTY(&s->nat_src_node->state_list)) 1564 TAILQ_REMOVE(&s->nat_src_node->state_list, s, srcnode_link); 1565 #endif 1566 } 1567 s->src_node = s->nat_src_node = NULL; Following s -> nat_src_node -> state_list: state_list = {tqh_first = 0xfffffe01fe360130, tqh_last = 0x0} s and nat_src_node seem ok for me, not any memory garbage, e.g. they have proper creation and expiration times. Do we hit a src_node that has the state_list uninitialised, and performing an early insert automatically fixes it? -- | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 15:52:59 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5B611546 for ; Wed, 13 Mar 2013 15:52:59 +0000 (UTC) (envelope-from lists@rewt.org.uk) Received: from abby.lhr1.as41113.net (unknown [IPv6:2001:b70:201:2::22]) by mx1.freebsd.org (Postfix) with ESMTP id EB530113 for ; Wed, 13 Mar 2013 15:52:58 +0000 (UTC) Received: from [172.16.9.23] (bella.stf.rewt.org.uk [91.208.177.62]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: joe@rewt.org.uk) by abby.lhr1.as41113.net (Postfix) with ESMTPSA id 3ZQyHk6ZJtzNG; Wed, 13 Mar 2013 15:52:50 +0000 (UTC) Message-ID: <5140A0CE.4030605@rewt.org.uk> Date: Wed, 13 Mar 2013 15:52:46 +0000 From: Joe Holden User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: Schrodinger Subject: Re: ipv6 default router Operation not permitted References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> <20130313131016.GE17859@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> In-Reply-To: <20130313135253.GA18992@defiant.konundrum.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 15:52:59 -0000 Just use router solicitation to ask for the link-local gateway, that is the "correct" way to do it. Schrodinger wrote: > Damien, > > I appreciate your replies very much, but I'm a subscriber so just reply > to the mailing list. Thanks. > > On 2013/03/13 14:19, Fleuriot Damien wrote: > > [SNARF] > >> >> These are indeed correct, thanks for clarifying. >> > > I thought that's what I said in my first email ;) Sorry for any > confusion. > >> Find below the config I'm using on an old OVH box. >> Said config might be outdated now (as per OVH's guide on setting up IPv6 [1]) , however that was at the time the only way to get things working properly. >> >> rc.conf >> === >> #Range IPv6: 2001:41D0:2:613b::/64 >> ipv6_enable="YES" >> ipv6_ifconfig_re0="fe80::21c:c0ff:fef3:31fa/64 scopeid 0x1" >> ipv6_ifconfig_re0_alias0="2001:41d0:2:613b::dead:beef/56" >> ipv6_defaultrouter="2001:41d0:2:61ff:ff:ff:ff:ff" >> === >> > > You have /56 and this is what I believe to be the incorrect way to get > this to Just Work. I think this assume that anyone else in this /56 is > in the same layer two segment as you.... > >> routing table >> === > [SNARF] >> === >> >> >> >> Notice that said config actually works: >> === >> $ ping6 www.google.com >> PING6(56=40+8+8 bytes) 2001:41d0:2:613b::dead:beef --> 2a00:1450:4007:804::1014 >> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=0 hlim=57 time=4.461 ms >> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=1 hlim=57 time=4.462 ms >> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=2 hlim=57 time=4.405 ms >> ^C >> --- www.google.com ping6 statistics --- >> 3 packets transmitted, 3 packets received, 0.0% packet loss >> round-trip min/avg/max/std-dev = 4.405/4.443/4.462/0.027 ms >> === >> >> Either way, you might want to have a look at OVH's guide [1] but in my own case, using a /56 was, at the time, the only way to get things working in a clean way. >> >> [1] http://help.ovh.com/Ipv4Ipv6#link10 >> > > I read this, I made sure to read this and then I read it a second time. > No where does it indicate the use of a /56. I am in the process of a > migration from an old OVH server to a new OVH server. My old box uses > the /56 prefix length "fix" but based on the documentation this is > incorrect and IMO this assumes that anyone else in the /56 is in the > same segment as me and if they are using /64 - well, There Be Dragons. > > Also from the information I have received, router advertisements may be > turned off in the future, my host should simply Neighbour Solicit for > the global scope unicast address of my default gateway. And as pointed > out in previous emails without ACCEPT_RTADV for re0 - FreeBSD does not > perform this action. > > So again, what is the correct way ? I think this is a debate of IPv6 > Protocol vs. IPv6 Policy vs. Network architecture. > > I'll go and get Tina Turner. You get Masterblaster and we'll meet in > Thunderdome. > > C. From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 15:59:40 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D86E96C2 for ; Wed, 13 Mar 2013 15:59:40 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id 698B518C for ; Wed, 13 Mar 2013 15:59:40 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id 5ADBD1CD7F4 for ; Wed, 13 Mar 2013 15:59:39 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C2xG+cOiQ32m for ; Wed, 13 Mar 2013 15:59:39 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id CB6361CD7F3 for ; Wed, 13 Mar 2013 15:59:38 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DFxbJQ019725 for ; Wed, 13 Mar 2013 15:59:37 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DFxb5J019724 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 15:59:37 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 15:59:36 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313155936.GC18992@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> <20130313131016.GE17859@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> <5140A0CE.4030605@rewt.org.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="IpbVkmxF4tDyP/Kb" Content-Disposition: inline In-Reply-To: <5140A0CE.4030605@rewt.org.uk> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 15:59:40 -0000 --IpbVkmxF4tDyP/Kb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 15:52, Joe Holden wrote: > Just use router solicitation to ask for the link-local gateway, that is= =20 > the "correct" way to do it. >=20 Hi Joe, If you read some of this thread you'll note that router advertisements are being disabled by the hosting provider. While their documentation indicates the use of router advertisments this does not solve the issue that I get "Operation not permitted" when trying to ping the default gateway. Without ACCEPT_RTADV on re0 FreeBSD does not even perform NEIGHBOUR solicitation for 2001:41d0:2:e7ff:ff:ff:ff:ff - presumably because it thinks that this is not on the same link as re0. C. > Schrodinger wrote: > > Damien,=20 > >=20 > > I appreciate your replies very much, but I'm a subscriber so just reply > > to the mailing list. Thanks. > >=20 > > On 2013/03/13 14:19, Fleuriot Damien wrote: > >=20 > > [SNARF] > >=20 > >> > >> These are indeed correct, thanks for clarifying. > >> > >=20 > > I thought that's what I said in my first email ;) Sorry for any > > confusion. > >=20 > >> Find below the config I'm using on an old OVH box. > >> Said config might be outdated now (as per OVH's guide on setting up IP= v6 [1]) , however that was at the time the only way to get things working p= roperly. > >> > >> rc.conf > >> =3D=3D=3D > >> #Range IPv6: 2001:41D0:2:613b::/64 > >> ipv6_enable=3D"YES" > >> ipv6_ifconfig_re0=3D"fe80::21c:c0ff:fef3:31fa/64 scopeid 0x1" > >> ipv6_ifconfig_re0_alias0=3D"2001:41d0:2:613b::dead:beef/56" > >> ipv6_defaultrouter=3D"2001:41d0:2:61ff:ff:ff:ff:ff" > >> =3D=3D=3D > >> > >=20 > > You have /56 and this is what I believe to be the incorrect way to get > > this to Just Work. I think this assume that anyone else in this /56 is > > in the same layer two segment as you....=20 > >=20 > >> routing table > >> =3D=3D=3D > > [SNARF] > >> =3D=3D=3D > >> > >> > >> > >> Notice that said config actually works: > >> =3D=3D=3D > >> $ ping6 www.google.com > >> PING6(56=3D40+8+8 bytes) 2001:41d0:2:613b::dead:beef --> 2a00:1450:400= 7:804::1014 > >> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D0 hlim=3D57 time=3D= 4.461 ms > >> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D1 hlim=3D57 time=3D= 4.462 ms > >> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D2 hlim=3D57 time=3D= 4.405 ms > >> ^C > >> --- www.google.com ping6 statistics --- > >> 3 packets transmitted, 3 packets received, 0.0% packet loss > >> round-trip min/avg/max/std-dev =3D 4.405/4.443/4.462/0.027 ms > >> =3D=3D=3D > >> > >> Either way, you might want to have a look at OVH's guide [1] but in my= own case, using a /56 was, at the time, the only way to get things working= in a clean way. > >> > >> [1] http://help.ovh.com/Ipv4Ipv6#link10 > >> > >=20 > > I read this, I made sure to read this and then I read it a second time. > > No where does it indicate the use of a /56. I am in the process of a > > migration from an old OVH server to a new OVH server. My old box uses > > the /56 prefix length "fix" but based on the documentation this is > > incorrect and IMO this assumes that anyone else in the /56 is in the=20 > > same segment as me and if they are using /64 - well, There Be Dragons. > >=20 > > Also from the information I have received, router advertisements may be > > turned off in the future, my host should simply Neighbour Solicit for > > the global scope unicast address of my default gateway. And as pointed > > out in previous emails without ACCEPT_RTADV for re0 - FreeBSD does not > > perform this action. > >=20 > > So again, what is the correct way ? I think this is a debate of IPv6 > > Protocol vs. IPv6 Policy vs. Network architecture. > >=20 > > I'll go and get Tina Turner. You get Masterblaster and we'll meet in > > Thunderdome. > >=20 > > C. >=20 --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --IpbVkmxF4tDyP/Kb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQKJnAAoJEBBi7cjNKnTjYm4P/32e0AfR7Hk2lotLaCvyQRP2 anmvN28RHawkOVLpbhW85aq9M2MQ6ExFHul/9FMlxJ2FjkFJhC5DPu8sjSeZRGY3 Ov6ojqJsEyv48QiImwfz1lMYWYJ3NU6EqCOgKAXIxh+d3NGXm1phX+fNnBjrvOJi vrwA70yml8ZyfVyatf/7avoOty21hrOav2rAXbDlM4OkF0YJpm6BjmRTpj9ghTZB FhhKXWDAUa7ep997oEr/4houcjqEL7EUTD3gd5/hIOyNWZHsRVcsZnlJ5hRAQLHr K5ixUmu18P1S8YnTcR26i7gYM1oi65g+YYIkMge9UFpb0rWWrM1SlwxeCFr/laAi XNmejXJkae/aGGkhJYKCiwxw+RXOnVQAQVJKmQGj3+M+e7IB9RXrlPwyyMYGcjW+ MbOs67C9Gv/N3Wl4jCh5YCy7orZPyL7Ntog2oKd51ClknJZxEU/1eUBykIuPx8BH 9+pvB0K0PvNaxCR/3AwC/8s/Wl4pJWpH0IcAL1VgYIhoWHOt11vhAsZh1ZuygqyL zqD0Mg/eSgBZ1AVEc+EwLeliXsXdmpqtQca2CpgXjAlihxmoYgHmYgUVBuwcn6dP oMH0ku6K/rlCvbmq5PPFYnL2NupVzXTHj8mx2MV2Oh5rzFiHgruP7u4XDsenIOAe q71ABUAGD4/CZHnXH4+X =7oK4 -----END PGP SIGNATURE----- --IpbVkmxF4tDyP/Kb-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 15:59:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CFE956C3 for ; Wed, 13 Mar 2013 15:59:45 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail4.ijs.si [193.2.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 5CE5618E for ; Wed, 13 Mar 2013 15:59:42 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3ZQyR25xrlzGMnK for ; Wed, 13 Mar 2013 16:59:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:in-reply-to:references:user-agent:date:date :subject:subject:organization:from:from:received:received :received:vbr-info; s=jakla2; t=1363190345; x=1365782346; bh=fFM AHv1rpH1PTpeEpdzhxbrBYVGRQXyoT7QGox6mlts=; b=JgXqOXzyHUqvniY41no xaZG8b+T7j065C4vYyUmpkVKamOaw+4R9E46E2zf46ohMO290Wq2iTtnBfotHpxV vb/j/o1H20UI2LeftEt5VHIktRI6cXbSFQrrWm2vXnr+itOF4kRCHVh/azNYg0xf qoCdiMgPGopoDemS2DIWl/Lc= VBR-Info: md=ijs.si; mc=all; mv=dwl.spamhaus.org; X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id U7abi5VfjDQf for ; Wed, 13 Mar 2013 16:59:05 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP for ; Wed, 13 Mar 2013 16:59:04 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id 733D26F3 for ; Wed, 13 Mar 2013 16:59:04 +0100 (CET) From: Mark Martinec Organization: J. Stefan Institute To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Date: Wed, 13 Mar 2013 16:59:04 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.1-STABLE; KDE/4.9.5; amd64; ; ) References: <20130312225018.GA13589@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> In-Reply-To: <20130313135253.GA18992@defiant.konundrum.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201303131659.04074.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 15:59:45 -0000 On Wednesday March 13 2013 Schrodinger wrote: > This isn't my network so I don't have any input into the matter. This > is the OVH configuration for their dedicated servers, at least in my > product range. > > > [1] http://help.ovh.com/Ipv4Ipv6#link10 > I read this, I made sure to read this and then I read it a second time. > No where does it indicate the use of a /56. I am in the process of a > migration from an old OVH server to a new OVH server. My old box uses > the /56 prefix length "fix" but based on the documentation this is > incorrect and IMO this assumes that anyone else in the /56 is in the > same segment as me and if they are using /64 - well, There Be Dragons. > I am informed that I must configure my interface to /64 by OVH. The same > as everyone else. So if everyone was on a /64 then we will send packets > to each other via our shared default gateway. > If I were to change my interface prefix length to /56 my host would no > longer consider the need to send packets to the default gateway for any > host within this /56. I would simply perform Neighbour Solicitation on > my link. > I am informed that I must configure my interface to /64 by OVH. The same > as everyone else. So if everyone was on a /64 then we will send packets > to each other via our shared default gateway. Fleuriot Damien writes: > OVH allocates a /64 per customer. > To avoid having to setup 1 gateway per customer, they set up a single one > within a /56, allowing for 256 /64s I see, nasty. > This mimics the situation where your host gives you a /32 ipv4 within > a /24 network and uses a single gateway, again for 250ish customers. No it doesn't. That would be true if a mask on the interface were /32, in which case there would again be a problem reaching a default router in absence of an ARP entry. Similarly if a net would be a /16 and an interface would be given a /24 mask and a router would be outside of this /24 subnet, even if on the same L2 link. > Does adding the interface route not put the default gateway on-link > though ? I don't think it does. The on-link state of an address comes from matching the address to a set of prefixes on an interface, or finding it in the ndp cache - perhaps as a result of receiving a redirect messages, or manually. RFC 4861 section 2.1.: on-link - an address that is assigned to an interface on a specified link. A node considers an address to be on-link if: - it is covered by one of the link's prefixes (e.g., as indicated by the on-link flag in the Prefix Information option), or - a neighboring router specifies the address as the target of a Redirect message, or - a Neighbor Advertisement message is received for the (target) address, or - any Neighbor Discovery message is received from the address. > Also from the information I have received, router advertisements may be > turned off in the future, my host should simply Neighbour Solicit for > the global scope unicast address of my default gateway. A host would not send a Neighbour Solicitation for an address which is not considered on-link: RFC 4861 section 7.2 Address resolution is the process through which a node determines the link-layer address of a neighbor given only its IP address. Address resolution is performed only on addresses that are determined to be on-link and for which the sender does not know the corresponding link-layer address (see Section 5.2). > was that because of the host route for the default gateway FreeBSD does > not solicit for the "on-link gateway" because the interface is not set > to ACCEPT_RTADV. But that doesn't make immediate sense. > > And as pointed out in previous emails without ACCEPT_RTADV for > re0 - FreeBSD does not perform this action. The router's link-layer address is available in RA messages. If you turn off ACCEPT_RTADV (or if a router does not send them), the only way to obtain a router's link-layer address would be by sending a Neighbour Solicitation (which is only sent to an address which is considered on-link), or by manually placing it in the NDP cache. > So again, what is the correct way ? I think this is a debate of IPv6 > Protocol vs. IPv6 Policy vs. Network architecture. I'm not sure, but it appears to me that adding the router's link-layer address to the NDP cache looks like the only reliable way, in absence or router advertisements. Mark From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 16:27:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 67AA9DB1 for ; Wed, 13 Mar 2013 16:27:04 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id D4D30361 for ; Wed, 13 Mar 2013 16:27:03 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id E96551CD7F4 for ; Wed, 13 Mar 2013 16:27:02 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3-NqGSpxr6nx for ; Wed, 13 Mar 2013 16:27:02 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 72DF71CD7F3 for ; Wed, 13 Mar 2013 16:27:02 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DGR1KT019855 for ; Wed, 13 Mar 2013 16:27:01 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DGR1vY019854 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 16:27:01 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 16:27:00 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313162700.GD18992@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> <201303131659.04074.Mark.Martinec+freebsd@ijs.si> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="osDK9TLjxFScVI/L" Content-Disposition: inline In-Reply-To: <201303131659.04074.Mark.Martinec+freebsd@ijs.si> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 16:27:04 -0000 --osDK9TLjxFScVI/L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 16:59, Mark Martinec wrote: Hi Mark, [...] >=20 > > Does adding the interface route not put the default gateway on-link > > though ? >=20 > I don't think it does. The on-link state of an address comes > from matching the address to a set of prefixes on an interface, > or finding it in the ndp cache - perhaps as a result of receiving > a redirect messages, or manually. >=20 >=20 > RFC 4861 section 2.1.: >=20 > on-link - an address that is assigned to an interface on a > specified link. A node considers an address to be > on-link if: >=20 > - it is covered by one of the link's prefixes (e.g., > as indicated by the on-link flag in the Prefix > Information option), or >=20 > - a neighboring router specifies the address as the > target of a Redirect message, or >=20 > - a Neighbor Advertisement message is received for > the (target) address, or >=20 > - any Neighbor Discovery message is received from > the address. >=20 Quite correct and I had read through this last night but I only now see that adding the route doesn't indicate the presence of the prefix on that link. And >this< is the problem. Duh. >=20 > > Also from the information I have received, router advertisements may be > > turned off in the future, my host should simply Neighbour Solicit for > > the global scope unicast address of my default gateway. >=20 > A host would not send a Neighbour Solicitation for an address > which is not considered on-link: >=20 > RFC 4861 section 7.2 >=20 > Address resolution is the process through which a node determines the > link-layer address of a neighbor given only its IP address. Address > resolution is performed only on addresses that are determined to be > on-link and for which the sender does not know the corresponding > link-layer address (see Section 5.2). >=20 Again quite correct, I had read this as well trying to understand what should happen. My thinking was that the host route should indicate to FreeBSD that the default gateway is on that link but it is the configuration of a prefix _ON_ an interface that does that. This would be a "No Shit Sherlock" moment for me, thanks! :) >=20 > > was that because of the host route for the default gateway FreeBSD does > > not solicit for the "on-link gateway" because the interface is not set= =20 > > to ACCEPT_RTADV. But that doesn't make immediate sense. > >=20 > > And as pointed out in previous emails without ACCEPT_RTADV for > > re0 - FreeBSD does not perform this action. >=20 > The router's link-layer address is available in RA messages. > If you turn off ACCEPT_RTADV (or if a router does not send them), > the only way to obtain a router's link-layer address would be > by sending a Neighbour Solicitation (which is only sent to an > address which is considered on-link), or by manually placing > it in the NDP cache. >=20 FreeBSD of course isn't performing ND for the default gateway because it is not on the same link as my interface, regardless of what the routing table says. What I am confused about is that without ACCEPT_RTADV on re0, FreeBSD doesn't perform Neighbour Solicitation for the default gateway but with ACCEPT_RTADV it does ..... Why ? This is Neighbour Solicitation and not Router Solicitation.... I understand that FreeBSD doesn't consider the defaulte gateway to be "on-link" so it does not perform ND for it but why does it perform ND when ACCEPT_RTADV is set on re0 ? "Surely" ACCEPT_RTADV only affects Router Advertisements / Solicitations and not ND. I have done packet captures and with ACCEPT_RTADV I see the initial Neighbour Solicitation and the Neighbour Advertisement to and from my default gateway. Without ACCEPT_RTADV - FreeBSD simply doesn't try to perform ND for the address. This is where I am uncertain if this is expected or not. > > So again, what is the correct way ? I think this is a debate of IPv6 > > Protocol vs. IPv6 Policy vs. Network architecture. >=20 > I'm not sure, but it appears to me that adding the router's > link-layer address to the NDP cache looks like the only > reliable way, in absence or router advertisements. >=20 Yes, this would work and there is a configuration directive in rc.conf to achieve this but what if the link layer address changes for my default gateway ? Say a hardware failure and it changes, I would loose IPv6 connectivity. I guess a solution is to just ask OVH if they can continue to perform RA. Cheers, C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --osDK9TLjxFScVI/L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQKjTAAoJEBBi7cjNKnTjSCUP/1UtM+L+9s7DwcTxSH3WSmxO k2bv+mhag5wv7HCMVNUC/SwVId8MnYtc/Fox0bUheGrXoyidp2a+jWDGR51zDfjz /hZHMxhdoJt4TNUySYDfJI1wFU0g3dNdKZSi4a8rCdj2ETv+Ce49XRJcRy/d8Oze wFjtvV9mP3xLEWgEO4ec0pYPxxUWSzBChMONYD4ZDdYacaw04tIFzlY0a6PIJbyG QdKbTdi6BCvfz4t++CWAPZ9rrnJgmQfVx7bw4r0co9iB5q+XlpOMSP2xrpI6FVH8 StTP04dEdN66gGtNDdZ4EVXeutrL4sZpXane2NK0x/wpRTMcndpS2xDWJoit0k5g /MkzZUstwxM6sgffoQHvrMYdedoKx5I/T69vSofSp2dokM4AaqrzzLJ5ryBabdDH 0iZlQFwUE+imtCE9CSqenzkpqbmOlnogXUGO3Ht2PcDHE90KinxIQXCiKSTVtPbB Oy+wGk1Oc2P5XC2yEtsKuIIhT639Q6Vq1HZdHh9Xhf8IVglasxtVW0OnMtdyvNkd 4aJTBHFqvsFiOGGcOREUqvYk8qwdjAJLBsximlp0Hjcvy9DCI+azUhJdr6JOhU/p wBcIU+GAIAaJ6ADhiNv6XYYZpr3y4XBuZrQwZyREmfxfvgdhU3kgheX9N09n8/qk 6nyL5RydofjRDMdeg6Dq =1zQx -----END PGP SIGNATURE----- --osDK9TLjxFScVI/L-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 16:27:41 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8E52EE43 for ; Wed, 13 Mar 2013 16:27:41 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail4.ijs.si [193.2.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 4843236E for ; Wed, 13 Mar 2013 16:27:41 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3ZQz3L47JyzGN5j for ; Wed, 13 Mar 2013 17:27:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:in-reply-to:references:user-agent:date:date :subject:subject:organization:from:from:received:received :received:vbr-info; s=jakla2; t=1363192029; x=1365784030; bh=b0a YKVzxvumJKr3o/bODLPVoDDNLEpPNKpRj6jNkwe8=; b=fmA8biM6WEesW2ZTq+E 2trtmmc1J7g3ZYPBrTMr96xNkgpxA9LPCVhdRsGKVelMmuEb1hjzPUqbxXsiXF0k YAO1yE42MUQJ+4rgzvVKfx+utCInXpvALXR4M4doACTk3cVQ42fVSRTZh6FwR6cP z+DHNEDo8Gr4VZ6bIKPIYZ98= VBR-Info: md=ijs.si; mc=all; mv=dwl.spamhaus.org; X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id VkpbC8Fkfj_O for ; Wed, 13 Mar 2013 17:27:09 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP for ; Wed, 13 Mar 2013 17:27:09 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id AC57D8B0 for ; Wed, 13 Mar 2013 17:27:08 +0100 (CET) From: Mark Martinec Organization: J. Stefan Institute To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Date: Wed, 13 Mar 2013 17:27:08 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.1-STABLE; KDE/4.9.5; amd64; ; ) References: <20130312225018.GA13589@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> <201303131659.04074.Mark.Martinec+freebsd@ijs.si> In-Reply-To: <201303131659.04074.Mark.Martinec+freebsd@ijs.si> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201303131727.08183.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 16:27:41 -0000 > > I am informed that I must configure my interface to /64 by OVH. The same > > as everyone else. So if everyone was on a /64 then we will send packets > > to each other via our shared default gateway. Btw, if the router responds to your subnet's (/64) anycast address (RFC 4291 section 2.6.1 Required Anycast Address) and you are not afraid of rougue routers on the same subnet, you could try pointing a default route to that address, in your case to 2001:41d0:2:e7c4:: Mark From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 16:29:32 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C368AF39 for ; Wed, 13 Mar 2013 16:29:32 +0000 (UTC) (envelope-from lists@rewt.org.uk) Received: from abby.lhr1.as41113.net (hosted.mx.as41113.net [91.208.177.22]) by mx1.freebsd.org (Postfix) with ESMTP id 5FBB33C8 for ; Wed, 13 Mar 2013 16:29:31 +0000 (UTC) Received: from [172.16.9.23] (bella.stf.rewt.org.uk [91.208.177.62]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: joe@rewt.org.uk) by abby.lhr1.as41113.net (Postfix) with ESMTPSA id 3ZQz622MfwzP0; Wed, 13 Mar 2013 16:29:30 +0000 (UTC) Message-ID: <5140A965.5090206@rewt.org.uk> Date: Wed, 13 Mar 2013 16:29:25 +0000 From: Joe Holden User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: Schrodinger Subject: Re: ipv6 default router Operation not permitted References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> <20130313131016.GE17859@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> <5140A0CE.4030605@rewt.org.uk> <20130313155936.GC18992@defiant.konundrum.org> In-Reply-To: <20130313155936.GC18992@defiant.konundrum.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 16:29:32 -0000 Strange, I used this setup on an OVH machine a while ago, seemed to work - perhaps something isn't properly configured at their end properly Schrodinger wrote: > On 2013/03/13 15:52, Joe Holden wrote: >> Just use router solicitation to ask for the link-local gateway, that is >> the "correct" way to do it. >> > > Hi Joe, > > If you read some of this thread you'll note that router advertisements > are being disabled by the hosting provider. While their documentation > indicates the use of router advertisments this does not solve the issue > that I get "Operation not permitted" when trying to ping the default > gateway. > > Without ACCEPT_RTADV on re0 FreeBSD does not even perform NEIGHBOUR > solicitation for 2001:41d0:2:e7ff:ff:ff:ff:ff - presumably because it > thinks that this is not on the same link as re0. > > C. > >> Schrodinger wrote: >>> Damien, >>> >>> I appreciate your replies very much, but I'm a subscriber so just reply >>> to the mailing list. Thanks. >>> >>> On 2013/03/13 14:19, Fleuriot Damien wrote: >>> >>> [SNARF] >>> >>>> These are indeed correct, thanks for clarifying. >>>> >>> I thought that's what I said in my first email ;) Sorry for any >>> confusion. >>> >>>> Find below the config I'm using on an old OVH box. >>>> Said config might be outdated now (as per OVH's guide on setting up IPv6 [1]) , however that was at the time the only way to get things working properly. >>>> >>>> rc.conf >>>> === >>>> #Range IPv6: 2001:41D0:2:613b::/64 >>>> ipv6_enable="YES" >>>> ipv6_ifconfig_re0="fe80::21c:c0ff:fef3:31fa/64 scopeid 0x1" >>>> ipv6_ifconfig_re0_alias0="2001:41d0:2:613b::dead:beef/56" >>>> ipv6_defaultrouter="2001:41d0:2:61ff:ff:ff:ff:ff" >>>> === >>>> >>> You have /56 and this is what I believe to be the incorrect way to get >>> this to Just Work. I think this assume that anyone else in this /56 is >>> in the same layer two segment as you.... >>> >>>> routing table >>>> === >>> [SNARF] >>>> === >>>> >>>> >>>> >>>> Notice that said config actually works: >>>> === >>>> $ ping6 www.google.com >>>> PING6(56=40+8+8 bytes) 2001:41d0:2:613b::dead:beef --> 2a00:1450:4007:804::1014 >>>> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=0 hlim=57 time=4.461 ms >>>> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=1 hlim=57 time=4.462 ms >>>> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=2 hlim=57 time=4.405 ms >>>> ^C >>>> --- www.google.com ping6 statistics --- >>>> 3 packets transmitted, 3 packets received, 0.0% packet loss >>>> round-trip min/avg/max/std-dev = 4.405/4.443/4.462/0.027 ms >>>> === >>>> >>>> Either way, you might want to have a look at OVH's guide [1] but in my own case, using a /56 was, at the time, the only way to get things working in a clean way. >>>> >>>> [1] http://help.ovh.com/Ipv4Ipv6#link10 >>>> >>> I read this, I made sure to read this and then I read it a second time. >>> No where does it indicate the use of a /56. I am in the process of a >>> migration from an old OVH server to a new OVH server. My old box uses >>> the /56 prefix length "fix" but based on the documentation this is >>> incorrect and IMO this assumes that anyone else in the /56 is in the >>> same segment as me and if they are using /64 - well, There Be Dragons. >>> >>> Also from the information I have received, router advertisements may be >>> turned off in the future, my host should simply Neighbour Solicit for >>> the global scope unicast address of my default gateway. And as pointed >>> out in previous emails without ACCEPT_RTADV for re0 - FreeBSD does not >>> perform this action. >>> >>> So again, what is the correct way ? I think this is a debate of IPv6 >>> Protocol vs. IPv6 Policy vs. Network architecture. >>> >>> I'll go and get Tina Turner. You get Masterblaster and we'll meet in >>> Thunderdome. >>> >>> C. > From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 16:33:26 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 21C6BEA for ; Wed, 13 Mar 2013 16:33:26 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id A87F75F4 for ; Wed, 13 Mar 2013 16:33:25 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id 0C79D1CD7F4 for ; Wed, 13 Mar 2013 16:33:25 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JtdZGDljx3Ch for ; Wed, 13 Mar 2013 16:33:24 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 752521CD7F3 for ; Wed, 13 Mar 2013 16:33:24 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DGXNBW019890 for ; Wed, 13 Mar 2013 16:33:23 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DGXNcX019889 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 16:33:23 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 16:33:23 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313163323.GE18992@defiant.konundrum.org> References: <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> <20130313125221.GD17859@defiant.konundrum.org> <20130313131016.GE17859@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> <5140A0CE.4030605@rewt.org.uk> <20130313155936.GC18992@defiant.konundrum.org> <5140A965.5090206@rewt.org.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="X3gaHHMYHkYqP6yf" Content-Disposition: inline In-Reply-To: <5140A965.5090206@rewt.org.uk> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 16:33:26 -0000 --X3gaHHMYHkYqP6yf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 16:29, Joe Holden wrote: > Strange, I used this setup on an OVH machine a while ago, seemed to work= =20 > - perhaps something isn't properly configured at their end properly >=20 I have a ticket opened with OVH since yesterday to confirm or deny the future of RA on their network. Thankfully this discussion has helped to further understand what *should* be happening and why RA is really more ideal. This is for a new box and I have time to experiment, my old host uses=20 /56 but it's not the right way to do it, IMHO. C. > Schrodinger wrote: > > On 2013/03/13 15:52, Joe Holden wrote: > >> Just use router solicitation to ask for the link-local gateway, that i= s=20 > >> the "correct" way to do it. > >> > >=20 > > Hi Joe, > >=20 > > If you read some of this thread you'll note that router advertisements > > are being disabled by the hosting provider. While their documentation > > indicates the use of router advertisments this does not solve the issue > > that I get "Operation not permitted" when trying to ping the default > > gateway. > >=20 > > Without ACCEPT_RTADV on re0 FreeBSD does not even perform NEIGHBOUR > > solicitation for 2001:41d0:2:e7ff:ff:ff:ff:ff - presumably because it > > thinks that this is not on the same link as re0. > >=20 > > C. > >=20 > >> Schrodinger wrote: > >>> Damien,=20 > >>> > >>> I appreciate your replies very much, but I'm a subscriber so just rep= ly > >>> to the mailing list. Thanks. > >>> > >>> On 2013/03/13 14:19, Fleuriot Damien wrote: > >>> > >>> [SNARF] > >>> > >>>> These are indeed correct, thanks for clarifying. > >>>> > >>> I thought that's what I said in my first email ;) Sorry for any > >>> confusion. > >>> > >>>> Find below the config I'm using on an old OVH box. > >>>> Said config might be outdated now (as per OVH's guide on setting up = IPv6 [1]) , however that was at the time the only way to get things working= properly. > >>>> > >>>> rc.conf > >>>> =3D=3D=3D > >>>> #Range IPv6: 2001:41D0:2:613b::/64 > >>>> ipv6_enable=3D"YES" > >>>> ipv6_ifconfig_re0=3D"fe80::21c:c0ff:fef3:31fa/64 scopeid 0x1" > >>>> ipv6_ifconfig_re0_alias0=3D"2001:41d0:2:613b::dead:beef/56" > >>>> ipv6_defaultrouter=3D"2001:41d0:2:61ff:ff:ff:ff:ff" > >>>> =3D=3D=3D > >>>> > >>> You have /56 and this is what I believe to be the incorrect way to get > >>> this to Just Work. I think this assume that anyone else in this /56 is > >>> in the same layer two segment as you....=20 > >>> > >>>> routing table > >>>> =3D=3D=3D > >>> [SNARF] > >>>> =3D=3D=3D > >>>> > >>>> > >>>> > >>>> Notice that said config actually works: > >>>> =3D=3D=3D > >>>> $ ping6 www.google.com > >>>> PING6(56=3D40+8+8 bytes) 2001:41d0:2:613b::dead:beef --> 2a00:1450:4= 007:804::1014 > >>>> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D0 hlim=3D57 time= =3D4.461 ms > >>>> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D1 hlim=3D57 time= =3D4.462 ms > >>>> 16 bytes from 2a00:1450:4007:804::1014, icmp_seq=3D2 hlim=3D57 time= =3D4.405 ms > >>>> ^C > >>>> --- www.google.com ping6 statistics --- > >>>> 3 packets transmitted, 3 packets received, 0.0% packet loss > >>>> round-trip min/avg/max/std-dev =3D 4.405/4.443/4.462/0.027 ms > >>>> =3D=3D=3D > >>>> > >>>> Either way, you might want to have a look at OVH's guide [1] but in = my own case, using a /56 was, at the time, the only way to get things worki= ng in a clean way. > >>>> > >>>> [1] http://help.ovh.com/Ipv4Ipv6#link10 > >>>> > >>> I read this, I made sure to read this and then I read it a second tim= e. > >>> No where does it indicate the use of a /56. I am in the process of a > >>> migration from an old OVH server to a new OVH server. My old box uses > >>> the /56 prefix length "fix" but based on the documentation this is > >>> incorrect and IMO this assumes that anyone else in the /56 is in the= =20 > >>> same segment as me and if they are using /64 - well, There Be Dragons. > >>> > >>> Also from the information I have received, router advertisements may = be > >>> turned off in the future, my host should simply Neighbour Solicit for > >>> the global scope unicast address of my default gateway. And as pointed > >>> out in previous emails without ACCEPT_RTADV for re0 - FreeBSD does not > >>> perform this action. > >>> > >>> So again, what is the correct way ? I think this is a debate of IPv6 > >>> Protocol vs. IPv6 Policy vs. Network architecture. > >>> > >>> I'll go and get Tina Turner. You get Masterblaster and we'll meet in > >>> Thunderdome. > >>> > >>> C. > >=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --X3gaHHMYHkYqP6yf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQKpSAAoJEBBi7cjNKnTjeI4QAJ+J3oL9+oVZQBXygZLs4Dkd I6PtuK7Si9QB8D1T17c60+v2ZC8zZacnnriI6tZVJ1V81HmHXQk2TjnIQ22EMN4l KPdUSf+YvkQurFZexCr/ZXaYJYiyjGPyAhGkVtO2vZ8uoXqU0LIASdm43gH+fXSS c6QyQ/i1394Mir5Gq2gBdsl9CshHIxkkYQeIyifDUDKxi2HNWrm4lKS2ED/DEzSQ uaGKns/tx6if5nyRnfZyLlCC9u2TiGBIX2mV2e7HDFtIqOeDJkLv9MfDdptC62yT /zn72k+eWMLtmLJPbwc2dfZds5wzczGj1KpQK5iW009uxtPFzlrQjboK8zN++2bP Ay46qVASgiOYYv5b20LFszVtZDCghaoijwRNOVCjg+2Bh8g50XhgsvJLqDPsJIkW 3rgaCp4vxsW/02hKxuCa1dEXH6fq8BdHAaOJQpOMJE3yvbvP2v5l2RljebK34979 2wHADbY828iao82CRTkHDA+xixjN5RCd2OW0vWA6zny/hggHILnhgkmQQ+JTv2dh bELOKjVXAmNp7vYj2QzTp3P++7ZJbVyR8oie30NmiaRpr/JSGhBWnGBXEQkbpGoi o/iffA7WA0SLwLfnRdyT6BhkFrErI98Od/7BPhM67ij0+lc1knTNmVXse0wP8Umk 3++elE8l2+q2PCRUkPFw =9swd -----END PGP SIGNATURE----- --X3gaHHMYHkYqP6yf-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 16:38:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EEC132E0 for ; Wed, 13 Mar 2013 16:38:36 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [IPv6:2001:41d0:1:c74c::1]) by mx1.freebsd.org (Postfix) with ESMTP id 841E1642 for ; Wed, 13 Mar 2013 16:38:36 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id B60A61CD7F4 for ; Wed, 13 Mar 2013 16:38:35 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HFQoLs33B649 for ; Wed, 13 Mar 2013 16:38:35 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 908401CD7F3 for ; Wed, 13 Mar 2013 16:38:34 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DGcTPr019916 for ; Wed, 13 Mar 2013 16:38:29 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DGcTCh019915 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 16:38:29 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 16:38:28 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313163827.GF18992@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <20130313135253.GA18992@defiant.konundrum.org> <201303131659.04074.Mark.Martinec+freebsd@ijs.si> <201303131727.08183.Mark.Martinec+freebsd@ijs.si> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="aPdhxNJGSeOG9wFI" Content-Disposition: inline In-Reply-To: <201303131727.08183.Mark.Martinec+freebsd@ijs.si> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 16:38:37 -0000 --aPdhxNJGSeOG9wFI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 17:27, Mark Martinec wrote: > > > I am informed that I must configure my interface to /64 by OVH. The s= ame > > > as everyone else. So if everyone was on a /64 then we will send packe= ts > > > to each other via our shared default gateway. >=20 > Btw, if the router responds to your subnet's (/64) anycast address > (RFC 4291 section 2.6.1 Required Anycast Address) and you are > not afraid of rougue routers on the same subnet, you could > try pointing a default route to that address, in your case > to 2001:41d0:2:e7c4:: >=20 15 packets transmitted, 0 packets received, 100.0% packet loss Unfortunately the wording of this might support the packet loss. "All routers are required to support the Subnet-Router anycast addresses for the subnets to which they have interfaces." I am assuming that the subnet for the routers interface is /56 or Something Else so it shouldn't need to answer for the Anycast in my /64. Thanks though, C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --aPdhxNJGSeOG9wFI Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQKuCAAoJEBBi7cjNKnTjUk0P/inmzXxbdBSd3t0ZtSWLLSLp Kxqt3ewUw5mufdlK7yU5MLv5SNXGP3mM6AeEA2n3NBpBUhpfPQc3rOk+Rq4N2GSm qEaJMuA8lRm/majvSiCZW8U/lkZyG67vbK7+21dy36AvTi4k/RfXq9N/8+kA/eDL W4sj2m0qr1jmzYEL+bNm377SRiZac41CHK03rK8TVGtZQgdv0sUeHFtVh9qrtmeS jqISpxG4jpoNA+4AZft+7Qn4JJeEYO5aTyv50sha8II5/XS5X9u7ON7BnJkIrHoR ZbzwizSeBEXgWr3OeKehF0Ad0o/+RtdKCMvlrrtfQM4G/ZZEyI/4QVzUfigvN2/W C1dYKpB4vXFV473Mn+PsV6ctxPTAHv3RDWr5bcT+hdqhxUOwNEYk1ES3mfc9oTgT w8na9BHWD9kEeGEUCdE4xVMBe430lMDobBQuzccFeasHtibQT8Oe9BHkggK41qlG tOab33AKqToKv2F2IS+I4khghih1km9nFJOUa4IzZHUxArOBoovZ6WBe7p6THw63 I+zX48hpLDyPjv9Wt/yvr2NfmaxmlwLDTX2tsteEgQAmOcWxRgo6Apq55b16FmNL A/7QutPK8+XNyMjqQ6CK9rydDhYTQZEeyPUxTFw3vtO3LP5FtHXP+Qye1iD7fprH xeUqZm0WEA16JqKyiBO0 =JA7X -----END PGP SIGNATURE----- --aPdhxNJGSeOG9wFI-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 17:13:20 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1E822AC0 for ; Wed, 13 Mar 2013 17:13:20 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [193.2.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id CC10D850 for ; Wed, 13 Mar 2013 17:13:17 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3ZR03v6T1NzGMp8 for ; Wed, 13 Mar 2013 18:12:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:in-reply-to:references:user-agent:date:date :subject:subject:organization:from:from:received:received :received:vbr-info; s=jakla2; t=1363194757; x=1365786758; bh=9z9 BMtra0KqiPe7+GeRLiKsV0NevRObq1oR2NgruCh0=; b=fW/Vko67KnEK0LbU1x2 9DHMSXSsFBeoR9NWAEBdfCxBxq9FMLDEegb+VgSdaecQve3+DAOkUrw0mMZMmaRS suiE1IlGcjqoYIVQAAr2EASUaAmzQV7oFZe1bpMplpaaad3dL51Dbi3IBrQoF0x8 WvL/0pan4PJiF9d0HlGhpkS4= VBR-Info: md=ijs.si; mc=all; mv=dwl.spamhaus.org; X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id 4Mzi1R2J_tR0 for ; Wed, 13 Mar 2013 18:12:37 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP for ; Wed, 13 Mar 2013 18:12:37 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id DA556B04 for ; Wed, 13 Mar 2013 18:12:36 +0100 (CET) From: Mark Martinec Organization: J. Stefan Institute To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Date: Wed, 13 Mar 2013 18:12:36 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.1-STABLE; KDE/4.9.5; amd64; ; ) References: <20130312225018.GA13589@defiant.konundrum.org> <201303131659.04074.Mark.Martinec+freebsd@ijs.si> <20130313162700.GD18992@defiant.konundrum.org> In-Reply-To: <20130313162700.GD18992@defiant.konundrum.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201303131812.36388.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 17:13:20 -0000 Schrodinger wrote: > What I am confused about is that without ACCEPT_RTADV on re0, FreeBSD > doesn't perform Neighbour Solicitation for the default gateway but with > ACCEPT_RTADV it does ..... Why ? This is Neighbour Solicitation and not > Router Solicitation.... > > I understand that FreeBSD doesn't consider the defaulte gateway to be > "on-link" so it does not perform ND for it but why does it perform ND > when ACCEPT_RTADV is set on re0 ? "Surely" ACCEPT_RTADV only affects > Router Advertisements / Solicitations and not ND. > > I have done packet captures and with ACCEPT_RTADV I see the initial > Neighbour Solicitation and the Neighbour Advertisement to and from my > default gateway. > > Without ACCEPT_RTADV - FreeBSD simply doesn't try to perform ND for the > address. This is where I am uncertain if this is expected or not. That is a good question and I'd be interested in an answer too. Perhaps FreeBSD is implementing a predecessor to RFC 4861, i.e. the now obsolete RFC 2461: RFC 4861, Appendix F: Changes from RFC 2461 o Removed the on-link assumption in Section 5.2 based on RFC 4943, "IPv6 Neighbor Discovery On-Link Assumption Considered Harmful". RFC 4943, Abstract This document describes the historical and background information behind the removal of the "on-link assumption" from the conceptual host sending algorithm defined in Neighbor Discovery for IP Version 6 (IPv6). According to the algorithm as originally described, when a host's default router list is empty, the host assumes that all destinations are on-link. Mark From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 18:54:54 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 927EAF7A for ; Wed, 13 Mar 2013 18:54:54 +0000 (UTC) (envelope-from schrodinger@konundrum.org) Received: from crux.konundrum.org (crux.konundrum.org [91.121.150.76]) by mx1.freebsd.org (Postfix) with ESMTP id 243BFF16 for ; Wed, 13 Mar 2013 18:54:53 +0000 (UTC) Received: from crux.konundrum.org (localhost [127.0.0.1]) by crux.konundrum.org (Postfix) with ESMTP id DABEA1CD7F4 for ; Wed, 13 Mar 2013 18:54:52 +0000 (GMT) X-Virus-Scanned: amavisd-new at konundrum.org Received: from crux.konundrum.org ([127.0.0.1]) by crux.konundrum.org (crux.konundrum.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZiUurOfi7jg5 for ; Wed, 13 Mar 2013 18:54:52 +0000 (GMT) Received: from defiant.konundrum.org (defiant.konundrum.org [IPv6:2001:770:146:2::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crux.konundrum.org (Postfix) with ESMTPS id 434A81CD7F3 for ; Wed, 13 Mar 2013 18:54:52 +0000 (GMT) Received: from defiant.konundrum.org (localhost [127.0.0.1]) by defiant.konundrum.org (8.14.5/8.14.5) with ESMTP id r2DIsoI6020480 for ; Wed, 13 Mar 2013 18:54:50 GMT (envelope-from schrodinger@konundrum.org) Received: (from schrodinger@localhost) by defiant.konundrum.org (8.14.5/8.14.5/Submit) id r2DIsolZ020479 for freebsd-net@freebsd.org; Wed, 13 Mar 2013 18:54:50 GMT (envelope-from schrodinger@konundrum.org) X-Authentication-Warning: defiant.konundrum.org: schrodinger set sender to schrodinger@konundrum.org using -f Date: Wed, 13 Mar 2013 18:54:49 +0000 From: Schrodinger To: freebsd-net@freebsd.org Subject: Re: ipv6 default router Operation not permitted Message-ID: <20130313185449.GA20353@defiant.konundrum.org> References: <20130312225018.GA13589@defiant.konundrum.org> <201303131659.04074.Mark.Martinec+freebsd@ijs.si> <20130313162700.GD18992@defiant.konundrum.org> <201303131812.36388.Mark.Martinec+freebsd@ijs.si> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6TrnltStXW4iwmi0" Content-Disposition: inline In-Reply-To: <201303131812.36388.Mark.Martinec+freebsd@ijs.si> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 18:54:54 -0000 --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2013/03/13 18:12, Mark Martinec wrote: > Schrodinger wrote: > > What I am confused about is that without ACCEPT_RTADV on re0, FreeBSD > > doesn't perform Neighbour Solicitation for the default gateway but with > > ACCEPT_RTADV it does ..... Why ? This is Neighbour Solicitation and not > > Router Solicitation.... > >=20 > > I understand that FreeBSD doesn't consider the defaulte gateway to be > > "on-link" so it does not perform ND for it but why does it perform ND > > when ACCEPT_RTADV is set on re0 ? "Surely" ACCEPT_RTADV only affects > > Router Advertisements / Solicitations and not ND. > >=20 > > I have done packet captures and with ACCEPT_RTADV I see the initial > > Neighbour Solicitation and the Neighbour Advertisement to and from my > > default gateway. > >=20 > > Without ACCEPT_RTADV - FreeBSD simply doesn't try to perform ND for the > > address. This is where I am uncertain if this is expected or not. >=20 > That is a good question and I'd be interested in an answer too. >=20 Great!!! It's not just me then. > Perhaps FreeBSD is implementing a predecessor to RFC 4861, > i.e. the now obsolete RFC 2461: >=20 >=20 > RFC 4861, Appendix F: Changes from RFC 2461 > o Removed the on-link assumption in Section 5.2 based on RFC 4943, > "IPv6 Neighbor Discovery On-Link Assumption Considered Harmful". >=20 >=20 > RFC 4943, Abstract > This document describes the historical and background information > behind the removal of the "on-link assumption" from the conceptual > host sending algorithm defined in Neighbor Discovery for IP Version 6 > (IPv6). According to the algorithm as originally described, when a > host's default router list is empty, the host assumes that all > destinations are on-link. >=20 Based on what is documented in RFC 4943 is FreeBSD not working as expected ? Since it does not perform ND for the default gateway because it isn't seen as on-link and FreeBSD is not making the on-link assumption? Correct? If I understand how FreeBSD should behave if applying RFC2461 when there is no default gateway configured it should assume the detination is on-link. Correct? I removed the default gateway, but left the interface host route for the default gateway address in the routing table and I still get "Operation not permitted" when I attempt to ping the address. # netstat -rn -f inet6 | fgrep ff:ff=20 default 2001:41d0:2:e7ff:ff:ff:ff:ff UGS = re0 2001:41d0:2:e7ff:ff:ff:ff:ff e0:69:95:88:0b:27 UHS = re0 # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff ping6: sendmsg: Operation not permitted ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 0 packets received, 100.0% packet loss # route delete -inet6 default =20 delete net default # netstat -rn -f inet6 | fgrep ff:ff 2001:41d0:2:e7ff:ff:ff:ff:ff e0:69:95:88:0b:27 UHS = re0 # ndp -c 2001:41d0:2:e7c4::1 (2001:41d0:2:e7c4::1) deleted fe80::e269:95ff:fe88:b27%re0 (fe80::e269:95ff:fe88:b27%re0) deleted # ping6 -c 1 2001:41d0:2:e7ff:ff:ff:ff:ff PING6(56=3D40+8+8 bytes) 2001:41d0:2:e7c4::1 --> 2001:41d0:2:e7ff:ff:ff:ff:= ff ping6: sendmsg: Operation not permitted ping6: wrote 2001:41d0:2:e7ff:ff:ff:ff:ff 16 chars, ret=3D-1 --- 2001:41d0:2:e7ff:ff:ff:ff:ff ping6 statistics --- 1 packets transmitted, 0 packets received, 100.0% packet loss Although this still does not immediately explain to me why ACCEPT_RTADV causes it to perform ND. Cheers, C. --=20 +---------------------------------------------------------------+ Quidquid latine dictum sit, altum sonatur. MSN: schro5@hotmail.com ICQ: 112562229 GPG: http://www.konundrum.org/schro.asc --6TrnltStXW4iwmi0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBCgAGBQJRQMt4AAoJEBBi7cjNKnTjRRYP/Rhe47oA/ufaX/bcHIyxTlnx k+TPwI106NmHBx0hdUckOlpBOmrnE+tvWH/ll2jgO48OqVxFjXxtcNYmWu7ULkCK mONs+4P4ezfCf/aNFp275REfq86tqVXETueOn5BEbMIMKIQlLY72p0aypBOm8Ra4 ZYD6KS20ZXcQjqZBUhPSt7UJuOuYPvRx3ezWkoWehDs4UuPyO9nntblXNic8JTf9 g3FN9QaxWjgOUd5kXKh8+RTiijsnlfX9tqyIFSThYhQRPlT/PSwfrwWOSlGFTG7n DcDrUmw0wS8/siKERVj+C24+LRrPzjo6SmN8xu9undIvm43u1/G3Tj/dVHU/4qGn zlS2WqnewBSC1AlSoTfRNji893KpdwNUX/UZowZY41Tcfg7pLccATC6g8JhX/u/r Wt3uN53AGqRhU+uayOnnhXQP03GFf5iZzJWYCkopLTa3aB43hCfQVQgx5dIo/95t 870pUPyJoN7xNJ9mm8vj8a3zKNLeZrwTeij9rHljAG2dXVpL0/RdgGVs34LEUjaw OtfSnU1JhAAeU46HelFNWSKMGviZP6sktufEzeGhUceIQ1rCezWxRar3QsKL6LV1 3xilpIis/6JmL8VBCm3Y5EyX27Pf+evFq8R5c+vorIeNv8nVmIykQQkc/Q2cXZql 9B6Qcn0ONinqOjjbpiy2 =zngn -----END PGP SIGNATURE----- --6TrnltStXW4iwmi0-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 20:36:03 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 88AEFB4A for ; Wed, 13 Mar 2013 20:36:03 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from nk11p00mm-asmtp008.mac.com (nk11p00mm-asmtp008.mac.com [17.158.161.7]) by mx1.freebsd.org (Postfix) with ESMTP id 759B4807 for ; Wed, 13 Mar 2013 20:36:03 +0000 (UTC) Received: from cswiger1.apple.com ([17.209.4.71]) by nk11p00mm-asmtp008.mac.com (Oracle Communications Messaging Server 7u4-26.01(7.0.4.26.0) 64bit (built Jul 13 2012)) with ESMTPSA id <0MJM004R26FW6M70@nk11p00mm-asmtp008.mac.com> for freebsd-net@freebsd.org; Wed, 13 Mar 2013 19:35:57 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8626,1.0.431,0.0.0000 definitions=2013-03-13_08:2013-03-13,2013-03-13,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1302030000 definitions=main-1303130134 Subject: Re: ip_output() Error Handling in tcp_output() MIME-version: 1.0 (Apple Message framework v1085) Content-type: text/plain; charset=us-ascii From: Chuck Swiger In-reply-to: Date: Wed, 13 Mar 2013 12:35:56 -0700 Content-transfer-encoding: quoted-printable Message-id: <8D29B9DE-A59B-483D-92AD-30B794BD5417@mac.com> References: To: Matt Miller X-Mailer: Apple Mail (2.1085) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 20:36:03 -0000 Hi-- On Mar 13, 2013, at 8:21 AM, Matt Miller wrote: > If we have a connection that has received a SYN and ip_output() > returns, say, EHOSTUNREACH, is there anything that guarantees the > connection would always eventually be dropped if the condition > persists? If the local TCP stack is unable to reply with a SYN-ACK, then it hasn't = established a connection yet because it cannot proceed through the 3WHS = to ESTAB. It will stay in LISTEN state. Regards, --=20 -Chuck From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 22:27:57 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 56C4CE31; Wed, 13 Mar 2013 22:27:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id D93BC328; Wed, 13 Mar 2013 22:27:56 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAH/8QFGDaFvO/2dsb2JhbABDiCG8NYFtdIIqAQEEASNWBRYYAgINGQJZBhOIDgawJ5JbgSONOTQHgi2BEwOWWJECgyYggWw X-IronPort-AV: E=Sophos;i="4.84,840,1355115600"; d="scan'208";a="18875801" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 13 Mar 2013 18:27:49 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id C28C7B3F36; Wed, 13 Mar 2013 18:27:49 -0400 (EDT) Date: Wed, 13 Mar 2013 18:27:49 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1814312198.3878858.1363213669787.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20800.2898.798984.157469@hergotha.csail.mit.edu> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, andre@freebsd.org, Ivan Voras X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 22:27:57 -0000 Garrett Wollman wrote: > < said: > > > Basically, this patch: > > - allows setting of the tcp timeout via vfs.nfsd.tcpcachetimeo > > (I'd suggest you go down to a few minutes instead of 12hrs) > > - allows TCP caching to be disabled by setting vfs.nfsd.cachetcp=0 > > - does the above 2 things you describe to try and avoid the > > livelock, > > although not quite using an lru list > > - increases the hash table size to 500 (still a compile time > > setting) > > (feel free to make it even bigger) > > - sets nfsrc_floodlevel to at least nfsrc_tcphighwater, so you can > > grow vfs.nfsd.tcphighwater as big as you dare > > Thanks, this looks very good. One quibble with the last bit: I'd do > that in a sysctl() handler rather than checking it every time through. > If somebody uses a debugger rather than sysctl to change tcphighwater, > they deserve what's coming to them. Also, I might suggest adding a > counter for how many times we had to go through the "try harder" > phase, so that the sysadmin has some indication that the defaults need > adjustment. > I agree w.r.t. both comments. I did the one line setting of nfsrc_floodlevel, just because I was getting lazy while doing the patch. And if/when a patch like this (I think if this works, it can easily be added to ivoras@'s patch) goes into head, some way for a sysadmin to monitor how well it's working would be good. A count of the "try harder" attempts seems like a good candidate. Good luck with testing of it, rick > I will test this out later this week and see how it performs. I have > a user who has been able to reproducibly clobber servers before, so if > he has time and cycles available it should be pretty easy to tell > whether it's working or not. > > -GAWollman From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 01:12:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BBC96753 for ; Thu, 14 Mar 2013 01:12:36 +0000 (UTC) (envelope-from mattmiller1@gmail.com) Received: from mail-qe0-f41.google.com (mail-qe0-f41.google.com [209.85.128.41]) by mx1.freebsd.org (Postfix) with ESMTP id 81950BE0 for ; Thu, 14 Mar 2013 01:12:36 +0000 (UTC) Received: by mail-qe0-f41.google.com with SMTP id 6so997872qeb.14 for ; Wed, 13 Mar 2013 18:12:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=Ohgbj/LZ+Z1EVf+O/qhI9BmJtQuN9+IPlUomswTuMow=; b=Dt1Tl8baZ+Env6LFLBnO9lgyRNIrMxW+tHY3wLdN5ELduG/VPX29GTJ5nwSDxS09qO vpdid9qjYncO47eVhsAqLhkE1bwgGMLb3W36ExU/owzbNCzBQ4Zr1jLH3dQH9cwcLXgF P56yTLR4CBG1u/1KiqzMN65o5sylSkXriP8KJ1NXFKu/czP6KW1yrf5jHgjLhF6qu1tY YjKjFjSfcmyjIFxlcM/YY6+BfLSZj0aS4+uiMnqOi/m9SNARdgnKwzk40ftihMu6NMW1 Cp091WK0dQ9J7nFkDddwC4Nq/p8pl0njnX5M5sSQbUsASSBY7BQiPSHm2mWUJTORn2kL WW+g== X-Received: by 10.224.10.141 with SMTP id p13mr1317684qap.91.1363223550087; Wed, 13 Mar 2013 18:12:30 -0700 (PDT) MIME-Version: 1.0 Sender: mattmiller1@gmail.com Received: by 10.49.24.166 with HTTP; Wed, 13 Mar 2013 18:11:49 -0700 (PDT) In-Reply-To: <8D29B9DE-A59B-483D-92AD-30B794BD5417@mac.com> References: <8D29B9DE-A59B-483D-92AD-30B794BD5417@mac.com> From: Matt Miller Date: Wed, 13 Mar 2013 21:11:49 -0400 X-Google-Sender-Auth: 2w9vkFeTdSsNJk7sRkCALe67hNs Message-ID: Subject: Re: ip_output() Error Handling in tcp_output() To: Chuck Swiger Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 01:12:36 -0000 On Wed, Mar 13, 2013 at 3:35 PM, Chuck Swiger wrote: > Hi-- > > On Mar 13, 2013, at 8:21 AM, Matt Miller wrote: >> If we have a connection that has received a SYN and ip_output() >> returns, say, EHOSTUNREACH, is there anything that guarantees the >> connection would always eventually be dropped if the condition >> persists? > > If the local TCP stack is unable to reply with a SYN-ACK, then it hasn't established a connection yet because it cannot proceed through the 3WHS to ESTAB. It will stay in LISTEN state. > Sorry, I should have been more specific: say the connection has already progressed to at least ESTABLISHED or beyond when the ip_output() error occurs. So, we'll hit the TCPS_HAVERCVDSYN if block below: case EHOSTDOWN: case EHOSTUNREACH: case ENETDOWN: case ENETUNREACH: if (TCPS_HAVERCVDSYN(tp->t_state)) { tp->t_softerror = error; return (0); } Thanks, Matt > Regards, > -- > -Chuck > From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 06:44:37 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CC76FE5 for ; Thu, 14 Mar 2013 06:44:37 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (unknown [IPv6:2001:470:1f09:14c0::2]) by mx1.freebsd.org (Postfix) with ESMTP id 69E587AA for ; Thu, 14 Mar 2013 06:44:36 +0000 (UTC) Received: from bsdrookie.norma.com. ([IPv6:fd00::726]) by elf.hq.norma.perm.ru (8.14.5/8.14.5) with ESMTP id r2E6iXiR000906 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 14 Mar 2013 12:44:33 +0600 (YEKT) (envelope-from emz@norma.perm.ru) Message-ID: <514171D1.50807@norma.perm.ru> Date: Thu, 14 Mar 2013 12:44:33 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: FreeBSD 9.1-RELEASE + bge0 == watchdog timeout References: <512DE968.4020409@quip.cz> <20130228053558.GA1474@michelle.cdnetworks.com> <5136D89D.4000902@norma.perm.ru> <20130306062658.GC1483@michelle.cdnetworks.com> <513713C2.1000007@norma.perm.ru> <20130307022446.GB3108@michelle.cdnetworks.com> <513820E2.806@norma.perm.ru> <20130307062335.GB1478@michelle.cdnetworks.com> <51383E3B.5030007@norma.perm.ru> <20130307081548.GD1478@michelle.cdnetworks.com> <20130313015753.GD3062@michelle.cdnetworks.com> In-Reply-To: <20130313015753.GD3062@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [IPv6:fd00::30a]); Thu, 14 Mar 2013 12:44:33 +0600 (YEKT) X-Spam-Status: No hits=-97.8 bayes=0.5 testhits RDNS_NONE=1.274, SPF_SOFTFAIL=0.972,USER_IN_WHITELIST=-100 autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 06:44:37 -0000 Hi. On 13.03.2013 07:57, YongHyeon PYUN wrote: > > If your controller supports ASF/IPMI access please apply r248226 > to stable/8 and let me know whether that makes any difference. > I believe ignoring ASF/IPMI firmware is not good idea since the > ASF/IPMI firmware will run regardless of hw.bge.allow_asf loader > tunable configuration. You may have to set hw.bge.allow_asf=1 > since it's off by default on stable/8. I'm sorry, but obviously my C skills are way to low to make a local MFC of if_bge.c - it differs way too much from the 8.3-STABLE version, and it affects lots of other stuff (mostly in /usr/src/dev/mii and /usr/src/sys/sys). I tried to do this, but I failed completely. Could you please do a patch for me or can this be really MFC'd ? I got a bunch of bge timeouts on a 8.3-STABLE machine we talked about earlier just this morning. I really think this may become a problem for other people too. Yup, there's another option - to upgrade to 9.x, but right now I have unresolved issues with 9.x. Thanks. Eugene. From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 07:29:54 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 75716B02 for ; Thu, 14 Mar 2013 07:29:54 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pb0-f48.google.com (mail-pb0-f48.google.com [209.85.160.48]) by mx1.freebsd.org (Postfix) with ESMTP id 4D94E914 for ; Thu, 14 Mar 2013 07:29:53 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id wy12so1822275pbc.21 for ; Thu, 14 Mar 2013 00:29:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:date:to:cc:subject:message-id:reply-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=I/IvLQcSvAME9hUqikjqIFzSBWmPEFYVgIKB/3lPCuM=; b=cHbKQ4xXgdBGYm1KS5j2S6GVLjcyu7wkhbARXCYAJr+pchlc+o3DzBnr70cCtFyoFz FONDhQhRjC30Pg7qlTmYxOiPkmiK+STgLl6A6B5y0Tm2ulkNjmQinwl4YOHgjXg2NK9V T8JuwoaMQhlddjDosayt4qNhdvMX3GvpWxDRoJpriOaFbWEX05DoMPsz2kcLBb3MRvX/ KiIWSU4OsS7Jh+Q6Rqdem6xazKSUucF6gKjOJGEGl+0al5pfQFb/tXo5xP4Y2hhHi7VI Zxyv6FktRuD1FfDhzRIdYpnNTTEqziX8kYEPZy5eX+UlfecM6R5XcHvOL9NNla/Urpv1 wOTQ== X-Received: by 10.68.204.10 with SMTP id ku10mr3395141pbc.41.1363246193825; Thu, 14 Mar 2013 00:29:53 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPS id rr14sm1952978pbb.34.2013.03.14.00.29.50 (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 14 Mar 2013 00:29:52 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 14 Mar 2013 16:29:46 +0900 From: YongHyeon PYUN Date: Thu, 14 Mar 2013 16:29:46 +0900 To: "Eugene M. Zheganin" Subject: Re: FreeBSD 9.1-RELEASE + bge0 == watchdog timeout Message-ID: <20130314072946.GA1519@michelle.cdnetworks.com> References: <5136D89D.4000902@norma.perm.ru> <20130306062658.GC1483@michelle.cdnetworks.com> <513713C2.1000007@norma.perm.ru> <20130307022446.GB3108@michelle.cdnetworks.com> <513820E2.806@norma.perm.ru> <20130307062335.GB1478@michelle.cdnetworks.com> <51383E3B.5030007@norma.perm.ru> <20130307081548.GD1478@michelle.cdnetworks.com> <20130313015753.GD3062@michelle.cdnetworks.com> <514171D1.50807@norma.perm.ru> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="YZ5djTAD1cGYuMQK" Content-Disposition: inline In-Reply-To: <514171D1.50807@norma.perm.ru> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 07:29:54 -0000 --YZ5djTAD1cGYuMQK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Mar 14, 2013 at 12:44:33PM +0600, Eugene M. Zheganin wrote: > Hi. > > On 13.03.2013 07:57, YongHyeon PYUN wrote: > > > > If your controller supports ASF/IPMI access please apply r248226 > > to stable/8 and let me know whether that makes any difference. > > I believe ignoring ASF/IPMI firmware is not good idea since the > > ASF/IPMI firmware will run regardless of hw.bge.allow_asf loader > > tunable configuration. You may have to set hw.bge.allow_asf=1 > > since it's off by default on stable/8. > I'm sorry, but obviously my C skills are way to low to make a local MFC > of if_bge.c - it differs way too much from the 8.3-STABLE version, and > it affects lots of other stuff (mostly in /usr/src/dev/mii and > /usr/src/sys/sys). I tried to do this, but I failed completely. Could I thought you were using stable/8 but it seems you have slightly older stable/8. The bge(4) code difference between CURRENT and stable9/stable8 is very minor. > you please do a patch for me or can this be really MFC'd ? I got a bunch > of bge timeouts on a 8.3-STABLE machine we talked about earlier just > this morning. I really think this may become a problem for other people > too. Yup, there's another option - to upgrade to 9.x, but right now I > have unresolved issues with 9.x. I've attached diff against stable/8 which will address ASF/IPMI issue but I'm not sure whether it helps watchdog timeout or not. I have plan to MFC the change but I need time for settlement before the MFC. --YZ5djTAD1cGYuMQK Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="bge.asf.stable8.diff" Index: sys/dev/bge =================================================================== --- sys/dev/bge (revision 248264) +++ sys/dev/bge (working copy) Property changes on: sys/dev/bge ___________________________________________________________________ Modified: svn:mergeinfo Merged /head/sys/dev/bge:r248226 Index: sys/dev/bge/if_bge.c =================================================================== --- sys/dev/bge/if_bge.c (revision 248264) +++ sys/dev/bge/if_bge.c (working copy) @@ -3637,15 +3637,15 @@ bge_attach(device_t dev) } bge_stop_fw(sc); - bge_sig_pre_reset(sc, BGE_RESET_START); + bge_sig_pre_reset(sc, BGE_RESET_SHUTDOWN); if (bge_reset(sc)) { device_printf(sc->bge_dev, "chip reset failed\n"); error = ENXIO; goto fail; } - bge_sig_legacy(sc, BGE_RESET_START); - bge_sig_post_reset(sc, BGE_RESET_START); + bge_sig_legacy(sc, BGE_RESET_SHUTDOWN); + bge_sig_post_reset(sc, BGE_RESET_SHUTDOWN); if (bge_chipinit(sc)) { device_printf(sc->bge_dev, "chip initialization failed\n"); @@ -3998,6 +3998,20 @@ bge_reset(struct bge_softc *sc) } else write_op = bge_writereg_ind; + if (sc->bge_asicrev != BGE_ASICREV_BCM5700 && + sc->bge_asicrev != BGE_ASICREV_BCM5701) { + CSR_WRITE_4(sc, BGE_NVRAM_SWARB, BGE_NVRAMSWARB_SET1); + for (i = 0; i < 8000; i++) { + if (CSR_READ_4(sc, BGE_NVRAM_SWARB) & + BGE_NVRAMSWARB_GNT1) + break; + DELAY(20); + } + if (i == 8000) { + if (bootverbose) + device_printf(dev, "NVRAM lock timedout!\n"); + } + } /* Take APE lock when performing reset. */ bge_ape_lock(sc, BGE_APE_LOCK_GRC); --YZ5djTAD1cGYuMQK-- From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 13:05:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DFA36931 for ; Thu, 14 Mar 2013 13:05:12 +0000 (UTC) (envelope-from bejoygthomas@yahoo.com) Received: from nm9-vm8.bullet.mail.sg3.yahoo.com (nm9-vm8.bullet.mail.sg3.yahoo.com [106.10.148.215]) by mx1.freebsd.org (Postfix) with SMTP id 344BCABC for ; Thu, 14 Mar 2013 13:05:11 +0000 (UTC) Received: from [106.10.166.114] by nm9.bullet.mail.sg3.yahoo.com with NNFMP; 14 Mar 2013 13:02:23 -0000 Received: from [106.10.167.144] by tm3.bullet.mail.sg3.yahoo.com with NNFMP; 14 Mar 2013 13:02:23 -0000 Received: from [127.0.0.1] by smtp117.mail.sg3.yahoo.com with NNFMP; 14 Mar 2013 13:02:23 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1363266143; bh=mi7zMZftvGEt7vIyDXDfj07LLDZ5RSkvTHspjyAslW0=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:From:Content-Type:Content-Transfer-Encoding:Subject:Date:Message-Id:To:Mime-Version:X-Mailer; b=OQTeeG+culLlhN3K5oIxzkcw4DkQwyecpdKDe3MBdF64QJgYzBrLi7YF0tQC+s3uIpik5ff4GWvVL4fhFofLXSyprV5xNzs9AGM8ZC0TErc74gd0TKjxy4+/k95cZEgqDJOtARFlaNhoA71bgw8xr5NRoYT5HVl9JopmMBS9ygE= X-Yahoo-Newman-Id: 597433.28861.bm@smtp117.mail.sg3.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 83JyW_8VM1lW6VbwPiocPcAlQcRdnw.tbZHcqvozWZn5GY. 6YsukBXYN0s3ZT.D6s6LfLScTBEECsiQvGcBCOHpuAYFL_Ky5C8lMUtyVWWt L3cpL0jaQwZbWLUxpIwLS9dtcRfRB4ZiM_.k8ZNiK7TBNOt2_NvEj0nK.r6f RlDCMl1sCHy2dxcXVKlcE1Yc_dYwuIzU8PwOYBViwd.3Hcs0x8043VYiNpF4 CBMpDmlaXw_8SI29Of9DZzciuJ.CtpuC_o_yvz007P85Sj5F1Fy4JR8VqBVE uIq9bdkQRNIqe5LnGOL7pw3O2Bgl0KUuyVYyONCV6YXM.bdA9SUPzQ069i6N eGmZBbQ6ZhqknQ2rMmm37kL9Vj0r_00JeDitEnTdgGV2VbASTX5J_ZjW2zBa 4QTSV25861m.y8JQ- X-Yahoo-SMTP: wUFYtX6swBBtUSvlG_G5EWi_ycgIVslBiQ-- X-Rocket-Received: from [192.168.0.3] (bejoygthomas@117.192.235.205 with plain) by smtp117.mail.sg3.yahoo.com with SMTP; 14 Mar 2013 06:02:23 -0700 PDT From: Bejoy Thomas Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Question regarding FreeBSD Congestion control Date: Thu, 14 Mar 2013 18:32:17 +0530 Message-Id: To: freebsd-net@freebsd.org Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 13:05:12 -0000 I have been looking at the FreeBSD congestion control code and had some = questions on the SACK enabled code flow . =20 With SACK the congestion window is retained as 1 MSS when the Fast = Retransmit is exited and Fast Recovery is entered,so cwnd growth seems = to be much slower than slow start, as we we are re transmitting drops = from the sack hole or new data. But in non-SACK case the congestion = window is restored to cwnd/2 +(dupacks-snd_limited)*MSS when Fast = retransmit is exited as per RFC3782. Could some one familiar with the = code please throw some light on why we fall back to a cwnd of 1MSS as we = enter fast recovery with SACK enabled. /*Congestion signal before ack. = */ cc_cong_signal(tp, th, = CC_NDUPACK); cc_ack_received(tp, th, = CC_DUPACK); tcp_timer_activate(tp, TT_REXMT, = 0); tp->t_rtttime =3D 0; if (tp->t_flags & = TF_SACK_PERMIT) { TCPSTAT_INC( = tcps_sack_recovery_episode); tp->sack_newdata =3D = tp->snd_nxt; tp->snd_cwnd =3D = tp->t_maxseg; <<<<<<<<< (void) tcp_output(tp); goto drop; = <<<<<<<<<< } tp->snd_nxt =3D th->th_ack; tp->snd_cwnd =3D tp->t_maxseg; (void) tcp_output(tp); KASSERT(tp->snd_limited <=3D 2, ("%s: tp->snd_limited too = big", __func__)); tp->snd_cwnd =3D = tp->snd_ssthresh + tp->t_maxseg * (tp->t_dupacks - = tp->snd_limited); <<<<<<<<< Bejoy Thomas= From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 14:47:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3DC824DC for ; Thu, 14 Mar 2013 14:47:19 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) by mx1.freebsd.org (Postfix) with ESMTP id CDE641D4 for ; Thu, 14 Mar 2013 14:47:18 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id hq4so3856049wib.12 for ; Thu, 14 Mar 2013 07:47:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:content-type:content-transfer-encoding:subject :message-id:date:to:mime-version:x-mailer:x-gm-message-state; bh=uo251IlTJxAddOLdWTarNEnOleFUyhmT+ogMRr9SPZI=; b=LqjmcNl5ATCFmXoGqvcU7L5I9oESoE0J/CX8L7u/KIhCuYeTmFiDUZc+/6W5/KKwLq YqWybTT5FOdShc00wbij9iWRvfTfTlSRkWS0b5nuDCYVHdvOxRbbN0BJCYSfOfos+r/x hIE2uxJ0TJmvCr4BBxlLwJ0Eotc+b6iBoeMdlwYN2XSnDH1HyUNfT3VtB0n23zPICqHg V3YmW1DCDq6TG3JFLhtuHcnLRGe9t54qmYCTmsB9FcH+MpZFdG7Aib13Trn7rUgTVumE Iie2H8JkXtWbbdxqId26duiSNbuPGTt4kIslt04su4iXUKtdzCkNxWhsTSEfcDXoziGb CLYQ== X-Received: by 10.180.100.10 with SMTP id eu10mr34591532wib.4.1363272437774; Thu, 14 Mar 2013 07:47:17 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id ex1sm4496765wib.7.2013.03.14.07.47.15 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 14 Mar 2013 07:47:15 -0700 (PDT) From: Fleuriot Damien Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: carp regression in 9.1 ? Message-Id: <3B04FCB1-D0D4-4BC9-BB15-5221F438738C@my.gd> Date: Thu, 14 Mar 2013 15:47:11 +0100 To: "freebsd-net@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQkUWjH7uuGM7lE1j/NoZ8Pmsmp++ZxZYEDkFj+AwC3klACJUVSxIcpKvRmtNgk9Kq0snQiz X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 14:47:19 -0000 Hello list, I'm experiencing this odd behavior with 9.1 r24791 for amd64. Find below exerpts from my rc.conf related to carp: =3D=3D=3D # production ifconfig_carp22=3D"inet 192.168.22.254/32 vhid 122 pass vrrp22 advskew = 100" ifconfig_carp32=3D"inet 192.168.32.254/32 vhid 132 pass vrrp32 advskew = 100" cloned_interfaces=3D"carp0 carp1 carp22 carp32" # test interfaces, use different syntaxes ifconfig_carp1=3D"vhid 124 advskew 100 192.168.22.240 netmask = 255.255.255.0" ifconfig_carp0=3D"vhid 140 advskew 50 pass dsds 192.168.22.250" ipv4_addrs_carp0=3D"192.168.22.241-249/24" =3D=3D=3D With all of these configuration lines, carp interfaces come up at boot. The problem I experience is when destroying the carp interfaces and = recreating them: root@pf1-drt:~ # ifconfig carp1 destroy root@pf1-drt:~ # ifconfig carp1 create root@pf1-drt:~ # ifconfig carp1=20 carp1: flags=3D8 metric 0 mtu 1500 nd6 options=3D21 FreeBSD 8.3 doesn't exhibit the same behaviour, when destroying then = recreating the interfaces, they are initialized correctly. Additionally, note the syntax I used for my test interface carp0: ipv4_addrs_carp0=3D"192.168.22.241-249/24" And the results at boot: carp0: flags=3D49 metric 0 mtu 1500 inet 192.168.22.250 netmask 0xffffff00=20 inet 192.168.22.241 netmask 0xffffff00=20 inet 192.168.22.242 netmask 0xffffffff=20 inet 192.168.22.243 netmask 0xffffffff=20 inet 192.168.22.244 netmask 0xffffffff=20 inet 192.168.22.245 netmask 0xffffffff=20 inet 192.168.22.246 netmask 0xffffffff=20 inet 192.168.22.247 netmask 0xffffffff=20 inet 192.168.22.248 netmask 0xffffffff=20 inet 192.168.22.249 netmask 0xffffffff I'm really confused here, is this intended behaviour ? FreeBSD 8.3 certainly doesn't act this way. Does anyone have an idea what's happening there ? From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 15:00:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8AC16A3B for ; Thu, 14 Mar 2013 15:00:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 7CAC02B2 for ; Thu, 14 Mar 2013 15:00:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2EF01h1079754 for ; Thu, 14 Mar 2013 15:00:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2EF01EQ079753; Thu, 14 Mar 2013 15:00:01 GMT (envelope-from gnats) Date: Thu, 14 Mar 2013 15:00:01 GMT Message-Id: <201303141500.r2EF01EQ079753@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: John Baldwin Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: John Baldwin List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 15:00:01 -0000 The following reply was made to PR kern/176446; it has been noted by GNATS. From: John Baldwin To: "Charbon, Julien" Cc: bug-followup@freebsd.org, "De La Gueronniere, Marc" , jfv@freebsd.org Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST Date: Thu, 14 Mar 2013 09:34:18 -0400 On Thursday, March 07, 2013 5:11:25 am Charbon, Julien wrote: > On 2/28/13 8:10 PM, Charbon, Julien wrote: > > On 2/28/13 4:57 PM, John Baldwin wrote: > >> Can you try the fixes from http://svnweb.freebsd.org/base?view=revision&revision=240968? > > > > Actually, Marc (I CC'ed him) did find the r240968 fix for concurrency > > between ixgbe_msix_que() and ixgbe_handle_que(), and made a backport for > > release-8.3.0 (see patch [1] below). However, the issue was still > > reproducible, then Marc found another place for concurrency from > > ixgbe_local_timer() and fix it (see patch [2]). But it was still not > > enough, and he found a last place for concurrency due to > > ixgbe_rearm_queues() call (see patch [3]). We all these patches > > applied, we were not able to reproduce this issue. > > Just for the record: As expected this issue is reproducible on > 9.1-RELEASE: > > # uname -a > FreeBSD atlas 9.1-RELEASE FreeBSD 9.1-RELEASE #1 r247851M: Wed Mar 6 > 11:17:43 UTC 2013 > jcharbon@atlas:/usr/obj/app/jcharbon/9.1.0/sys/GENERIC amd64 > > Enable TCP debug log: > > # sysctl net.inet.tcp.log_debug=1 > > Load enough a TCP service and due to ixgbe race conditions between > ixgbe_msix_que() and ixgbe_handle_que(), you will get: > > Mar 7 10:01:04 atlas kernel: TCP: [192.168.100.21]:12918 to > [192.168.100.152]:8080; syncache_socket: in_pcbconnect failed with error 48 > Mar 7 10:01:04 atlas kernel: TCP: [192.168.100.21]:12918 to > [192.168.100.152]:8080 tcpflags 0x10; tcp_input: Listen socket: > Socket allocation failed due to limits or memory shortage, sending RST > Mar 7 10:01:04 atlas kernel: TCP: [192.168.100.21]:12918 to > [192.168.100.152]:8080 tcpflags 0x4; syncache_chkrst: Spurious RST > without matching syncache entry (possibly syncookie only), segment ignored > > We will provide our current fix patch for 9.1-RELEASE. The place you noticed in 2) is broken, though your fix isn't quite correct. I've been hesitant to reply yet as it requires a long reply. The short version is that the task to handle rx/tx processing should never be queued by anything other than an interrupt handler or itself (when it reschedules itself). Anything else that schedules it is going to result in lock contention and out-of-order packet delivery. Your 3rd case is also correct. We should not re-enable interrupts on every timer tick since the rx/tx task might already be running. Similarly, re- enabling all queues anytime one queue processes RX interrupts can trigger an interrupt on another queue while it's rx/tx task is already running. Both of these are pointless as each queue will rearm itself when the rx/tx task finds no more pending RX packets to process. Now, some more details on the 2nd one which is due to watchdog handling which is broken in both igb and ixgbe. First, some background on how watchdog handling works in nearly all other drivers (and specifically in single-queue drivers): First, each device maintains a 'timer' field in the softc which is a count of seconds until the transmit watchdog should expire. Whenever a packet is queued for transmit in the descriptor ring, it is set to the 'N' seconds (e.g. 5). Whenever the transmit completion interrupt fully drains the descriptor ring such that the ring is idle the timer is set to 0. Second, each device runs a periodic stats timer that fires once a second while the interface is "up" (so it is started in the foo_init() routine and stopped in foo_stop()). Part of this timer's job is to check the transmit watchdog. It uses logic like this to do so: if (timer > 0) { timer--; if (timer == 0) { /* watchdog expired */ } } The typical implementation for the watchdog expiring is to just reset the chip by doing 'foo_stop()' followed by 'foo_init_locked()'. However, if you have a NIC whose hardware is known to have a quirk where it can lose interrupts, then a driver can decide to scan the TX ring to see if it makes any progress. It should do this synchronously from the timer, not by scheduling another task. Also, if you do make progress, then you should reset the watchdog timer if there are still any pending transmits. In this case I would suggest only setting it to '1' so you check it on the next second. The psuedo-code for this would look something like: if (timer > 0) { timer--; if (timer == 0) { /* Have this return true if it finds any completions */ if (foo_txintr()) { if (tx ring is not empty) timer = 1; return; } foo_stop(); foo_init_locked(); } However, most drivers don't need that sort of complication at all as working hardware shouldn't be regularly failing to schedule interrupts. The one wrinkle that multiple queues throw into this is that it is hard to know when your transmit interrupt routine can clear 'timer' to 0 to disable it as you should only do so if all transmit queues are empty. One easy solution is to simply make the 'timer' field per transmit queue and have your stats timer check each queue. The remaining question then is when you do actually reset the chip. I think you should do it as soon as one queue becomes "hung" vs waiting for all of the queues to be hung. The issues I see with the igb/ixgbe driver's watchdog handling: 1) Rather than managing 'timer' as described above, they store the current walltime when a packet is transmitted and then try to decide in the stats timer if too much time has gone by. This makes things more complicated as you have to deal with 'ticks' rolling over, etc. Also, igb tries to determine this in the tx interrupt handler rather than doing it directly in the stats timer. 2) At least igb(4) tries to set IFF_DRV_OACTIVE when it thinks the chip is in the hung state. This is not what OACTIVE means and shouldn't be there at all. OACTIVE should be set in the if_start() case if the transmit ring is full and should be cleared either when the chip is reset or if a transmit completion interrupt frees up TX descriptors. 3) The watchdog handler queues a RX/TX task on every stats timer tick if there are any pending TX frames. a) It should only do this if the queue is hung, and b) it should only do TX processing, and c) it doesn't need to schedule a task. 4) At least igb(4) attempts to maintain a separate transmit queue status. I find that this actually makes things more complex and harder to understand and that is simpler to check the relevant flag instead. My patch for igb(4) to fix this watchdog handling is below. The fixes to ixgbe are probably similar since these drivers share many algorithms: Index: if_igb.c =================================================================== --- if_igb.c (revision 248162) +++ if_igb.c (working copy) @@ -856,7 +856,8 @@ igb_resume(device_t dev) !drbr_empty(ifp, txr->br)) igb_mq_start_locked(ifp, txr); #else - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) + if (((txr->queue_status & IGB_QUEUE_DEPLETED) == 0) && + !IFQ_DRV_IS_EMPTY(&ifp->if_snd)) igb_start_locked(txr, ifp); #endif IGB_TX_UNLOCK(txr); @@ -913,8 +914,10 @@ igb_start_locked(struct tx_ring *txr, struct ifnet if (igb_xmit(txr, &m_head)) { if (m_head != NULL) IFQ_DRV_PREPEND(&ifp->if_snd, m_head); - if (txr->tx_avail <= IGB_MAX_SCATTER) + if (txr->tx_avail <= IGB_MAX_SCATTER) { txr->queue_status |= IGB_QUEUE_DEPLETED; + ifp->if_drv_flags |= IFF_DRV_OACTIVE; + } break; } @@ -922,7 +925,7 @@ igb_start_locked(struct tx_ring *txr, struct ifnet ETHER_BPF_MTAP(ifp, m_head); /* Set watchdog on */ - txr->watchdog_time = ticks; + txr->watchdog_time = IGB_WATCHDOG; txr->queue_status |= IGB_QUEUE_WORKING; } } @@ -1018,7 +1021,7 @@ igb_mq_start_locked(struct ifnet *ifp, struct tx_r if (enq > 0) { /* Set the watchdog */ txr->queue_status |= IGB_QUEUE_WORKING; - txr->watchdog_time = ticks; + txr->watchdog_time = IGB_WATCHDOG; } if (txr->tx_avail <= IGB_TX_CLEANUP_THRESHOLD) igb_txeof(txr); @@ -1393,7 +1396,8 @@ igb_handle_que(void *context, int pending) !drbr_empty(ifp, txr->br)) igb_mq_start_locked(ifp, txr); #else - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) + if (((txr->queue_status & IGB_QUEUE_DEPLETED) == 0) && + !IFQ_DRV_IS_EMPTY(&ifp->if_snd)) igb_start_locked(txr, ifp); #endif IGB_TX_UNLOCK(txr); @@ -1444,7 +1448,8 @@ igb_handle_link_locked(struct adapter *adapter) !drbr_empty(ifp, txr->br)) igb_mq_start_locked(ifp, txr); #else - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) + if (((txr->queue_status & IGB_QUEUE_DEPLETED) == 0) && + !IFQ_DRV_IS_EMPTY(&ifp->if_snd)) igb_start_locked(txr, ifp); #endif IGB_TX_UNLOCK(txr); @@ -1581,7 +1586,8 @@ igb_msix_que(void *arg) !drbr_empty(ifp, txr->br)) igb_mq_start_locked(ifp, txr); #else - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) + if (((txr->queue_status & IGB_QUEUE_DEPLETED) == 0) && + !IFQ_DRV_IS_EMPTY(&ifp->if_snd)) igb_start_locked(txr, ifp); #endif IGB_TX_UNLOCK(txr); @@ -2055,7 +2061,7 @@ retry: tx_buffer = &txr->tx_buffers[first]; tx_buffer->next_eop = last; /* Update the watchdog time early and often */ - txr->watchdog_time = ticks; + txr->watchdog_time = IGB_WATCHDOG; /* * Advance the Transmit Descriptor Tail (TDT), this tells the E1000 @@ -2174,10 +2180,9 @@ igb_local_timer(void *arg) { struct adapter *adapter = arg; device_t dev = adapter->dev; - struct ifnet *ifp = adapter->ifp; struct tx_ring *txr = adapter->tx_rings; struct igb_queue *que = adapter->queues; - int hung = 0, busy = 0; + int hung = 0; IGB_CORE_LOCK_ASSERT(adapter); @@ -2185,28 +2190,28 @@ igb_local_timer(void *arg) igb_update_link_status(adapter); igb_update_stats_counters(adapter); + /* + * Don't check for any TX timeouts if the adapter received + * pause frames since the last tick or if the link is down. + */ + if (adapter->pause_frames != 0 || adapter->link_active == 0) + goto out; + /* ** Check the TX queues status - ** - central locked handling of OACTIVE - ** - watchdog only if all queues show hung + ** - watchdog if any queue hangs */ for (int i = 0; i < adapter->num_queues; i++, que++, txr++) { - if ((txr->queue_status & IGB_QUEUE_HUNG) && - (adapter->pause_frames == 0)) - ++hung; - if (txr->queue_status & IGB_QUEUE_DEPLETED) - ++busy; - if ((txr->queue_status & IGB_QUEUE_IDLE) == 0) - taskqueue_enqueue(que->tq, &que->que_task); + IGB_TX_LOCK(txr); + if (txr->watchdog_time >= 0) + if (--txr->watchdog_time == 0) + ++hung; + IGB_TX_UNLOCK(txr); } - if (hung == adapter->num_queues) + if (hung != 0) goto timeout; - if (busy == adapter->num_queues) - ifp->if_drv_flags |= IFF_DRV_OACTIVE; - else if ((ifp->if_drv_flags & IFF_DRV_OACTIVE) && - (busy < adapter->num_queues)) - ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; +out: adapter->pause_frames = 0; callout_reset(&adapter->timer, hz, igb_local_timer, adapter); #ifndef DEVICE_POLLING @@ -2349,13 +2354,13 @@ igb_stop(void *arg) callout_stop(&adapter->timer); /* Tell the stack that the interface is no longer active */ - ifp->if_drv_flags &= ~IFF_DRV_RUNNING; - ifp->if_drv_flags |= IFF_DRV_OACTIVE; + ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); /* Disarm watchdog timer. */ for (int i = 0; i < adapter->num_queues; i++, txr++) { IGB_TX_LOCK(txr); txr->queue_status = IGB_QUEUE_IDLE; + txr->watchdog_time = 0; IGB_TX_UNLOCK(txr); } @@ -3566,6 +3571,7 @@ igb_initialize_transmit_units(struct adapter *adap E1000_READ_REG(hw, E1000_TDBAL(i)), E1000_READ_REG(hw, E1000_TDLEN(i))); + txr->watchdog_time = 0; txr->queue_status = IGB_QUEUE_IDLE; txdctl |= IGB_TX_PTHRESH; @@ -3930,7 +3936,7 @@ igb_txeof(struct tx_ring *txr) tx_buffer->m_head = NULL; } tx_buffer->next_eop = -1; - txr->watchdog_time = ticks; + txr->watchdog_time = IGB_WATCHDOG; if (++first == adapter->num_tx_desc) first = 0; @@ -3955,24 +3961,20 @@ igb_txeof(struct tx_ring *txr) txr->next_to_clean = first; - /* - ** Watchdog calculation, we know there's - ** work outstanding or the first return - ** would have been taken, so none processed - ** for too long indicates a hang. - */ - if ((!processed) && ((ticks - txr->watchdog_time) > IGB_WATCHDOG)) - txr->queue_status |= IGB_QUEUE_HUNG; /* * If we have a minimum free, * clear depleted state bit */ - if (txr->tx_avail >= IGB_QUEUE_THRESHOLD) + if (txr->tx_avail >= IGB_QUEUE_THRESHOLD) { txr->queue_status &= ~IGB_QUEUE_DEPLETED; +#if __FreeBSD_version >= 800000 + ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; +#endif + } /* All clean, turn off the watchdog */ if (txr->tx_avail == adapter->num_tx_desc) { - txr->queue_status = IGB_QUEUE_IDLE; + txr->watchdog_time = 0; return (FALSE); } -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Thu Mar 14 21:59:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C299D943; Thu, 14 Mar 2013 21:59:50 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-oa0-f50.google.com (mail-oa0-f50.google.com [209.85.219.50]) by mx1.freebsd.org (Postfix) with ESMTP id 64994E17; Thu, 14 Mar 2013 21:59:50 +0000 (UTC) Received: by mail-oa0-f50.google.com with SMTP id l20so2748231oag.23 for ; Thu, 14 Mar 2013 14:59:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=FXG5JHFzahylqQuq+HBb2IMDwWJcXJIDSyNN/Eky6iw=; b=nbyEH1IoKeNcnF3NN4Dl1uYM3Yhu2EMjiDbBJOgDOrjMw9n3+uEMSYr9XcZAJ7WMPt kCKToZxu4S+c7QQ2KuaDxp4Lsj4hRDxnm/3Wl+AhOSUf9sAjZVuSaXuMGu9n0Y6x0knV dHAwTB7PE4/Hp456O9226SsAaS98hIvcw9XcI5AziUHYDOyA6tnDjn6ORV+7mkGhkVg3 Cyt3EcWg3DmgbqLUKTHVPyw8Owmk07zXWs8pqXtgojeZtdmdv4vud9c24Dk1ruCOcsUo woZfe+Bc8v6p/TfdSQBnNFkgnc2EKHd7GSmeWSrLrj5v0DS/Upd+4R91ZvijHYaNH/WP RJpQ== MIME-Version: 1.0 X-Received: by 10.60.12.41 with SMTP id v9mr1985133oeb.75.1363298384256; Thu, 14 Mar 2013 14:59:44 -0700 (PDT) Received: by 10.76.109.236 with HTTP; Thu, 14 Mar 2013 14:59:44 -0700 (PDT) In-Reply-To: <201303141500.r2EF01EQ079753@freefall.freebsd.org> References: <201303141500.r2EF01EQ079753@freefall.freebsd.org> Date: Thu, 14 Mar 2013 17:59:44 -0400 Message-ID: Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST From: Ryan Stone To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 21:59:50 -0000 What's the benefit in having a both an interrupt thread and task that performs the same function? It seems to me that having two threads that do the same job is what is making this so complicated. From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 07:33:10 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 191681A0; Fri, 15 Mar 2013 07:33:10 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-vc0-f176.google.com (mail-vc0-f176.google.com [209.85.220.176]) by mx1.freebsd.org (Postfix) with ESMTP id 82E867C4; Fri, 15 Mar 2013 07:33:09 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id ib11so1011944vcb.21 for ; Fri, 15 Mar 2013 00:33:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=3tT7O3YN/aOhZZNsL2FRqo1UFho6WPy+hHf57wA2mko=; b=aqRVPufMqMaIQEpPXa2I1rMcG2IJj6bcgF6yb+t0Bm5GYdiqvK9Fg0EctO7G8R/mS+ HTlNotTAX9avHsE/RNwgz1N7aVbR3Bzmh+DqGMfNH1PyYPaQztDGiH9/DD6sz05tWZYe vNav2qbMdf1NQOWJoNH/S1qof/+IiCMwRLC1DVWGOIg1pNA2Z0JFKqe5aRAhQftCB6Kd xGuBnzkTLPDGCpNgq+aNI70r9iTMqXyon2LlqE+apmFNSOfdlJ5uRb0uEdAZawL3jkAz nO0jGZM19kzsBmijn5IKvXULIttx43vtPZLO372s8pBEFJSBcB99tdjmes4N8WndQir5 /w6Q== MIME-Version: 1.0 X-Received: by 10.52.92.225 with SMTP id cp1mr4854739vdb.41.1363332783658; Fri, 15 Mar 2013 00:33:03 -0700 (PDT) Received: by 10.220.232.6 with HTTP; Fri, 15 Mar 2013 00:33:03 -0700 (PDT) Date: Fri, 15 Mar 2013 03:33:03 -0400 Message-ID: Subject: vlan on em0 cannot set MAC address. From: Zaphod Beeblebrox To: FreeBSD Net , FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 07:33:10 -0000 I have a FreeBSD-8.3 machine with an em0 interface in it. em0: port 0xd400-0xd43f mem 0xcffa0000-0xcffbffff,0xcff80000-0xcff9ffff irq 12 at device 17.0 on pci0 em0: [FILTER] em0: Ethernet address: 00:0e:0c:bc:6f:87 For various reasons, I have more than one DSL interface, and for some time I have run two PPPoE connections (using mpd4) and I run those PPPoE connections over VLANs on the em0 interface. There are two switches on the path and both switches are sufficiently competent to understand that two different VLANs include the same MAC address. I'm convinced that I used to be connected to two separate DSLAM devices and that after some line trouble, I was reassigned to two ports on a single device. This is because using the SAME MAC address on both VLANs stopped working. To fix this, I tried setting the MAC address on one of the VLANs. This does not work. em0 no longer transmits the packet. Expected behavior? Are there drivers that support different MAC addresses for different VLAN sub-devices? From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 09:04:38 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D305988A for ; Fri, 15 Mar 2013 09:04:38 +0000 (UTC) (envelope-from lucy.linkmanager@gmail.com) Received: from mail-pb0-f65.google.com (mail-pb0-f65.google.com [209.85.160.65]) by mx1.freebsd.org (Postfix) with ESMTP id B17C6B0E for ; Fri, 15 Mar 2013 09:04:38 +0000 (UTC) Received: by mail-pb0-f65.google.com with SMTP id rp8so2457548pbb.4 for ; Fri, 15 Mar 2013 02:04:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:to:subject:date:message-id:mime-version :content-type:x-mailer:thread-index:x-mimeole; bh=7scyvvD3jKud8vEfuRaipFdVvjSNPJJ2xkbUYnTOSNM=; b=m2JHlfsoK7tr9QMEBGyzcy3MP+83YDceEx19Rr9BgM2kY/JhKO5nXtY9lasxYHnxlG 2z+vI00qcU9SQmMFBLnawCfiHaX4Q0R83Go+flujVT+aR5iHWUe3ax7Zo0B07NArLGIV 2tjQ2FeVjqgh7d1bM+aS5ngXgjCKSnromjeDxaSG76JubeTAZh7Xs6R6PrnVrAkXiwW6 3kPH6njKdeccFgrWIFlgQh2q5yo4I98FJODQLA/G+//yoavAO9wSaGI0gUHUopPCcpj9 +qaNynCkVxoTjUnHOsnTwPzmxyjN/HHs1I0zta3W6w38sBDSif887hY48S/+JT2fH2ax jX1Q== X-Received: by 10.68.125.169 with SMTP id mr9mr13901538pbb.74.1363338277989; Fri, 15 Mar 2013 02:04:37 -0700 (PDT) Received: from viral ([122.177.21.88]) by mx.google.com with ESMTPS id y1sm7759364pbg.10.2013.03.15.02.04.34 (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 15 Mar 2013 02:04:37 -0700 (PDT) From: To: Subject: Web Design Date: Fri, 15 Mar 2013 14:34:19 +0530 Message-ID: <0d4301ce215c$1e5fb660$c305a8c0@viral> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 11 Thread-Index: Ac4hXBAk99+TGTrjQg6/m57u5zoEMA== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 09:04:38 -0000 freebsd-net@freebsd.org I am Lucy, Business Development Manager. I was on your website. Would you be interested in a possible redesigning of the site or addition feature that might benefit the overall usability and user experience which usually leads to better sales? We can execute the Web Design or SEO projects on a white label basis for you. Our aim is to be the best in service and as such we offer a premium service at very competitive prices than what it might be in house. We have a three tier quality check system so No compromise on quality! Our primary focus is:- SEO Web Design Web Development PPC Flash Design Graphic Designs Link Building Database programming E-Commerce Solutions Content Writing Blog Writing Mobile Web Design Local Directories Local Classifieds Local Yellow Pages Local Press Release Submission Google LBC Listing Local Reviews/Citations We provide Web Design & Development to a number of agencies and establishments and have expertise in- Developing highly effective corporate websites, flash designs, E- Commerce/Online Web Stores to name a few. PHP, MySQL, .NET, Joomla, Wordpress, Perl, Druple, ASP, JSP, HTML, CSS. Design- Flash, Photoshop, Corel E-Commerce - OS Commerce, Zen Cart, Virtue Mart, Pay Pal , Fedex and any checkout payment gateway systems. Do let me know if you are interested and I would be happy to share our Methodologies, past work details and client Testimonials and Prices. Kind Regards, Lucy Business Development Manager Email ID: lucy.linkmanager@gmail.com Disclaimer: The CAN-SPAM Act of 2003 (Controlling the Assault of Non-Solicited Pornography and Marketing Act) establishes requirements for those who send commercial email, spells out penalties for spammers and companies whose products are advertised in spam if they violate the law, and gives consumers the right to ask mailers to stop spamming them. The above mail is in accordance to the Can Spam act of 2003: There are no deceptive subject lines and is a manual process through our efforts on World Wide Web. You can opt out by sending id lucy.linkmanager@gmail.com and we ensure you will not receive any such mails. From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 14:43:26 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 96F9EC2 for ; Fri, 15 Mar 2013 14:43:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 7529568B for ; Fri, 15 Mar 2013 14:43:26 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id ED20EB943; Fri, 15 Mar 2013 10:43:25 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST Date: Fri, 15 Mar 2013 09:57:26 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <201303141500.r2EF01EQ079753@freefall.freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201303150957.26192.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 15 Mar 2013 10:43:26 -0400 (EDT) Cc: Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 14:43:26 -0000 On Thursday, March 14, 2013 5:59:44 pm Ryan Stone wrote: > What's the benefit in having a both an interrupt thread and task that > performs the same function? It seems to me that having two threads that do > the same job is what is making this so complicated. Yes, yes it is. I have a branch that has changes to interrupt threads where you can have an interrupt handler reschedule itself. That prevents this class of problems as the handler always runs in the interrupt thread. I really should get that patch into HEAD someday. I've posted it to arch@ twice now I think. :( It also fixes interrupt filters to really work properly and be on by default. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 14:48:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0B1003DC for ; Fri, 15 Mar 2013 14:48:28 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 567A66E7 for ; Fri, 15 Mar 2013 14:48:27 +0000 (UTC) Received: (qmail 68610 invoked from network); 15 Mar 2013 16:00:27 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 15 Mar 2013 16:00:27 -0000 Message-ID: <514334B6.9080001@freebsd.org> Date: Fri, 15 Mar 2013 15:48:22 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: John Baldwin Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST References: <201303141500.r2EF01EQ079753@freefall.freebsd.org> <201303150957.26192.jhb@freebsd.org> In-Reply-To: <201303150957.26192.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 14:48:28 -0000 On 15.03.2013 14:57, John Baldwin wrote: > On Thursday, March 14, 2013 5:59:44 pm Ryan Stone wrote: >> What's the benefit in having a both an interrupt thread and task that >> performs the same function? It seems to me that having two threads that do >> the same job is what is making this so complicated. > > Yes, yes it is. I have a branch that has changes to interrupt threads where > you can have an interrupt handler reschedule itself. That prevents this class > of problems as the handler always runs in the interrupt thread. > > I really should get that patch into HEAD someday. I've posted it to arch@ > twice now I think. :( It also fixes interrupt filters to really work properly > and be on by default. Do you have a link to that patch? -- Andre From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 16:28:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B17AD2B3 for ; Fri, 15 Mar 2013 16:28:36 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-pb0-f52.google.com (mail-pb0-f52.google.com [209.85.160.52]) by mx1.freebsd.org (Postfix) with ESMTP id 917CBB56 for ; Fri, 15 Mar 2013 16:28:36 +0000 (UTC) Received: by mail-pb0-f52.google.com with SMTP id ma3so4048825pbc.39 for ; Fri, 15 Mar 2013 09:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=5yGpbIBKKlcY5fvE2m0j6+pW71ar+8+piVfhOvdD5N4=; b=ACPtoN0uu9gyTYntI17mERpAELkgDE9VdadbiHJVH4/iOzoCUlyqsT32tN2V+j2kO4 W7mlU3+VZpLAlVhxSIH+7LE9o/Ow0svVy37rnQiLVwUJ5Gf97UCPsbhHiPXLJHZuFGSA iu6NajqxXfBQhIa3A6l8csfK5TpIbrEthFca8bb7NcyO1NJiDw7/AxxZL8OTaJ+gQbQV amgnRNmgrfLxeLvQu8BgYkONVrUr8JOQNv9lHjlcl4ZJzpnaxhMZbYhD7ghjHBvzqJdg GJqBkqA8Nznjs4EQZHNmI5HITUb/P2OC8sY6D5llUqHHwLRF+5sQIw9av0HTixrqFIzQ x1kQ== MIME-Version: 1.0 X-Received: by 10.68.189.99 with SMTP id gh3mr16988319pbc.203.1363364909824; Fri, 15 Mar 2013 09:28:29 -0700 (PDT) Received: by 10.70.34.103 with HTTP; Fri, 15 Mar 2013 09:28:29 -0700 (PDT) Received: by 10.70.34.103 with HTTP; Fri, 15 Mar 2013 09:28:29 -0700 (PDT) In-Reply-To: References: Date: Fri, 15 Mar 2013 18:28:29 +0200 Message-ID: Subject: MPLS From: Sami Halabi To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 16:28:36 -0000 Hi, Are there ongoing job of mpls in freebsd? I saw thd site http://freebsd.mpls.in for aboug a year now and I don't see much progress. ITOH OpenBSD has a complete implementation of MPLS out of the box, maybe porting it would be short and more straight forward than porting linux LDP implementation of BIRD. Thanks in advance, Sami From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 21:54:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E22FFF4E for ; Fri, 15 Mar 2013 21:54:51 +0000 (UTC) (envelope-from m-freebsd@fuglos.org) Received: from m.fuglos.org (m.fuglos.org [217.11.61.114]) by mx1.freebsd.org (Postfix) with ESMTP id A57C5857 for ; Fri, 15 Mar 2013 21:54:51 +0000 (UTC) Received: by m.fuglos.org (Postfix, from userid 1001) id 0243235C411; Fri, 15 Mar 2013 22:54:42 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by m.fuglos.org (Postfix) with ESMTP id A4D9135C3FA for ; Fri, 15 Mar 2013 22:54:42 +0100 (CET) Date: Fri, 15 Mar 2013 22:54:42 +0100 (CET) From: "M. Schulte" X-X-Sender: mel@m.fuglos.org To: freebsd-net@freebsd.org Subject: Re: ntpd bind() failure: Can't assign requested address In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 21:54:51 -0000 Hi, > What is the address space is the address you are seeing on the interface? > Is it is a link-local address or one assigned by the router, a static from > rc.conf or DHCPv6? You should have two inet6 addresses, one is link-local > (starts with "fe") and another should start with "2". There may be more > than two. I fear I cannot answer all of these questions. I don't have any IPv6 addresses configured in rc.conf. Also, I am not using dhcp. Basically I only have IPv4 networking configured with static addresses -- neither have I actively _disabled_ IPv6. This is my ifconfig output: re0: flags=8843 metric 0 mtu 1500 options=8009b ether 00:16:36:74:20:76 inet XXX.XXX.XXX.XXX netmask 0xfffffff0 broadcast XXX.XXX.XXX.XXX inet6 fe80::216:36ff:fe74:2076%re0 prefixlen 64 scopeid 0x2 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 lo1: flags=8049 metric 0 mtu 16384 options=600003 inet 192.168.1.1 netmask 0xffffff00 inet 192.168.1.2 netmask 0xffffff00 inet 192.168.1.3 netmask 0xffffff00 nd6 options=29 Also, I have noticed something: when i disable ntpd_enable in rc.conf and start ntpd manually after boot up, no such errors are logged. So I had the impression that is must have something to do with the timing. I have changed /etc/rc.d/ntpd in such a way that it logs the output of ifconfig at the beginning. But... there's no difference to the ifconfig output above. Hence I am not sure what's going on. Thanks for any help! ~ melanie From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 23:29:00 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4E360697; Fri, 15 Mar 2013 23:29:00 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 298C65F7; Fri, 15 Mar 2013 23:29:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r2FNSxHX053040; Fri, 15 Mar 2013 23:28:59 GMT (envelope-from eadler@freefall.freebsd.org) Received: (from eadler@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r2FNSxL2053039; Fri, 15 Mar 2013 23:28:59 GMT (envelope-from eadler) Date: Fri, 15 Mar 2013 23:28:59 GMT Message-Id: <201303152328.r2FNSxL2053039@freefall.freebsd.org> To: fming@borderware.com, eadler@FreeBSD.org, freebsd-net@FreeBSD.org From: eadler@FreeBSD.org Subject: Re: kern/85320: [gre] [patch] possible depletion of kernel stack in ip_gre.c when net.isr.enable = 1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 23:29:00 -0000 Synopsis: [gre] [patch] possible depletion of kernel stack in ip_gre.c when net.isr.enable = 1 State-Changed-From-To: patched->closed State-Changed-By: eadler State-Changed-When: Fri Mar 15 23:28:59 UTC 2013 State-Changed-Why: per Zeising http://www.freebsd.org/cgi/query-pr.cgi?pr=85320 From owner-freebsd-net@FreeBSD.ORG Fri Mar 15 23:53:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2517EA46 for ; Fri, 15 Mar 2013 23:53:06 +0000 (UTC) (envelope-from m-freebsd@fuglos.org) Received: from m.fuglos.org (m.fuglos.org [217.11.61.114]) by mx1.freebsd.org (Postfix) with ESMTP id E10146D4 for ; Fri, 15 Mar 2013 23:53:05 +0000 (UTC) Received: by m.fuglos.org (Postfix, from userid 1001) id EB8D235C411; Sat, 16 Mar 2013 00:53:04 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by m.fuglos.org (Postfix) with ESMTP id E275735C3FA for ; Sat, 16 Mar 2013 00:53:04 +0100 (CET) Date: Sat, 16 Mar 2013 00:53:04 +0100 (CET) From: "M. Schulte" X-X-Sender: mel@m.fuglos.org To: freebsd-net@freebsd.org Subject: Re: ntpd bind() failure: Can't assign requested address In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 23:53:06 -0000 To add a few notes: I did some more debugging/testing and this behaviour seems to be a bit random. Sometimes I boot the system and no error is triggered.... the number of 'cannot assign requested errors' varies between 0 and 2 on my system and I have no idea what might influence it. :-/ Thanks, melanie From owner-freebsd-net@FreeBSD.ORG Sat Mar 16 08:59:15 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 481C8FE3 for ; Sat, 16 Mar 2013 08:59:15 +0000 (UTC) (envelope-from ume@mahoroba.org) Received: from mail.mahoroba.org (ent.mahoroba.org [IPv6:2001:2f0:104:8010::1]) by mx1.freebsd.org (Postfix) with ESMTP id A447980A for ; Sat, 16 Mar 2013 08:59:13 +0000 (UTC) Received: from yuga.mahoroba.org (ume@yuga-m.mahoroba.org [IPv6:2001:2f0:104:8016:7258:12ff:fe22:d94b]) (user=ume mech=DIGEST-MD5 bits=0) by mail.mahoroba.org (8.14.6/8.14.6) with ESMTP/inet6 id r2G8x1pX004503 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 16 Mar 2013 17:59:07 +0900 (JST) (envelope-from ume@mahoroba.org) Date: Sat, 16 Mar 2013 17:58:59 +0900 Message-ID: From: Hajimu UMEMOTO To: "M. Schulte" Subject: Re: ntpd bind() failure: Can't assign requested address In-Reply-To: References: User-Agent: xcite1.60> Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-2022-JP-2?B?R29qGyQoRCtXGyhC?=) APEL/10.8 Emacs/24.2 (i386-portbld-freebsd9.1) MULE/6.0 (HANACHIRUSATO) X-Operating-System: FreeBSD 9.1-STABLE X-PGP-Key: http://www.mahoroba.org/~ume/publickey.asc X-PGP-Fingerprint: 1F00 0B9E 2164 70FC 6DC5 BF5F 04E9 F086 BF90 71FE MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: multipart/mixed; boundary="Multipart_Sat_Mar_16_17:58:59_2013-1" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.mahoroba.org [IPv6:2001:2f0:104:8010::1]); Sat, 16 Mar 2013 17:59:07 +0900 (JST) X-Virus-Scanned: clamav-milter 0.97.7 at asuka.mahoroba.org X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on asuka.mahoroba.org Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 08:59:15 -0000 --Multipart_Sat_Mar_16_17:58:59_2013-1 Content-Type: text/plain; charset=US-ASCII Hi, >>>>> On Fri, 15 Mar 2013 22:54:42 +0100 (CET) >>>>> "M. Schulte" said: m-freebsd> I fear I cannot answer all of these questions. I don't have any IPv6 m-freebsd> addresses configured in rc.conf. Also, I am not using dhcp. Basically m-freebsd> I only have IPv4 networking configured with static addresses -- m-freebsd> neither have I actively _disabled_ IPv6. m-freebsd> This is my ifconfig output: m-freebsd> re0: flags=8843 metric 0 mtu 1500 m-freebsd> options=8009b m-freebsd> ether 00:16:36:74:20:76 m-freebsd> inet XXX.XXX.XXX.XXX netmask 0xfffffff0 broadcast XXX.XXX.XXX.XXX m-freebsd> inet6 fe80::216:36ff:fe74:2076%re0 prefixlen 64 scopeid 0x2 m-freebsd> nd6 options=29 m-freebsd> media: Ethernet autoselect (100baseTX ) m-freebsd> status: active It seems IPv6 is disabled by IFDISABLED nd6 option. Please try the attached patch to ignore it. Sincerely, --Multipart_Sat_Mar_16_17:58:59_2013-1 Content-Type: text/x-patch; type=patch; charset=US-ASCII Content-Disposition: attachment; filename="ntpd-ifdisabled.diff" Content-Transfer-Encoding: 7bit Index: contrib/ntp/ntpd/ntp_io.c diff -u -p contrib/ntp/ntpd/ntp_io.c.orig contrib/ntp/ntpd/ntp_io.c --- contrib/ntp/ntpd/ntp_io.c.orig 2012-10-01 02:28:37.157635135 +0900 +++ contrib/ntp/ntpd/ntp_io.c 2013-03-16 17:30:51.384215167 +0900 @@ -69,6 +69,7 @@ #include #include #include +#include #endif /* !INCLUDE_IPV6_SUPPORT */ extern int listen_to_virtual_ips; @@ -1202,6 +1203,28 @@ is_anycast(struct sockaddr *sa, char *na #endif /* !SIOCGIFAFLAG_IN6 || !IN6_IFF_ANYCAST */ return ISC_FALSE; } + +static isc_boolean_t +is_ifdisabled(char *name) +{ +#if defined(SIOCGIFINFO_IN6) && defined(ND6_IFF_IFDISABLED) + struct in6_ndireq nd; + int fd; + + if ((fd = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) + return ISC_FALSE; + memset(&nd, 0, sizeof(nd)); + strlcpy(nd.ifname, name, sizeof(nd.ifname)); + if (ioctl(fd, SIOCGIFINFO_IN6, &nd) < 0) { + close(fd); + return ISC_FALSE; + } + close(fd); + if ((nd.ndi.flags & ND6_IFF_IFDISABLED) != 0) + return ISC_TRUE; +#endif /* !SIOCGIFAFLAG_IN6 || !IN6_IFF_ANYCAST */ + return ISC_FALSE; +} #endif /* !INCLUDE_IPV6_SUPPORT */ /* @@ -1335,6 +1358,16 @@ update_interfaces( continue; } +#ifdef INCLUDE_IPV6_SUPPORT + if (interface.sin.ss_family == AF_INET6 && + is_ifdisabled(interface.name)) { + DPRINTF(4, + ("skipping interface %s (%s) - IFDISABLED\n", + interface.name, stoa(&interface.sin))); + continue; + } +#endif /* !INCLUDE_IPV6_SUPPORT */ + /* * skip any interfaces UP and bound to a wildcard * address - some dhcp clients produce that in the -- Hajimu UMEMOTO ume@mahoroba.org ume@{,jp.}FreeBSD.org http://www.mahoroba.org/~ume/ --Multipart_Sat_Mar_16_17:58:59_2013-1--